Abstractions and Contracts

Audience: anyone designing or implementing a service, integration, or driver. Read after 00-overview.md and 01-architecture-summary.md.

Purpose: to define the interface boundaries that let WiqAIa+ accept “anything” — any camera vendor, any fire panel protocol, any NVR brand, any LLM, any 3D model format — without rewriting the platform. The pilot ships one production-quality driver per type; the slot is real and additional drivers slot in cleanly later.

This is the doc that prevents the platform from becoming a tangle. Read it before adding any external integration.

1. The packages/contracts/ package

A single shared package, in the monorepo at packages/contracts/, owns every wire contract. It is the only package that both the TypeScript side (frontend + cloud API) and the Python side (edge app + AI worker) and the Go side (edge supervisor) all depend on.

Drivers depend on contracts. Consumers depend on contracts. Drivers and consumers never depend on each other. That is the rule that keeps this clean.

packages/contracts/
├── proto/                           — protobuf source files (canonical wire definitions)
│   ├── events/                      — event types (frame, alarm, alert, telemetry, audit)
│   ├── api/                         — gRPC service definitions (intra-cluster service-to-service)
│   ├── edge/                        — edge ↔ cloud tunnel messages
│   └── ai/                          — AI inference inputs / outputs
├── schemas/                         — JSON Schema (configuration, REST DTOs, dashboard data)
├── ts/                              — generated TypeScript types + zod validators
├── py/                              — generated Python dataclasses + pydantic validators
├── go/                              — generated Go structs
├── CHANGELOG.md                     — SemVer history, with migration notes per breaking change
└── README.md

A single pnpm build (or make target) in this package runs:

1. protoc → TypeScript / Python / Go bindings
2. quicktype or json-schema-to-typescript → TS types + zod validators
3. datamodel-code-generator → Python pydantic models
4. SemVer check against the previous release (breaking changes fail unless a major version bump is in the PR)

Every other workspace pulls the generated bindings as a normal package dependency.

2. Wire format choices

We do not use Avro, BSON, MessagePack, or other formats unless a compelling reason emerges; the cognitive cost of mixing wire formats outweighs marginal compression gains.

3. Versioning policy

packages/contracts/ follows SemVer:

• Patch: doc-only changes, comments, formatting.
• Minor: backward-compatible additions — new optional fields, new message types, new enum values added at the end.
• Major: breaking changes — removed or renamed fields, type changes, semantic changes to existing fields. Triggers a coordinated migration plan and an entry in CHANGELOG.md with explicit migration guidance.

Producers (services that emit messages) can emit the latest minor schema; consumers should be tolerant of older schemas within the same major version. Cross-major-version interop requires an explicit translation layer (rare; planned ahead).

Edge gateways in the field may run an older minor version while the cloud runs a newer one. This is supported as long as no breaking change has shipped. The fleet manager (M09) tracks each edge’s contracts version and surfaces stale ones for update.

4. Canonical events

The event types that flow through Pub/Sub and the edge tunnel. These are the platform’s language. Adding a new event type is a contracts package PR; adding a new field is a minor bump.

4.1 FrameEvent

A camera frame, post-Tier-0, en route to cloud AI.

message FrameEvent {
  string event_id = 1;               // UUID
  string tenant_id = 2;
  string building_id = 3;
  string camera_id = 4;
  google.protobuf.Timestamp captured_at = 5;
  bytes jpeg_data = 6;               // ~50 KB typical
  int32 width_px = 7;
  int32 height_px = 8;
  float motion_score = 9;            // Tier 0 output, 0..1
  optional bytes reference_frame_hash = 10;   // For dedup
  EdgeMetadata edge = 11;            // edge gateway info
}

4.2 AnalysisEvent

Output from Tier 1 — what the fast models concluded about the frame.

message AnalysisEvent {
  string event_id = 1;
  string frame_event_id = 2;
  string tenant_id = 3;
  string building_id = 4;
  string camera_id = 5;
  string model_version_id = 6;
  AnalysisType type = 7;             // PERSON_DETECTION, ANOMALY, CROWD_DENSITY, etc.
  int32 person_count = 8;
  float anomaly_score = 9;           // 0..1
  float confidence = 10;
  repeated Detection detections = 11;   // bounding boxes
  optional HeatmapGrid heatmap = 12;
  int32 processing_ms = 13;
  google.protobuf.Timestamp analyzed_at = 14;
  EscalationDecision escalation = 15;   // ESCALATE_TIER_2, NO_ESCALATE, ESCALATE_PRIORITY
}

4.3 Tier2AnalysisEvent

Output from the multimodal LLM. The signature deep-analysis result.

message Tier2AnalysisEvent {
  string event_id = 1;
  string frame_event_id = 2;
  string analysis_event_id = 3;
  string tenant_id = 4;
  string building_id = 5;
  string camera_id = 6;
  string llm_model_id = 7;
  string prompt_template_id = 8;
  bool threat_detected = 9;
  ThreatType threat_type = 10;       // FIRE, SMOKE, CROWD_HAZARD, FALL, INTRUSION, etc.
  Severity severity = 11;            // INFO, LOW, MODERATE, HIGH, CRITICAL
  float confidence = 12;
  float false_alarm_likelihood = 13;
  string reasoning = 14;             // human-readable explanation
  string recommended_action = 15;
  int32 input_tokens = 16;
  int32 output_tokens = 17;
  int32 processing_ms = 18;
  google.protobuf.Timestamp analyzed_at = 19;
}

4.4 AlarmEvent

Fire panel, smoke detector, or other building-safety system event.

message AlarmEvent {
  string event_id = 1;
  string tenant_id = 2;
  string building_id = 3;
  string source_system_id = 4;       // which adapter / panel
  AlarmType alarm_type = 5;          // FIRE, SMOKE, HEAT, MANUAL_PULL, SUPERVISORY, TROUBLE, ELEVATOR_FAULT
  string zone_id = 6;                // panel-native zone identifier
  string mapped_zone_id = 7;         // our canonical Zone UUID, post-mapping
  Severity severity = 8;
  AlarmState state = 9;              // ACTIVE, ACKNOWLEDGED, CLEARED
  google.protobuf.Timestamp occurred_at = 10;
  map<string, string> protocol_attributes = 11;   // BACnet / Modbus / vendor-specific extras
}

4.5 Alert

The platform’s user-visible output: a correlated, prioritized incident with evidence.

message Alert {
  string alert_id = 1;
  string tenant_id = 2;
  string building_id = 3;
  string zone_id = 4;
  AlertType type = 5;                // FIRE, SMOKE, CROWD_EXCESS, RESTRICTED_ZONE, ANOMALY, EQUIPMENT_FAULT, CONNECTIVITY
  Severity severity = 6;
  AlertStatus status = 7;            // NEW, ACKNOWLEDGED, INVESTIGATING, ESCALATED, RESOLVED, FALSE_ALARM
  float confidence = 8;
  string title = 9;
  string description = 10;
  repeated EvidenceItem evidence = 11;        // camera screenshots, LLM verdicts, alarm refs
  repeated string source_event_ids = 12;      // FrameEvent / AnalysisEvent / AlarmEvent IDs
  google.protobuf.Timestamp created_at = 13;
  google.protobuf.Timestamp acknowledged_at = 14;
  google.protobuf.Timestamp resolved_at = 15;
  string resolved_by_user_id = 16;
  string resolution_notes = 17;
}

4.6 EdgeTelemetryEvent

Heartbeat + health from each edge gateway.

message EdgeTelemetryEvent {
  string event_id = 1;
  string edge_gateway_id = 2;
  string tenant_id = 3;
  string building_id = 4;
  google.protobuf.Timestamp captured_at = 5;
  EdgeStatus status = 6;             // ONLINE, DEGRADED, OFFLINE
  int32 connected_cameras = 7;
  int32 healthy_cameras = 8;
  int32 alarm_panels_connected = 9;
  float cpu_percent = 10;
  float memory_percent = 11;
  float disk_percent = 12;
  int32 queued_events = 13;
  string software_version = 14;
  map<string, string> container_versions = 15;
}

4.7 AuditLogEntry

Every state-changing or sensitive-access action.

message AuditLogEntry {
  string log_id = 1;
  string tenant_id = 2;
  string actor_user_id = 3;          // null for system actions
  string actor_service = 4;          // for system actions
  AuditCategory category = 5;        // AUTH, AUTHZ, DATA_ACCESS, DATA_MUTATION, CONFIG, FLEET, KEY_CUSTODY
  string action = 6;                 // canonical verb: "camera.view", "alert.acknowledge", etc.
  string entity_type = 7;
  string entity_id = 8;
  google.protobuf.Struct changes = 9;     // before/after diff for mutations
  string ip_address = 10;
  string user_agent = 11;
  google.protobuf.Timestamp performed_at = 12;
  bytes prev_hash = 13;              // hash of the previous audit row (chain)
  bytes self_hash = 14;              // hash of this row's contents
}

5. Driver interfaces

For each pluggable type, the interface a driver implements. We define these once in packages/contracts/; the pilot ships one driver per interface; later drivers conform to the same shape.

5.1 CameraTransport

How the system gets video frames and live streams from a camera (or from an NVR proxying cameras).

class CameraTransport(Protocol):
    async def discover(self, network_subnet: str) -> list[CameraInfo]: ...
    async def connect(self, camera_id: str, credentials: Credentials) -> CameraConnection: ...
    async def stream_url(self, camera_id: str, profile: StreamProfile) -> str: ...   # RTSP URL or equivalent
    async def snapshot(self, camera_id: str) -> bytes: ...
    async def ptz(self, camera_id: str, command: PtzCommand) -> None: ...   # may raise NotSupported
    async def health(self, camera_id: str) -> HealthStatus: ...

Pilot driver: OnvifRtspTransport (ONVIF discovery, RTSP streams). Future drivers: HikvisionIsapiTransport, DahuaRpcTransport, MilestoneMipTransport, OnvifNvrTransport (NVR-routed).

5.2 FirePanelAdapter

Read alarm events from a fire alarm control panel (FACP). Read-only: we never command fire panels.

class FirePanelAdapter(Protocol):
    async def connect(self, config: FirePanelConfig) -> None: ...
    async def subscribe(self) -> AsyncIterator[AlarmEvent]: ...   # streaming
    async def list_zones(self) -> list[ZoneInfo]: ...
    async def panel_health(self) -> HealthStatus: ...
    async def disconnect(self) -> None: ...

Pilot driver: TBD on confirmation of the Saudi Airlines fire panel model (see reminders.md). Most likely BacnetIpAdapter (Honeywell, Siemens, Johnson Controls) or ModbusTcpAdapter. Future drivers: NotifierProprietaryAdapter, EstProprietaryAdapter, MqttSmartPanelAdapter, ContactClosureAdapter, HochikiAdapter, GstAdapter.

5.3 NvrAdapter (optional — only for buildings where cameras route through an NVR)

class NvrAdapter(Protocol):
    async def connect(self, config: NvrConfig) -> None: ...
    async def list_cameras(self) -> list[CameraInfo]: ...
    async def stream_url(self, camera_id: str, profile: StreamProfile) -> str: ...
    async def search_recordings(self, camera_id: str, time_range: TimeRange) -> list[RecordingSegment]: ...
    async def playback_url(self, recording_id: str) -> str: ...
    async def health(self) -> HealthStatus: ...

Pilot driver: TBD on NVR access (see reminders.md). If we get NVR access, likely OnvifProfileGAdapter first. Future drivers: HikvisionIsapiNvrAdapter, DahuaRpcNvrAdapter, MilestoneMipAdapter.

5.4 ElevatorAdapter (out of pilot scope — interface defined for forward compatibility)

class ElevatorAdapter(Protocol):
    async def subscribe(self) -> AsyncIterator[ElevatorTelemetry]: ...
    async def list_cars(self) -> list[ElevatorCar]: ...
    async def health(self) -> HealthStatus: ...

Interface exists so future buildings with elevators slot in cleanly. No pilot driver.

5.5 LlmProvider

The Tier 2 multimodal model.

class LlmProvider(Protocol):
    async def initialize(self, config: LlmConfig) -> None: ...
    async def analyze(self, request: LlmAnalysisRequest) -> LlmAnalysisResponse: ...
    async def health(self) -> HealthStatus: ...
    @property
    def supports_vision(self) -> bool: ...
    @property
    def model_id(self) -> str: ...
    @property
    def context_window(self) -> int: ...

LlmAnalysisRequest carries the frame, the prompt template ID, and structured context. LlmAnalysisResponse is parsed by a pydantic model with the canonical threat_detected / threat_type / severity / confidence / reasoning / recommended_action / false_alarm_likelihood shape — no matter which model is behind it.

Pilot driver: GemmaProvider (Gemma 4, falling back to Gemma 3 if Gemma 4 fails vision verification). Future drivers: Qwen25VlProvider, KimiProvider, LlavaProvider, future SOTA.

5.6 ObjectDetector

Tier 1 fast object detection.

class ObjectDetector(Protocol):
    async def initialize(self, config: DetectorConfig) -> None: ...
    async def detect(self, frame: Frame) -> list[Detection]: ...
    async def detect_batch(self, frames: list[Frame]) -> list[list[Detection]]: ...
    @property
    def supported_classes(self) -> list[str]: ...
    @property
    def model_id(self) -> str: ...

Pilot driver: YoloV8Detector (AGPL license requires resolution before scale-out). Future drivers: RtDetrDetector (Apache 2.0, license-clean alternative), YoloNasDetector, vendor-specific.

5.7 AnomalyDetector

Per-camera adaptive anomaly scoring.

class AnomalyDetector(Protocol):
    async def initialize(self, config: AnomalyConfig) -> None: ...
    async def score(self, frame: Frame, camera_baseline: CameraBaseline) -> AnomalyScore: ...
    async def update_baseline(self, camera_id: str, frames: list[Frame]) -> CameraBaseline: ...

Pilot driver: AutoencoderWithAdaptiveKmeansDetector (the per-camera calibration approach from the original design).

5.8 IdentityProvider

Auth integration.

interface IdentityProvider {
  initialize(config: IdpConfig): Promise<void>;
  validateToken(token: string): Promise<TokenClaims | null>;
  refreshToken(refreshToken: string): Promise<TokenPair>;
  initiateLogin(redirectUri: string): Promise<string>;   // auth URL
  completeLogin(code: string, state: string): Promise<TokenPair>;
  logout(token: string): Promise<void>;
  supportsMfa(): boolean;
}

Pilot driver: KeycloakProvider. Future drivers: AzureAdProvider, OktaProvider, CustomerSamlProvider.

5.9 MapTileSource

Geospatial mapping.

interface MapTileSource {
  initialize(config: MapTileConfig): Promise<void>;
  tileUrl(zoom: number, x: number, y: number): string;
  geocode(query: string): Promise<GeocodeResult[]>;
  reverseGeocode(lat: number, lon: number): Promise<Address>;
}

Pilot driver: EsriArcGisSource (SGA basemap data). Future drivers: Customer-supplied tile servers, OpenStreetMap proxies.

5.10 ThreeDModelLoader

How a 3D building model gets into the digital twin.

interface ThreeDModelLoader {
  supportedFormats(): string[];   // ["glb", "gltf", "obj", "ifc", "bim", ...]
  load(modelData: Blob, metadata: ModelMetadata): Promise<ThreeDScene>;
  validate(modelData: Blob): Promise<ValidationResult>;
}

Pilot driver: GlbLoader (assuming the Clock Tower modeler delivers glTF/glb; pending format confirmation per reminders.md). Future drivers: IfcLoader (BIM), ObjLoader, ManualExtrusionLoader (from 2D floor plans).

5.11 StorageBackend

Object storage abstraction (useful for the on-prem GPU contingency where GCS isn’t available).

class StorageBackend(Protocol):
    async def put(self, key: str, data: bytes, metadata: dict[str, str]) -> str: ...
    async def get(self, key: str) -> bytes: ...
    async def delete(self, key: str) -> None: ...
    async def signed_url(self, key: str, expires_in: int) -> str: ...
    async def list(self, prefix: str) -> AsyncIterator[ObjectInfo]: ...

Pilot driver: GcsBackend with CMEK + per-tenant bucket layout. Contingency driver: MinioBackend (S3-compatible self-hosted) for the on-prem deployment mode.

6. The shape of an integration PR

When someone adds a new driver — say, a Honeywell Notifier fire-panel adapter — the PR looks like:

1. New file apps/edge-app/drivers/fire_panel/notifier.py implementing FirePanelAdapter.
2. Unit tests using a recorded protocol trace (we don’t require a live panel for CI).
3. A row in the drivers/REGISTRY.md documenting: which protocols supported, which firmware versions tested, deployment notes, known quirks.
4. No changes in apps/edge-app/main.py or anywhere outside the driver directory and the registry — the driver is loaded by configuration, not by import.
5. No changes in packages/contracts/ unless the driver requires a contract change (rare; usually a sign the contract is wrong).

This shape is what plug-and-play concretely means.

7. Driver selection at runtime

Each pluggable slot has a config field that names the driver:

# building config (per tenant, per building)
camera_transport: onvif_rtsp
fire_panel:
  driver: bacnet_ip
  config:
    panel_ip: 10.0.40.12
    network_number: 1
    objects: [...]
nvr:
  driver: null            # no NVR for this building
llm_provider:
  driver: gemma_4
  config:
    endpoint: http://ai-tier2.wiqaia.svc.cluster.local:8000
    model_id: gemma-4-9b-vision
    temperature: 0.1

The supervisor (edge) and the cloud services load the named driver at startup from a registry. Misnamed drivers fail fast with a clear error.

8. What’s intentionally not abstracted

Not everything pluggable is a good abstraction target. Some examples of things we don’t abstract, and why:

• PostgreSQL itself. We’ve committed to Postgres + PostGIS. An abstraction over the DB engine costs more than it saves; if we ever change DB engines, that’s a major migration that warrants explicit attention.
• Pub/Sub vs Kafka. Same reasoning. We don’t paper over message-bus differences.
• Three.js itself. The frontend uses Three.js directly; we don’t abstract “3D rendering library” because there’s no realistic alternative for our use case.
• TLS / mTLS implementation. Standard libraries; not our problem.
• Audit log format. It’s fixed by the contracts package; no driver story.

The rule: we abstract things where multiple drivers will plausibly exist in the next 12 months, and where consumers benefit from a uniform interface. We don’t abstract for theoretical future flexibility.

9. How this doc gets maintained

When you add a new abstraction:

1. Define the interface in packages/contracts/ (Python Protocol or TypeScript interface).
2. Add a section to this file under §5.
3. Ship at least one driver, even if it’s a stub for testing.
4. Add the driver registry row.

When you add a new driver to an existing abstraction:

1. Code in the appropriate driver directory.
2. Update the relevant §5 section to note the new driver.
3. Update drivers/REGISTRY.md.

When you remove an abstraction (rare, costly): write an ADR, update every consumer, then remove the contracts file.

The contracts package and this document are kept in sync by code review. PRs that add drivers without updating either fail review.