M05 — Edge AI (Tier 0 motion + anomaly autoencoder)

Architecture revision (2026-05-13): the original M05 plan ran YOLO + the anomaly autoencoder on a cloud GPU as a “Tier 1” gate before the LLM. That tier moved to the edge device — CPU-only, ONNX Runtime INT8 — and YOLO was demoted out of the pilot critical path. The cloud LLM (M06) is now the primary safety classifier and runs against every frame the edge flags as anomalous plus a baseline sampling cadence. See docs/plan/01-architecture-summary.md §0 for the rationale.

One-line goal: frames flowing through the edge device are filtered by motion + scored by an anomaly autoencoder, and the flagged frames (plus a baseline sampling cadence) are escalated up the mTLS tunnel to the cloud Tier-2 LLM. Per-camera calibration produces a stable baseline within 48 hours of registration. AnalysisEvent records that operators see on the dashboard come from M06 (the cloud LLM verdict); this milestone is the gate that decides which frames the LLM sees.

After M05, the platform watches every camera, learns what’s “normal” per camera, and escalates anomalies + a baseline sample for cloud-LLM analysis.

Tracks involved

• AI Worker (edge-side) — primary. The anomaly autoencoder + escalation gate on the edge device, CPU-only.
• Edge — Tier 0 motion filter integrated into the camera manager + the new autoencoder container.
• Backend — analysis-events Pub/Sub topic (consumed by M06’s LLM service); camera_calibration and calibration_cluster tables; dashboard query endpoints for calibration status.
• Frontend — anomaly-score indicators on camera tiles; calibration status UI.

Dependencies

• M01 (Pub/Sub topics exist; edge supervisor + edge-app containers running).
• M03 (camera_calibration, calibration_cluster tables ready; analysis_event table exists for future M06 verdicts).
• M04 (frames flowing from cameras through the edge camera manager).

Deliverables

1. Tier 0 — Edge motion filter

A Python module in the camera-manager container (already running per M04). Algorithm per the Submission Docs Digest §9 algo 1:

• Maintain a per-camera reference frame (sliding short-window average, or last-N-frame-difference baseline).
• For each new frame:
  • Convert to grayscale (resize to 320×180 for speed).
  • Gaussian blur (5×5 kernel) to suppress noise.
  • Frame-difference against reference, threshold pixel-wise at 25/255.
  • Compute change_ratio = (pixels above threshold) / total pixels.
  • If change_ratio > 0.005 (configurable per camera), mark frame as “motion detected” and emit to the cloud tunnel; otherwise discard.
• Output target: ~80% of frames discarded in typical scenes.
• Performance target: < 5 ms per frame on a modern CPU; < 5% of one core for a 1-fps camera.
• Per-camera motion threshold tunable via config and via a manual recalibration command (operator can override if a camera is in a high-motion zone or vice versa).

2. Edge-side anomaly autoencoder + escalation gate

apps/edge-app/wiqaia_edge_app/anomaly/ as a Python module running inside the edge-app containers on the building’s edge device. CPU only — ONNX Runtime with the encoder quantized to INT8. No GPU on the edge device.

2.1 Models loaded

• AutoencoderAnomalyDetector (driver via AnomalyDetector). Frozen pre-trained encoder (MobileNetV3-small or EfficientNet-Lite0-class for CPU efficiency, INT8) producing 128-dim embeddings; per-camera adaptive K-means clusters held locally on the edge device (SQLite) with a periodic sync to cloud Redis for visibility.
• (Optional) CrowdDensityEstimator — moved to M06 / Tier-2: the cloud LLM now produces person counts + bounding boxes directly in its structured JSON output, and the heatmap is computed from those. The edge does not run object detection in the 2-tier architecture.

2.2 Frame flow (no Pub/Sub between Tier 0 and the autoencoder — they’re in the same edge container)

• The camera manager hands each motion-flagged frame directly to the autoencoder via an in-process queue.
• The autoencoder scores it (target: <30 ms per frame on one CPU core at INT8).
• The escalation gate decides whether to send the frame to the cloud:
  • Anomaly-driven: anomaly_score >= per_camera_threshold (default 0.7).
  • Baseline cadence: one frame per camera every 30 s regardless of score (so the cloud LLM never relies entirely on the autoencoder).
  • Alarm-correlated: if a BACnet/Modbus alarm just fired and this camera correlates to the alarm zone, escalate immediately (bypasses the score).
• The escalation gate enforces a per-camera rate cap (default 4/min) to prevent a misbehaving camera from running up Tier-2 cost.

2.3 Edge → cloud escalation

For each frame that passes the gate:

1. Encode JPEG (already done by camera manager).
2. Encrypt the JPEG with the application-layer key (per 03-data-protection.md §5) before it leaves the building.
3. Publish to the cloud analysis-events Pub/Sub topic as a FrameEscalation event with: camera_id, building_id, frame_id, anomaly_score, escalation_reason (anomaly / baseline_sample / alarm_correlated), frame_payload (encrypted JPEG).
4. The cloud Tier-2 LLM service (M06) subscribes and produces the actual AnalysisEvent / Tier2AnalysisEvent rows.
5. If the cloud is unreachable, the edge’s store-and-forward queue holds escalations on local NVMe (up to the configured retention) and replays them when the tunnel comes back. Alarm-correlated escalations get priority in the queue.

3. Per-camera calibration scaffold

Algorithm per the Submission Docs Digest §10:

3.1 Initial calibration (48-hour window)

• New camera enters calibrating state on registration.
• Frames sampled every ~20 s (randomized ±5 s) for 48 hours = ~8,000 frames.
• Embeddings collected via the autoencoder encoder (no full inference; encoder only).
• K-means clustering applied at the end of the window with auto-K selection (K=2..8, choosing K that maximizes silhouette score above a threshold).
• Cluster centroids stored as calibration_cluster rows; the camera’s is_active=true calibration record references them.
• Camera transitions to active state; per-frame anomaly scoring uses these clusters from this point.

3.2 Continuous drift adaptation

• A background job samples 200 frames per active camera every 24 hours, computes embeddings, and tests them against current clusters.
• If a sustained shift is detected (median anomaly score drifts >0.1 over a week), the job re-runs K-means and updates the centroids.
• Manual recalibration trigger available in the camera admin UI for cases where the environment changed (renovation, camera repositioned).

3.3 Cold-start handling

• For the first 48 hours, anomaly scoring uses a global baseline (a centroid set computed across a large held-out dataset, shipped with the edge image). Crude but better than no signal.
• During this period the baseline sampling cadence to the cloud LLM stays on; the LLM is the safety classifier and does not depend on the autoencoder’s correctness.
• After calibration completes, switch to per-camera centroids.

4. Dashboard overlays (M05 contribution)

• Anomaly score indicator (small colored dot — green / yellow / red based on score) on each camera tile. Updates as the edge emits an AnalysisEvent.anomaly_score over the WebSocket fan-out.
• Calibration badge showing calibrating (xx%) / active (k clusters) / paused / drifted.
• Recent escalations list on the camera detail view: last 10 frames the edge escalated, each with the reason (anomaly / baseline_sample / alarm_correlated). Once M06 lands, each entry gains the cloud LLM verdict.

Bounding boxes, person counts, and heatmap overlays land in M06 when the cloud LLM starts producing them.

Performance: rendering anomaly-score updates for 16 simultaneous cameras at 1-fps update rate must not drop below 30 fps on the dashboard’s host browser (tested on a mid-range Chromebook representative of Civil Defence machines).

5. WebSocket real-time feed

• API exposes a /v1/realtime WebSocket endpoint.
• After auth, the socket subscribes to analysis-events Pub/Sub (server-side) and forwards filtered events to the client based on the client’s current view (building scope, camera selection, etc.).
• Reconnection logic with exponential backoff in the frontend.

6. Calibration UI

• Admin view at /buildings/:id/cameras/:id/calibration showing:
  • Current calibration status (calibrating / active / paused).
  • Frames collected so far.
  • Cluster count.
  • Current anomaly threshold and sensitivity multiplier.
  • “Recalibrate” button (requires confirmation; resets to 48-hour collection).
  • Last drift-detection timestamp and verdict.

7. Observability

• Per-building edge metrics (shipped over the telemetry topic to Cloud Monitoring):
  • Edge CPU utilization (Tier 0 + autoencoder), per-camera frame rate, per-camera anomaly-score distribution, store-and-forward queue depth.
  • Autoencoder inference latency p50 / p95 / p99 on the edge CPU.
  • Frames discarded by Tier 0 / scored by autoencoder / escalated to cloud (counts per minute, per camera).
• Alert on: edge CPU sustained >70% (indicates spec mismatch or too many cameras); store-and-forward queue depth >1000 (cloud tunnel issue); per-camera frame rate deviation >50% from expected; per-camera escalation rate hitting the cap (suggests a misbehaving camera or a real event in progress).

8. Cost considerations

• No cloud GPU cost from M05. The edge-side autoencoder runs on hardware the customer already provides (or that we ship as part of the install). Cloud GPU cost shows up in M06 (Tier-2 LLM, one always-warm L4 at ~$1,000/mo).
• Edge CPU budget: the autoencoder targets <30 ms/frame at INT8, ~20-30% of one core for 40 cameras × 1 fps. This leaves headroom for Tier 0 + the camera manager + the fire-panel adapter on the minimum-spec edge device.
• Escalation rate caps (per-camera, configurable; default 4/min) and the baseline sampling cadence (1 frame per camera per 30 s, configurable) are the primary cost controls. They bound the cloud LLM’s throughput requirement to a known maximum regardless of how anomalous any single camera gets.
• Frame snapshots: only persisted in the cloud when an escalation actually fires. The edge keeps a short-rolling local cache for the dashboard’s “recent frames” view; nothing else is uploaded.

Verification

1. Tier 0 discards ~80% of frames on a static-scene test camera over a 1-hour window. Measured by frames_handed_to_autoencoder / frames_emitted_by_camera_manager.
2. Anomaly score behaves sensibly. Throw an unusual object (oddly colored, large) into the scene; anomaly score spikes above 0.7. Remove it; score returns to baseline.
3. Edge → cloud escalation works. A frame with anomaly_score > 0.7 (manually injected) results in a FrameEscalation event on the analysis-events topic with escalation_reason = anomaly. The encrypted payload arrives in cloud Pub/Sub within 2 s of the score being computed.
4. Baseline sampling cadence works. Even with no anomalies for an hour, the cloud analysis-events topic receives ~120 escalations per camera (one per 30 s) over the hour, all with escalation_reason = baseline_sample.
5. Per-camera rate cap works. Forcing a continuous anomaly stream on one camera (50 frames/min over the cap) results in only 4/min reaching the cloud topic, with the excess dropped and logged in the edge’s audit trail.
6. Calibration completes for a new camera. Register a new test camera; after 48 hours the camera’s is_active=true calibration record references 2–6 cluster centroids; status flips to active.
7. Drift adaptation runs. Manually inject a sustained scene change (cover the lens with a different background for >24h); the drift-detection job triggers a recalibration within the configured window.
8. Store-and-forward survives a cloud outage. Pull the WAN cable on the edge device for 30 minutes during normal operation; escalations queue on local NVMe; reconnecting drains the queue to the cloud topic with no loss.
9. Multi-camera load test. With 40 cameras at 1 fps, edge CPU usage stays under 70% of available cores. Tier 0 keeps frames-into-autoencoder at ~8 fps (80% filtered). Autoencoder p95 latency <30 ms per frame.
10. Cross-tenant isolation in escalations. Frames from Tenant A’s camera produce escalations tagged with Tenant A’s building_id; the cloud-side LLM service (in M06) only persists AnalysisEvent rows scoped to that tenant.

Risks

Open questions

• Exact escalation thresholds. Anomaly > 0.7 is the starting value. Hajj loads will require zone-aware thresholds (a prayer hall during Jummah looks dramatically different than the same hall empty — both can be “normal” depending on context). Per-zone thresholds in M05; full context-aware in M07.
• Baseline sampling cadence — is 30 s the right number? Lower cadence (e.g. 10 s) means the cloud LLM sees every camera 3× more often, raising cost. Higher cadence (e.g. 60 s) means a 1-minute worst-case delay between when a fire is visible and when the LLM evaluates it (if the autoencoder happened to miss it). 30 s feels right for the pilot; revisit after we see real escalation patterns.
• Whether to enable nighttime IR mode for anomaly detection. Some cameras switch to monochrome IR at night; their embeddings differ substantially from daytime. Per-camera calibration should handle this with two clusters (day / night) but verify.

Exit criteria

All 10 verification items pass on the pilot’s actual cameras (or a representative test setup). Performance benchmarks captured in docs/runbooks/ai-performance.md. M05 sign-off entry in docs/plan/COMPLETION_LOG.md.