M04 — Camera Ingestion and Live Viewing

One-line goal: an operator opens the dashboard, sees a registered camera’s live feed within 2 seconds via WebRTC, and the camera-manager service is extracting one frame per second to the AI pipeline.

After M04, real cameras are producing real frames. The platform is no longer abstract.

Tracks involved

• Edge — primary. Camera Manager (Python container under the supervisor), ONVIF discovery, RTSP connection management, MediaMTX integration, frame extraction.
• Cloud Infra — MediaMTX deployment in the cloud (the WebRTC relay).
• Backend — camera registration endpoints (extend M03’s CameraService), stream-start signalling.
• Frontend — camera grid with WebRTC playback, online/offline indicators.

Dependencies

• M01 complete (edge supervisor scaffolding, MediaMTX containers in registry).
• M02 complete (operators are authenticated; only authorized users see live streams).
• M03 complete (camera registry exists in Postgres).

Deliverables

1. Edge Camera Manager container

A Python service in apps/edge-app/camera_manager/ implementing the CameraTransport abstraction (packages/contracts/04-abstractions §5.1).

• OnvifRtspTransport as the pilot driver:
  • WS-Discovery for finding cameras on the LAN.
  • ONVIF over digest auth for capability discovery and profile selection.
  • RTSP client (using onvif-zeep + aiortsp or similar) for stream URL retrieval.
• Reconnect-with-backoff on stream failure: 1 s → 2 s → 4 s → 8 s → 16 s → 30 s (max), with jitter.
• Camera-offline detection: 5 minutes without a successful frame → mark camera offline, emit CameraOfflineEvent to the tunnel.
• Per-camera health metrics: frame rate observed, decode errors per minute, last-frame timestamp.
• Credential storage: per-camera credentials encrypted at rest in the edge supervisor’s SQLite store, fetched from Secret Manager during registration.

2. Frame extraction pipeline

• At 1 fps (configurable per camera, default 1 fps) the camera manager grabs a frame from each active stream.
• Frame is JPEG-encoded at 75% quality, typically ~50 KB.
• Frame plus metadata (camera_id, building_id, tenant_id, captured_at) becomes a FrameEvent per the contracts package.
• Sent to a local Tier-0 motion filter (M05) — frames that pass go through the edge tunnel to the cloud frame-events Pub/Sub topic. Frames that fail are discarded with a count maintained for observability.

3. Cloud-side MediaMTX deployment

• MediaMTX deployed in the cluster (streaming namespace), exposing a WHEP endpoint behind the mesh gateway.
• Two roles:
  • Live stream relay — receives WebRTC streams pushed from the edge MediaMTX (over the mTLS tunnel via WebRTC’s standard signalling) and forwards to browser viewers via WHEP.
  • On-demand stream coordination — receives stream-start commands from the API, signals the edge to start the stream, manages teardown when no viewers remain.
• TLS via the mesh gateway; clients see only HTTPS.
• HPA on concurrent viewer count.

4. Edge-side MediaMTX

• Runs as a container under the edge supervisor.
• Configuration generated from the camera registry — each camera has a path like /cam-{camera_id}.
• Converts RTSP-pulled-from-camera to WebRTC-pushed-to-cloud, and to WebRTC-served-to-local-LAN (for the local building dashboard).
• Stream activation: idle by default; the supervisor receives a stream-start command (via the tunnel) and instructs MediaMTX to begin pulling from the camera. On stream-stop (or 5 min of no viewers), the camera-side connection closes.

5. Stream-start signalling

The flow when an operator clicks a camera:

1. Browser calls POST /v1/cameras/{id}/stream/start on the API.
2. API authenticates, verifies operator has access to this camera’s building.
3. API publishes a StreamStartCommand to a tenant-scoped command channel on the edge tunnel.
4. Edge supervisor receives the command, validates it, instructs its MediaMTX to start.
5. Edge MediaMTX establishes a WebRTC publishing connection to the cloud MediaMTX.
6. Cloud MediaMTX advertises the stream is available via WHEP.
7. Browser begins WebRTC negotiation via WHEP and receives video within ~2 s of the original click.
8. When the browser closes the stream (or session ends), reverse signalling tears it down.

Stream-start commands are audit-logged with actor_user_id, camera_id, and timestamp.

6. Camera registration UX

• Admin user navigates to a building → “Cameras” tab → “Discover cameras”.
• The API forwards a discovery request to the edge supervisor; the supervisor runs ONVIF discovery; results come back as a list.
• Admin selects which cameras to register; enters credentials per camera (or per-batch).
• Cameras saved to Postgres with status = 'pending_verification'.
• Edge supervisor verifies each by attempting a snapshot; success → status = 'online'; failure → status reflects the error, retry available.

7. Camera health monitoring

• Each edge supervisor emits EdgeTelemetryEvent per 01-architecture-summary.md and the contracts package.
• The API consumes these and updates the camera registry’s last_seen_at and status fields.
• An alert engine rule (basic version exists from M01; full version in M07) emits a camera_offline alert if a camera’s last_seen_at is older than its expected check-in interval.

8. Frontend — camera grid

• apps/dashboard/src/views/CameraGrid.tsx:
  • 1 / 4 / 9 / 16 / 25 grid layouts.
  • Per-tile: camera name, online status indicator, current FPS estimate.
  • WebRTC player using a minimal WHEP client (or video.js with WHEP plugin).
  • Click a tile to enlarge (or to go full-screen).
  • Multi-stream support: opening multiple tiles concurrently triggers a stream-start per camera (each independently authorized).
• Saved layout presets per user (in Postgres user.preferences JSONB).
• Spatial-aware “auto-layout” hook (used by M07 — show cameras near the active alert), placeholder implementation in M04.

9. Frontend — single camera detail

• Larger viewer.
• PTZ controls if the camera supports them (calls API → command → edge MediaMTX → camera ONVIF PTZ).
• Snapshot button (calls API → edge → returns a fresh frame).
• Recent AI events for this camera (populated in M05).
• Audit log: who has viewed this camera recently (auditor / admin only).

10. Bandwidth management

• Default profile: 1080p / H.264 / 2-5 Mbps per stream.
• Adaptive fallback: if a client’s WebRTC reports packet loss, automatically renegotiate at 720p / 1.5-3 Mbps.
• Only active when a viewer is connected — idle = 0 bandwidth.

Verification

1. A registered camera shows up in the dashboard. Discovery finds the camera; admin registers it; within 30 seconds the camera appears in the operator’s grid with the online indicator.
2. Live video plays within 2 seconds. Click → first frame visible in ≤ 2 s, measured on a representative network connection (Saudi internal, not transcontinental).
3. Cable-pull test. Disconnecting the camera from the network triggers a camera_offline alert within 5 minutes. Reconnecting clears the alert and resumes streaming within 30 seconds.
4. Frame extraction confirmed. While streaming, frame-events Pub/Sub topic receives one message per second per camera (or whatever the configured rate is). Messages contain valid JPEG data and correct metadata.
5. Multi-camera grid works. 4 cameras streaming simultaneously in a 2×2 grid; CPU on the edge box stays under 50%; cloud MediaMTX handles all four; browser plays smoothly.
6. Stream teardown is automatic. Close the dashboard tab; within 10 seconds, the edge supervisor reports the stream is stopped and the camera-side RTSP connection is closed.
7. Cross-tenant denial. Operator from Building A attempts POST /v1/cameras/{building-B-camera-id}/stream/start. Returns 404. Audit log captures the attempt.
8. PTZ works on a camera that supports it. Sending a pan-left command via the UI rotates the camera as expected. PTZ command is audit-logged.
9. Snapshot endpoint works. POST /v1/cameras/{id}/snapshot returns a fresh JPEG within 1 second.
10. Reconnect handles network blip. Manually toggle the edge box’s network interface for 30 seconds; cameras reconnect afterwards without manual intervention; no permanent state loss.

Risks

Open questions

• Specific camera vendor at the Arafat site — affects which ONVIF quirks we’ll hit. Site survey reveals; contingency for vendor-specific drivers in 04-abstractions-and-contracts.md §5.1.
• NVR vs direct access — Q2 in the reminders list. Affects whether we use OnvifRtspTransport directly or OnvifNvrTransport. Pilot driver should be writeable for either case; deploy decision at site time.
• PTZ availability — some cameras have it, some don’t. The UI hides PTZ controls when is_ptz = false.
• Frame rate per camera — default 1 fps is conservative. Crowd-monitoring zones (prayer halls) may warrant 2-3 fps; static corridors maybe 0.5 fps. Per-camera config available; tuning in M05.

Exit criteria

All 10 verification items pass on a real camera connected to a real (or developer-emulated) edge box. Sign-off entry in docs/plan/COMPLETION_LOG.md.