Architecture Summary

Audience: anyone touching the codebase or the infrastructure. Read after 00-overview.md.

Purpose: the locked-in tech stack with the why behind each choice. Decisions are not relitigated casually; if you want to change one, write an ADR in docs/architecture/ and bring it for review.

0. Architecture revision (2026-05-13)

The original plan described a three-tier AI pipeline: Tier 0 (edge motion filter) → Tier 1 (cloud GPU running YOLO + anomaly autoencoder) → Tier 2 (cloud GPU running multimodal LLM). After cost analysis at me-central2 GPU prices and re-examining what each tier actually buys, we moved Tier 1 to the edge and demoted YOLO out of the critical path:

• Edge device runs Tier 0 motion filter + the anomaly autoencoder (CPU-only inference via ONNX Runtime + INT8 quantization, ~20-30% of one core on a modern desktop).
• Cloud runs only Tier 2 — the multimodal LLM (always-warm L4) — as the primary safety classifier. The LLM evaluates every frame the edge flags as anomalous, plus a baseline sampling cadence on every camera.
• YOLO is no longer a gate. It can be added back as a parallel metadata feed (person counting, exact bounding boxes) if needed for the dashboard, but it does not decide whether the LLM sees a frame. This eliminates the failure mode where YOLO’s training-distribution blind spots could suppress fire-detection escalations.

Net effect: one cloud GPU instead of two ($1,000/mo saved), simpler architecture, safer false-negative posture. The remainder of this doc still uses “Tier 2” to refer to the cloud LLM; references to “Tier 1” should be read as either the edge-side anomaly autoencoder or removed from the pilot, depending on context.

Required minimum edge hardware spec (per the architecture decision above):

• OS: Ubuntu Server 22.04 LTS (24.04 LTS also acceptable)
• CPU: 8+ cores, x86-64, modern generation (Intel 12th gen or newer, AMD Ryzen 5000-series or newer)
• RAM: 32 GB
• Storage: 512 GB NVMe SSD (1 TB recommended to give the store-and-forward queue room during cloud outages)
• Network: 2× Gigabit Ethernet (one for camera LAN, one for uplink to the customer’s internet handoff)
• Camera capacity: this spec handles 40 cameras at 1 fps with Tier 0 + autoencoder comfortably. Beyond ~80 cameras per device, scale to a larger CPU or split across multiple edge boxes.

Customers either provide a machine meeting this spec or purchase one we ship. Raspberry-Pi / mini-NUC / older-hardware deployments are not supported; this is a safety-critical platform and weak hardware introduces failure modes we won’t accept.

1. Stack at a glance

Two languages on the runtime side (TypeScript + Python) plus Go for the edge supervisor. The TypeScript ↔ Python ↔ Go integration boundary is handled exclusively through packages/contracts/ (protobuf + JSON Schema). See 04-abstractions-and-contracts.md.

2. Why each major decision

2.1 Cloud provider — Google Cloud, Dammam (me-central2)

Saudi Arabia mandates in-Kingdom data residency for surveillance video of Hajj pilgrims, Civil Defence operations, and any system likely to be classified under NCA Critical/Sensitive controls. The only hyperscaler with a Saudi Arabia region offering full GPU + GKE + managed Postgres + managed Pub/Sub is GCP Dammam. AWS has no KSA region (Bahrain is outside KSA). Azure is in Qatar. Alibaba is in Riyadh but its AI/ML services are weaker. GCP Dammam is the only viable answer.

Customer/reseller relationship: Saudi customers cannot self-serve sign-up with Google — purchase must go through CNTXT, Google’s exclusive Saudi reseller. The same SKUs as global GCP, but billed through a Saudi entity. Our account is in setup as of 2026-05-12.

We do not use Vertex AI managed inference, because Google has confirmed Vertex processing happens outside KSA. All AI is self-hosted on GKE in Dammam.

2.2 Frontend — React + Vite, NOT a full-stack framework

Our dashboards are auth-gated control rooms — there is no SEO requirement, no content marketing pages, no anonymous traffic. The Next.js / Remix / SvelteKit value proposition (SSR, edge rendering, hybrid static/dynamic) gives us nothing. The dominant runtime concerns are heavy client-side work: live WebRTC video, Three.js scene rendering, WebSocket fan-out for real-time alerts. A plain SPA built with Vite serves these cleanly with the smallest possible framework surface.

React specifically (vs Svelte / Vue / Solid) because the ecosystem alignment is overwhelming for what we’re building: @react-three/fiber and @react-three/xr for the digital twin, @arcgis/core React bindings for Esri, i18next with RTL plugins for Arabic, mature live-video components, and — critically — substantially more LLM training data, which means AI agents make fewer mistakes in React than in Svelte.

Vite (vs CRA, Webpack, Parcel) because: instant HMR, ESM-native, small config surface, zero ceremony, well-supported by Turborepo.

2.3 Cloud API — Node + TypeScript + NestJS

Node + TypeScript shares the language with the frontend (zero ceremony around shared types via packages/contracts/), is an excellent fit for the I/O-bound workload (WebSocket fan-out, Pub/Sub plumbing, REST APIs, mTLS connection management), and has the broadest LLM training coverage.

NestJS specifically (vs Express, Fastify, Hono) because:

• It gives us strong opinions about module boundaries, dependency injection, and lifecycle, which prevents collisions between multiple engineers and agents working on different modules.
• Decorators + DI map naturally to RBAC enforcement, audit-log interception, and request-scoped tenant resolution.
• The conventions are AI-agent-friendly: a new module follows a predictable shape (*.module.ts, *.controller.ts, *.service.ts, *.dto.ts), so an agent generating one module doesn’t have to discover structure from scratch.

Express alone is fine for a 1,000-line API; for the volume we’re heading toward (auth, RBAC, multi-tenant, audit, alerts, fleet management, integration adapters), the structure tax pays for itself within a week.

2.4 Edge supervisor — Go

The edge supervisor is the long-running systemd service on every building’s Linux box that holds the cloud tunnel, manages Docker containers, and runs the auto-update logic. The requirements are:

• Single static binary, no runtime install, hardenable.
• Talks to Docker, holds a long-lived mTLS connection, does TLS rotation, manages systemd-controlled processes.
• Must be debuggable in the field with minimal tooling on the host.

Go is purpose-built for this: cross-compiles to a single ~10 MB binary, excellent standard library for TLS / HTTP / file IO, mature Docker SDK, easy systemd integration. Distribution is apt install wiqaia-edge-supervisor with our own apt repo, or a single curl … | sudo bash for the bootstrap.

We considered Python here (consistency with the application containers) but the deployment story is worse: shipping a Python runtime + dependencies as a system service is materially more painful than shipping one Go binary, and the supervisor doesn’t need any of Python’s ML/CV libraries (the application containers do).

2.5 Edge application containers — Python

The application code on the edge — camera manager, ONVIF discovery, RTSP handling, Tier 0 motion filter, fire-panel adapter, store-and-forward queue — leans heavily on Python-dominant libraries:

• ONVIF clients (onvif-zeep, python-onvif)
• OpenCV for motion detection
• BAC0 / bacpypes3 for BACnet/IP
• pymodbus for Modbus TCP
• pyserial for RS-485 contact-closure scenarios
• The MediaMTX client patterns

These exist in other languages but are weaker. Python’s combined library coverage for our exact mix of CV + industrial protocols is unmatched.

The Python containers never see the host system directly — they run under the Go supervisor in Docker. The supervisor handles networking, restarts, updates. Python deals only with frames and events. This separation keeps responsibilities tight.

2.6 AI worker — Python

The AI inference services (edge-side anomaly autoencoder, cloud-side Tier 2 multimodal LLM) live in Python because:

• vLLM, the highest-throughput open-source LLM serving framework, is Python-native.
• ONNX Runtime Python bindings are first-class.
• Hugging Face Transformers, model loading, tokenizers, post-processing — all Python.
• Per-camera calibration logic (clustering, embedding storage) sits naturally next to the model code.

Performance-critical inference happens inside C++ / CUDA underneath these Python wrappers — the language is a coordination layer, not a hot path.

2.7 Relational database — PostgreSQL 15 + PostGIS

The data model is fundamentally relational (Building → Floor → Zone → Camera, with foreign keys everywhere) and contains spatial geometry (camera positions, fire-zone polygons, building footprints). Postgres + PostGIS handles both natively. Row-level security (RLS) on building_id gives us multi-tenant isolation at the database layer, not just the application layer — so an SQL injection in a controller can’t leak across tenants.

Cloud SQL for Postgres in Dammam runs as managed (no patching, no failover scripting) with synchronous replication to a standby in the same region for HA, plus an async read replica for read-heavy analytics. Backups are automatic.

Why not MongoDB / DynamoDB / Cockroach? The schema is genuinely relational with hard referential integrity needs (audit trails on every mutation, foreign keys we’d be implementing manually in a NoSQL store, multi-table consistency for alert lifecycle). Postgres is simpler and more correct. Cockroach is overkill for our load profile.

2.8 Cache, sessions, real-time state — Redis (Memorystore)

Redis serves four distinct roles:

• Session storage for the API (with sliding expiry).
• Real-time alert state for the dashboard’s WebSocket fan-out (so a reconnecting client gets the current alert snapshot, not just future deltas).
• Per-camera anomaly baselines (compact embeddings + cluster centroids — fast lookup during inference).
• Lightweight pub/sub for in-cluster signalling (Pub/Sub is for cross-service events; Redis is for “tell every WebSocket gateway about this alert now”).

Memorystore is managed Redis in Dammam. We use the M1 standard tier (4 GB) to start; resize when metrics demand.

2.9 Event bus — Google Pub/Sub

Pub/Sub gives us durable, at-least-once delivery for the camera-frame and AI-event streams that flow between services. The bus decouples the edge (anomaly-flagged frames) from the cloud Tier-2 LLM, and decouples AI from the alert engine. If the Tier-2 pod crashes, frames queue up; when a new pod comes online, it drains the queue.

Why not Kafka? Operational complexity. Kafka requires cluster management, broker tuning, partition sizing — none of which buys us anything over Pub/Sub at our volume. We can reconsider if we ever exceed Pub/Sub’s per-topic throughput, but at projected pilot rates we’re nowhere close.

2.10 Object storage — GCS with CMEK on every bucket

Surveillance frame snapshots, video clips for incident review, 3D models, floor plans, and other binary blobs live in GCS. Customer-managed encryption keys (CMEK) are mandatory on every bucket — Google stores the data but never holds the key in plaintext form long enough to read it without our authorization. For the most sensitive blobs (video clips of pilgrims), we add application-layer encryption on top using keys we hold outside GCP (KMS-wrapped, custodied per 03-data-protection.md).

Bucket layout: one bucket per (tenant, data-class) pair (e.g. wiqaia-bld-{id}-frames, wiqaia-bld-{id}-clips, wiqaia-bld-{id}-models), each with its own lifecycle policy (frames expire fast, clips medium, models permanent until building offboarded).

2.11 Analytics — BigQuery

KPI dashboards, historical trend analysis, and long-term metrics (crowd counts, anomaly scores, alert volumes by hour-of-day-of-week-of-month, etc.) live in BigQuery. Serverless, billed on query bytes, costs scale with use not provisioned capacity. We pipe AnalysisEvent and Alert tables to BigQuery via scheduled CDC for analytical queries; the operational store is Postgres.

2.12 Orchestration — GKE

Kubernetes lets us:

• Run heterogeneous workloads (CPU pods for API, GPU pods for AI, stateful pods for media server) in one cluster.
• Auto-scale on multiple signals (CPU, GPU utilization, custom metrics like Pub/Sub queue depth).
• Use node taints / tolerations to keep AI workloads on GPU nodes and other workloads off them.
• Get rolling updates, horizontal pod autoscaling, and pod disruption budgets for free.

GKE Autopilot is tempting for cost simplicity but limits GPU node-pool configuration. We use GKE Standard with explicit node pools — one for general CPU workloads, one for GPU (a single always-warm L4 in the 2-tier architecture; see §0). GPU SKU note for me-central2 (Dammam): only NVIDIA L4 is available in-region, and only in zones me-central2-a and me-central2-c (verified 2026-05-13). T4 / A100 / H100 / H200 are not available. The Tier-2 LLM runs on L4 (g2-standard-8 or g2-standard-12). If a future workload genuinely needs A100-class compute, the on-prem-GPU contingency in docs/plan/05-contingencies.md §1 becomes the path — not a workaround, but a deliberate deployment mode.

2.13 Identity — Keycloak (leaning)

Self-hosted Keycloak in Dammam keeps auth data in-Kingdom (a soft requirement under NCA CCC, a strict one under some interpretations of PDPL for identity records linked to Civil Defence personnel). Supports OAuth 2.0, OIDC, SAML, social/SSO federation, MFA via TOTP / WebAuthn. Mature, battle-tested. The main downside is operational weight (it has more knobs than we’ll use), but the alternative — building auth ourselves — is far worse.

We considered SuperTokens (lighter, more modern but younger and less proven) and Ory Kratos + Hydra (excellent but more infrastructure). Keycloak wins on the in-Kingdom self-host story and feature completeness. Final decision deferred to M02 — a one-day evaluation against SuperTokens before committing.

2.14 Multimodal LLM — Gemma 4 (target), Gemma 3 / Qwen 2.5-VL (fallbacks)

The Tier 2 model performs the deep contextual visual analysis that distinguishes our platform: “is this smoke actually a fire, or just incense?“. Melissa has chosen Gemma 4 to start — contingent on verifying it has multimodal vision capability before integration (Gemma generations have sometimes shipped text-only first). If Gemma 4 is text-only at integration time, fall back to Gemma 3 (current production candidate, already tested in the prior prototype) or Qwen 2.5-VL (Apache 2.0, fully permissive license, comparable quality).

All multimodal LLMs sit behind a LlmProvider interface (see 04-abstractions-and-contracts.md). Swapping models is a config change, not a code change. We A/B and benchmark whatever’s interesting.

2.15 Object detection — YOLOv8 (with caveat)

Status: demoted from pilot critical path (see §0 for the architecture revision).

YOLOv8 was originally planned as the Tier-1 gate (cheap fast filter before the expensive LLM). After re-examining the cost math (L4 throughput is ~5-7 LLM inferences/sec sustained, not ~0.2/sec as the old plan assumed) and the safety implications (YOLO wasn’t trained on fire / smoke, so gating LLM access on YOLO’s verdict means YOLO’s blind spots become the system’s blind spots), the decision was to make the LLM the primary safety classifier and run the lightweight anomaly autoencoder at the edge for cheap independent screening.

If a future need surfaces (e.g., we need exact person counts for crowd-capacity-limit alerts), YOLOv8 can return as a parallel metadata producer — but not as a gate, and not in the pilot. The license question (AGPL-3.0) gets resolved at that point. For the pilot, the LLM’s structured JSON output includes “approximate person count” and “bounding boxes of detected people,” which is sufficient.

The ObjectDetector interface in 04-abstractions-and-contracts.md makes a future model swap (YOLO commercial license, RT-DETR Apache 2.0, or YOLO-NAS) a one-config-change if and when this gets revisited.

2.16 Edge OS — Ubuntu 22.04 LTS Server

Headless Ubuntu Server because:

• Long support (security updates through 2027 standard, 2032 with ESM).
• Universal hardware support, well-known automation surface.
• Docker, systemd, networking, mTLS all rock-solid.
• Free and redistributable — no per-device license fee even at scale.
• The bootstrap install script is one bash file.

We require Linux on the edge for hard reasons documented in 00-overview.md and the prior conversation: Docker is native (not virtualized), hardware passthrough for serial fire-panel adapters works cleanly, 24/7 unattended operation is the design point, security surface is minimal, and the operational story for headless fleet management is mature only on Linux.

2.17 Repo & build — Monorepo, pnpm workspaces, Turborepo

One repo at wiqaia/. apps/* for deployables, packages/* for shared libraries, infra/ for Terraform + Helm + CI configs, docs/ for everything else. pnpm workspaces handles the JS link-graph; Turborepo provides change-aware builds with caching (so CI only rebuilds what changed); each app has its own Dockerfile and its own deployment. Python apps live in the same repo with their own pyproject.toml; Go apps with their own go.mod. The languages don’t fight because each app’s container build is hermetic.

Monorepo over polyrepo because:

• Cross-cutting refactors (rename a contract field, propagate to TS + Python + Go) are one PR, not three coordinated releases.
• One CI configuration, one lint config, one dependency graph to reason about.
• Sub-agents and engineers see the full system in one tree, which dramatically reduces context-switching cost.
• The classic “monorepo deploy hell” comes from people deploying the whole monorepo as a unit — we don’t. Each app deploys on its own cadence.

2.18 Source hosting — self-hosted Gitea in-Kingdom from day 1

We run our own Gitea instance at git.rizoma.sa, on the wiqaia-ops VM in me-central2 (Dammam). All source code, all issue tracking, all docs — in-Kingdom from day 1. No GitHub dependency.

This avoids the data-residency question entirely. Under NCA Critical/Sensitive classification, source code that contains customer-context comments, deployment configs, or internal logic should not sit on US-based servers without scrutiny. Self-hosting on wiqaia-ops makes that a non-question.

• Source of truth: rizoma/wiqaia at https://git.rizoma.sa
• Issue tracking: Gitea milestones + issues, mirroring the M01–M10 plan. See Issue tracking.
• Docs hosting: git.rizoma.sa/docs/ — built by SvelteKit, deployed via post-receive hook (no external CI).
• CI/CD: Cloud Build (planned) — fully managed inside GCP me-central2. Until that’s set up, a pre-push hook runs the same gates locally. Gitea Actions is deliberately not used — the 2 GiB ops VM cannot safely host a runner without OOM risk.

The historical “GitHub initially → Saudi later” plan in earlier drafts is obsolete; the migration step we’d have deferred didn’t need to happen because we never went to GitHub in the first place.

2.19 Error tracking — Sentry SaaS (dev/staging) → self-hosted production

Same data-residency reasoning as source hosting. Sentry’s SaaS is wonderful for developer experience but error reports can contain stack traces with sensitive variable values. For production:

• Self-hosted GlitchTip (Apache 2.0, Sentry-compatible) on the Dammam GKE cluster, or
• Self-hosted Sentry on Dammam GKE (heavier but more features).

Dev and staging environments can use Sentry SaaS because they should never see real customer data.

3. Deployment topology

Two deployment modes, same code, selected by config:

3.1 Cloud mode (default)

                   Operator Browser
                          │
                          │ HTTPS / WebRTC
                          ▼
                   Cloud Load Balancer (Dammam)
                          │
        ┌─────────────────┼──────────────────────┐
        ▼                 ▼                      ▼
   API Gateway       MediaMTX Relay         Static SPA bundle
   (Kong/Envoy)      (cloud)                (Cloud CDN)
        │
        ▼
  NestJS API ── Postgres + PostGIS
        │     ── Redis
        │     ── GCS (CMEK)
        │     ── Pub/Sub
        ▼
   AI Worker (Tier 2 LLM, always-warm L4 GPU)
        │
        ▼
   Alert Engine ── Notification Service
        │
        ▼
   Building Edge (one per building)
        │
        │ outbound mTLS tunnel only
        ▼
   Edge Supervisor (Go)
        │
   ┌────┼─────────────┬─────────────────────┐
   ▼    ▼             ▼                     ▼
Camera  Tier 0       Anomaly Autoencoder   Fire Panel
Mgr.    Motion       (CPU, ONNX INT8)      Adapter
   │    Filter             │                     │
   ▼                       ▼                     ▼
IP Cameras           (flags frames           FACP / BACnet
(ONVIF/RTSP)          for cloud LLM)

3.2 On-prem GPU mode (contingency)

Identical except the Tier-2 LLM worker runs on a customer-site GPU box managed by the same edge supervisor (the edge-side autoencoder is already there in both modes). The cloud retains the API, data layer, dashboard, and fleet management. Used only if GCP Dammam GPU quota becomes unavailable in time. See 05-contingencies.md.

4. Camera-frame data flow

A single frame’s journey from sensor to operator alert:

1. Camera → Edge. IP camera streams RTSP to the camera-manager container on the edge box. Frame extracted at 1 fps for AI; full stream available for on-demand viewing.
2. Tier 0 (edge, CPU). The frame passes through OpenCV motion detection. ~80% of frames in typical buildings show no motion and are discarded immediately. The remaining ~20% advance.
3. Anomaly autoencoder (edge, CPU). Surviving frames are scored by the per-camera autoencoder + adaptive cluster baseline. Frames that match the camera’s “normal” distribution are logged as a routine sample but do not escalate. Frames with anomaly scores above the per-camera threshold are flagged. Independently, a small baseline sampling cadence (e.g. one frame per camera every 30 seconds) is always escalated regardless of score, so the cloud LLM never relies entirely on the autoencoder to spot things.
4. Edge → Cloud. Flagged frames (anomaly-driven or baseline sample) travel up the mTLS tunnel as encrypted JPEG payloads to a Pub/Sub topic in Dammam. Frame size is ~50 KB; one camera’s bandwidth contribution is bounded.
5. Tier 2 (cloud, GPU). The Tier-2 AI worker pulls from Pub/Sub, runs the multimodal LLM, and returns structured JSON: threat_detected / confidence / severity / reasoning / false_alarm_likelihood / person_count / detected_objects. Latency target end-to-end (frame capture → Tier 2 verdict): under 10 seconds.
6. Alert engine. Confirmed threats become Alert records. The alert engine performs spatial correlation against the digital twin (PostGIS query) to find all cameras with line-of-sight to the incident location and assembles an evidence package (camera screenshots, AI verdicts, fire-panel state).
7. Notification. Alerts fan out via WebSocket to live operator dashboards, into the FD-view feed for the relevant fire department, and into mobile notifications (M07 / M08).
8. Operator action. Operator acknowledges / dismisses / escalates / resolves. Each action persists an AlertAction row in the audit-logged trail.

Fire-alarm events follow the same path with one twist: the alarm event arrives via the BACnet/Modbus adapter, the spatial correlation step pulls relevant cameras’ most recent frames, and Tier-2 is invoked at maximum priority on those frames (bypassing the autoencoder gate). The whole sequence (alarm → evidence on operator screen) targets under 30 seconds.

5. What we explicitly are NOT using

For each, a one-line reason. These are not relitigated without an ADR:

• Electron — was right for the desktop PoC; wrong for a cloud platform with browser-based access.
• Next.js / Remix / SvelteKit — full-stack frameworks add complexity that an auth-gated real-time control room doesn’t benefit from.
• Express (for cloud API) — too loose; NestJS structure is worth the learning tax for multi-engineer / multi-agent work.
• SQLite in production — straight to Postgres. SQLite is used only inside the edge supervisor’s local store-and-forward queue.
• Vertex AI managed inference — Google has confirmed Vertex processing happens outside KSA.
• Google Maps in production — Esri ArcGIS with SGA basemap data for data-sovereignty alignment.
• MongoDB / DynamoDB / Cockroach — relational with spatial extensions is the right model.
• Kafka — Pub/Sub is simpler at our volume.
• Lambda Labs / RunPod / etc. for production GPU — they’re outside KSA. Only used for dev experiments.
• Sentry SaaS for production — data residency.
• Auth0 / Okta / Firebase Auth managed — same.
• GitHub or any non-KSA git hosting — source lives in our self-hosted Gitea at git.rizoma.sa (Dammam). Never used GitHub; no migration needed.
• Facial recognition / biometric identification — explicitly out of scope by design (PDPL alignment, plus NCA + ethical considerations). Person detection produces anonymous bounding boxes.

6. Future-state considerations (not pilot blockers)

Documented here so we don’t paint into corners during the pilot:

• Multi-region failover. The Dammam region is single-region today. When Google opens an additional Saudi/GCC region with full GKE+GPU, we replicate. Edge gateways’ store-and-forward queues already buy time during a regional outage.
• In-Kingdom git, CI, and error tracking. Migration triggered by compliance audit; design choices above already keep the path open.
• Hardware-rooted device identity for edge boxes. TPM-backed certificates rather than file-stored mTLS keys. Adds supply-chain complexity; deferred.
• Per-tenant KMS keys for application-layer encryption. Already wired in at the data-protection layer (03-data-protection.md); the per-tenant version becomes meaningful at 50+ tenants.
• Federation with Civil Defence dispatch systems. Currently we provide an FD view; in the future, structured alerts can pipe directly to their dispatch software via REST or message queue. Requires a discussion with Civil Defence about their existing systems.
• VR walkthrough for remote building familiarization. WebXR support is already in the design; the actual UX implementation is a polish-phase deliverable.
• YOLOv8 license resolution. Only relevant if/when YOLO returns as a parallel metadata producer (see §2.15). Not a pilot blocker in the 2-tier architecture.

7. Where this doc fits

The decisions captured here are committed. They drive the milestone files (M01-… through M10-…). When implementing, write code that conforms to these choices; if a choice needs to change, write an ADR in docs/architecture/, get review, then update this doc and the relevant milestones in the same change.