Deployment Topology

Formation runs on Azure Container Apps with Bicep-defined infrastructure. One subscription hosts three environments (dev, uat, prod), each with its own resource group, SQL database, Key Vault, and Container Apps environment. Shared infrastructure (container registry) lives in a common resource group.

This page maps the deployed topology — what runs where, how the pieces talk, how identity and secrets flow, and how CI/CD reaches into each environment.

Big-Picture Topology
Infrastructure Layout
Compute
Data Tier
Identity and Auth
Observability
CI/CD
Local Dev vs Deployed
Gotchas

Big-Picture Topology

    Internet (formdev.pma.co.uk, formuat…, formprod…)
       │
       ▼
  ┌────────────────────────────────────────────────────────┐
  │ ingrs (Traefik) — ca-ingrs-01                          │
  │ External ingress, custom domain + managed cert         │
  │ Routes /api/* → api,  /* → web                         │
  └──────┬──────────────────────┬──────────────────────────┘
         │                      │
         ▼                      ▼
  ┌────────────────────────────────────────────────────────┐
  │ Container Apps Environment — cae-01                    │
  │  ┌──────────────┐  ┌──────────────┐  ┌──────────────┐  │
  │  │ api          │  │ web          │  │ docs         │  │
  │  │ .NET 10      │  │ SvelteKit 5  │  │ Static docs  │  │
  │  │ 1–5 replicas │  │ 1–5 replicas │  │ 1–2 replicas │  │
  │  └──────────────┘  └──────────────┘  └──────────────┘  │
  │                                                        │
  │  ┌──────────────────────────────────────────────────┐  │
  │  │ Container App Jobs (event/schedule/manual):      │  │
  │  │  jobload (queue scaler)                          │  │
  │  │  jobcompletionscore                              │  │
  │  │  jobrebuildqueryviews                            │  │
  │  │  jobcurimp  (currency import)                    │  │
  │  │  jobdedup   (duplicate detection)                │  │
  │  └──────────────────────────────────────────────────┘  │
  └────────┬──────────────┬──────────────┬─────────────────┘
           │              │              │
           ▼              ▼              ▼
   ┌─────────────┐ ┌───────────┐ ┌───────────────────┐
   │ SQL Server  │ │ Key Vault │ │ Storage Accounts  │
   │  sql-01     │ │  kv-01    │ │   stloadq01       │
   │  sqldb-01   │ │           │ │   stdocs01        │
   │  Standard   │ │ 10+ app   │ │   Blob + Queue    │
   │  400 DTU    │ │ secrets   │ │   VNet-restricted │
   │  AAD-only   │ │           │ │                   │
   └─────────────┘ └───────────┘ └───────────────────┘
           │
           ▼
   ┌─────────────────────────────────────────┐
   │ Observability                           │
   │   Log Analytics → Application Insights  │
   │   OpenTelemetry sources: Formation.*    │
   └─────────────────────────────────────────┘

Everything inside the box is in one resource group (frmpmauks-{env}-rg-01); the container registry is shared across environments in the common resource group (frmpmauks-common-rg-01).

Infrastructure Layout

Bicep Modules

Infrastructure as code lives in infrastructure/. Composition is subscription-scoped:

infrastructure/
  main.bicep                    # Entry point (subscription scope)
  configuration/
    common/                     # Shared across environments
    dev/main.parameters.json    # Per-environment inputs
    uat/main.parameters.json
  modules/
    common.bicep                # Container registry + resource group locks
    core.bicep                  # VNet, SQL, KV, Storage, Log Analytics,
                                # App Insights, Container Apps Environment,
                                # Container Apps (api, web, docs, ingrs, jobs)
    aca_job.bicep               # Container App Job template
    acr_role_assignment.bicep   # ACR Pull role assignment
  deploy/                       # Deployment helper scripts

main.bicep is targetScope='subscription' — it creates both resource groups (common + environment) and dispatches to the module scopes:

// infrastructure/main.bicep (opening parameters)
targetScope='subscription'

@description('Region for all resources.')
param region string = 'uks'

@description('The short name of the organisation.')
param org string = 'pma'

@allowed(['dev', 'uat', 'prod'])
param environment string = 'dev'

param app string = 'frm'
param addressSpace string
param azureADAdminGroupSid string
param sqlDatabases object = {}
param appConfig object = {}

core.bicep is where the bulk of the platform lives: the VNet with subnets, Azure SQL server + database, Key Vault, storage accounts, Log Analytics + Application Insights, the Container Apps environment, and every Container App / Job. It’s a single large module (~1000+ lines) rather than one-module-per-resource; the trade-off is less composition ceremony at the cost of a bigger file.

Naming Convention

All resources follow {app}{org}{region}-{environment}-{resource-type}-{sequence}:

Part	Value	Meaning
app	`frm`	Formation
org	`pma`	Property Market Analysts
region	`uks`	UK South
environment	`dev` / `uat` / `prod`	Environment tier
resource-type	`ca`, `sql`, `kv`, `st`, `log`, `appi`, `mi`, `acr`, …	Short Azure resource abbreviation
sequence	`01`, `02`, …	Usually `01` — future-proofing for cost of re-keying

Examples:

frmpmauks-dev-sql-01 — Azure SQL Server (dev)
frmpmauks-dev-sqldb-01 — Database inside that server
frmpmauks-dev-ca-api-01 — API Container App
frmpmauks-dev-kv-01 — Key Vault
frmpmauks-dev-mi-api-01 — User-assigned managed identity for the API
frmpmaukscommonacr01 — Shared container registry (no hyphens — ACR naming rules forbid them)

The prefix frmpmauks is fixed across environments; the environment segment differentiates.

Resource Groups

Common RG (frmpmauks-common-rg-01) — one per subscription. Hosts the shared container registry (frmpmaukscommonacr01) and resource-group-level locks.
Environment RGs (frmpmauks-{env}-rg-01) — one per environment. Hosts every workload resource: VNet, Container Apps Environment, Container Apps, SQL, Key Vault, storage, Log Analytics, Application Insights, managed identities.

Keeping the registry in a common RG avoids paying for three copies of the same images; environment-specific RGs give each environment its own blast radius for IAM and cost reporting.

Compute

Container Apps

Every runtime component is a Container App, deployed into a single shared Container Apps Environment (cae-01) per environment tier. Current apps:

Container App	Purpose	Ingress	Port
`ca-api-01`	.NET 10 REST / OData API, LiteBus CQRS	Internal	8080
`ca-web-01`	SvelteKit 5 frontend, OAuth session	Internal	3000 (+ 9464 metrics)
`ca-docs-01`	Static documentation site (OpenAPI, Storybook)	Internal	3000
`ca-ingrs-01`	Traefik reverse proxy — external entry point	External	80

“Internal” means the Container App is only reachable inside the Container Apps Environment (via the private DNS zone). Only the Traefik ingress has an external endpoint; all public traffic goes through it.

Each app has Dapr enabled (appId = service name) but Formation does not currently use Dapr pub/sub or state bindings — the sidecar is running for future use.

Scaling

Scaling rules are configured per-app in infrastructure/configuration/{env}/main.parameters.json. Example for the API in dev:

"api": {
  "scaleSettings": {
    "maxReplicas": 5,
    "minReplicas": 1,
    "cooldownPeriod": 500,
    "pollingInterval": 30,
    "rules": [
      {
        "name": "http-rule",
        "http": {
          "metadata": {
            "concurrentRequests": "5"
          }
        }
      }
    ]
  }
}

Min replicas = 1 across the HTTP-facing apps. Cold starts are 30–60 seconds; always keeping one replica warm avoids that on the first request after idle.
HTTP concurrency target = 5 for the API. KEDA scales out when average in-flight requests per replica crosses that threshold.
Revision strategy is single active revision — no canary or traffic splitting configured. Rollback requires redeploying the previous image version.

Jobs have no active scaling; they’re triggered events/schedules rather than steady load.

Ingress and Traefik Routing

ca-ingrs-01 is the only externally-exposed Container App. It runs Traefik configured with file-based dynamic routing at src/services/ingrs/dynamic/. The config fans traffic out by path:

Traefik (ca-ingrs-01, port 80)
  │
  ├─ /api/*   → https://frmpmauks-dev-ca-api-01.internal.{DNS_SUFFIX}
  ├─ /docs/*  → https://frmpmauks-dev-ca-docs-01.internal.{DNS_SUFFIX}
  └─ /*       → https://frmpmauks-dev-ca-web-01.internal.{DNS_SUFFIX}

The internal.{DNS_SUFFIX} hostnames are Container Apps Environment-assigned DNS names that resolve only inside the environment. External requests arrive at formdev.pma.co.uk (managed-cert HTTPS), hit Traefik, and are routed to the appropriate internal service.

Traefik also terminates TLS and adds a managed certificate via the custom-domain binding on the Container App.

Jobs

Container App Jobs are defined in aca_job.bicep. Each job has its own lifecycle independent of the long-running apps:

Job	Trigger	Purpose
`ca-jobload`	Event (Azure Queue Storage)	Data-load ingestion — processes files dropped into the `data-load` container
`ca-jobcompscore`	Manual / Scheduled	Re-computes completeness scores, writes to `[query].*List.CompletenessScore`
`ca-jobqueryvws`	Manual / Scheduled	`RebuildQueryViewsWorker` — full rebuild of query views (see query-views.md)
`ca-jobcurimp`	Scheduled (daily)	Currency import — pulls ECB exchange rates from the BI lakehouse (`WarehouseDb` connection string, `[ECBExchangeRates].[CurrencyConversion]`), pivots direct/cross-rate pairs, upserts into `[app].CurrencyConversion`. See Jobs → Currency Import.
`ca-jobdedup`	Manual	Duplicate detection run

Job execution:

Event-triggered (jobload) uses a KEDA Azure Storage Queue scaler. New message → job replica spun up.
Manual/Scheduled jobs can be kicked off via the API (the API’s system-assigned managed identity has the Container Apps Jobs Operator role on every job) or by a cron schedule in the Bicep config.
All jobs have replicaTimeout: 4h — long-running batch jobs (rebuild, currency import) need time.

Data Tier

Azure SQL Database

Single server per environment:

Field	Value (dev)
Server	`frmpmauks-dev-sql-01.database.windows.net`
Database	`frmpmauks-dev-sqldb-01`
Tier	Standard, 400 DTU
Max size	10 GB
Backup	Locally redundant (`Local`)
Auth	Azure AD only (`azureADOnlyAuthentication: true`)
TLS	1.2 minimum
Firewall	Allow Azure services (0.0.0.0/0) + VNet rule for app subnet

AAD-only means no SQL logins / passwords. All connections authenticate via managed identity using the Active Directory Default authentication mode on the connection string. The Key Vault secret formation-db-connection-string stores the connection string template; the Container App picks up the actual managed identity token at runtime.

The Allow Azure services firewall rule opens the server to any Azure tenant — it’s a big hammer. The rule is there to permit CI/CD DACPAC deployments (GitHub Actions runners are on azure IPs) without pinning to a changing runner pool. Production tightens this down via a dedicated deployment service principal + temporarily added runner IPs.

Key Vault

Every environment has one Key Vault (frmpmauks-{env}-kv-01). Container Apps reference secrets by name; the Container App configuration resolves each reference to a secret value at startup using its managed identity + Key Vault Secrets User RBAC role.

Secrets maintained by Bicep:

Secret	Purpose
`formation-db-connection-string`	SQL connection string (constructed in Bicep)
`warehouse-db-connection-string`	BI lakehouse (read-only) — source of ECB exchange rates for the currency-import job (nullable — see gotchas)
`formation-api-auth-client-id`, `-scopes`	API Azure AD app registration
`formation-web-auth-client-id`, `-secret`, `-scopes`	Web Azure AD app registration (OAuth)
`formation-web-session-secret`	Session cookie encryption key
`formation-web-google-maps-api-key`	Google Maps embedding key
`formation-users-group-id`, `formation-admins-group-id`	Azure AD group SIDs for role mapping

Container App config wires these into environment variables via secretRef:

// From appConfig.api.containers[].env
{ "name": "ConnectionStrings__FormationDB", "secretRef": "formation-db-connection-string" },
{ "name": "AzureAd__ClientId",             "secretRef": "formation-api-auth-client-id"    }

Secrets do not hot-reload. Updating a Key Vault secret value does not propagate to running replicas — a new revision deploy is required. This is why warehouse-db-connection-string is nullable: Bicep skips writing it unless a value is explicitly provided, so routine redeploys don’t overwrite a stable credential.

Storage Accounts

Two storage accounts per environment:

frmpmauksdevstloadq01 — data ingestion
- Blob container data-load — inbound files dropped here trigger jobload via queue message.
- Blob container data-external — staging for external data pulls.
- Queue data-load — KEDA scaler target for the load job.
- Network: deny-by-default; VNet subnet + AzureServices bypass are the only allowed access paths.
frmpmauksdevstdocs01 — static docs hosting
- Blob container generated-docs — API docs, Storybook builds, DB schema docs.
- Network: deny-by-default; CI temporarily adds the runner IP to the allow list when publishing new content, then revokes.

Both accounts enforce TLS 1.2 minimum and block public blob access.

Identity and Auth

Managed Identities

Every runtime component has its own user-assigned managed identity. User-assigned (rather than system-assigned) lets multiple components share identities where appropriate and simplifies lifecycle management — the identity survives a Container App redeploy.

Identity	Attached to	RBAC roles
`mi-api-01`	API Container App	Key Vault Secrets User, Storage Queue Data Contributor, Storage Blob Data Contributor, ACR Pull
`mi-web-01`	Web Container App	Key Vault Secrets User, ACR Pull
`mi-docs-01`	Docs Container App	Key Vault Secrets User, ACR Pull
`mi-job-01`	jobload	Key Vault Secrets User, Storage Queue Data Contributor, Storage Blob Data Reader, ACR Pull
`mi-jobcompscore-01`	completion-score job	(similar subset)
`mi-jobrebuildqueryviews-01`	query-view rebuild job	(similar subset)
`mi-jobcurimp-01`, `mi-jobdedup-01`	currency / dedup jobs	(similar subset)
`mi-sql-01`	SQL AAD admin proxy	Used as `AuthenticationManagedIdentity` on the SQL connection string
`aca-user-identity-*`	Shared ACR-pull identity	ACR Pull (used by every Container App)

The API also has a system-assigned identity with two unusual roles:

Reader on the subscription — lets the API introspect Azure resource metadata if needed.
Container Apps Jobs Operator on each job — lets the API kick off a manual job run from the admin UI.

Azure AD App Registrations

Two app registrations, mirrored per environment:

API app — JWT bearer audience. Tokens with audience {api-client-id} or api://{api-client-id} are accepted.
Web app — OAuth client for the frontend session. Acquires tokens for the API scope (api://api-client-id/.default) and proxies them on each request.

Client IDs live in Key Vault so Bicep can inject them into Container App env vars. The client secret for the web app is Key Vault-backed; rotation requires updating the secret and redeploying web.

Mock auth in development bypasses both registrations — see Local Dev vs Deployed.

Role Mapping — Users and Admins

The API exposes two Formation roles: User and Admin. These are derived from Azure AD group membership.

Two groups are provisioned per environment: formation-{env}-users and formation-{env}-admins.
Group object IDs are passed into Bicep via parameters and stored as Key Vault secrets (formation-users-group-id, formation-admins-group-id).
On each request, the GroupRoleClaimsTransformation reads group claims from the JWT and emits role claims: members of the users group get role=User, members of the admins group get role=Admin.

The default authorisation policy requires RequireRole("User", "Admin"), so an Azure AD user who authenticates but isn’t in either group sees HTTP 403 on protected endpoints (the /Roles endpoint is the exception — it’s readable by any authenticated user so new users can see what roles they need).

Observability

Every environment has its own Log Analytics workspace (log-01) and Application Insights instance (appi-01). Application Insights is linked to the workspace so logs and metrics share a store.

OpenTelemetry is wired up in Program.cs:

builder.Services.AddOpenTelemetry()
    .ConfigureResource(resource => resource.AddService(serviceName))
    .WithLogging()
    .WithTracing(tracing => tracing
        .AddSource("Formation.Browser")
        .AddSource("Formation.Handlers")
        .AddSource("Formation.Search"))
    .WithMetrics(metrics =>
    {
        metrics.AddAspNetCoreInstrumentation().AddMeter("Formation.API");
        if (isDevelopment) metrics.AddConsoleExporter();
    });

if (!isDevelopment)
{
    builder.Services.AddOpenTelemetry()
        .UseAzureMonitor(options => { })
        .AddAzureMonitorProfiler();
}

Traces: sources Formation.Handlers (per-command spans from InstrumentedCommandMediator), Formation.Search (search pipeline), Formation.Browser (frontend-originated spans). See CQRS flow → Telemetry.
Metrics: meter Formation.API (command / event / search duration histograms) plus standard AspNetCore.* meters.
Profiler: enabled in non-dev environments via AddAzureMonitorProfiler().

In dev, metrics go to the console (AddConsoleExporter) rather than Azure Monitor so they’re visible during dotnet run.

Health: the API exposes /liveness on port 8080; the Container App liveness probe hits it every 10 s with a 3 s timeout.

CI/CD

GitHub Actions workflows in .github/workflows/:

Workflow	Trigger	Deploys
`infrastructure-validate.yml`	PR / manual	Bicep lint + what-if
`infrastructure-deploy.yml`	Manual dispatch	Bicep subscription-scope deployment
`dotnet-service-validate.yml`	PR	.NET build + unit tests
`dotnet-service-deploy.yml`	Push to `main` (src/services/api, src/services/job/**)	Build → push to ACR → update Container App
`web-service-deploy.yml`	Push to `main` (src/services/web)	Build → push to ACR → update Container App + Storybook to blob
`docs-service-deploy.yml`	Push to `main` (src/services/docs)	Build → push to ACR → update Container App
`ingress-deploy.yml`	Push to `main` (src/services/ingrs)	Build → push to ACR → update Container App
`database-docs-deploy.yml`	Manual	Generate DB schema docs → upload to blob
`sql-deploy.yml`	Manual dispatch	DACPAC publish + pre-deploy scripts (per environment)

The deploy pattern is consistent:

- name: Build + push
  uses: docker/build-push-action@v5
  with:
    context: src/services/api
    tags: frmpmaukscommonacr01.azurecr.io/app.api:${{ steps.version.outputs.version }}

- name: Deploy to Container App
  uses: azure/container-apps-deploy-action@v2
  with:
    acrName: frmpmaukscommonacr01
    containerAppName: frmpmauks-dev-ca-api-01
    resourceGroup: frmpmauks-dev-rg-01
    imageToDeploy: frmpmaukscommonacr01.azurecr.io/app.api:${{ steps.version.outputs.version }}

Authentication uses OIDC federation. GitHub secrets AZURE_CLIENT_ID, AZURE_TENANT_ID, AZURE_SUBSCRIPTION_ID + federated credential on a service principal give workflows short-lived tokens without long-lived client secrets in repo.

Semantic versioning is handled by paulhatch/semantic-version — each workflow computes its own version, tags the image, and stamps the revision.

Local Dev vs Deployed

Local development runs on a devcontainer:

# .devcontainer/docker-compose.yml (abridged)
services:
  app:          # Dev container — .NET + Node + tooling
    image: mcr.microsoft.com/devcontainers/dotnet:1-10.0
  sqlserver:    # SQL Server 2022 (Linux)
    image: mcr.microsoft.com/mssql/server:2022-latest
    environment:
      SA_PASSWORD: YourStrong!Passw0rd

Key overrides for dev:

Concern	Local	Deployed
SQL	`localhost:1433` with SA login	Azure SQL with AAD managed identity
Auth	`AzureAd:UseMockAuthentication=true` (defaults to `Admin` role)	Real JWT bearer against Entra ID
Secrets	`appsettings.Development.json` + user secrets	Key Vault references
Search	Local SQL Server 2022 FTI	Azure SQL FTI
Storage	None (imports disabled locally unless configured)	Storage accounts with VNet-restricted access
Telemetry	Console exporter	Azure Monitor + Profiler

Mock auth is enabled via builder.Configuration["AzureAd:UseMockAuthentication"] — when true, a MockAuthHandler synthesises a principal with the role from AzureAd:MockRole (defaults to Admin). This lets the frontend hit the API without configuring a real OAuth flow locally.

Gotchas

RBAC propagation delay. Managed-identity role assignments can take 2–5 minutes to propagate to Azure. A freshly-deployed Container App may fail with 403 on its first request to Key Vault or Storage. Redeploys don’t hit this; new environments do. Wait it out or re-trigger the workflow.
Secrets do not hot-reload. Updating a Key Vault secret value has no effect on running Container Apps — a new revision is required. Automation scripts that rotate secrets must trigger a redeploy too.
warehouseDbConnectionString is nullable-on-purpose. Bicep only writes it when a value is supplied. Routine deploys omit it to avoid overwriting the stable BI-warehouse credential. Only set it on the first deploy or when rotating.
Container Apps have no geo-replication. Backups are locally redundant. This is a cost trade-off — re-add geo-replication in prod if RTO/RPO requirements change.
Single revision per Container App. Traffic splitting and canary deploys are not configured. Rollback is “deploy the previous image tag” — keep old tags around in ACR.
The Azure-Services firewall rule on SQL is broad. It allows any Azure tenant, not just yours. Production tightens via a deployment SP + runner-IP allowlisting. Don’t assume SQL is tenant-private just because it’s VNet-enabled.
Dapr sidecars are running but unused. Every app has Dapr enabled (appId) but no state store, pub/sub, or binding is wired up. Removing Dapr would shave a small amount of memory per replica but requires re-deploying all apps.
KEDA queue scaler metadata uses storageAccountResourceId. The API version (2024-10-02-preview) is required for that field; older Bicep resource versions silently ignore it and the job never scales.
ACR is admin-disabled. There’s no admin_user on the registry. Pulls are identity-based via the user-assigned aca-user-identity-* with ACR Pull role. docker login acr … -u admin -p … will never work.
/api/* routing is path-based, not subdomain-based. The API is reachable at https://formdev.pma.co.uk/api/..., never https://api.formdev.pma.co.uk/.... Clients that need a separate host should go through the ingress, not around it.
System-assigned identity on the API has subscription-scope Reader. This is intentional for future resource-metadata queries but grants more than is strictly needed today. Audit before expanding the surface.
GitHub Actions runners temporarily allowlist into docs storage. The workflow adds the runner IP, waits 30 seconds for rule propagation, uploads, then revokes. If you see transient “storage access denied” during a deploy, the sleep may need extending on a slow region day.
Mock auth doesn’t map groups. The dev MockAuthHandler produces a principal with a single role, not the full group → role transformation. UI behaviours that rely on “not-yet-assigned-any-role” state can’t be tested locally.
Image tag :latest in parameter files is illustrative only. The actual image pinned at deploy time is app.api:${version} from the workflow. Don’t rely on the parameter-file tag — the Container App’s current image is whatever the last deploy set.
No separate prod environment exists yet. dev and uat are the currently-deployed tiers. The Bicep accepts prod as a valid environment value, but the parameter file isn’t wired up. Adding prod requires a new parameter file, AD group provisioning, and a prod-targeted deploy workflow.