Backups and recovery
Incomplete
This page records what is known and marks what still needs deciding. The modernization initiative lists "automated database backups and tested restoration procedures" as a risk-mitigation item; until that is done, treat the gaps below as open work.
What exists
| Component | State today | Notes |
|---|---|---|
Production database (emawweb-server, Azure Database for PostgreSQL flexible server) | Azure's built-in automated backups with point-in-time restore (default retention 7 days unless changed) | Confirm the retention setting in the portal and record it here |
Dev database (development-database-vm) | No scheduled backup known | Holds the imported Planning Center contacts; a loss would mean re-importing |
Azure Blob emawblob/media | Azure storage redundancy; soft delete status unknown | Cover images, generated audio, embeddings parquet |
Cloudflare R2 emaw-public | No versioning | Resource media; keys are immutable, so accidental overwrite is unlikely, deletion is not recoverable |
| Redis | redis_data volume; not backed up | Cache and queues only; nothing durable lives here |
| Qdrant | qdrant_storage volume; rebuildable from the parquet in blob | load_bible_embeddings recreates it |
| Code | GitHub | |
| Secrets | Password manager, Portainer, Key Vault | Ensure the password manager has the production .env too |
| Stripe, Deposco, ACS, Planning Center | Vendor-held | Stripe is the source of truth for payments |
Recovery procedures
Backend stack (VM lost)
- New VM in the resource group, Docker installed, on the vnet (prod) or tailnet (dev).
- Clone the repo at
main(prod) ordevelop(dev); restore.envfrom the password manager. docker compose up -d --build;load_bible_embeddings.- Install the Cloudflare tunnel token (same token: the tunnel is remotely managed, so the hostname follows the connector).
- Prod: install Portainer and re-import the stack; re-whitelist IPs.
Production database
Azure Portal → the flexible server → Backup and restore → point-in-time restore to a new server; update DB_HOST in the production .env; redeploy. Test this once before it is needed.
Dev database
Decide: nightly pg_dump from the VM to the blob account, or convert dev to a flexible server with built-in backups. Open item.
Blob or R2 object deleted
Blob: enable soft delete and record the retention. R2: re-upload from the source file; the resource's media_key in the admin app will need the new key.
Open items
- Record production backup retention and test a restore
- Schedule dev database dumps
- Enable blob soft delete
- Decide on R2 bucket versioning or an off-site copy of resource media
- Write the incident contact list and escalation on Who to ask