Upgrades & Maintenance

This guide covers upgrading Terragnos Core, handling breaking changes, and performing routine maintenance.

Upgrade Process

Pre-Upgrade Checklist

1. Review release notes – Check docs/releases/ for the target version
2. Backup database – Create a full database backup
3. Check compatibility – Verify your configuration is compatible
4. Test in staging – Test the upgrade in a non-production environment first
5. Schedule maintenance window – Plan for potential downtime

Upgrade Steps

1. Pull new images:

   docker compose pull api worker

2. Stop services:

   docker compose stop api worker

3. Run database migrations (if needed):

   docker compose --profile setup up db-setup
   docker compose --profile setup down

4. Start services with new images:

   docker compose up -d api worker

5. Verify upgrade:

   curl http://localhost:3000/v1/health/ready

6. Check version:

   curl -H "Authorization: Bearer <token>" \
     http://localhost:3000/v1/health/ready
   # Response includes version information

Zero-Downtime Upgrades

For zero-downtime upgrades:

1. Deploy new API instances alongside old ones
2. Update load balancer to route traffic to new instances
3. Wait for old connections to drain
4. Stop old instances
5. Upgrade worker instances (can run in parallel with old workers)

Breaking Changes

Breaking changes only occur in major version updates (v1 → v2, v2 → v3, etc.). Minor and patch versions maintain backward compatibility.

Handling Breaking Changes

1. Review migration guide – Check release notes for migration steps
2. Update API calls – Modify code to use new API contracts
3. Update SDKs – Upgrade to compatible SDK versions
4. Test thoroughly – Verify all integrations work with new version
5. Update configuration – Adjust environment variables if needed

Database Migrations

Automatic Migrations

The db-setup container automatically runs migrations:

docker compose --profile setup up db-setup

Manual Migrations

If you need to run migrations manually:

docker compose exec api node dist/scripts/migrate.js

Migration Safety

• Migrations are idempotent (safe to run multiple times)
• Migrations are backward-compatible within the same major version
• Always backup before running migrations

Maintenance Windows

Recommended Schedule

• Weekly – Review logs and metrics
• Monthly – Database maintenance (VACUUM, ANALYZE)
• Quarterly – Full system review and optimization
• As needed – Security patches and critical updates

Database Maintenance

-- Analyze tables for query optimization
ANALYZE;

-- Vacuum to reclaim space
VACUUM ANALYZE;

-- Check for bloat
SELECT schemaname, tablename,
       pg_size_pretty(pg_total_relation_size(schemaname||'.'||tablename)) AS size,
       n_dead_tup
FROM pg_stat_user_tables
WHERE n_dead_tup > 1000
ORDER BY n_dead_tup DESC;

Rollback Procedure

If an upgrade causes issues:

1. Stop new services:

   docker compose stop api worker

2. Restore previous images:

   # Update docker-compose.yml with previous image tags
   docker compose pull api worker

3. Start previous version:

   docker compose up -d api worker

4. Verify rollback:

   curl http://localhost:3000/v1/health/ready

Note: Database migrations cannot be automatically rolled back. If migrations were run, you may need to restore from backup.

Release Cadence

• Major versions – Breaking changes, new features (approximately annually)
• Minor versions – New features, backward-compatible (approximately quarterly)
• Patch versions – Bug fixes, security patches (as needed)

Best Practices

1. Always backup – Create database backups before upgrades
2. Test first – Test upgrades in staging before production
3. Read release notes – Review changes and migration guides
4. Monitor after upgrade – Watch logs and metrics closely
5. Keep images updated – Regularly update for security patches
6. Document changes – Keep track of configuration changes
7. Plan maintenance – Schedule regular maintenance windows