feat: backup cron
This commit is contained in:
@@ -4,6 +4,11 @@ APP_KEY=
|
||||
APP_DEBUG=true
|
||||
APP_URL=http://localhost
|
||||
APP_PORT=8003
|
||||
IMAGE_TAG=
|
||||
POSTGRES_VERSION=15-alpine
|
||||
REDIS_VERSION=7.4-alpine
|
||||
MEILISEARCH_VERSION=1.49.0
|
||||
ADMINER_VERSION=5.5.0-standalone
|
||||
APP_LOCALE=fr
|
||||
APP_FALLBACK_LOCALE=en
|
||||
APP_FAKER_LOCALE=en_US
|
||||
@@ -60,10 +65,20 @@ STRAVA_CLIENT_SECRET=
|
||||
|
||||
AWS_ACCESS_KEY_ID=
|
||||
AWS_SECRET_ACCESS_KEY=
|
||||
AWS_SESSION_TOKEN=
|
||||
AWS_DEFAULT_REGION=us-east-1
|
||||
AWS_BUCKET=
|
||||
AWS_USE_PATH_STYLE_ENDPOINT=false
|
||||
|
||||
# Production database backups (use a private bucket or a private prefix)
|
||||
BACKUP_CRON="0 3 * * *"
|
||||
BACKUP_LOCAL_RETENTION_DAYS=7
|
||||
BACKUP_RUN_ON_START=true
|
||||
BACKUP_S3_BUCKET=bowli-backup
|
||||
BACKUP_S3_PREFIX=database-backups/postgres
|
||||
BACKUP_S3_ENDPOINT=
|
||||
TZ=Europe/Paris
|
||||
|
||||
OLLAMA_API_KEY=
|
||||
GEMINI_API_KEY=
|
||||
OPENAI_API_KEY=
|
||||
|
||||
@@ -0,0 +1,202 @@
|
||||
# Daily Meal API
|
||||
|
||||
API Laravel de Daily Meal.
|
||||
|
||||
## Docker en production
|
||||
|
||||
Le fichier source du dépôt est `docker-compose.prod.yml`. Sur le serveur, il est
|
||||
déployé sous le nom `docker-compose.yml`. Les commandes de ce guide sont donc
|
||||
exécutées depuis le dossier de l'application sans option `-f`.
|
||||
|
||||
### Configuration
|
||||
|
||||
Le Compose charge la configuration applicative depuis le `.env` de production.
|
||||
Les variables Laravel ne sont pas répétées dans le fichier Compose afin de
|
||||
conserver une seule source de vérité.
|
||||
|
||||
Exemple des variables propres à l'infrastructure :
|
||||
|
||||
```dotenv
|
||||
IMAGE_TAG=2026-08-13.1
|
||||
|
||||
APP_ENV=production
|
||||
APP_DEBUG=false
|
||||
LOG_CHANNEL=stderr
|
||||
LOG_LEVEL=warning
|
||||
|
||||
DB_CONNECTION=pgsql
|
||||
DB_HOST=postgres
|
||||
DB_PORT=5432
|
||||
DB_DATABASE=bowli
|
||||
DB_USERNAME=bowli
|
||||
DB_PASSWORD=change-me
|
||||
|
||||
REDIS_HOST=redis
|
||||
QUEUE_CONNECTION=redis
|
||||
SCOUT_DRIVER=meilisearch
|
||||
MEILISEARCH_HOST=http://meilisearch:7700
|
||||
MEILISEARCH_KEY=change-me
|
||||
|
||||
FILESYSTEM_DISK=s3
|
||||
AWS_ACCESS_KEY_ID=change-me
|
||||
AWS_SECRET_ACCESS_KEY=change-me
|
||||
AWS_DEFAULT_REGION=eu-west-3
|
||||
```
|
||||
|
||||
`IMAGE_TAG` est obligatoire. Utiliser un tag de release immuable permet de savoir
|
||||
exactement quelle version est déployée et évite l'utilisation accidentelle de
|
||||
`latest`.
|
||||
|
||||
### Volumes et réseau externes
|
||||
|
||||
PostgreSQL, Redis, Meilisearch et les sauvegardes locales utilisent des volumes
|
||||
externes. Ils restent présents quand le projet Compose est recréé.
|
||||
|
||||
À effectuer une seule fois sur un nouveau serveur :
|
||||
|
||||
```bash
|
||||
docker volume create bemeal_pgsql_data
|
||||
docker volume create bemeal_redis_data
|
||||
docker volume create bemeal_meilisearch_data
|
||||
docker volume create bemeal_database_backups
|
||||
docker network create proxy
|
||||
```
|
||||
|
||||
Ces commandes sont idempotentes. Le réseau `proxy` peut déjà avoir été créé par
|
||||
le reverse proxy. Ne jamais supprimer `bemeal_pgsql_data` sans avoir validé une
|
||||
sauvegarde.
|
||||
|
||||
### Premier démarrage
|
||||
|
||||
```bash
|
||||
docker compose config -q
|
||||
docker compose build db-backup
|
||||
docker compose pull app postgres redis meilisearch
|
||||
docker compose up -d
|
||||
docker compose ps
|
||||
```
|
||||
|
||||
`docker compose build db-backup` construit l'image locale contenant `pg_dump`,
|
||||
`pg_restore`, AWS CLI, le cron et les scripts de `docker/database-backup`. Il faut
|
||||
l'exécuter au premier déploiement, après une modification de ces scripts ou après
|
||||
un changement de version de PostgreSQL.
|
||||
|
||||
Pour reconstruire et redémarrer uniquement ce service :
|
||||
|
||||
```bash
|
||||
docker compose up -d --build db-backup
|
||||
```
|
||||
|
||||
### Mettre à jour l'application
|
||||
|
||||
Après avoir changé `IMAGE_TAG` dans `.env` :
|
||||
|
||||
```bash
|
||||
docker compose config -q
|
||||
docker compose pull app
|
||||
docker compose up -d --remove-orphans
|
||||
docker compose ps
|
||||
```
|
||||
|
||||
Si les migrations ne sont pas déjà gérées par le pipeline ou l'image :
|
||||
|
||||
```bash
|
||||
docker compose exec app php artisan migrate --force
|
||||
```
|
||||
|
||||
## Sauvegardes PostgreSQL vers S3
|
||||
|
||||
Le service `db-backup` lance une sauvegarde au démarrage, puis chaque jour à
|
||||
03:00. Chaque archive est compressée, contrôlée avec `pg_restore`, accompagnée
|
||||
d'un checksum SHA-256 puis envoyée vers S3 avec le chiffrement AES-256. Les
|
||||
copies locales sont conservées 7 jours par défaut.
|
||||
|
||||
```dotenv
|
||||
BACKUP_CRON="0 3 * * *"
|
||||
BACKUP_RUN_ON_START=true
|
||||
BACKUP_LOCAL_RETENTION_DAYS=7
|
||||
BACKUP_S3_BUCKET=bowli-backup
|
||||
BACKUP_S3_PREFIX=database-backups/postgres
|
||||
BACKUP_S3_ENDPOINT=
|
||||
TZ=Europe/Paris
|
||||
```
|
||||
|
||||
Ces valeurs sont les valeurs par défaut et peuvent être omises du `.env`. Le
|
||||
bucket `bowli-backup` doit rester entièrement privé. `BACKUP_S3_ENDPOINT` reste
|
||||
vide avec AWS S3.
|
||||
|
||||
### Autorisations IAM minimales
|
||||
|
||||
```json
|
||||
{
|
||||
"Version": "2012-10-17",
|
||||
"Statement": [
|
||||
{
|
||||
"Sid": "ListDatabaseBackups",
|
||||
"Effect": "Allow",
|
||||
"Action": "s3:ListBucket",
|
||||
"Resource": "arn:aws:s3:::bowli-backup",
|
||||
"Condition": {
|
||||
"StringLike": {
|
||||
"s3:prefix": [
|
||||
"database-backups/postgres",
|
||||
"database-backups/postgres/*"
|
||||
]
|
||||
}
|
||||
}
|
||||
},
|
||||
{
|
||||
"Sid": "ManageDatabaseBackupObjects",
|
||||
"Effect": "Allow",
|
||||
"Action": ["s3:PutObject", "s3:GetObject"],
|
||||
"Resource": "arn:aws:s3:::bowli-backup/database-backups/postgres/*"
|
||||
}
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
Le conteneur ne supprime pas les sauvegardes distantes. Configurer une règle de
|
||||
cycle de vie S3 sur ce préfixe, par exemple après 30 ou 90 jours.
|
||||
|
||||
### Sauvegarde manuelle et contrôle
|
||||
|
||||
```bash
|
||||
docker compose exec db-backup /usr/local/bin/backup-database manual
|
||||
docker compose ps db-backup
|
||||
docker compose logs --tail=100 db-backup
|
||||
```
|
||||
|
||||
## Restaurer une sauvegarde
|
||||
|
||||
Pour restaurer la sauvegarde S3 la plus récente :
|
||||
|
||||
```bash
|
||||
./docker/database-backup/restore-production latest-s3
|
||||
```
|
||||
|
||||
Autres sources acceptées :
|
||||
|
||||
```bash
|
||||
# Dernière sauvegarde du volume local
|
||||
./docker/database-backup/restore-production latest-local
|
||||
|
||||
# Sauvegarde S3 précise
|
||||
./docker/database-backup/restore-production \
|
||||
s3://bowli-backup/database-backups/postgres/20260813T030000Z-bowli-scheduled.dump
|
||||
```
|
||||
|
||||
La restauration demande une confirmation explicite. Elle arrête l'application
|
||||
et les workers, crée une sauvegarde de sécurité, vérifie l'archive, restaure dans
|
||||
une base temporaire puis échange les bases. L'ancienne base est conservée sous
|
||||
un nom daté pour permettre un rollback.
|
||||
|
||||
Les services redémarrent uniquement si toute l'opération réussit. En cas
|
||||
d'échec, ils restent arrêtés afin d'éviter d'écrire dans une base potentiellement
|
||||
incomplète.
|
||||
|
||||
Pour utiliser exceptionnellement un autre nom de fichier Compose :
|
||||
|
||||
```bash
|
||||
COMPOSE_FILE=docker-compose.prod.yml \
|
||||
./docker/database-backup/restore-production latest-s3
|
||||
```
|
||||
|
||||
+103
-44
@@ -1,50 +1,67 @@
|
||||
name: daily-meal-api
|
||||
|
||||
x-app-image: &app-image git.leonmorival.xyz/leonm/daily-meal-api:${IMAGE_TAG:-latest}
|
||||
x-app-image: &app-image git.leonmorival.xyz/leonm/daily-meal-api:${IMAGE_TAG:?IMAGE_TAG is required}
|
||||
|
||||
x-app-environment: &app-environment
|
||||
APP_ENV: production
|
||||
APP_DEBUG: "false"
|
||||
OCTANE_SERVER: frankenphp
|
||||
OCTANE_HTTPS: "true"
|
||||
OCTANE_PORT: "80"
|
||||
OCTANE_ADMIN_PORT: "2019"
|
||||
OCTANE_MAX_REQUESTS: "500"
|
||||
LOG_CHANNEL: stderr
|
||||
LOG_LEVEL: "${LOG_LEVEL:-warning}"
|
||||
DB_CONNECTION: pgsql
|
||||
DB_HOST: postgres
|
||||
DB_PORT: "5432"
|
||||
REDIS_HOST: redis
|
||||
REDIS_CLIENT: predis
|
||||
QUEUE_CONNECTION: redis
|
||||
MAIL_MAILER: resend
|
||||
MAIL_FROM_ADDRESS: "${MAIL_FROM_ADDRESS:?MAIL_FROM_ADDRESS is required}"
|
||||
MAIL_FROM_NAME: "${MAIL_FROM_NAME:-Bowli}"
|
||||
RESEND_API_KEY: "${RESEND_API_KEY:?RESEND_API_KEY is required}"
|
||||
SCOUT_DRIVER: meilisearch
|
||||
MEILISEARCH_HOST: http://meilisearch:7700
|
||||
|
||||
x-app-volumes: &app-volumes
|
||||
- storage-data:/app/storage/app
|
||||
- storage-public-data:/app/storage/app/public
|
||||
x-default-logging: &default-logging
|
||||
driver: json-file
|
||||
options:
|
||||
max-size: "10m"
|
||||
max-file: "5"
|
||||
|
||||
x-app-service: &app-service
|
||||
image: *app-image
|
||||
restart: unless-stopped
|
||||
init: true
|
||||
env_file:
|
||||
- .env
|
||||
environment: *app-environment
|
||||
volumes: *app-volumes
|
||||
depends_on:
|
||||
postgres:
|
||||
condition: service_healthy
|
||||
redis:
|
||||
condition: service_started
|
||||
condition: service_healthy
|
||||
meilisearch:
|
||||
condition: service_started
|
||||
condition: service_healthy
|
||||
networks:
|
||||
- internal
|
||||
stop_grace_period: 30s
|
||||
logging: *default-logging
|
||||
|
||||
x-database-tools: &database-tools
|
||||
image: daily-meal-postgres-tools:${POSTGRES_VERSION:-15-alpine}
|
||||
build:
|
||||
context: ./docker/database-backup
|
||||
args:
|
||||
POSTGRES_IMAGE: postgres:${POSTGRES_VERSION:-15-alpine}
|
||||
environment:
|
||||
AWS_ACCESS_KEY_ID: ${AWS_ACCESS_KEY_ID:?AWS_ACCESS_KEY_ID is required}
|
||||
AWS_DEFAULT_REGION: ${AWS_DEFAULT_REGION:?AWS_DEFAULT_REGION is required}
|
||||
AWS_ENDPOINT: ${AWS_ENDPOINT:-}
|
||||
AWS_SECRET_ACCESS_KEY: ${AWS_SECRET_ACCESS_KEY:?AWS_SECRET_ACCESS_KEY is required}
|
||||
AWS_SESSION_TOKEN: ${AWS_SESSION_TOKEN:-}
|
||||
BACKUP_CRON: ${BACKUP_CRON:-0 3 * * *}
|
||||
BACKUP_LOCAL_RETENTION_DAYS: ${BACKUP_LOCAL_RETENTION_DAYS:-7}
|
||||
BACKUP_RUN_ON_START: ${BACKUP_RUN_ON_START:-true}
|
||||
BACKUP_S3_BUCKET: ${BACKUP_S3_BUCKET:-bowli-backup}
|
||||
BACKUP_S3_ENDPOINT: ${BACKUP_S3_ENDPOINT:-}
|
||||
BACKUP_S3_PREFIX: ${BACKUP_S3_PREFIX:-database-backups/postgres}
|
||||
RESTORE_CONFIRM: ${RESTORE_CONFIRM:-}
|
||||
RESTORE_SKIP_SAFETY_BACKUP: ${RESTORE_SKIP_SAFETY_BACKUP:-false}
|
||||
DB_DATABASE: ${DB_DATABASE:?DB_DATABASE is required}
|
||||
DB_HOST: postgres
|
||||
DB_PASSWORD: ${DB_PASSWORD:?DB_PASSWORD is required}
|
||||
DB_PORT: "5432"
|
||||
DB_USERNAME: ${DB_USERNAME:?DB_USERNAME is required}
|
||||
TZ: ${TZ:-Europe/Paris}
|
||||
volumes:
|
||||
- database-backups:/backups
|
||||
depends_on:
|
||||
postgres:
|
||||
condition: service_healthy
|
||||
networks:
|
||||
- internal
|
||||
logging: *default-logging
|
||||
security_opt:
|
||||
- no-new-privileges:true
|
||||
|
||||
services:
|
||||
app:
|
||||
@@ -81,13 +98,13 @@ services:
|
||||
command: ["php", "artisan", "schedule:work"]
|
||||
|
||||
postgres:
|
||||
image: postgres:15-alpine
|
||||
image: postgres:${POSTGRES_VERSION:-15-alpine}
|
||||
container_name: daily-meal-api-postgres
|
||||
restart: unless-stopped
|
||||
environment:
|
||||
POSTGRES_DB: "${DB_DATABASE:?DB_DATABASE is required}"
|
||||
POSTGRES_USER: "${DB_USERNAME:?DB_USERNAME is required}"
|
||||
POSTGRES_PASSWORD: "${DB_PASSWORD:?DB_PASSWORD is required}"
|
||||
POSTGRES_DB: ${DB_DATABASE:?DB_DATABASE is required}
|
||||
POSTGRES_USER: ${DB_USERNAME:?DB_USERNAME is required}
|
||||
POSTGRES_PASSWORD: ${DB_PASSWORD:?DB_PASSWORD is required}
|
||||
volumes:
|
||||
- postgres-data:/var/lib/postgresql/data
|
||||
healthcheck:
|
||||
@@ -98,15 +115,26 @@ services:
|
||||
retries: 10
|
||||
networks:
|
||||
- internal
|
||||
stop_grace_period: 60s
|
||||
logging: *default-logging
|
||||
|
||||
redis:
|
||||
image: redis:alpine
|
||||
image: redis:${REDIS_VERSION:-7.4-alpine}
|
||||
container_name: daily-meal-api-redis
|
||||
restart: unless-stopped
|
||||
command: ["redis-server", "--appendonly", "yes", "--appendfsync", "everysec"]
|
||||
volumes:
|
||||
- redis-data:/data
|
||||
healthcheck:
|
||||
test: ["CMD", "redis-cli", "ping"]
|
||||
start_period: 10s
|
||||
interval: 5s
|
||||
timeout: 3s
|
||||
retries: 10
|
||||
networks:
|
||||
- internal
|
||||
stop_grace_period: 30s
|
||||
logging: *default-logging
|
||||
|
||||
meilisearch:
|
||||
image: getmeili/meilisearch:v${MEILISEARCH_VERSION:-1.49.0}
|
||||
@@ -114,24 +142,54 @@ services:
|
||||
restart: unless-stopped
|
||||
environment:
|
||||
MEILI_NO_ANALYTICS: "true"
|
||||
MEILI_MASTER_KEY: "${MEILISEARCH_KEY:?MEILISEARCH_KEY is required}"
|
||||
MEILI_MASTER_KEY: ${MEILISEARCH_KEY:?MEILISEARCH_KEY is required}
|
||||
volumes:
|
||||
- meilisearch-data:/meili_data
|
||||
healthcheck:
|
||||
test: ["CMD-SHELL", "curl --fail --silent http://127.0.0.1:7700/health >/dev/null"]
|
||||
start_period: 20s
|
||||
interval: 10s
|
||||
timeout: 5s
|
||||
retries: 10
|
||||
networks:
|
||||
- internal
|
||||
stop_grace_period: 30s
|
||||
logging: *default-logging
|
||||
|
||||
db-backup:
|
||||
<<: *database-tools
|
||||
container_name: daily-meal-api-db-backup
|
||||
restart: unless-stopped
|
||||
healthcheck:
|
||||
test: ["CMD-SHELL", "pg_isready -h $$DB_HOST -p $$DB_PORT -U $$DB_USERNAME -d $$DB_DATABASE && test -f /backups/.last-success && find /backups/.last-success -mtime -2 -print -quit | grep -q ."]
|
||||
start_period: 5m
|
||||
interval: 30s
|
||||
timeout: 5s
|
||||
retries: 3
|
||||
|
||||
db-restore:
|
||||
<<: *database-tools
|
||||
profiles:
|
||||
- restore
|
||||
restart: "no"
|
||||
entrypoint: ["/usr/local/bin/restore-database"]
|
||||
|
||||
adminer:
|
||||
image: adminer:latest
|
||||
image: adminer:${ADMINER_VERSION:-5.5.0-standalone}
|
||||
container_name: daily-meal-api-adminer
|
||||
restart: unless-stopped
|
||||
restart: "no"
|
||||
profiles:
|
||||
- debug
|
||||
ports:
|
||||
- "127.0.0.1:${ADMINER_PORT:-8080}:8080"
|
||||
environment:
|
||||
ADMINER_DEFAULT_SERVER: postgres
|
||||
depends_on:
|
||||
postgres:
|
||||
condition: service_healthy
|
||||
networks:
|
||||
- internal
|
||||
logging: *default-logging
|
||||
|
||||
volumes:
|
||||
postgres-data:
|
||||
@@ -139,20 +197,21 @@ volumes:
|
||||
name: bemeal_pgsql_data
|
||||
|
||||
redis-data:
|
||||
external: true
|
||||
name: bemeal_redis_data
|
||||
|
||||
meilisearch-data:
|
||||
external: true
|
||||
name: bemeal_meilisearch_data
|
||||
|
||||
storage-data:
|
||||
name: bemeal_storage_data
|
||||
|
||||
storage-public-data:
|
||||
name: bemeal_storage_public_data
|
||||
database-backups:
|
||||
external: true
|
||||
name: bemeal_database_backups
|
||||
|
||||
networks:
|
||||
proxy:
|
||||
external: true
|
||||
name: proxy
|
||||
|
||||
internal:
|
||||
driver: bridge
|
||||
|
||||
@@ -0,0 +1,15 @@
|
||||
ARG POSTGRES_IMAGE=postgres:15-alpine
|
||||
FROM ${POSTGRES_IMAGE}
|
||||
|
||||
RUN apk add --no-cache aws-cli tzdata
|
||||
|
||||
COPY backup-cron-entrypoint /usr/local/bin/backup-cron-entrypoint
|
||||
COPY backup-database /usr/local/bin/backup-database
|
||||
COPY restore-database /usr/local/bin/restore-database
|
||||
|
||||
RUN chmod 0755 \
|
||||
/usr/local/bin/backup-cron-entrypoint \
|
||||
/usr/local/bin/backup-database \
|
||||
/usr/local/bin/restore-database
|
||||
|
||||
ENTRYPOINT ["/usr/local/bin/backup-cron-entrypoint"]
|
||||
+32
@@ -0,0 +1,32 @@
|
||||
#!/bin/sh
|
||||
set -eu
|
||||
|
||||
: "${BACKUP_CRON:=0 3 * * *}"
|
||||
|
||||
case "$BACKUP_CRON" in
|
||||
*"
|
||||
"*)
|
||||
echo "BACKUP_CRON must contain a single cron expression." >&2
|
||||
exit 1
|
||||
;;
|
||||
esac
|
||||
|
||||
cron_field_count="$(printf "%s\n" "$BACKUP_CRON" | awk '{ print NF }')"
|
||||
|
||||
if [ "$cron_field_count" -ne 5 ]; then
|
||||
echo "BACKUP_CRON must contain exactly five fields." >&2
|
||||
exit 1
|
||||
fi
|
||||
|
||||
mkdir -p /backups
|
||||
umask 077
|
||||
|
||||
printf "%s /usr/local/bin/backup-database scheduled >> /proc/1/fd/1 2>> /proc/1/fd/2\n" \
|
||||
"$BACKUP_CRON" > /etc/crontabs/root
|
||||
|
||||
if [ "${BACKUP_RUN_ON_START:-false}" = "true" ]; then
|
||||
/usr/local/bin/backup-database startup
|
||||
fi
|
||||
|
||||
echo "Database backup cron configured: $BACKUP_CRON"
|
||||
exec crond -f -l 2
|
||||
Executable
+122
@@ -0,0 +1,122 @@
|
||||
#!/bin/sh
|
||||
set -eu
|
||||
|
||||
umask 077
|
||||
|
||||
require_variable() {
|
||||
variable_name="$1"
|
||||
eval "variable_value=\${$variable_name:-}"
|
||||
|
||||
if [ -z "$variable_value" ]; then
|
||||
echo "$variable_name is required." >&2
|
||||
exit 1
|
||||
fi
|
||||
}
|
||||
|
||||
aws_cli() {
|
||||
endpoint="${BACKUP_S3_ENDPOINT:-${AWS_ENDPOINT:-}}"
|
||||
|
||||
if [ -n "$endpoint" ]; then
|
||||
aws --endpoint-url "$endpoint" "$@"
|
||||
else
|
||||
aws "$@"
|
||||
fi
|
||||
}
|
||||
|
||||
for variable in \
|
||||
AWS_ACCESS_KEY_ID \
|
||||
AWS_DEFAULT_REGION \
|
||||
AWS_SECRET_ACCESS_KEY \
|
||||
BACKUP_S3_BUCKET \
|
||||
DB_DATABASE \
|
||||
DB_HOST \
|
||||
DB_PASSWORD \
|
||||
DB_USERNAME
|
||||
do
|
||||
require_variable "$variable"
|
||||
done
|
||||
|
||||
backup_directory="${BACKUP_DIRECTORY:-/backups}"
|
||||
backup_prefix="${BACKUP_S3_PREFIX:-database-backups/postgres}"
|
||||
backup_prefix="${backup_prefix#/}"
|
||||
backup_prefix="${backup_prefix%/}"
|
||||
backup_label="${1:-manual}"
|
||||
backup_label="$(printf "%s" "$backup_label" | tr -cd "A-Za-z0-9_-")"
|
||||
retention_days="${BACKUP_LOCAL_RETENTION_DAYS:-7}"
|
||||
timestamp="$(date -u +"%Y%m%dT%H%M%SZ")"
|
||||
database_name="$(printf "%s" "$DB_DATABASE" | tr -cd "A-Za-z0-9_-")"
|
||||
backup_name="${timestamp}-${database_name}-${backup_label:-manual}.dump"
|
||||
backup_path="$backup_directory/$backup_name"
|
||||
temporary_path="$backup_path.partial"
|
||||
checksum_path="$backup_path.sha256"
|
||||
lock_directory="/tmp/bowli-database-backup.lock"
|
||||
|
||||
case "$retention_days" in
|
||||
""|*[!0-9]*)
|
||||
echo "BACKUP_LOCAL_RETENTION_DAYS must be a positive integer." >&2
|
||||
exit 1
|
||||
;;
|
||||
esac
|
||||
|
||||
if ! mkdir "$lock_directory" 2>/dev/null; then
|
||||
echo "Another database backup is already running." >&2
|
||||
exit 1
|
||||
fi
|
||||
|
||||
cleanup() {
|
||||
rm -f "$temporary_path"
|
||||
rmdir "$lock_directory" 2>/dev/null || true
|
||||
}
|
||||
|
||||
trap cleanup EXIT INT TERM
|
||||
|
||||
mkdir -p "$backup_directory"
|
||||
|
||||
export PGPASSWORD="$DB_PASSWORD"
|
||||
export PGCONNECT_TIMEOUT="${PGCONNECT_TIMEOUT:-10}"
|
||||
|
||||
echo "Creating PostgreSQL backup: $backup_name"
|
||||
pg_dump \
|
||||
--host="$DB_HOST" \
|
||||
--port="${DB_PORT:-5432}" \
|
||||
--username="$DB_USERNAME" \
|
||||
--dbname="$DB_DATABASE" \
|
||||
--format=custom \
|
||||
--compress=9 \
|
||||
--no-owner \
|
||||
--no-acl \
|
||||
--file="$temporary_path"
|
||||
|
||||
pg_restore --list "$temporary_path" >/dev/null
|
||||
mv "$temporary_path" "$backup_path"
|
||||
|
||||
(
|
||||
cd "$backup_directory"
|
||||
sha256sum "$backup_name" > "$backup_name.sha256"
|
||||
)
|
||||
|
||||
s3_directory="s3://$BACKUP_S3_BUCKET"
|
||||
if [ -n "$backup_prefix" ]; then
|
||||
s3_directory="$s3_directory/$backup_prefix"
|
||||
fi
|
||||
|
||||
echo "Uploading backup to $s3_directory/"
|
||||
aws_cli s3 cp \
|
||||
"$backup_path" \
|
||||
"$s3_directory/$backup_name" \
|
||||
--sse AES256 \
|
||||
--only-show-errors
|
||||
aws_cli s3 cp \
|
||||
"$checksum_path" \
|
||||
"$s3_directory/$backup_name.sha256" \
|
||||
--sse AES256 \
|
||||
--only-show-errors
|
||||
|
||||
touch "$backup_directory/.last-success"
|
||||
|
||||
find "$backup_directory" -type f \
|
||||
\( -name "*.dump" -o -name "*.dump.sha256" \) \
|
||||
-mtime "+$retention_days" \
|
||||
-delete
|
||||
|
||||
echo "Database backup completed: $backup_name"
|
||||
Executable
+230
@@ -0,0 +1,230 @@
|
||||
#!/bin/sh
|
||||
set -eu
|
||||
|
||||
umask 077
|
||||
|
||||
confirmation_phrase="replace-bowli-production-database"
|
||||
|
||||
if [ "${RESTORE_CONFIRM:-}" != "$confirmation_phrase" ]; then
|
||||
echo "Database restoration was not confirmed." >&2
|
||||
echo "Run it with RESTORE_CONFIRM=$confirmation_phrase" >&2
|
||||
exit 1
|
||||
fi
|
||||
|
||||
require_variable() {
|
||||
variable_name="$1"
|
||||
eval "variable_value=\${$variable_name:-}"
|
||||
|
||||
if [ -z "$variable_value" ]; then
|
||||
echo "$variable_name is required." >&2
|
||||
exit 1
|
||||
fi
|
||||
}
|
||||
|
||||
aws_cli() {
|
||||
endpoint="${BACKUP_S3_ENDPOINT:-${AWS_ENDPOINT:-}}"
|
||||
|
||||
if [ -n "$endpoint" ]; then
|
||||
aws --endpoint-url "$endpoint" "$@"
|
||||
else
|
||||
aws "$@"
|
||||
fi
|
||||
}
|
||||
|
||||
for variable in DB_DATABASE DB_HOST DB_PASSWORD DB_USERNAME
|
||||
do
|
||||
require_variable "$variable"
|
||||
done
|
||||
|
||||
case "$DB_DATABASE" in
|
||||
*[!A-Za-z0-9_-]*)
|
||||
echo "DB_DATABASE may only contain letters, numbers, underscores and hyphens for restoration." >&2
|
||||
exit 1
|
||||
;;
|
||||
esac
|
||||
|
||||
source_name="${1:-latest-s3}"
|
||||
backup_directory="${BACKUP_DIRECTORY:-/backups}"
|
||||
backup_prefix="${BACKUP_S3_PREFIX:-database-backups/postgres}"
|
||||
backup_prefix="${backup_prefix#/}"
|
||||
backup_prefix="${backup_prefix%/}"
|
||||
temporary_directory="$(mktemp -d)"
|
||||
restore_path=""
|
||||
|
||||
cleanup() {
|
||||
rm -rf "$temporary_directory"
|
||||
}
|
||||
|
||||
trap cleanup EXIT INT TERM
|
||||
|
||||
download_s3_backup() {
|
||||
s3_uri="$1"
|
||||
file_name="${s3_uri##*/}"
|
||||
restore_path="$temporary_directory/$file_name"
|
||||
|
||||
aws_cli s3 cp "$s3_uri" "$restore_path" --only-show-errors
|
||||
aws_cli s3 cp "$s3_uri.sha256" "$restore_path.sha256" --only-show-errors
|
||||
}
|
||||
|
||||
case "$source_name" in
|
||||
latest-local)
|
||||
restore_path="$(find "$backup_directory" -maxdepth 1 -type f -name "*.dump" | sort | tail -n 1)"
|
||||
;;
|
||||
latest-s3)
|
||||
require_variable AWS_ACCESS_KEY_ID
|
||||
require_variable AWS_DEFAULT_REGION
|
||||
require_variable AWS_SECRET_ACCESS_KEY
|
||||
require_variable BACKUP_S3_BUCKET
|
||||
|
||||
s3_directory="s3://$BACKUP_S3_BUCKET"
|
||||
if [ -n "$backup_prefix" ]; then
|
||||
s3_directory="$s3_directory/$backup_prefix"
|
||||
fi
|
||||
|
||||
latest_key="$(aws_cli s3 ls "$s3_directory/" --recursive \
|
||||
| awk '$4 ~ /\.dump$/ { print $4 }' \
|
||||
| sort \
|
||||
| tail -n 1)"
|
||||
|
||||
if [ -n "$latest_key" ]; then
|
||||
download_s3_backup "s3://$BACKUP_S3_BUCKET/$latest_key"
|
||||
fi
|
||||
;;
|
||||
s3://*)
|
||||
require_variable AWS_ACCESS_KEY_ID
|
||||
require_variable AWS_DEFAULT_REGION
|
||||
require_variable AWS_SECRET_ACCESS_KEY
|
||||
download_s3_backup "$source_name"
|
||||
;;
|
||||
/*)
|
||||
restore_path="$source_name"
|
||||
;;
|
||||
*)
|
||||
restore_path="$backup_directory/$source_name"
|
||||
;;
|
||||
esac
|
||||
|
||||
if [ -z "$restore_path" ] || [ ! -f "$restore_path" ]; then
|
||||
echo "Backup not found: $source_name" >&2
|
||||
exit 1
|
||||
fi
|
||||
|
||||
if [ ! -f "$restore_path.sha256" ]; then
|
||||
echo "Checksum not found for: $restore_path" >&2
|
||||
exit 1
|
||||
fi
|
||||
|
||||
(
|
||||
cd "$(dirname "$restore_path")"
|
||||
sha256sum -c "$(basename "$restore_path").sha256"
|
||||
)
|
||||
pg_restore --list "$restore_path" >/dev/null
|
||||
|
||||
if [ "$(dirname "$restore_path")" != "$temporary_directory" ]; then
|
||||
cp "$restore_path" "$temporary_directory/"
|
||||
cp "$restore_path.sha256" "$temporary_directory/"
|
||||
restore_path="$temporary_directory/$(basename "$restore_path")"
|
||||
fi
|
||||
|
||||
export PGPASSWORD="$DB_PASSWORD"
|
||||
export PGCONNECT_TIMEOUT="${PGCONNECT_TIMEOUT:-10}"
|
||||
|
||||
if [ "${RESTORE_SKIP_SAFETY_BACKUP:-false}" != "true" ]; then
|
||||
echo "Creating a safety backup before restoration."
|
||||
/usr/local/bin/backup-database pre-restore
|
||||
fi
|
||||
|
||||
timestamp="$(date -u +"%Y%m%dT%H%M%SZ")"
|
||||
database_prefix="$(printf "%s" "$DB_DATABASE" | cut -c1-28)"
|
||||
restore_database="${database_prefix}_restore_${timestamp}_$$"
|
||||
previous_database="${database_prefix}_before_${timestamp}_$$"
|
||||
|
||||
createdb \
|
||||
--host="$DB_HOST" \
|
||||
--port="${DB_PORT:-5432}" \
|
||||
--username="$DB_USERNAME" \
|
||||
--owner="$DB_USERNAME" \
|
||||
"$restore_database"
|
||||
|
||||
restore_failed=true
|
||||
current_database_renamed=false
|
||||
|
||||
cleanup_restore_database() {
|
||||
if [ "$restore_failed" = "true" ]; then
|
||||
if [ "$current_database_renamed" = "true" ]; then
|
||||
psql \
|
||||
--host="$DB_HOST" \
|
||||
--port="${DB_PORT:-5432}" \
|
||||
--username="$DB_USERNAME" \
|
||||
--dbname=postgres \
|
||||
--command="ALTER DATABASE \"$previous_database\" RENAME TO \"$DB_DATABASE\";" \
|
||||
>/dev/null 2>&1 || true
|
||||
fi
|
||||
|
||||
dropdb \
|
||||
--host="$DB_HOST" \
|
||||
--port="${DB_PORT:-5432}" \
|
||||
--username="$DB_USERNAME" \
|
||||
--force \
|
||||
--if-exists \
|
||||
"$restore_database" 2>/dev/null || true
|
||||
fi
|
||||
|
||||
cleanup
|
||||
}
|
||||
|
||||
trap cleanup_restore_database EXIT INT TERM
|
||||
|
||||
echo "Restoring and validating temporary database $restore_database."
|
||||
pg_restore \
|
||||
--host="$DB_HOST" \
|
||||
--port="${DB_PORT:-5432}" \
|
||||
--username="$DB_USERNAME" \
|
||||
--dbname="$restore_database" \
|
||||
--exit-on-error \
|
||||
--no-owner \
|
||||
--no-acl \
|
||||
"$restore_path"
|
||||
vacuumdb \
|
||||
--host="$DB_HOST" \
|
||||
--port="${DB_PORT:-5432}" \
|
||||
--username="$DB_USERNAME" \
|
||||
--dbname="$restore_database" \
|
||||
--analyze-in-stages
|
||||
|
||||
database_exists="$(psql \
|
||||
--host="$DB_HOST" \
|
||||
--port="${DB_PORT:-5432}" \
|
||||
--username="$DB_USERNAME" \
|
||||
--dbname=postgres \
|
||||
--tuples-only \
|
||||
--no-align \
|
||||
--command="SELECT 1 FROM pg_database WHERE datname = '$DB_DATABASE';")"
|
||||
|
||||
if [ "$database_exists" = "1" ]; then
|
||||
psql \
|
||||
--host="$DB_HOST" \
|
||||
--port="${DB_PORT:-5432}" \
|
||||
--username="$DB_USERNAME" \
|
||||
--dbname=postgres \
|
||||
--set=ON_ERROR_STOP=1 \
|
||||
--command="SELECT pg_terminate_backend(pid) FROM pg_stat_activity WHERE datname = '$DB_DATABASE' AND pid <> pg_backend_pid();" \
|
||||
--command="ALTER DATABASE \"$DB_DATABASE\" RENAME TO \"$previous_database\";"
|
||||
current_database_renamed=true
|
||||
fi
|
||||
|
||||
psql \
|
||||
--host="$DB_HOST" \
|
||||
--port="${DB_PORT:-5432}" \
|
||||
--username="$DB_USERNAME" \
|
||||
--dbname=postgres \
|
||||
--set=ON_ERROR_STOP=1 \
|
||||
--command="ALTER DATABASE \"$restore_database\" RENAME TO \"$DB_DATABASE\";"
|
||||
|
||||
restore_failed=false
|
||||
trap cleanup EXIT INT TERM
|
||||
|
||||
echo "Database restoration completed successfully."
|
||||
if [ "$database_exists" = "1" ]; then
|
||||
echo "Previous database kept for rollback as: $previous_database"
|
||||
fi
|
||||
Executable
+37
@@ -0,0 +1,37 @@
|
||||
#!/bin/sh
|
||||
set -eu
|
||||
|
||||
compose_file="${COMPOSE_FILE:-docker-compose.yml}"
|
||||
source_name="${1:-latest-s3}"
|
||||
confirmation_phrase="replace-bowli-production-database"
|
||||
|
||||
if [ ! -f "$compose_file" ]; then
|
||||
echo "Run this command from the daily-meal-api directory." >&2
|
||||
exit 1
|
||||
fi
|
||||
|
||||
echo "This operation will replace the production database."
|
||||
echo "Backup source: $source_name"
|
||||
printf "Type %s to continue: " "$confirmation_phrase"
|
||||
read -r confirmation
|
||||
|
||||
if [ "$confirmation" != "$confirmation_phrase" ]; then
|
||||
echo "Restoration cancelled."
|
||||
exit 1
|
||||
fi
|
||||
|
||||
docker compose -f "$compose_file" stop app horizon scheduler db-backup
|
||||
|
||||
if ! RESTORE_CONFIRM="$confirmation_phrase" docker compose \
|
||||
-f "$compose_file" \
|
||||
--profile restore \
|
||||
run --rm db-restore "$source_name"
|
||||
then
|
||||
echo "Restoration failed. The application remains stopped to protect the database." >&2
|
||||
echo "Inspect the logs, then restart it manually only when the database is safe." >&2
|
||||
exit 1
|
||||
fi
|
||||
|
||||
docker compose -f "$compose_file" up -d app horizon scheduler db-backup
|
||||
|
||||
echo "Application restarted after database restoration."
|
||||
Reference in New Issue
Block a user