Running schema migrations as Jobs

Tested with: Kubernetes 1.31 · Helm 3.16 · Argo CD 2.12 · Prisma 6 · Flyway 10 · Alembic 1.13

Schema migrations should run once, before the new application version receives traffic, against the direct endpoint (not the pooled one: migrations use locks and session state). A Job with the same Secret and CA mount as the application does that.

The Job

apiVersion: batch/v1
kind: Job
metadata:
  name: migrate-1.4.2
  namespace: app
  annotations:
    helm.sh/hook: pre-upgrade,pre-install
    helm.sh/hook-weight: "0"
    helm.sh/hook-delete-policy: before-hook-creation,hook-succeeded
spec:
  backoffLimit: 2
  activeDeadlineSeconds: 900
  template:
    spec:
      restartPolicy: Never
      containers:
        - name: migrate
          image: ghcr.io/acme/api:1.4.2
          command: ["npx", "prisma", "migrate", "deploy"]
          env:
            - name: DATABASE_URL
              valueFrom: { secretKeyRef: { name: zb-pg-prod, key: DATABASE_URL } }
          volumeMounts:
            - { name: databasezy-ca, mountPath: /etc/databasezy, readOnly: true }
      volumes:
        - name: databasezy-ca
          configMap: { name: databasezy-ca }

DATABASE_URL from the ExternalSecret template already carries sslmode=verify-full&sslrootcert=/etc/databasezy/ca.crt.

apiVersion: batch/v1
kind: Job
metadata:
  name: migrate-1.4.2
  namespace: app
  annotations:
    helm.sh/hook: pre-upgrade,pre-install
    helm.sh/hook-delete-policy: before-hook-creation,hook-succeeded
spec:
  backoffLimit: 2
  template:
    spec:
      restartPolicy: Never
      containers:
        - name: flyway
          image: flyway/flyway:10
          args: ["migrate"]
          env:
            - name: FLYWAY_URL
              value: jdbc:postgresql://$(PGHOST):5432/app?sslmode=verify-full&sslrootcert=/etc/databasezy/ca.crt
            - name: PGHOST
              valueFrom: { secretKeyRef: { name: zb-pg-prod, key: host } }
            - name: FLYWAY_USER
              valueFrom: { secretKeyRef: { name: zb-pg-prod, key: username } }
            - name: FLYWAY_PASSWORD
              valueFrom: { secretKeyRef: { name: zb-pg-prod, key: password } }
          volumeMounts:
            - { name: sql, mountPath: /flyway/sql, readOnly: true }
            - { name: databasezy-ca, mountPath: /etc/databasezy, readOnly: true }
      volumes:
        - name: sql
          configMap: { name: migrations-sql }
        - name: databasezy-ca
          configMap: { name: databasezy-ca }

Same Job; only the command changes:

# Alembic
command: ["alembic", "upgrade", "head"]
# Django
command: ["python", "manage.py", "migrate", "--noinput"]
# Rails
command: ["bin/rails", "db:migrate"]
# golang-migrate
command: ["migrate", "-path", "/migrations", "-database", "$(DATABASE_URL)", "up"]

Ordering with Helm and Argo CD

Helm: the helm.sh/hook: pre-upgrade annotation above runs the Job before the Deployment changes; the upgrade fails if the Job fails.
Argo CD: use argocd.argoproj.io/hook: PreSync and argocd.argoproj.io/hook-delete-policy: BeforeHookCreation instead (Argo ignores Helm hooks when rendering charts through its own Helm support unless enabled). See Argo CD and Helm patterns.
Give the Job a name that changes per release (migrate-{{ .Chart.AppVersion }}) so it always runs, and a hook-delete-policy so successful Jobs do not pile up.

Practical settings

Point at the direct endpoint; the pooler does not support LOCK, advisory locks or session SET.
Set activeDeadlineSeconds so a stuck lock cannot block a deploy forever.
Databasezy takes a backup before resizes and version upgrades, not before your migrations. For risky migrations, create a manual snapshot first: zb backups create pg-7f3k --label pre-1.4.2.
The Job needs the same egress NetworkPolicy allowance as the app; label it accordingly (app: api) or add a selector for job-name.