Skip to content

Implement deferred schema cleanup when applying multiple migrations#1025

Open
blakewatters wants to merge 3 commits intoxataio:mainfrom
blakewatters:implement-deferred-schema-cleanup
Open

Implement deferred schema cleanup when applying multiple migrations#1025
blakewatters wants to merge 3 commits intoxataio:mainfrom
blakewatters:implement-deferred-schema-cleanup

Conversation

@blakewatters
Copy link
Copy Markdown

Summary

When pgroll migrate applies multiple outstanding migrations, it breaks zero-downtime guarantees by dropping the previous version schema during each intermediate Complete() call. If an application is still connected via that schema, its queries break immediately.

Example: App runs on public_v1. Migrations v2, v3, v4 are pending:

  1. Start v2 → creates public_v2
  2. Complete v2 → drops public_v1 (app breaks here!)
  3. Start v3, Complete v3 → drops public_v2
  4. Start v4 → creates public_v4

This PR implements deferred schema cleanup so that intermediate Complete() calls skip the DROP SCHEMA step, and a batch cleanup runs after all migrations finish — dropping only the intermediate schemas while preserving the original (app-connected) and latest versions.

Changes

  • pkg/roll/execute.go: Added CompleteOption functional option type with WithSkipSchemaDrop() to make Complete() conditionally skip dropping the previous version schema. Added DropVersionSchemasExcept() method to drop all version schemas except a specified keep list.
  • cmd/start.go: Threaded CompleteOption through runMigration() so callers can forward options to Complete().
  • cmd/migrate.go: Updated batch orchestration to record the original version before the batch, pass WithSkipSchemaDrop() for intermediate migrations, and call DropVersionSchemasExcept() after the batch to clean up intermediates while preserving the original and latest schemas.
  • pkg/roll/execute_test.go: Added four integration tests covering skip-drop behavior, full multi-migration batch flow with cleanup, the cleanup method itself, and unchanged single-migration behavior.

Behavior after this change

Scenario Before After
Multi-migration batch Each intermediate Complete() drops the previous schema Intermediate drops are skipped; cleanup runs after the batch
Single migration start/complete Drops previous schema Unchanged — WithSkipSchemaDrop() is never passed
Batch failure midway Intermediate schemas dropped, app broken Intermediate schemas left behind (harmless views), cleaned up on next successful migrate

Test plan

  • Existing test suite passes (go test ./...)
  • TestCompleteWithSkipSchemaDropPreservesPreviousVersionSchema — verifies WithSkipSchemaDrop prevents schema drop
  • TestMultiMigrationBatchPreservesOriginalSchema — end-to-end batch flow: original schema persists through intermediates, cleanup drops only intermediates
  • TestDropVersionSchemasExceptKeepsSpecifiedSchemas — cleanup method keeps specified schemas and drops the rest
  • TestSingleMigrationCompleteStillDropsPreviousSchema — normal single-migration behavior is unchanged
  • Manual test: create a schema with a table, apply 3+ migrations via pgroll migrate ./migrations, verify original schema persists until cleanup

🤖 Generated with Claude Code

@github-actions github-actions bot temporarily deployed to Docs Preview March 11, 2026 09:47 Inactive
@github-actions github-actions bot temporarily deployed to Docs Preview March 12, 2026 23:01 Inactive
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant