Skip to content

Paginate delete blockers to prevent OOM#7939

Closed
foozleface wants to merge 1 commit intospecify:mainfrom
calacademy-research:cas/perf-delete-blockers-7515
Closed

Paginate delete blockers to prevent OOM#7939
foozleface wants to merge 1 commit intospecify:mainfrom
calacademy-research:cas/perf-delete-blockers-7515

Conversation

@foozleface
Copy link
Copy Markdown
Collaborator

Fixes #7515
Contributed by @foozleface

The delete_blockers endpoint used Django's Collector.collect() which loads ALL related objects into memory before returning. For a Taxon like "Life" with 900K+ determinations, this causes out-of-memory kills. This PR replaces the Collector with targeted COUNT queries and limited ID lists per PROTECT relationship, keeping memory bounded regardless of how many blocking rows exist.

Implementation

  • Replace Collector.collect() in _collect_delete_blockers with iteration over obj._meta.related_objects, filtering for protect_with_blockers on_delete
  • For each PROTECT relationship, run a COUNT query and fetch at most id_limit (default 100) primary keys
  • Add total_count field to the response so the frontend knows the full count even when IDs are capped
  • Update frontend test fixtures to match the new isstrict: undefined serialization
  • Add comprehensive backend tests for the pagination behavior

Testing instructions

  • Navigate to a record with many dependent records (e.g., a high-level Taxon node) and attempt to delete it
  • Verify the delete blocker dialog shows the correct count and a reasonable number of linked record IDs
  • Confirm no OOM or timeout on records with very large numbers of blockers
  • Run the test suite: python manage.py test specifyweb.backend.delete_blockers.tests.test_pagination

@acwhite211
Copy link
Copy Markdown
Member

Move to this PR with additional fixes #8018

@acwhite211 acwhite211 closed this Apr 23, 2026
@github-project-automation github-project-automation Bot moved this from 📋Back Log to ✅Done in General Tester Board Apr 23, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

Status: ✅Done

Development

Successfully merging this pull request may close these issues.

[Large Databases]: Paginate the delete blockers

2 participants