fix(spp_api_v2): serialize concurrent fastapi_endpoint sync across workers#175
fix(spp_api_v2): serialize concurrent fastapi_endpoint sync across workers#175
Conversation
…rkers After a registry reload (e.g. -u all) every Odoo worker independently invokes routing_map(), and each one races to UPDATE the same fastapi_endpoint rows when calling action_sync_registry(). Under PostgreSQL's REPEATABLE READ isolation this surfaces as N-1 noisy "could not serialize access due to concurrent update" ERRORs in the SQL log before the outer try/except swallows them. Gate the sync block with a pg_try_advisory_xact_lock keyed off a stable SHA-256-derived int8 so only one worker performs the sync per reload. The lock is transaction-scoped and released automatically at COMMIT. Losers log a DEBUG line and skip; the registry-cache invalidation signal from the winning worker propagates to them via the standard Odoo signaling mechanism. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
There was a problem hiding this comment.
Code Review
This pull request introduces a transaction-scoped advisory lock to serialize FastAPI endpoint synchronization across multiple workers. By using pg_try_advisory_xact_lock with a deterministic 64-bit key derived from the module name, the implementation prevents SerializationFailure errors during concurrent registry reloads. I have no feedback to provide.
Codecov Report❌ Patch coverage is
❌ Your patch check has failed because the patch coverage (50.00%) is below the target coverage (70.00%). You can increase the patch coverage or adjust the target coverage. Additional details and impacted files@@ Coverage Diff @@
## 19.0 #175 +/- ##
==========================================
+ Coverage 71.48% 71.64% +0.16%
==========================================
Files 932 933 +1
Lines 54840 55382 +542
==========================================
+ Hits 39201 39679 +478
- Misses 15639 15703 +64
Flags with carried forward coverage won't be shown. Click here to find out more.
🚀 New features to boost your workflow:
|
Why is this change needed?
After
Upgrade All Modules(or any other event that triggers a registry reload), every Odoo worker independently invokes the patchedir.http.routing_map(). Each worker enters the FastAPI endpoint sync block and races toUPDATE fastapi_endpoint SET registry_sync = TRUE WHERE id IN (1, 2). Under PostgreSQL's REPEATABLE READ isolation, all but one of those concurrent updates abort with:odoo.sql_dblogs each failed query at ERROR before the outertry/except Exceptionswallows it at DEBUG. With N workers, that produces N−1 noisy ERRORs in the SQL log on every registry reload — alarming to operators, and it makes real errors harder to spot. The system still works (one worker succeeds, hence the trailingSynced N FastAPI endpointsinfo line), but the noise is unnecessary and N−1 transactions are wasted on every cold start.How was the change implemented?
Gate the entire FastAPI endpoint sync block in
spp_api_v2/models/ir_http_patch.pybehind a PostgreSQL transaction-scoped advisory lock:_FASTAPI_SYNC_ADVISORY_LOCK_KEYis a deterministic 64-bit signed int derived fromSHA-256("spp_api_v2.fastapi_endpoint_sync")— stable across processes and unlikely to collide with any other module's advisory lock in the same database.with registry.cursor() as cr:block, the cursor first callsSELECT pg_try_advisory_xact_lock(<key>). The non-blockingtry_variant means losers return immediately rather than serialize on the lock and then re-attempt the same losingUPDATE.else:branch).DEBUGline (FastAPI endpoint sync skipped — another worker is syncing) and skips the sync.COMMITorROLLBACKwhen the cursor block exits — no manual unlock and no risk of leaking state.This is safe because skipping workers don't end up with a broken routing map: when the lock-holder commits,
_handle_registry_synccallsself.env.registry.clear_cache("routing"), and the cache-invalidation signal propagates to other workers through Odoo's standard registry-signaling mechanism. The nextrouting_map()call on any worker rebuilds with the synced routes.New unit tests
None. The race is inherently a multi-process / multi-cursor scenario, and reproducing it deterministically inside Odoo's standard test runner (which runs single-process with a single cursor) would require either coordinating two real PostgreSQL connections in a test or mocking the cursor's
execute/fetchone. Both add significant test scaffolding for limited value — happy to add one if reviewers feel strongly.The function-level guarantee (lock acquired ⇒ sync runs; lock not acquired ⇒ sync skips) is a 4-line code path that is straightforward by inspection.
Unit tests executed by the author
python3 -m py_compile spp_api_v2/models/ir_http_patch.py→ passes (also viaast.parsein development).spp_api_v2/tests/exerciseir_http_patch.routing_map()(it's a workaround for an Odoo core cache bug, not a normal Odoo model), so the existing test suite is unaffected.How to test manually
Reproduce the baseline (without this PR). Start the devcontainer and run:
Expect multiple
ERROR: could not serialize access due to concurrent updatelines (one per losing worker), followed by a singleSynced N FastAPI endpoints for database <db>.Apply this PR and rerun the same command. Expect zero
serialize accesserrors and a singleSynced N FastAPI endpointsline.Confirm the skip path runs (proves it's the lock doing the work, not a coincidence):
Expect ≥1
FastAPI endpoint sync skipped … another worker is syncingDEBUG line per registry reload.Functional smoke test — endpoints must still actually be registered after the upgrade:
Expect a non-404 response (200/401/403 depending on the route's auth — anything except 404 confirms routing is live).
Single-worker regression — start Odoo with
--workers=0and confirm sync still happens (the only worker always acquires the lock):Related links
pg_try_advisory_xact_lock)spp_api_v2/models/ir_http_patch.py— the patchedrouting_map()workaround for the Odoo 19 ormcache bugUPDATE:endpoint_route_handler/models/endpoint_route_sync_mixin.py—EndpointRouteSyncMixin.writeflipsregistry_syncand queues the post-commit hook