Skip to content

feat(core/db): drop redundant idx_core_transactions_tx_hash#205

Open
RolfAris wants to merge 1 commit intoOpenAudio:mainfrom
RolfAris:feat/drop-redundant-tx-hash-index
Open

feat(core/db): drop redundant idx_core_transactions_tx_hash#205
RolfAris wants to merge 1 commit intoOpenAudio:mainfrom
RolfAris:feat/drop-redundant-tx-hash-index

Conversation

@RolfAris
Copy link
Copy Markdown
Contributor

The shipped tx-hash lookup (GetTx) uses lower(tx_hash) = lower($1) and rides idx_core_transactions_tx_hash_lower. The plain tx_hash index is not referenced by any query in pkg/core or pkg/etl; it is pure write-amp and disk overhead.

Drops online with CONCURRENTLY to avoid blocking block ingestion (a non-concurrent DROP INDEX would take ACCESS EXCLUSIVE on core_transactions). Rollback recreates the index online.

Operators running ad-hoc WHERE tx_hash = '…' in psql should switch to WHERE lower(tx_hash) = lower('…') — the functional index has been the canonical shipped pattern for a while.

Fleet evidence and EXPLAIN plans in comment below.

The shipped tx-hash lookup (GetTx) uses lower(tx_hash) = lower($1) and
rides idx_core_transactions_tx_hash_lower. The plain tx_hash index is
not referenced by any query in pkg/core or pkg/etl; it is pure write-amp
and disk overhead.

Drops online with CONCURRENTLY to avoid blocking block ingestion on a
live validator (a non-concurrent DROP would take ACCESS EXCLUSIVE on
core_transactions). Rollback recreates the index online.

Operators running ad-hoc WHERE tx_hash = '...' in psql should switch
to WHERE lower(tx_hash) = lower('...'); the functional index has been
the canonical shipped pattern for a while.
@RolfAris
Copy link
Copy Markdown
Contributor Author

Evidence

Code path audit (pkg/core/db/sql/reads.sql, main @ 2026-04-16)

Every core_transactions access:

query predicate index used
GetTx lower(tx_hash) = lower($1) idx_core_transactions_tx_hash_lower
GetBlockTransactions block_id = $1 idx_core_transactions_block_id
GetRecentTxs order by created_at desc idx_core_transactions_created_at
TotalTxResults count(tx_hash) any

No query in pkg/core or pkg/etl does exact-case tx_hash = $1 against core_transactions. Other tx_hash = $1 lookups in the tree target different tables (core_etl_tx, core_rewards, core_ern, core_mead, core_pie) and are untouched.

Live evidence — 20 independent validator nodes, Postgres 15.16

Stats cumulative since pg_postmaster_start_time ≈ 2026-04-13 (~3.2 days).

Per node (the universal unit — operators run 1, 3, 30 nodes):

metric (per node) value
idx_core_transactions_tx_hash size ~5.20 GiB
idx_core_transactions_tx_hash scans 0 (on all 20/20)
idx_core_transactions_tx_hash_lower scans > 0 14/20 nodes
core_transactions rows ~58.5–58.8M
core_transactions inserts / day ~50–120K
redundant btree inserts / day eliminated ~50–120K

Per node: ~5.20 GiB reclaimed; one fewer btree touched per core_transactions insert — smaller insert-path WAL, fewer dirty buffers, faster block apply. The write-amp reduction scales with a node's share of chain throughput, not with operator size.

EXPLAIN (ANALYZE, BUFFERS) — val001, val002, val020

```
Limit (cost=0.69..8.71 rows=1 width=557) (actual time=0.6–1.1 ms)
-> Index Scan using idx_core_transactions_tx_hash_lower on core_transactions
Index Cond: (lower(tx_hash) = lower($1))
```

Planner picks the functional index on every node tested. Sub-ms exec.

Our rollout plan

Canary one validator first, 24h soak, then staggered fleet rollout. Will report back if anything looks off.

@RolfAris
Copy link
Copy Markdown
Contributor Author

Canary report — val001, T+24h

DROP INDEX CONCURRENTLY IF EXISTS idx_core_transactions_tx_hash applied to val001 at 2026-04-16T10:42Z. 24h observation complete, posted automatically from the canary host.

Index state

  • idx_core_transactions_tx_hash: GONE
  • idx_core_transactions_tx_hash_lower: present (5313 MB)
  • Planner still selects the functional index for GetTx (EXPLAIN match: 1)
  • _lower scans: 583 → 742 (+159 in the 24h window)

Chain state

  • live=true, height=23042673, total_tx=59478326
  • peers healthy: 71/71

Logs (last 24h)

  • Postgres / migration / core_transactions errors: 0

No regressions observed. Ready to stagger-roll the remaining 19 nodes once this PR merges.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant