fix: assign distinct semver labels to legacy benchmark versions #40

[doubledare704]

The legacy benchmark version IDs (git hashes and two-component names like "1.0") all shared the same displayed semver/label (0.0.1 or 0.9.0), making them indistinguishable in the API and UI.

This change:

• Adds a forward-only corrective migration that assigns deterministic 1.0.0-beta.N labels to every legacy/non-strict semver ID, ordered by created_at ASC, id ASC so chronological order is preserved and ties are broken.
• Restores strict three-component semver IDs (e.g. 1.2.1, 2.0.0-rc1, 1.2.3-rc.1+build.5) so that semver and label equal the ID.
• Updates submission handling so new non-semver benchmark versions receive the next available 1.0.0-beta.N label instead of the shared 0.0.1 default.
• Adds unit tests covering beta label ordering and strict/legacy classification.

The public API shape is unchanged. Existing filtering by benchmark version ID continues to work.

Deployment order: deploy code first, then apply the D1 migration to production. Closes #40

Local validation on a D1 instance
You can validate the full change against a representative dataset without accessing production by seeding a local D1 database from the public API.

Step 1: Prerequisites
From the repo root, ensure Wrangler is set up:

npx wrangler --version
Step 2: Create and migrate the local database

npx wrangler d1 create prod-pinchbench --local
npx wrangler d1 migrations apply prod-pinchbench --local

Step 3: Seed from the public API response
curl -s https://api.pinchbench.com/api/benchmark_versions > /tmp/benchmark_versions.json
Generate an INSERT script:

jq -r '
  .versions[]
  | "INSERT INTO benchmark_versions (id, created_at, current, hidden, semver, label, release_notes, release_url) VALUES (" +
    (.id | @json) + ", " +
    (.created_at | @json) + ", " +
    (if .is_current then "1" else "0" end) + ", " +
    "0, " +
    (.semver | @json) + ", " +
    (.label | @json) + ", " +
    "NULL, NULL);"
' /tmp/benchmark_versions.json > /tmp/seed_benchmark_versions.sql

Load the seed data:

npx wrangler d1 execute prod-pinchbench --local --file /tmp/seed_benchmark_versions.sql
Step 4: Verify duplicates exist before migration

npx wrangler d1 execute prod-pinchbench --local --command "
SELECT semver, label, COUNT(*) AS count
FROM benchmark_versions
GROUP BY semver, label
HAVING COUNT(*) > 1
ORDER BY count DESC, semver;
"

You should see repeated labels such as 0.0.1 and 0.9.0.

Step 5: Apply the corrective migration
npx wrangler d1 execute prod-pinchbench --local --file migrations/20260425_correct_benchmark_version_labels.sql
Step 6: Verify duplicates are resolved

npx wrangler d1 execute prod-pinchbench --local --command "
SELECT semver, COUNT(*) AS count
FROM benchmark_versions
WHERE id NOT GLOB '[0-9]*.[0-9]*.[0-9]*'
GROUP BY semver
HAVING COUNT(*) > 1;
"

Expected: no rows.

Also check the ordering and labels:

npx wrangler d1 execute prod-pinchbench --local --command "
SELECT id, created_at, semver, label
FROM benchmark_versions
ORDER BY created_at ASC, id ASC
LIMIT 10;
"

Legacy (non-semver) IDs should appear as 1.0.0-beta.1, 1.0.0-beta.2, … while strict IDs like 1.0.0, 1.2.1, 2.0.0-rc1 should show their own ID as semver/label.

Step 7: Verify strict IDs were restored

npx wrangler d1 execute prod-pinchbench --local --command "
SELECT id, semver, label
FROM benchmark_versions
WHERE id IN ('0.1.0', '1.0.0', '1.2.1', '2.0.0-rc1', '2.0.0-rc2');
"

Expected: each row’s semver and label equal the id.

Step 8: Run the test suite
npm test
Or run only the affected tests:

npx vitest run src/routes/benchmarkVersions.test.ts src/routes/results.test.ts
All tests should pass.

Notes

• The runtime getNextLegacyBenchmarkVersionLabel() is best-effort and not fully concurrency-safe; the corrective migration is deterministic and will repair any future duplicates if they arise.
• Deploy order: deploy the code changes first, then apply the migration to the production D1 database.
• Hidden versions are not affected by public endpoint queries but are included in the migration.
• Would it be better to perform this migration on a copy or backup of the main database? If everything appears to be in order, then proceed with the migration on the main database.
• if this change (https://github.com/pinchbench/api/pull/41/changes) will be accepted, then this PR can be closed.

[kilo-code-bot]

Code Review Summary

Status: No Issues Found | Recommendation: Merge

This is a well-considered fix. The migration carefully handles the strict-semver classification using SQLite-compatible logic (no temp tables, no CTEs), the beta label assignment is deterministic and ordered, and the runtime getNextLegacyBenchmarkVersionLabel gracefully handles the acknowledged best-effort concurrency limitation. The INSERT OR IGNORE guarantee downstream prevents duplicate primary key conflicts regardless. Test coverage is thorough.

Files Reviewed (4 files)

• migrations/20260425_correct_benchmark_version_labels.sql
• src/routes/results.ts
• src/routes/results.test.ts
• src/routes/benchmarkVersions.test.ts

---

Fix issues in Kilo Cloud

---

_{Reviewed by claude-4.6-sonnet-20260217 · 118,041 tokens}