fix(#58): Timeline data now accurately reflects usage patterns#74
fix(#58): Timeline data now accurately reflects usage patterns#74
Conversation
…API endpoints - modelClassifier.ts: session complexity classifier (simple/moderate/complex) based on message length, tool usage patterns, and feature analysis - modelRecommendations.ts: maps complexity to cheaper model alternatives with cost savings estimates - routes.ts: GET /api/recommendations/:sessionId and /api/recommendations/summary Implements backend for AI-powered model recommendations (issue #51)
…ge cases - 23 tests covering classifier and recommendation mapper - Edge cases: zero-cost sessions, empty models, no messages, unknown models - Savings rounding to 4 decimal places - Bulk recommendations filter completed sessions, respect limits - Added vitest as test framework - All tests passing, build clean
Prevents Express from matching 'summary' as a sessionId param.
Sessions spanning multiple hours now spread cost/tokens proportionally across all active hours instead of spiking at startedAt. Root cause: getBucketKey(s.startedAt) put 100% of a session's cost into the start hour, creating misleading spikes in the chart. Fix: for hourly grouping, generate bucket keys for every hour between startedAt and lastActivityAt, distribute cost/tokens evenly. Daily/weekly views unchanged (sessions rarely span multiple days). Safety cap at 168 hours (1 week) to prevent runaway loops. Session count only incremented in the first (start) bucket.
There was a problem hiding this comment.
Pull request overview
Adjusts backend analytics bucketing so the “Total Usage Over Time” chart reflects session activity across hours (fixing misleading spikes), and introduces model recommendation APIs with supporting recommendation/classification logic and tests.
Changes:
- Distributes session cost/tokens across multiple hourly buckets between
startedAtandlastActivityAtin/api/analytics. - Adds
/api/recommendations/summaryand/api/recommendations/:sessionIdendpoints plus recommendation + classification modules. - Adds Vitest test setup and new unit tests for model classification/recommendation logic.
Reviewed changes
Copilot reviewed 6 out of 7 changed files in this pull request and generated 9 comments.
Show a summary per file
| File | Description |
|---|---|
| backend/src/routes.ts | Updates hourly analytics aggregation logic and adds recommendation API routes. |
| backend/src/modelRecommendations.ts | Implements recommendation generation + savings estimation helpers. |
| backend/src/modelClassifier.ts | Adds heuristic session complexity classification (detail + summary). |
| backend/src/tests/modelRecommendations.test.ts | Unit tests for recommendation logic and edge cases. |
| backend/src/tests/modelClassifier.test.ts | Unit tests for classifier heuristics. |
| backend/package.json | Adds Vitest scripts/dependency for backend tests. |
| backend/package-lock.json | Locks new Vitest dependency tree. |
Files not reviewed (1)
- backend/package-lock.json: Language not supported
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| // Distribute cost/tokens evenly across active hours | ||
| const costPerBucket = s.costUsd / numBuckets; | ||
| const tokensPerBucket = Math.round(s.tokenCount / numBuckets); | ||
|
|
There was a problem hiding this comment.
tokensPerBucket uses Math.round(s.tokenCount / numBuckets), which can change the total tokenCount when summed across buckets (e.g., 1 token over 2 buckets becomes 2). Use integer distribution that preserves the total (e.g., Math.floor plus distributing the remainder), or keep fractional tokens internally and round once at the end.
| // Distribute cost/tokens evenly across active hours | ||
| const costPerBucket = s.costUsd / numBuckets; | ||
| const tokensPerBucket = Math.round(s.tokenCount / numBuckets); |
There was a problem hiding this comment.
PR description says hourly cost/tokens are distributed proportionally across active hours, but the implementation distributes them evenly per bucket (costUsd / numBuckets, tokenCount / numBuckets) and doesn’t weight partial first/last hours. Either update the PR description or change the allocation to weight buckets by actual overlap duration per hour.
| import { SessionDetail, SessionSummary } from "./sessions.js"; | ||
| import { classifySession, classifySessionSummary, SessionComplexity } from "./modelClassifier.js"; |
There was a problem hiding this comment.
These imports use a .js extension and are not type-only. In this repo, TS files import local modules without extensions (e.g., backend/src/index.ts:7-11), and tsconfig.json uses module: "commonjs". Using ./sessions.js / ./modelClassifier.js can break TS/tsx resolution and also forces a runtime import for what are mostly types—prefer extensionless paths and import type for SessionDetail/SessionSummary/SessionComplexity.
| @@ -0,0 +1,148 @@ | |||
| import { SessionDetail, SessionSummary } from "./sessions.js"; | |||
There was a problem hiding this comment.
This file imports from ./sessions.js. The rest of the backend TS uses extensionless local imports (e.g., backend/src/index.ts:7-11) and tsconfig.json is module: "commonjs". Prefer extensionless imports and import type here to avoid resolution issues and unintended runtime imports.
| import { SessionDetail, SessionSummary } from "./sessions.js"; | |
| import type { SessionDetail, SessionSummary } from "./sessions"; |
| // Calculate potential savings | ||
| const currentCost = session.costUsd; | ||
| const estimatedNewCost = estimateSessionCost(session.tokenCount, recommendation.model); | ||
| const savings = Math.max(0, currentCost - estimatedNewCost); | ||
| const savingsPercentage = currentCost > 0 ? (savings / currentCost) * 100 : 0; |
There was a problem hiding this comment.
If recommendation.model equals session.model (no change), this still computes savings by comparing actual cost to an estimated cost for the same model, which can yield misleading positive savings. Consider short-circuiting savings/savingsPercentage to 0 when no model change is recommended.
| const limit = parseInt(req.query.limit as string) || 20; | ||
|
|
There was a problem hiding this comment.
limit is taken directly from the querystring with parseInt(...) || 20, which treats 0 as “use default” and allows negatives/very large values. Clamp/validate the value (e.g., min 1, max reasonable) and return a 400 for invalid input.
| const limit = parseInt(req.query.limit as string) || 20; | |
| const rawLimit = req.query.limit as string | undefined; | |
| let limit = 20; | |
| if (rawLimit !== undefined) { | |
| const parsedLimit = Number.parseInt(rawLimit, 10); | |
| const MIN_LIMIT = 1; | |
| const MAX_LIMIT = 1000; | |
| if (Number.isNaN(parsedLimit) || parsedLimit < MIN_LIMIT || parsedLimit > MAX_LIMIT) { | |
| res.status(400).json({ error: "Invalid 'limit' parameter" }); | |
| return; | |
| } | |
| limit = parsedLimit; | |
| } |
| keys.push(cursor.toISOString().slice(0, 13) + ":00"); | ||
| cursor.setUTCHours(cursor.getUTCHours() + 1); | ||
| // Safety: cap at 168 hours (1 week) to avoid runaway loops | ||
| if (keys.length > 168) break; |
There was a problem hiding this comment.
The “168 hours” safety cap is off by one: the loop breaks only when keys.length > 168, so it can return 169 buckets. If the intent is a strict 168-hour cap, break when keys.length === 168 (or check before pushing).
| if (keys.length > 168) break; | |
| if (keys.length >= 168) break; |
| const endKey = getBucketKey(s.lastActivityAt); | ||
| while (cursor.toISOString().slice(0, 13) + ":00" <= endKey) { | ||
| keys.push(cursor.toISOString().slice(0, 13) + ":00"); |
There was a problem hiding this comment.
The while (... <= endKey) loop will include the bucket for lastActivityAt even when the session ends exactly on an hour boundary (e.g., 10:30→11:00 yields both 10:00 and 11:00), over-distributing cost/tokens into an extra hour. Prefer looping with Date comparisons (e.g., push while cursor < end) or make the end boundary exclusive when aligned to :00.
| const endKey = getBucketKey(s.lastActivityAt); | |
| while (cursor.toISOString().slice(0, 13) + ":00" <= endKey) { | |
| keys.push(cursor.toISOString().slice(0, 13) + ":00"); | |
| while (cursor < end) { | |
| const key = cursor.toISOString().slice(0, 13) + ":00"; | |
| keys.push(key); |
| const estimatedNewCost = estimateSessionCost(session.tokenCount, recommendation.model); | ||
| const savings = Math.max(0, currentCost - estimatedNewCost); | ||
| const savingsPercentage = currentCost > 0 ? (savings / currentCost) * 100 : 0; | ||
|
|
There was a problem hiding this comment.
In bulk recommendations, if recommendation.model equals session.model, the computed savings is just “actual vs estimated” for the same model and can be misleading. Consider forcing savings to 0 when there’s no model change and/or filtering out entries where recommendedModel === currentModel.
| const estimatedNewCost = estimateSessionCost(session.tokenCount, recommendation.model); | |
| const savings = Math.max(0, currentCost - estimatedNewCost); | |
| const savingsPercentage = currentCost > 0 ? (savings / currentCost) * 100 : 0; | |
| const isSameModel = recommendation.model === session.model; | |
| let savings = 0; | |
| let savingsPercentage = 0; | |
| if (!isSameModel) { | |
| const estimatedNewCost = estimateSessionCost(session.tokenCount, recommendation.model); | |
| savings = Math.max(0, currentCost - estimatedNewCost); | |
| savingsPercentage = currentCost > 0 ? (savings / currentCost) * 100 : 0; | |
| } |
Problem
The "Total Usage Over Time" chart showed misleading spikes because all of a session's cost/tokens were bucketed at the session's
startedAttime. A session that starts at 05:00 but runs until 12:00 would show $X at 05:00 and nothing for 06:00-12:00.Root Cause
getBucketKey(s.startedAt)was the sole bucket assignment — 100% of cost went to the start hour.Fix
For hourly grouping, sessions now distribute their cost and tokens proportionally across all active hours (from
startedAttolastActivityAt).Testing
tsc --noEmit)Closes #58