Skip to content

fix(#58): Timeline data now accurately reflects usage patterns#74

Closed
GalDayan wants to merge 4 commits intomainfrom
fix/issue-58-timeline-accuracy
Closed

fix(#58): Timeline data now accurately reflects usage patterns#74
GalDayan wants to merge 4 commits intomainfrom
fix/issue-58-timeline-accuracy

Conversation

@GalDayan
Copy link
Copy Markdown
Contributor

Problem

The "Total Usage Over Time" chart showed misleading spikes because all of a session's cost/tokens were bucketed at the session's startedAt time. A session that starts at 05:00 but runs until 12:00 would show $X at 05:00 and nothing for 06:00-12:00.

Root Cause

getBucketKey(s.startedAt) was the sole bucket assignment — 100% of cost went to the start hour.

Fix

For hourly grouping, sessions now distribute their cost and tokens proportionally across all active hours (from startedAt to lastActivityAt).

  • A 4-hour session with $1.00 total → $0.25 in each of the 4 hourly buckets
  • Session count is only incremented in the start bucket (no double-counting)
  • Safety cap at 168 hours (1 week) to prevent runaway loops
  • Daily/weekly views unchanged (sessions rarely span multiple days)

Testing

  • Build clean (tsc --noEmit)
  • All 23 existing tests passing
  • Manual verification: sessions now show activity distributed across their actual working hours

Closes #58

…API endpoints

- modelClassifier.ts: session complexity classifier (simple/moderate/complex)
  based on message length, tool usage patterns, and feature analysis
- modelRecommendations.ts: maps complexity to cheaper model alternatives
  with cost savings estimates
- routes.ts: GET /api/recommendations/:sessionId and /api/recommendations/summary

Implements backend for AI-powered model recommendations (issue #51)
…ge cases

- 23 tests covering classifier and recommendation mapper
- Edge cases: zero-cost sessions, empty models, no messages, unknown models
- Savings rounding to 4 decimal places
- Bulk recommendations filter completed sessions, respect limits
- Added vitest as test framework
- All tests passing, build clean
Prevents Express from matching 'summary' as a sessionId param.
Sessions spanning multiple hours now spread cost/tokens proportionally
across all active hours instead of spiking at startedAt.

Root cause: getBucketKey(s.startedAt) put 100% of a session's cost
into the start hour, creating misleading spikes in the chart.

Fix: for hourly grouping, generate bucket keys for every hour between
startedAt and lastActivityAt, distribute cost/tokens evenly.
Daily/weekly views unchanged (sessions rarely span multiple days).

Safety cap at 168 hours (1 week) to prevent runaway loops.
Session count only incremented in the first (start) bucket.
Copilot AI review requested due to automatic review settings March 25, 2026 03:31
Copy link
Copy Markdown

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adjusts backend analytics bucketing so the “Total Usage Over Time” chart reflects session activity across hours (fixing misleading spikes), and introduces model recommendation APIs with supporting recommendation/classification logic and tests.

Changes:

  • Distributes session cost/tokens across multiple hourly buckets between startedAt and lastActivityAt in /api/analytics.
  • Adds /api/recommendations/summary and /api/recommendations/:sessionId endpoints plus recommendation + classification modules.
  • Adds Vitest test setup and new unit tests for model classification/recommendation logic.

Reviewed changes

Copilot reviewed 6 out of 7 changed files in this pull request and generated 9 comments.

Show a summary per file
File Description
backend/src/routes.ts Updates hourly analytics aggregation logic and adds recommendation API routes.
backend/src/modelRecommendations.ts Implements recommendation generation + savings estimation helpers.
backend/src/modelClassifier.ts Adds heuristic session complexity classification (detail + summary).
backend/src/tests/modelRecommendations.test.ts Unit tests for recommendation logic and edge cases.
backend/src/tests/modelClassifier.test.ts Unit tests for classifier heuristics.
backend/package.json Adds Vitest scripts/dependency for backend tests.
backend/package-lock.json Locks new Vitest dependency tree.
Files not reviewed (1)
  • backend/package-lock.json: Language not supported

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread backend/src/routes.ts
Comment on lines +896 to +899
// Distribute cost/tokens evenly across active hours
const costPerBucket = s.costUsd / numBuckets;
const tokensPerBucket = Math.round(s.tokenCount / numBuckets);

Copy link

Copilot AI Mar 25, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

tokensPerBucket uses Math.round(s.tokenCount / numBuckets), which can change the total tokenCount when summed across buckets (e.g., 1 token over 2 buckets becomes 2). Use integer distribution that preserves the total (e.g., Math.floor plus distributing the remainder), or keep fractional tokens internally and round once at the end.

Copilot uses AI. Check for mistakes.
Comment thread backend/src/routes.ts
Comment on lines +896 to +898
// Distribute cost/tokens evenly across active hours
const costPerBucket = s.costUsd / numBuckets;
const tokensPerBucket = Math.round(s.tokenCount / numBuckets);
Copy link

Copilot AI Mar 25, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

PR description says hourly cost/tokens are distributed proportionally across active hours, but the implementation distributes them evenly per bucket (costUsd / numBuckets, tokenCount / numBuckets) and doesn’t weight partial first/last hours. Either update the PR description or change the allocation to weight buckets by actual overlap duration per hour.

Copilot uses AI. Check for mistakes.
Comment on lines +1 to +2
import { SessionDetail, SessionSummary } from "./sessions.js";
import { classifySession, classifySessionSummary, SessionComplexity } from "./modelClassifier.js";
Copy link

Copilot AI Mar 25, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These imports use a .js extension and are not type-only. In this repo, TS files import local modules without extensions (e.g., backend/src/index.ts:7-11), and tsconfig.json uses module: "commonjs". Using ./sessions.js / ./modelClassifier.js can break TS/tsx resolution and also forces a runtime import for what are mostly types—prefer extensionless paths and import type for SessionDetail/SessionSummary/SessionComplexity.

Copilot uses AI. Check for mistakes.
@@ -0,0 +1,148 @@
import { SessionDetail, SessionSummary } from "./sessions.js";
Copy link

Copilot AI Mar 25, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This file imports from ./sessions.js. The rest of the backend TS uses extensionless local imports (e.g., backend/src/index.ts:7-11) and tsconfig.json is module: "commonjs". Prefer extensionless imports and import type here to avoid resolution issues and unintended runtime imports.

Suggested change
import { SessionDetail, SessionSummary } from "./sessions.js";
import type { SessionDetail, SessionSummary } from "./sessions";

Copilot uses AI. Check for mistakes.
Comment on lines +149 to +153
// Calculate potential savings
const currentCost = session.costUsd;
const estimatedNewCost = estimateSessionCost(session.tokenCount, recommendation.model);
const savings = Math.max(0, currentCost - estimatedNewCost);
const savingsPercentage = currentCost > 0 ? (savings / currentCost) * 100 : 0;
Copy link

Copilot AI Mar 25, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If recommendation.model equals session.model (no change), this still computes savings by comparing actual cost to an estimated cost for the same model, which can yield misleading positive savings. Consider short-circuiting savings/savingsPercentage to 0 when no model change is recommended.

Copilot uses AI. Check for mistakes.
Comment thread backend/src/routes.ts
Comment on lines +1139 to +1140
const limit = parseInt(req.query.limit as string) || 20;

Copy link

Copilot AI Mar 25, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

limit is taken directly from the querystring with parseInt(...) || 20, which treats 0 as “use default” and allows negatives/very large values. Clamp/validate the value (e.g., min 1, max reasonable) and return a 400 for invalid input.

Suggested change
const limit = parseInt(req.query.limit as string) || 20;
const rawLimit = req.query.limit as string | undefined;
let limit = 20;
if (rawLimit !== undefined) {
const parsedLimit = Number.parseInt(rawLimit, 10);
const MIN_LIMIT = 1;
const MAX_LIMIT = 1000;
if (Number.isNaN(parsedLimit) || parsedLimit < MIN_LIMIT || parsedLimit > MAX_LIMIT) {
res.status(400).json({ error: "Invalid 'limit' parameter" });
return;
}
limit = parsedLimit;
}

Copilot uses AI. Check for mistakes.
Comment thread backend/src/routes.ts
keys.push(cursor.toISOString().slice(0, 13) + ":00");
cursor.setUTCHours(cursor.getUTCHours() + 1);
// Safety: cap at 168 hours (1 week) to avoid runaway loops
if (keys.length > 168) break;
Copy link

Copilot AI Mar 25, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The “168 hours” safety cap is off by one: the loop breaks only when keys.length > 168, so it can return 169 buckets. If the intent is a strict 168-hour cap, break when keys.length === 168 (or check before pushing).

Suggested change
if (keys.length > 168) break;
if (keys.length >= 168) break;

Copilot uses AI. Check for mistakes.
Comment thread backend/src/routes.ts
Comment on lines +882 to +884
const endKey = getBucketKey(s.lastActivityAt);
while (cursor.toISOString().slice(0, 13) + ":00" <= endKey) {
keys.push(cursor.toISOString().slice(0, 13) + ":00");
Copy link

Copilot AI Mar 25, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The while (... <= endKey) loop will include the bucket for lastActivityAt even when the session ends exactly on an hour boundary (e.g., 10:30→11:00 yields both 10:00 and 11:00), over-distributing cost/tokens into an extra hour. Prefer looping with Date comparisons (e.g., push while cursor < end) or make the end boundary exclusive when aligned to :00.

Suggested change
const endKey = getBucketKey(s.lastActivityAt);
while (cursor.toISOString().slice(0, 13) + ":00" <= endKey) {
keys.push(cursor.toISOString().slice(0, 13) + ":00");
while (cursor < end) {
const key = cursor.toISOString().slice(0, 13) + ":00";
keys.push(key);

Copilot uses AI. Check for mistakes.
Comment on lines +186 to +189
const estimatedNewCost = estimateSessionCost(session.tokenCount, recommendation.model);
const savings = Math.max(0, currentCost - estimatedNewCost);
const savingsPercentage = currentCost > 0 ? (savings / currentCost) * 100 : 0;

Copy link

Copilot AI Mar 25, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In bulk recommendations, if recommendation.model equals session.model, the computed savings is just “actual vs estimated” for the same model and can be misleading. Consider forcing savings to 0 when there’s no model change and/or filtering out entries where recommendedModel === currentModel.

Suggested change
const estimatedNewCost = estimateSessionCost(session.tokenCount, recommendation.model);
const savings = Math.max(0, currentCost - estimatedNewCost);
const savingsPercentage = currentCost > 0 ? (savings / currentCost) * 100 : 0;
const isSameModel = recommendation.model === session.model;
let savings = 0;
let savingsPercentage = 0;
if (!isSameModel) {
const estimatedNewCost = estimateSessionCost(session.tokenCount, recommendation.model);
savings = Math.max(0, currentCost - estimatedNewCost);
savingsPercentage = currentCost > 0 ? (savings / currentCost) * 100 : 0;
}

Copilot uses AI. Check for mistakes.
@GalDayan
Copy link
Copy Markdown
Contributor Author

Closing — this PR is mis-scoped: labeled as fix for #58 but diff includes modelClassifier/modelRecommendations files that belong to #51 (covered by #73). Keeping #70 as the clean fix for #58 (focused on backend/src/routes.ts + sessions.ts).

@GalDayan GalDayan closed this Apr 15, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Timeline data does not accurately reflect actual usage patterns

2 participants