Skip to content

feat: opt-in usage telemetry + community intelligence platform (v0.8.6)#210

Open
garrytan wants to merge 16 commits intomainfrom
garrytan/usage-telemetry
Open

feat: opt-in usage telemetry + community intelligence platform (v0.8.6)#210
garrytan wants to merge 16 commits intomainfrom
garrytan/usage-telemetry

Conversation

@garrytan
Copy link
Owner

@garrytan garrytan commented Mar 19, 2026

Summary

  • Local telemetry: gstack-telemetry-log appends JSONL events per skill invocation with privacy tiers (off/anonymous/community). gstack-analytics shows a personal usage dashboard.
  • Opt-in prompt: First-time users are asked once whether to share anonymous data. No code, file paths, or repo names are ever sent.
  • Supabase backend: 3 tables (telemetry_events, installations, update_checks), 2 SQL views (crash_clusters, skill_sequences), RLS insert+select policies. Publishable key committed (safe — like Firebase public config).
  • Remote sync: gstack-telemetry-sync batches local events to Supabase with field stripping per privacy tier.
  • Community dashboard: gstack-community-dashboard CLI shows skill popularity, crash clusters, and version distribution.
  • Install ping: Dual-attempt update check fires a parallel Supabase ping (non-blocking) alongside the GitHub version fetch.
  • Upgrade funnel: Logs upgrade_prompted events on fresh fetches (not cached replays).
  • /retro integration: Weekly retrospectives now include gstack skill usage metrics.
  • Session-specific .pending markers: Prevents race conditions between concurrent gstack sessions.
  • Codex review fix: crash_clusters view correctly counts anonymous events.

Test Coverage

396 tests pass (0 failures). 21 new telemetry integration tests covering:

  • JSONL append, tier=off, invalid tier, installation_id generation
  • .pending finalization (cross-session, same-session, tier=off)
  • Analytics dashboard rendering, time window filtering
  • Sync (no endpoint, no JSONL), community dashboard (no config, with config)

Pre-Landing Review

CEO Review: CLEAR (SCOPE EXPANSION, all 7 expansions + 3 TODOs accepted)
Eng Review: CLEAR (FULL_REVIEW, 0 critical gaps)
Codex Review: 14 findings triaged — session_id uniqueness fixed, .pending race fixed, crash_clusters view fixed, jq dependency eliminated.

Eval Results

No prompt-related files changed — evals skipped.

TODOS

  • "Skill usage telemetry" moved to Completed (was P3)

Test plan

  • All tests pass (396 tests, 0 failures)
  • Manual: telemetry-log appends valid JSONL
  • Manual: Supabase insert returns HTTP 201
  • Manual: community-dashboard queries live data
  • Manual: analytics CLI renders bar chart

🤖 Generated with Claude Code

garrytan and others added 16 commits March 18, 2026 23:58
@garrytan @claude
feat: add gstack-telemetry-log and gstack-analytics scripts
03d866b 
Local telemetry infrastructure for gstack usage tracking.
gstack-telemetry-log appends JSONL events with skill name, duration,
outcome, session ID, and platform info. Supports off/anonymous/community
privacy tiers. gstack-analytics renders a personal usage dashboard
from local data.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
@garrytan @claude
feat: add telemetry preamble injection + opt-in prompt + epilogue
4833b1d 
Extends generatePreamble() with telemetry start block (config read,
timer, session ID, .pending marker), opt-in prompt (gated by
.telemetry-prompted), and epilogue instructions for Claude to log
events after skill completion. Adds 5 telemetry tests.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
@garrytan @claude
chore: regenerate all SKILL.md files with telemetry blocks
f7fd45a 
Automated regeneration from gen-skill-docs.ts changes. All skills
now include telemetry start block, opt-in prompt, and epilogue.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
@garrytan @claude
feat: add Supabase schema, edge functions, and SQL views
b43229a 
Telemetry backend infrastructure: telemetry_events table with RLS
(insert-only), installations table for retention tracking,
update_checks for install pings. Edge functions for update-check
(version + ping), telemetry-ingest (batch insert), and
community-pulse (weekly active count). SQL views for crash
clustering and skill co-occurrence sequences.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
@garrytan @claude
feat: add telemetry-sync, community-dashboard, and integration tests
5ea95c5 
gstack-telemetry-sync: fire-and-forget JSONL → Supabase sync with
privacy tier field stripping, batch limits, and cursor tracking.
gstack-community-dashboard: CLI tool querying Supabase for skill
popularity, crash clusters, and version distribution.
19 integration tests covering all telemetry scripts.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
@garrytan @claude
fix: session-specific .pending markers + crash_clusters view fix
bd24581 
Addresses Codex review findings:
- .pending race condition: use .pending-$SESSION_ID instead of
  shared .pending file to prevent concurrent session interference
- crash_clusters view: add total_occurrences and anonymous_occurrences
  columns since anonymous tier has no installation_id
- Added test: own session pending marker is not finalized

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
@garrytan @claude
feat: dual-attempt update check with Supabase install ping
75e7aad 
Fires a parallel background curl to Supabase during the slow-path
version fetch. Logs upgrade_prompted event only on fresh fetches
(not cached replays) to avoid overcounting. GitHub remains the
primary version source — Supabase ping is fire-and-forget.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
@garrytan @claude
feat: integrate telemetry usage stats into /retro output
6ae842a 
Retro now reads ~/.gstack/analytics/skill-usage.jsonl and includes
gstack usage metrics (skill run counts, top skills, success rate)
in the weekly retrospective output.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
@garrytan @claude
chore: move 'Skill usage telemetry' to Completed in TODOS.md
9b980d9 
Implemented in this branch: local JSONL logging, opt-in prompt,
privacy tiers, Supabase backend, community dashboard, /retro
integration.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
@garrytan @claude
feat: wire Supabase credentials and expose tables via Data API
ebac41b 
Add supabase/config.sh with project URL and publishable key (safe to
commit — RLS restricts to INSERT only). Update telemetry-sync,
community-dashboard, and update-check to source the config and
include proper auth headers for the Supabase REST API.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
@garrytan @claude
fix: add SELECT RLS policies to migration for community dashboard reads
7d6fc52 
All telemetry data is anonymous (no PII), so public reads via the
publishable key are safe. Needed for the community dashboard to
query skill popularity, crash clusters, and version distribution.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
@garrytan
Merge remote-tracking branch 'origin/main' into garrytan/usage-telemetry
790bd18 
# Conflicts:
#	SKILL.md
#	TODOS.md
#	browse/SKILL.md
#	design-consultation/SKILL.md
#	design-review/SKILL.md
#	document-release/SKILL.md
#	plan-ceo-review/SKILL.md
#	plan-design-review/SKILL.md
#	plan-eng-review/SKILL.md
#	qa-only/SKILL.md
#	qa/SKILL.md
#	retro/SKILL.md
#	retro/SKILL.md.tmpl
#	review/SKILL.md
#	scripts/gen-skill-docs.ts
#	setup-browser-cookies/SKILL.md
#	ship/SKILL.md
@garrytan @claude
chore: bump version and changelog (v0.8.6)
7eb27af 
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
@garrytan @claude
fix: analytics backward-compatible with old JSONL format
9c85821 
Handle old-format events (no event_type field) alongside new format.
Skip hook_fire events. Fix grep -c whitespace issues and unbound
variable errors.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
@garrytan @claude
fix: map JSONL field names to Postgres columns in telemetry-sync
c0e3c17 
Local JSONL uses short names (v, ts, sessions) but the Supabase
table expects full names (schema_version, event_timestamp,
concurrent_sessions). Add sed mapping during field stripping.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
@garrytan @claude
fix: address Codex adversarial findings — cursor, opt-out, queries
5467164 
- Sync cursor now advances on HTTP 2xx (not grep for "inserted")
- Update-check respects telemetry opt-out before pinging Supabase
- Dashboard queries use correct view column names (total_occurrences)
- Sync strips old-format "repo" field to prevent privacy leak

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

1 participant

@garrytan