Usage & cost stats: metering, pricing, and the stats drawer (0047)

glen commented

2026-10-17 17:49:27 -04:00

Collaborator

Implements docs/0047-usage-and-cost-stats.md (the approved spec, merged in #121): per-ticket usage/cost metering across a conversation group — Gloo costs exact from the public /platform/v2/models price table, tokens/steps/time for everything, and a stats drawer in the chat header.

Capture

Chat turns: per-slice LanguageModelUsage summed in turn.ts, one usage_events row per turn (u_<conv>_<skip>), written after the turn stream closes — never fails the turn, recorded even when a slice throws.
Harness calls: finish-step usage read through a new onUsage hook in the shared sink, checkpointed into the u_hc_<callId> row after EVERY advance step (idempotent upsert) so a cancelled call keeps its completed metering; settle makes the final pass. Pi is priced; claude/cursor record tokens with no dollars (plan badge).
Titles excluded per spec.

Pricing

lib/usage/pricing.ts: pure cost function (fresh×input + cached×cache_read + output×output per-1M, long-context tier selection, platform-fee multiplier, cent rounding, null = unmetered), Gloo payload parser (cache_write 5m/1h/default tiers, deprecated→replacement attribution), normalizeUsage for both AI SDK usage shapes (verified against installed types).
model_prices refreshed daily: Gloo public endpoint (no key) + OpenRouter pricing from the existing catalog payload. Manual rows survive refreshes. Cold-start closed: startChatTurn and startHarnessCall fire-and-forget ensureModelPrices (the models API alone left a null-cost window on fresh deploys).
Gloo's 6.5% guarded-fee multiplier: model_prices.platform_fee defaults 1.0; calibrate once against Gloo Studio's Usage dashboard and set via a manual row.

Read + UI

GET /api/x/stats/<id> rolls up the 0046 locked group (root+descendants): totals, wall/work/wait time, touches (real user messages only), median answer latency, byModel (labels from the catalogs), byChild (conversations + harness calls with links), cache hit rate.
components/stats-drawer.tsx: overlay-aside off a header button — hand-rolled SVG donut (chart-1..5 tokens, no chart dependency), plan rows, stacked time strip, per-model bars, per-child links. Semantic tokens only, mono numerals. Empty-state friendly. QA'd visually on clickhouse + Cal themes with a seeded fixture (numbers hand-verified: $8.06 group, per-model split, 70% cache hit).

Review — fresh-eyes start_thread review (review skill) before this PR; both should-fix findings addressed in the last commit (cancelled-call metering, price cold-start); its id-math, double-count, workflow-bundle-hygiene, and test-quality hunts all came back sound. Remaining known items (noted by the review, deliberately not here): drizzle snapshot drift from 0048–0053 hand-written migrations (pre-existing), and the Gloo fee calibration above.

Tests: 33 new (pricing math, parser, store, turn capture via the scripted mock, harness settle + cancel-survival, API rollup, OpenRouter prices). Full gate green: typecheck, lint, 777 passing / 13 skipped (one pre-existing flaky x-chat test passes in isolation and passed the full run too). Migration 0054 verified applying on a fresh db.

Implements docs/0047-usage-and-cost-stats.md (the approved spec, merged in #121): per-ticket usage/cost metering across a conversation group — Gloo costs exact from the public `/platform/v2/models` price table, tokens/steps/time for everything, and a stats drawer in the chat header. **Capture** - Chat turns: per-slice `LanguageModelUsage` summed in `turn.ts`, one `usage_events` row per turn (`u_<conv>_<skip>`), written after the turn stream closes — never fails the turn, recorded even when a slice throws. - Harness calls: `finish-step` usage read through a new `onUsage` hook in the shared sink, checkpointed into the `u_hc_<callId>` row after EVERY advance step (idempotent upsert) so a cancelled call keeps its completed metering; settle makes the final pass. Pi is priced; claude/cursor record tokens with no dollars (plan badge). - Titles excluded per spec. **Pricing** - `lib/usage/pricing.ts`: pure cost function (fresh×input + cached×cache_read + output×output per-1M, long-context tier selection, platform-fee multiplier, cent rounding, null = unmetered), Gloo payload parser (cache_write 5m/1h/default tiers, deprecated→replacement attribution), `normalizeUsage` for both AI SDK usage shapes (verified against installed types). - `model_prices` refreshed daily: Gloo public endpoint (no key) + OpenRouter pricing from the existing catalog payload. Manual rows survive refreshes. Cold-start closed: `startChatTurn` and `startHarnessCall` fire-and-forget `ensureModelPrices` (the models API alone left a null-cost window on fresh deploys). - Gloo's 6.5% guarded-fee multiplier: `model_prices.platform_fee` defaults 1.0; calibrate once against Gloo Studio's Usage dashboard and set via a manual row. **Read + UI** - `GET /api/x/stats/<id>` rolls up the 0046 locked group (root+descendants): totals, wall/work/wait time, touches (real user messages only), median answer latency, byModel (labels from the catalogs), byChild (conversations + harness calls with links), cache hit rate. - `components/stats-drawer.tsx`: overlay-aside off a header button — hand-rolled SVG donut (chart-1..5 tokens, no chart dependency), plan rows, stacked time strip, per-model bars, per-child links. Semantic tokens only, mono numerals. Empty-state friendly. QA'd visually on clickhouse + Cal themes with a seeded fixture (numbers hand-verified: $8.06 group, per-model split, 70% cache hit). **Review** — fresh-eyes `start_thread` review (review skill) before this PR; both should-fix findings addressed in the last commit (cancelled-call metering, price cold-start); its id-math, double-count, workflow-bundle-hygiene, and test-quality hunts all came back sound. Remaining known items (noted by the review, deliberately not here): drizzle snapshot drift from 0048–0053 hand-written migrations (pre-existing), and the Gloo fee calibration above. Tests: 33 new (pricing math, parser, store, turn capture via the scripted mock, harness settle + cancel-survival, API rollup, OpenRouter prices). Full gate green: typecheck, lint, 777 passing / 13 skipped (one pre-existing flaky x-chat test passes in isolation and passed the full run too). Migration 0054 verified applying on a fresh db.

glen added 8 commits

2026-10-17 17:49:27 -04:00

feat(usage): 0047 schema + pricing module + usage store 665a12ef5d

Migration 0054 (usage_events, model_prices), the pure cost function and
Gloo /platform/v2/models parser (list prices, cache segments, long-context
tiers, deprecated→replacement attribution), and the db store: event
writes, price upserts (manual rows win), group rollup sums.

test(usage): pricing math + store coverage (0047) e557559aee

computeCost segments/tiers/fee/clamps; parseGlooPrices tiers, deprecated
→replacement attribution, malformed payloads; store upserts (manual
wins), idempotent event writes, group rollup incl. null-cost plan rows,
FK cascade.

feat(organize): export the group walk for the stats read (0047) f5e9269753

rootConversationId and conversationGroupIds were module-private; the
usage rollup resolves the same locked group the sidebar does.

feat(usage): capture chat-turn token usage acacdc516e

Sum each slice's LanguageModelUsage into one usage_events row after the turn stream closes, including zero-token turns that actually called a model.

Co-authored-by: Cursor <cursoragent@cursor.com>

feat(usage): capture harness-call token usage a4532c20d1

Read finish-step usage in the shared sink and write one u_hc row at settle. Pi is priced; Claude and Cursor stay unmetered.

Co-authored-by: Cursor <cursoragent@cursor.com>

feat(usage): refresh gloo and openrouter model prices e4a27c8790

Pull Gloo's public price table daily and store OpenRouter per-token rates from the catalog payload the models page already fetches.

Co-authored-by: Cursor <cursoragent@cursor.com>

feat(usage): add the stats API and drawer 1f6a9a7ac3

Roll a conversation's group up at GET /api/x/stats and open it from the chat header: cost, wall time, plan rows, and per-thread links.

Co-authored-by: Cursor <cursoragent@cursor.com>

fix(usage): review fixes — cancelled-call metering + price cold-start (0047)

deploy / checks (pull_request) Failing after 50s

Details

deploy / deploy (pull_request) Has been skipped

Details

662cee132b

Fresh-eyes review findings on the pre-PR branch:

- Cancelled harness calls dropped all metering (the usage write lived in
  the settle step, which a racing cancel skips). The advance step now
  checkpoints running totals into the u_hc row after every step via an
  idempotent upsert — a cancel at minute 29 keeps 29 minutes of tokens.
  Settle's final write reuses the same writer.
- Price cold-start: the models JSON API was the only lazy price fetcher,
  so a fresh deploy's first turns/calls metered null-cost forever
  (cost-at-capture never backfills). startChatTurn and startHarnessCall
  now fire-and-forget ensureModelPrices.
- Journal idx-54 'when' timestamp regressed below idx 53's (cosmetic).

glen added 1 commit

2026-10-17 20:50:47 -04:00

merge main: mock ask-id fix (0047 follow-up)

deploy / checks (pull_request) Successful in 41s

Details

deploy / deploy (pull_request) Has been skipped

Details

069bf89e77

chad merged commit 1ee4dc0c63 into main

2026-10-17 21:06:35 -04:00

chad referenced this pull request from a commit

2026-10-17 21:06:36 -04:00

Merge pull request 'Usage & cost stats: metering, pricing, and the stats drawer (0047)' (#122) from glen/0047-usage-stats into main

Rows
Columns

Usage & cost stats: metering, pricing, and the stats drawer (0047) #122