How I Track Model, Harness, and Effort Spend to Tune My Hive Profiles

Every Hive lead and worker I run leaves a transcript behind. A script pulls the harness, model, effort, turns, and tokens out of each one and stores them in SQLite. Once a week, my Hive lead, the session that plans work and supervises the workers, reads the results and brings me one decision about how my profile should change. The profile is the set of instructions and settings my leads and workers run on. New to Hive? Start with the launch post.

Hive ships the store, the workers, and a couple of opt-in budget settings. The tracking skill, the ledger, and the routing rules live in my private profile. Every number here comes from my machine, across all my projects, with small samples. Read them as an example of the method, not as benchmarks.

Why I Started Counting

Claude Code and Codex both run on consumer subscriptions with usage windows. Claude has a 5 hour window and a weekly one, and Codex has a weekly one. Every project on my machine shares them, so a heavy afternoon in one repo is a slow day in another.

In mid-August, an afternoon with three workers running used 20% of a week's usage in a single day. The workers were cheap. What I'd stacked on top wasn't: a PR review bot on every commit and /simplify on by default. Three workers each spun up four to seven subagents for one real finding. I cut the review bot and made /simplify opt-in. Two weeks later, running only Claude was still emptying the weekly window by midweek, so I started sending a share of every batch of work to Codex.

Both of those were reactions to a usage bar. I could see the window draining, but I couldn't see where it went.

The First Attempt: a Spend Row Per Lane

My first fix was to have the lead record what each lane cost when the lane finished. A lane is one piece of work: a todo, a worker, a PR. The lead already wrote a ledger line for every lane it merged, so I added a spend row next to it.

It lasted 11 days. A shared window can't tell you which lane used what, so the lead was guessing, and I had no way to check the guesses. Everything I wanted was already on disk in the session transcripts. I just wasn't reading them.

Reading the Transcripts Instead

Claude Code writes every session to a JSONL file under ~/.claude/projects. Codex writes rollouts under ~/.codex/sessions. They record every API response with its token usage, model, and effort level.

So I wrote a skill called evaluate-leads. It's a standard-library Python script that reads every transcript, matches worker sessions to Hive's agents table, and saves the results to SQLite. It skips transcripts that haven't changed, and it labels each session as a lead, a worker, or neither. A worker is any session Hive spawned. A lead is a session that spawned or woke workers and didn't run from a worktree.

# collect, then report the last 7 days
python3 ~/.agents/skills/evaluate-leads/evaluate_leads.py

# zoom out
python3 ~/.agents/skills/evaluate-leads/evaluate_leads.py --days 90

# report from what's already collected
python3 ~/.agents/skills/evaluate-leads/evaluate_leads.py --no-collect --since 2026-09-26

The report shows each lead session, median and p90 turns per worker model, how full each worker's context got, and which workers passed the context checkpoint (more on that below). It answers "where did the week go" in one screen.

A few definitions matter, because I got the first one wrong:

• A turn is one API response, counted by unique message id.
• Spend is input plus cache creation plus output, per response. Cache reads are tracked separately and left out.
• Context is everything sent on a response: input, cache creation, and cache reads together. Peak context is the highest it got in a session.

The first numbers were double

Claude Code writes each content block of a response as its own line in the transcript, and every one of those lines carries the same usage data. My first version counted lines. One worker I checked had 279 lines but only 142 turns, and every token count was roughly doubled.

That mattered, because my profile already had a rule built on the bad number. The rule told the lead its turns were the cost, citing "lead sessions produce 1.0M to 1.75M output tokens." That was a per-line sum. Counted per response across 98 lead sessions, the median output was 193k, and the biggest was 443k.

The rule survived anyway, for a better reason. Those same 98 sessions re-read a median of 81M tokens of cached context. Every turn re-sends the whole conversation, so with 81M tokens read against 193k written, the lead's bill is its turn count.

That's the most useful number I've found so far. A lead that takes 400 turns to supervise five workers costs more than all five workers combined, even if it writes very little.

Transcripts disappear

Two things were quietly deleting my data. Claude Code removes transcripts after cleanupPeriodDays, which defaults to 30. My oldest transcript was exactly 30 days old when I noticed. I raised it to 90, and the skill runs well inside that window. Anything it has parsed stays in SQLite after the transcript is gone.

The second one was worse. Hive used to delete a Codex worker's whole home directory when it closed the worker, rollout included. Every Codex transcript before 2026-09-10 was gone, 146 workers in all. Hive now moves the rollout aside before cleanup, but those 146 only show how long they ran.

What the Numbers Changed

The Hive lead runs the skill every Sunday night, posts the read-out as a comment on a tracking todo, and brings me one decision with a recommended answer.

A turn budget for leads. The 2026-09-20 read-out covered 29 lead sessions over two weeks. The median was 272 turns, and the top tenth ran 573 to 1,236 turns at 3M to 5M tokens each. So leads now get a budget: at 300 turns they hand off to a fresh session at the next clean stopping point, and at 600 they stop. Before that landed on 2026-09-21, 22 lead sessions passed 60% of their context before turn 300. Since then, none have, and no lead has reached 600.

A context checkpoint for workers. When a worker's context passes 60%, it commits, writes its state to the todo, reports, and waits. I didn't give workers a turn budget, because turns are only a stand-in for context. That's holding up: since 2026-09-26, 37 of 419 workers passed 60%, and almost all of them were one model in one project. GPT-6 Luna accounts for 33. Sonnet 5.5 accounts for none in 136 workers. Luna works long and fills its context, and Sonnet finishes small.

Screenshots out of the lead. One read-out showed that 60% to 72% of a UI project lead's tool traffic was the lead opening screenshots itself, at 30k to 124k tokens each. Now workers look at images and long logs, and the lead reads their summary.

Rules the lead didn't follow. The lead was supposed to check its own turn count after every merge. The data showed it did that once in 16 merges. Around the same time, a lead told me a 176-turn session was "well past" its 300-turn budget without checking the count at all. So the budget moved out of the instructions and into Hive, which now shows the turn count in its statusline and can hand off for the lead. Instructions are for judgment. Anything that has to happen every time goes in code.

Grade the Pick, Not the Spend

This is the part I'd most like other people to steal.

Once I had per-model numbers, I wanted to rank models. Cheapest for mechanical work? Fewest fix rounds on a debug lane? The problem is that models aren't assigned at random. The lead picks one for each lane, often for reasons that have nothing to do with how good it is. Sometimes the Claude window is tight, so the lane goes to Codex. Sometimes a model is on trial and needs lanes. Sometimes a standing rule decides it.

So on 2026-09-26, the ledger line every lead writes at the end of a lane got three new keys:

2026-09-23 todo-1397 version-pinned test | PR #12 | c454d6d | log: session-end | sonnet | eval: agents=impl:1261 fix_rounds=1 premise_miss=no findings=1/1 kind=debug why=fit alt=claude:opus:default

• kind is the type of work: mechanical, debug, design, docs, or test.
• why is the reason for the pick: fit (the best model for the job), quota (window pressure, or my Codex split), trial, or rule.
• alt is the runner-up, if one came close. Otherwise it's none.

Only fit picks say anything about which model is better. Everything else measures my routing.

Here's what that looks like on two weeks of data. GPT-6 Luna at medium effort did 52 mechanical lanes, but only 12 were fit picks. Another 33 were quota picks and 7 came from a rule. Sonnet 5.5 at medium did 34 mechanical lanes, and 33 of those were fit picks.

Ranked on all picks, Luna gets graded on lanes it only got because the Claude window was tight. Even on fit picks alone, I wouldn't trust a ranking yet. Twelve lanes is not enough, and cheap models get the easy lanes, a bias kind only partly corrects. The skill's own docs say to treat any pattern as a question for me, never as a change to the routing rules.

What the Data Can't Tell Me Yet

Writing this post turned up a gap. The transcripts recorded lead effort, but the skill only saved it for workers, so lead rows showed the model but not how hard it was thinking. The Hive lead fixed that the same day, and every lead session back to the start of September now has its effort filled in.

The ledger has a similar blind spot. It records that a lane was a quota pick, not what the window looked like that day. When the Claude and Codex mix shifts from week to week, I can see it but can't always say why.

And sometimes I make the call before the data comes in. When I trialed GPT-6.1 Sol as a replacement for GPT-6 Sol, I'd already decided to switch if nothing clearly regressed. The read came back with "no signal either way," and I switched. The tracking informed that call. It didn't make it.

What Hive Ships, and What's Mine

Here's what Hive ships as of 1.11.0:

• A turn count in the statusline, and an optional lead_turn_budget with warn and stop levels.
• An optional context_checkpoint_percent. Hive measures each worker's context and tells it when it crosses your threshold.
• Context use per worker in hive crew and the crew sidebar.
• Codex rollouts kept when a worker closes, so its transcript survives.
• Auto-handoff at the turn budget (new in 1.11.0, opt-in). A lead past its budget is replaced by a fresh session in the same pane, keeping its workers and scheduled wake-ups.

Both budget settings are off by default.

Everything else stays in my private profile: the evaluate-leads skill, the ledger keys, the weekly read-out, the Codex load-balance rule, and the model routing.

If you want to try this without my skill, start small. Point a script at ~/.claude/projects, count unique message ids per session instead of lines, and sum input, cache creation, and output, keeping cache reads separate. That alone shows you which sessions are expensive, and it'll probably be your leads.

What's Next

The Sol versus Luna comparison is waiting on more fit picks. Two weeks of kind, why, and alt data is a start. A couple more months will be worth reading.

If you want to try the budget settings, Hive is on GitHub. And if you're tracking your own agent spend, I'd love to hear how you're doing it, especially what you're counting that I'm not.