Product manager · developer tools & AI
Saurabh
Sharan
I'm a product manager. I build things out of curiosity.
ctx started with a question about my own AI bill. This page is the record of chasing it: why I built the tool, what I learned along the way, and what shipped. The source is public.
Case study · Evidence-driven product pivot
ctx.
I built a tool to cut AI coding costs. My proof that it was safe turned out to be no proof at all, so I rebuilt the product around what I could actually show.
Evidence note: the figures below come from one developer's machine. They informed the pivot; they are not market validation or a causal safety guarantee. The source is public, so you can check them.
Why I built it. My own AI coding bill made no sense. My agent was carrying about 77,000 tokens of tool definitions into every single request, and most of them were for tools it never used. I wanted to know exactly what that cost me and whether a tool could cut it without breaking anything. I couldn't find a good answer, so I built one.
What I learned. My proof wasn't proof. For 25 days my safety metric said trimming caused zero problems. When I dug in, the metric could not have caught a problem if there had been one. So I threw out the claim, kept the honest numbers, and rebuilt the product around showing evidence instead of promising safety.
What shipped. A local tool, written in Rust, that sits between your coding agent and its tools. It shows an itemized bill of what your context window costs, trims only what it can prove safe from your own usage, and keeps a byte-for-byte undo for every change. Once I had my answer, I ended the project and made the source public. You can install it and see everything for yourself.
Home. One number the moment you open it: tokens reclaimed so far, with a line that says autopilot has it handled.
The problem
My agent was spending most of its memory on things it never used.
A coding agent like Claude Code or Cursor works inside a fixed context window. Every token it spends carrying something it does not need is a token it cannot spend on your actual problem. Two kinds of waste dominate, and neither is visible to the person paying for it.
The output tax is what the agent reads back. Call a tool and the full result lands in the window: a file dump, a long test log, a large API response. Most of it is read once and never needed again, yet it sits there for the rest of the session.
The input tax is quieter, and on my machine it was larger. Every tool server you connect loads its entire menu into every request, whether the agent uses those tools or not. That came to about 77,000 tokens on every single request, roughly 64,000 of them for tools the agent never once called. A fixed cost, paid on every turn, all day.
I first framed CTX as an optimizer: learn what each tool result could lose, then treat a lack of observed correction as proof that the cut was harmless. The observability gap was real. That causal leap was the risky assumption.
The evidence-driven pivot
The data said zero problems. It really meant my measurement could not catch problems.
After 25 days, the correction label had fired zero times across 2,731 decisions. The tempting story was “CTX caused no corrections.” The defensible story was that the label was too sparse to validate the claim.
Other evidence still mattered. On this single machine CTX had actually removed about 475,000 output tokens, and a separate observational “needed whole” model reached 0.89 holdout AUC, later 0.95. Useful engineering evidence, but not causal proof and not evidence that a team would pay.
I changed the product around what the data could support: a Context Bill before optimization, verbatim rewind for every applied trim, randomized safety checks, a “suspected trim cost” view, and neutral reporting across agent surfaces.
The product principle became: See the bill. Reclaim with a way back. Describe the evidence honestly.
The product
See, Save, Trust, across four surfaces.
The pivot is not a disclaimer buried in the docs. It changes the information architecture, controls, and claims.
See, the itemized bill.
The two taxes side by side, then every tool ranked by what it costs, biggest first. The green share is eligible under CTX's current transform, not an earned-safety claim. Activation and observed harm are shown separately.
See. Two taxes, one context window, itemized from your own sessions.
Save, the evidence, tool by tool.
Every tool climbs the same ladder: watched, tested on a randomized slice of real work, evaluated, then earned. CTX only trims a tool once comparable runs pass its safety check, and the page shows where each one stands. At the product level the promise is narrow: compare behavior on the user's own sessions, keep output unchanged when evidence is unclear, and keep the change only while the measured thresholds hold.
Save. The earn-it ladder. A tool is trimmed only once it reaches Earned.
Trust, made concrete.
Everything is local by default, with no account or background telemetry. Issue reports show the exact payload before an explicit Send action. Export, purge, or delete remain under the user's control.
Settings. Local storage, no background telemetry, and explicit preview-before-send controls.
The process
Five personas, and a coherence check I could run every iteration.
The hard part of the redesign was not visual. It was knowing whether each version was actually better, and better for whom. Five people open this dashboard for very different reasons.
They pull against each other. Sam wants one calm number; Priya wants depth. Too simple and Priya leaves, too dense and Sam and Alex leave. The old dashboard served exactly one of them: me.
So I built a repeatable check I named fitcheck. It role-plays all five personas through any version of the dashboard, in their own voice and their own patience window, and scores what they experience across seven dimensions: comprehension, time to value, trust and safety, cognitive load, action clarity, journey coherence, and delight. Out comes a score per persona, an overall, a separate coherence score, and a verdict: ship, iterate, or rework. The rule that made it useful: a version ships only if it beats the last one and no single persona regresses. A better average was never allowed to hide a worse experience for the skeptic or the first-run user.
That gate is the most transferable thing I built here. It turned "does this feel better" into a call I could defend, iteration over iteration.
Before / after · Home, scored cold
The two personas the redesign was for, Sam and Alex, moved most. No persona regressed on any dimension. The same language then rolled across See, Save, and Settings, each gated the same way, each landing 4.3 to 4.4 with full coherence.
Hard calls
Good redesigns are mostly subtraction.
Merge
Eight tabs to four.
The old nav had eight destinations: Home, Context bill, Tool tax, Tool report, Compaction, Surfaces, Activity, Settings. Each was individually defensible, and together they were a wall. I grouped them into four that match the product's own story: Home, See, Save, and Settings, with Activity folded into Settings. Fewer places to go, and the nav now teaches the model instead of listing features.
Delete
The metrics that padded the case.
The proof surface used to carry a stat strip, a sparkline, a suspected-cost section, a compaction grid, and a correlation footnote. Individually interesting, together noise. I cut them. A skeptic reads a page that shows less as more honest, not less capable. I also retired a manual switcher once its job was handled better elsewhere, and kept the underlying system working. A UI cut, not a capability cut.
Admit
A data gap, stated plainly.
ctx can show how many tools on a server are dead weight, but it cannot name every removed tool, because it never stored the full menu it was pruning. The easy move is to hide that. Instead the UI says it outright: here is the count, and here is why we cannot list each one. For the skeptic, admitting the limit is exactly what earns trust. I scoped the fuller version as a follow-up, not a bug to paper over.
Under the hood, briefly
The calls behind the four surfaces.
Four screens on top. Behind them, a few decisions I had to reason through, each a real tradeoff and not a free win.
Local, including the parts that adapt.
CTX builds private fingerprints on the user's machine. There is no account or background telemetry. Learning comes only from that user's own history, and nothing leaves the machine without an explicit, previewed send.
Protect first, cut second.
CTX never trims on a model score alone. Comparable randomized runs must pass its safety check, and the original remains available for rewind. That reduces risk; it does not guarantee that every future trim will be sufficient.
An audit trail you can actually read.
Every action ctx takes is recorded locally, so you can see what it did to which tool and reverse any of it. The cost: one more surface to keep honest, which is the whole point.
Every number is measured, including the misses.
The dashboard only shows figures ctx computed from your own runs, the times it had to give something back included. The cost: the honest number is sometimes smaller than the one I would like to show.
Light and dark, because it should read like your editor, not a control panel.
What I learned
Four product decisions the evidence changed.
Check that your metric can fail before you trust it.
Zero corrections across 2,731 decisions looked like a perfect record. Then I asked a simpler question: if trimming had broken something, would this number have moved? It would not have. A metric that can only say "fine" proves nothing, so I stopped using it as proof.
When the evidence changes, change the product, not the pitch.
I could have kept the safety claim and softened the wording. Instead I redesigned around what I could show: the bill you can see, the undo that always works, the honest label on every number. A different product, not a different sentence.
Internal scores are quality gates, not customer truth.
I built five personas and an automated check (fitcheck) that scores every redesign through their eyes. It made "is this version better" a repeatable call. It still says nothing about whether anyone will adopt or pay, so I stopped letting it pretend to.
End it on purpose.
I started this to answer a question about my own machine, and the question got answered. Turning it into a business would have meant chasing adoption I never set out to chase. So I stopped cleanly: source public, installers shipped, findings written down. Ending a project deliberately is a decision too, and it beats letting it fade.
Where it runs, honestly
Full where the hooks are. Insight everywhere else.
Trimming rides on Claude Code hooks, so I show the coverage plainly rather than imply it is everywhere.
Surfaces
Platforms
On Claude Desktop you get the ctx tools and the dashboard, not automatic trimming. The proxy that would bring trimming there puts ctx in the request path, so it stays a deliberate opt-in, not the default.
Where it ended
I got my answer. The project is done, and the code is public.
I built ctx to understand what my agent's context actually cost and whether trimming it could be proven safe. Both questions are answered on this page. Rather than chase a business I didn't set out to build, I ended it cleanly: the source is public and you can install it.
$ brew install saurabh0392/ctx/ctxA walkthrough of what ctx does on your machine. 3.5 minutes, with sound.
Contact
Get in touch.
If you want to talk about ctx, developer tools, or a role, email me.