If you’re a tech leader walking into H2 planning, you already feel it:
Product wants tokens, Engineering wants tokens, Design wants tokens.
And Meta is openly saying an engineer’s burn rate may soon equal their salary.
Six weeks ago I gave you five habits for controlling AI spend. But habits don’t survive hundreds or thousands of engineers at scale. Systems do. And policy needs somewhere to live.
So this week Jonathan and I did the homework so you don’t have to.
We went through everything the leading engineering orgs have published about running AI at scale—Uber, Stripe, DoorDash, Shopify.
We compiled it into an executive brief:
the architecture,
the methodology,
the enforcement patterns,
what worked and what got reversed within a year.
Let’s dive in.
Informing my thinking this week:
- Steve Yegge’s Google Platforms Rant — the canonical public account of Bezos’s 2002 API mandate
- Uber’s GenAI Gateway — one internal service, the standard path to every vendor
- Stripe’s Minions/minions-stripes-one-shot-end-to-end-coding-agents) — 1,300+ agent-written PRs a week
- DoorDash’s AI code reviewer/doordash-built-an-ai-code-reviewer-engineers-actually-listen-to/) — findings in 7 minutes, ~$3 a review
- Mosseri on per-engineer token caps — the budget pressure arriving at Meta
- Kimi K3 — 18th to 1st on the Frontend Code Arena in one release
- Nadella on token capital — the ecosystem argument
In 2002, Jeff Bezos sent a memo
No original text has ever surfaced.
What we have is Steve Yegge’s account, written inside Google in 2011 and posted in public by accident.
The rules:
The rule that mattered said every interface, without exception, must be designed from the ground up to be externalizable.
Ready for strangers.
That killed tribal knowledge as a load-bearing dependency: no more gluing two systems together with a conversation.
And when Amazon later decided to sell infrastructure, the infrastructure was already in sellable shape.
AWS was a byproduct of this memo.
I spent years as an enterprise architect at Workday and Dayforce, and API-first was something we always wanted to strive for. I never felt like we were close to the rigor I’ve read about from the AWS stories.
And for tech teams, this is directly related to the tokenmaxxing and budget problems surfacing at enterprise scale.
Let me explain.
The memo I would send today
Bezos wrote his memo in 2002 without knowing AWS was on the other side of it.
**This week an engineering leader shared his leadership team’s agenda with me. **
The items, near verbatim:
Which harnesses will we use?
Which engineering methodology is the best — is SDD here to stay?
Which workflows get which models, and how do we enforce that?
How do we protect our IP from the frontier companies?
How do we measure AI ROI — velocity, quality, rework?
How does intelligence compound so all agent runs make future agent runs better?
Every engineering leader I talk to is circling the same whiteboard.
You do not answer these questions one at a time. This requires taking a step back and putting on your visionary hat, then your tactical hat.
So here is an exercise for every engineering leader reading this.
Stand in 2036.
Every function of your enterprise — product, engineering, finance, support, sales — runs as a closed loop: agents discover, build, verify, and learn from outcomes without waiting on a meeting.
Now look backwards.
What must have been true?
Write the memo that made it true.
Here is mine.
- Measurable and queryable. All work and outcomes must be measurable and queryable. This includes the durable artifacts from the recorded decisions, discussions, and debates around the topics of risk management, planning, and execution on business goals.
- CI for CI. All model calls route through one layer, such as a controllable internal service like a gateway. Nobody talks to a vendor directly, and no workflow depends on whose model answers. Direct vendor calls risk intellectual property theft. Centralized, compounded intelligence is the solution.
- Right context, right time. Meaning context is delivered, not hunted. Every agent and every human receives the right context at the right time for the workflow’s goal — pulled from the centralized intelligence layer for the entire business. Think of it like a living company brain. Cross-functional expertise and learning is queryable, instantly, by all decision makers.
- Compound the company brain. This means that every run, eval, harness, and dataset around every workflow for the enterprise, without exception, must be designed from the ground up to compound as a dataset for agentic workflow optimization. This powers the centralized intelligence layer which means every future workflow benefits from every one before it. They are first-class intellectual property, curated like capital, because they are capital. This empowers speed, velocity, and acceleration to all parts of the organization.
I’ll check back on this and see how far this survives.
Here is my reasoning, starting with what the leading orgs already converged on.
Why every leading org converged on one AI gateway
Last month I sat with a tech executive managing a token budget north of fifty million dollars.
He had been reading his engineers’ prompts, and some of them were asking a frontier model what the weather was.
But his question was not “how do I stop that.”
His question was “how do I forecast spend without becoming the token police.”
Every leading org we examined in our briefing converged on the same answer:
From the brief: one enforced infrastructure point between every caller and every model.
An AI gateway (sometimes called an LLM gateway) is one internal service that model calls route through by default.
No sanctioned path reaches a vendor API directly; access goes through a security review, not a personal API key.
Cheap open-weight models for mechanical work where a test suite catches most failures, frontier models where being confidently wrong costs a week, specialized models where your domain demands it.
On the pod I put it this way: right now your engineers are spending tokens at a casino.
The gateway takes the casino out of it.
Same chips, but now something sits between the player and the table optimizing what you get back.
The word that matters in that diagram is ENFORCED. A gateway you can route around is a suggestion.
The leaders are building harnesses behind a mandatory gateway
Not vendor decks — public engineering blogs:
From the brief: Stripe, Uber, DoorDash, with the receipts.
The pattern across all three: they do not hand engineers off-the-shelf tools and hope. They build harnesses behind the chokepoint, so every improvement one team encodes routes to everyone, and the whole org compounds instead of the one engineer who figured it out.
That is the moat conversation hiding inside the budget conversation. Kimi K3 just went 18th to 1st on the Frontend Code Arena in a single release, open weights landing July 27.
Taking a moment to just say, the AI landscape is pretty damn wild. It’s an amazing time to build.
The model layer reprices on a ten-day clock — but domain-specific harnesses around your economically valuable workflows are a little island you can plant a root on.
Grow some trees, start farming.
Good waterfront protection.
Your gateway is the waterfront.
Spec-driven development is what flows through it
The gateway is infrastructure. The methodology is the other half.
From the brief: ad-hoc prompting reworks in loops. A versioned spec makes agent execution repeatable and traceable.
Spec-driven development means curating the input artifact that informs the code generation — research, requirements, test strategy, the spec — then holding the agent to it as acceptance criteria. Most of your brainpower moves upstream.
Me and Jonathan ship every spec as a PR to each other.
Two-person company, and we still do it, because a spec in the repo is your reasoning made ready for strangers. Even when the stranger is your co-founder.
In the early 2000s, it was API first.
Now, this is one tactical method towards AI first principles.
Our reasoning should be queryable.
Mandate AI in workflow gates, not performance reviews
This is the part of the brief I would put in front of a board.
From the brief: Shopify and Uber enforced through infrastructure and it scaled. Duolingo tied AI to performance reviews and reversed the policy within a year.
Enforcement through infrastructure survives. Enforcement through individual scoring gets reversed, publicly, while your best people update their resumes.
The gateway compounds intelligence
From the brief: the intelligence layer is its own service behind the gateway — it injects the right context, at the right time, into every call.
Jonathan said something profound on the podcast that all engineering leaders should think on:
“It’s not just that we’re routing requests, it’s actually that we’re comparing cross domain company knowledge, you know, to really make sure that the context itself is augmented in a way that takes advantage of the entire corpus of the company brain.”
The chokepoint you build for cost governance turns out to be the one insertion point you control for every call.
You cannot enrich calls you cannot see.
What should it record?
Log tokens, model, latency, and cost, and you have built a billing system with a routing table attached.
This is necessary for operations.
Log who asked, what it served, what answered, and what happened next — merged, reverted, shipped, rolled back — and you will realize this is an intelligence layer abstracted from the reliance on models and frontier labs.
This layer becomes YOUR compounded intelligence layer that stays safe, no matter what changes in the AI landscape.
My prediction: within twelve months the interesting gateway conversations stop being about rate limits and start being about what the gateway knows.
The orgs that logged outcomes from day one will have the makings of a training set. They will have a stacked harness architecture that builds loops from discovery in product through engineering PRs through business outcomes like revenue.
True autonomous closed loops, around the whole enterprise.
Everyone else will have a billing system.
And that intelligence layer is exactly what we’re building with Clarity.
Exciting times.
Key takeaways
Steal these for your next engineering leadership meeting:
- One gateway, enforced. Every model call through one internal service. A gateway you can route around is a suggestion. It’s the API Gateway pattern, extended to the AI era.
- Route by verifiability. Open-weight models for work a test suite can check. Frontier models where being wrong costs a week.
- Log four fields, not two. Who asked, what it served, what answered, what happened next. Three fields make a ledger. Four make a training set.
- Ship the spec with the PR. The spec is your reasoning made externalizable — and it is the acceptance criteria you hold the agent to.
- Enforce in infrastructure and workflow gates. Never in performance reviews. Duolingo already ran that experiment for you.
- The harness is the moat, not the model. Models reprice in weeks. Your record of which calls produced value does not.
Want the full brief?
The executive brief covers the gateway architecture in depth, the harness patterns, spec-driven development, ROI measurement that survives CFO scrutiny, and how Uber ran its agentic pods—every source cited.
Want the full brief? Email me the word “brief” and I’ll send it over.
—Robert
P.S. Here’s the latest episode of Weekly AI Clarity, our series bringing AI tech executives and builders signal from the frontlines of AI. New episodes weekly. We talk through this brief on the latest pod coming out this week. Stay tuned and subscribe.
Build In Public
I’m trying something new with the newsletter game.
We’re getting traction focusing on serious product and engineering tech teams, responsible for delivering internal and external facing agents.
So I decided to evolve the website and newsletter design further for a cleaner and rigorous aesthetic.
Let me know what you think!
$10M ARR
The content engine is finally getting a bigger flywheel effect! I’ve been grinding on this forEVER and now am seeing results in the experimentation grind.
Here’s a recent short clip that went relatively viral for my account size!
Dog Dadding
Okay so I found some key beta (an umbrella term climbers use to refer to all tactics for sending or accomplishing the goal) for Kenji’s longevity protocol.
We’re out in the sun a lot and especially in the mountains the UV can be killer.
Doggle boi
So I got him doggles. LOL. It’s so cute. Here he is at the beach getting used to them, and I’m excited to next put a googly eye decal on it.
Not Kenji, shoutout Rex Specs (no affiliation, I just like the product)
Climb V10, Run 100 miles
I’ve still been recovering from my recent 50 mile ultra that I wrote about last time.
I’m feeling more fit and healthy than ever though.
It makes me so endlessly happy that I’m continuing to advance my health and fitness goals while advancing my business goals.
It takes a lot of discipline and me constantly being on top of my systems—ordering protein powder, eating it , not snoozing the alarm for early AM workouts, etc.
BUT IT’S SO WORTH IT.
I’ve been doing more yoga because my body got so tight from that ultra, and that’s been a great change.
Recently hit PRs in my key strength exercises.
Things are good.
Gotta enjoy it until the next injury, which I’ve learned is inevitable.
Just gotta keep PTing.
References
- Steve Yegge, “Google Platforms Rant” (2011) — the canonical public account of the 2002 mandate. gist.github.com/chitchcock/1281611
- Uber Engineering, “Navigating the LLM Landscape: Uber’s GenAI Gateway” (Jul 2024). uber.com/blog/genai-gateway
- Uber Engineering, “uReview: Scalable, Trustworthy GenAI for Code Review” — 90%+ of ~65K weekly diffs, ≈39 developer-years/yr. uber.com/blog/ureview
- Stripe Engineering, “Minions: One-Shot, End-to-End Coding Agents,” Parts 1–2 (2026) — 1,300+ agent PRs/week, up from ~1,000. stripe.dev/blog
- DoorDash Engineering, “How DoorDash built an AI code reviewer engineers actually listen to” (2026) — ~7 min, ~$3/review. careersatdoordash.com/blog
- TechCrunch, “Meta’s Adam Mosseri says AI token budgets could soon be capped per engineer” (Jul 14, 2026).
- Tom’s Hardware / Arena.ai, Kimi K3 release — 1,679 pts, 18th→1st, Frontend Code Arena; open weights Jul 27 (2026).
- Satya Nadella on X (Jun 2026) — “the future of the firm is a learning loop in which human capital and token capital compound.”
- Tobi Lütke memo via BetaKit (Apr 2025) — AI mandated in Shopify’s GSD Prototype phase; AI-first before headcount.
- Fortune, “Duolingo CEO walks back AI usage requirement tied to performance evaluations” (Apr 2026).
- BCG, “AI at Work: Why Strategy Matters More Than Tools” (2026) — 1,488 workers; productivity drops beyond 3–4 tools.
Liked this article?
Click the like button.
Feedback or addition?
Add a comment.
Know someone that would find this helpful?
Share this post.
P.S. Want reminders on entrepreneurship, growth, leadership, empathy, and product?
Follow me on..