What it takes to make agents reliable enough to run in production. The focus is the runtime around the model, and the design choices that keep failures contained once an agent acts on real systems.
Habits most codebases rely on without ever stating them. I write each one down and test it against real code and data, to see whether it deserves to stay.
How a voice call actually works, one layer at a time, from a single packet to a group call. If you build on WebRTC or voice AI, these are the protocols underneath the API, and the reasons they behave the way they do.
Distributed systems at the point where the theory meets the pager. I measure what common defaults and mechanisms actually do under load, and what each one costs. Useful if you run services in production and would rather have numbers than rules of thumb.
The internals of toolbench, the framework behind mainLog's tools: what the runner guards against, why a worker exists only to be terminated, how a graph is laid out and routed the same in every browser, how known answers and size budgets keep tools honest, and why every migration is the identity function.
Toolbench is a small framework for interactive tools in web pages. A tool is a pure function and a JSON manifest, and the runtime builds the form, runs the function and draws the answer. What it is, how it fits together, and the choices behind it, shown with tools that run live on mainLog.
Agent memory stores mostly decide what to forget by time: a TTL, or deleting what hasn't been used. In a 90-day scripted scenario, that kept a cancelled booking alive for weeks and stored a restaurant's sentence as the user's preference. Two fields fix both: who wrote the memory, and which event would make it false.
Put an AI agent in a group call and two questions decide the design: does the model hear each person, or one blend, and who has to hold the keys? Both come down to where the agent's audio gets decoded.
Six AI agents each designed one extension to my framework, and each design held up on its own. Side by side they disagreed about names every user touches. One question sorted them out, and writing down the rules my existing code already followed made the answer countable.
Timeouts, retries and budgets for model calls that finish too slowly, produce unusable answers or keep repeating themselves. With the real Anthropic SDK, timeout=3.0 let an 18-second stream finish. In a simulated support chat, with invented failure rates, five checks took the conversations waiting over two minutes from 249 to 0.
Add Bob and Carol to Alice's call and someone has to copy her voice twice. Mesh, an SFU and an MCU put that copy in three different places, and the cost moves with it: upload, server bandwidth, or a decode and an encode per person. Then back to Alice's assistant, where the server is simply the other end of the call.
On three busy repositories the wait for a first review grows much more slowly than a PR's size, so a change split into eight PRs reviewed one after another waits three to eight times as long. Opening them side by side helps less than it looks, and only one reviewer taking them together is no worse than one PR. The measurement, the rule that falls out of it, and a tool to run it on your own PRs.
The same dinner booking, sent to Claude as a free-form tool loop and as a state machine. Per-state tool lists sent 61% fewer tokens, but with prompt caching on they cost 23% more on input, and with output counted the two came out about even. The real saving was code running the steps that need no model, 8 calls down to 2 and about 72% off the total, plus a table of every state against every event.
Your shop now runs 14 services in three languages, and security wants every call between them encrypted and identified. A mesh does that without touching the code, for about a fifth of a millisecond per hop and a tenth of a core per proxy per 1,000 requests a second, measured here with real Envoy. It also brings retry and queue defaults nobody chose. Over plain HTTP they retry less than most people expect; over gRPC, or after one common edit, a failing service gets 27 calls for one checkout.
Alice's laptop sends fifty packets a second into a network that never answers. A few dozen bytes from the receiver every few seconds carry the loss, the jitter and enough timestamps to measure the round trip without synchronised clocks. Enough to back off, but not fast enough to back off well, which is why browsers add feedback for every packet.
Framework debates trade concept counts that measure different things. Count what the smallest real program makes its user write, in separate columns and per reader, find the rules the count can't see by misspelling them, and keep an extra concept only when it moves a check earlier or gives users a seam they can see.
A shop's checkout service can answer 100 requests a second, and for one minute 150 arrive. Left at the usual default, it answers 8% of those people in time; refusing the overflow at once answers two thirds. Capacity decides how many can be served. The policy decides whether they are, and only the question of who is a product decision.
The path is found, and every router on it could hand Alice's voice to a decoder. How two machines that have never met agree on keys over a network anyone can read, what SRTP does to her packet and what it deliberately leaves alone, and how long a call takes from cold start, measured.
My agent could already write Mermaid. Diagrams worth reading took a skill that chooses the picture, keeps the facts intact, and checks the drawing before handing it back.
Alice's laptop doesn't know the address the internet sees it at, and the assistant can't reply to an address it was never told. How one STUN round trip finds it, why some networks need a TURN relay, and how ICE checks every path and picks one.
A deploy makes checkout slow for 0.9% of requests, and the dashboard's median doesn't move. Writing down the question you'd ask if it broke, and making a test prove the telemetry can answer it, is what finds them. Plus what that telemetry costs, in series and in microseconds.
Packets arrive unevenly, and a packet that misses its slot is as good as lost. How a jitter buffer trades delay for dropouts, what concealment can and can't hide, and what Opus FEC and RED buy against random loss and against bursts.
A pytest plugin passed every test from the source checkout and failed twice in CI, because only the installed wheel told pytest the plugin existed.
Twenty milliseconds of speech becomes an 80-byte Opus frame inside a 120-byte packet. Why each header is there, and why one lost packet costs 20 ms over UDP and 80 or more over TCP.
A handoff carries two things: the words the next model reads, and the state the next runtime has to enforce. Every common handoff style moves only the words. In a small scripted demo, even the full transcript, 1,838 bytes, left the restaurant holding two tables, because the key that names the in-doubt booking was never in it. A 310-byte record written by the sending runtime fixed it.
Converting audio to a higher sample rate makes it fit the next program; it cannot bring back sound an earlier step removed. Three test tones show why.
A plugin you can see in pip list comes back “not found”. Python hosts turn installed entry points into loaded code in four checks, and the order of those checks decides whether a plugin's code runs before anyone asks if it fits, and whether the error your code catches says why.
When one agent calls another with a sentence, the caller's model writes the callee's prompt, and a stranger's note can ride along. In a small scripted demo a bus seemed to fix that, but a typed fact sent by call was just as safe, and a bus event with a note was not. What the bus adds is that one agent being down doesn't leave another's work half done.
A test that checks importing a library leaves its optional dependency unloaded, and proves it got far enough to notice.
The user says “actually, Saturday” while the agent is still booking Friday. Cancelling the task and starting again leaves the restaurant holding two tables. What a runtime has to do instead: find out what the old plan already did, secure the new one, then undo the old, and tell the user what happened.
A checkout page hangs while the dashboard says 23 ms. The mean, a percentile and a share under a threshold each answer a different question, only two of them add up across hosts, and an averaged p99 can be wrong by a factor of three in either direction.
Halving an audio bitrate from 32 to 16 kb/s took traffic from 48 to 32, because every packet's headers stay the same size. Packet rate is the other control.
Agent runtimes are built around the user's message, so every input ends up acting with the user's authority. A restaurant's webhook, a timer and a second message mid-task each break that, and each gets a rule, shown in a small runtime you can run.
Whether two similar functions should become one depends on whether they change together, and git already knows. A real history from httpx, a script that counts co-changes across a repository, a browser tool for one pair, and a third answer that is neither copying nor abstracting: generate one copy and check it.
The internals of toolbench, the framework behind mainLog's tools: what the runner guards against, why a worker exists only to be terminated, how a graph is laid out and routed the same in every browser, how known answers and size budgets keep tools honest, and why every migration is the identity function.
Toolbench is a small framework for interactive tools in web pages. A tool is a pure function and a JSON manifest, and the runtime builds the form, runs the function and draws the answer. What it is, how it fits together, and the choices behind it, shown with tools that run live on mainLog.
Agent memory stores mostly decide what to forget by time: a TTL, or deleting what hasn't been used. In a 90-day scripted scenario, that kept a cancelled booking alive for weeks and stored a restaurant's sentence as the user's preference. Two fields fix both: who wrote the memory, and which event would make it false.
Put an AI agent in a group call and two questions decide the design: does the model hear each person, or one blend, and who has to hold the keys? Both come down to where the agent's audio gets decoded.
Six AI agents each designed one extension to my framework, and each design held up on its own. Side by side they disagreed about names every user touches. One question sorted them out, and writing down the rules my existing code already followed made the answer countable.
Timeouts, retries and budgets for model calls that finish too slowly, produce unusable answers or keep repeating themselves. With the real Anthropic SDK, timeout=3.0 let an 18-second stream finish. In a simulated support chat, with invented failure rates, five checks took the conversations waiting over two minutes from 249 to 0.
Add Bob and Carol to Alice's call and someone has to copy her voice twice. Mesh, an SFU and an MCU put that copy in three different places, and the cost moves with it: upload, server bandwidth, or a decode and an encode per person. Then back to Alice's assistant, where the server is simply the other end of the call.
On three busy repositories the wait for a first review grows much more slowly than a PR's size, so a change split into eight PRs reviewed one after another waits three to eight times as long. Opening them side by side helps less than it looks, and only one reviewer taking them together is no worse than one PR. The measurement, the rule that falls out of it, and a tool to run it on your own PRs.
The same dinner booking, sent to Claude as a free-form tool loop and as a state machine. Per-state tool lists sent 61% fewer tokens, but with prompt caching on they cost 23% more on input, and with output counted the two came out about even. The real saving was code running the steps that need no model, 8 calls down to 2 and about 72% off the total, plus a table of every state against every event.
Your shop now runs 14 services in three languages, and security wants every call between them encrypted and identified. A mesh does that without touching the code, for about a fifth of a millisecond per hop and a tenth of a core per proxy per 1,000 requests a second, measured here with real Envoy. It also brings retry and queue defaults nobody chose. Over plain HTTP they retry less than most people expect; over gRPC, or after one common edit, a failing service gets 27 calls for one checkout.
Alice's laptop sends fifty packets a second into a network that never answers. A few dozen bytes from the receiver every few seconds carry the loss, the jitter and enough timestamps to measure the round trip without synchronised clocks. Enough to back off, but not fast enough to back off well, which is why browsers add feedback for every packet.
Framework debates trade concept counts that measure different things. Count what the smallest real program makes its user write, in separate columns and per reader, find the rules the count can't see by misspelling them, and keep an extra concept only when it moves a check earlier or gives users a seam they can see.
A shop's checkout service can answer 100 requests a second, and for one minute 150 arrive. Left at the usual default, it answers 8% of those people in time; refusing the overflow at once answers two thirds. Capacity decides how many can be served. The policy decides whether they are, and only the question of who is a product decision.
The path is found, and every router on it could hand Alice's voice to a decoder. How two machines that have never met agree on keys over a network anyone can read, what SRTP does to her packet and what it deliberately leaves alone, and how long a call takes from cold start, measured.
My agent could already write Mermaid. Diagrams worth reading took a skill that chooses the picture, keeps the facts intact, and checks the drawing before handing it back.
Alice's laptop doesn't know the address the internet sees it at, and the assistant can't reply to an address it was never told. How one STUN round trip finds it, why some networks need a TURN relay, and how ICE checks every path and picks one.
A deploy makes checkout slow for 0.9% of requests, and the dashboard's median doesn't move. Writing down the question you'd ask if it broke, and making a test prove the telemetry can answer it, is what finds them. Plus what that telemetry costs, in series and in microseconds.
Packets arrive unevenly, and a packet that misses its slot is as good as lost. How a jitter buffer trades delay for dropouts, what concealment can and can't hide, and what Opus FEC and RED buy against random loss and against bursts.
A pytest plugin passed every test from the source checkout and failed twice in CI, because only the installed wheel told pytest the plugin existed.
Twenty milliseconds of speech becomes an 80-byte Opus frame inside a 120-byte packet. Why each header is there, and why one lost packet costs 20 ms over UDP and 80 or more over TCP.
A handoff carries two things: the words the next model reads, and the state the next runtime has to enforce. Every common handoff style moves only the words. In a small scripted demo, even the full transcript, 1,838 bytes, left the restaurant holding two tables, because the key that names the in-doubt booking was never in it. A 310-byte record written by the sending runtime fixed it.
Converting audio to a higher sample rate makes it fit the next program; it cannot bring back sound an earlier step removed. Three test tones show why.
A plugin you can see in pip list comes back “not found”. Python hosts turn installed entry points into loaded code in four checks, and the order of those checks decides whether a plugin's code runs before anyone asks if it fits, and whether the error your code catches says why.
When one agent calls another with a sentence, the caller's model writes the callee's prompt, and a stranger's note can ride along. In a small scripted demo a bus seemed to fix that, but a typed fact sent by call was just as safe, and a bus event with a note was not. What the bus adds is that one agent being down doesn't leave another's work half done.
A test that checks importing a library leaves its optional dependency unloaded, and proves it got far enough to notice.
The user says “actually, Saturday” while the agent is still booking Friday. Cancelling the task and starting again leaves the restaurant holding two tables. What a runtime has to do instead: find out what the old plan already did, secure the new one, then undo the old, and tell the user what happened.
A checkout page hangs while the dashboard says 23 ms. The mean, a percentile and a share under a threshold each answer a different question, only two of them add up across hosts, and an averaged p99 can be wrong by a factor of three in either direction.
Halving an audio bitrate from 32 to 16 kb/s took traffic from 48 to 32, because every packet's headers stay the same size. Packet rate is the other control.
Agent runtimes are built around the user's message, so every input ends up acting with the user's authority. A restaurant's webhook, a timer and a second message mid-task each break that, and each gets a rule, shown in a small runtime you can run.
Whether two similar functions should become one depends on whether they change together, and git already knows. A real history from httpx, a script that counts co-changes across a repository, a browser tool for one pair, and a third answer that is neither copying nor abstracting: generate one copy and check it.
Melanie Mitchell's honest assessment of where AI actually stands, the cycles of hype and winter, and why the barrier of meaning still separates narrow AI from anything resembling human intelligence.
A practical introduction to statistical modeling and simple linear regression, from mathematical foundations to a real-world housing price prediction.
Melanie Mitchell's honest assessment of where AI actually stands, the cycles of hype and winter, and why the barrier of meaning still separates narrow AI from anything resembling human intelligence.
A practical introduction to statistical modeling and simple linear regression, from mathematical foundations to a real-world housing price prediction.
John Brockman assembles 25 of the sharpest minds across physics, robotics, and philosophy to debate AI's future.
Max Tegmark's balanced exploration of AI's future, from the three stages of life to superintelligence risks.
How I built a domain-specific language for runtime-configurable boolean expressions, with optimized .NET expression trees and variable binding.
A sweeping history of humankind that challenges how we think about cooperation, fiction, and the revolutions that shaped our species.
A survey of how philosophers, scientists, and transhumanists have tried to answer the oldest question.
Julian Baggini on why the meaning of life is not one profound answer, and why looking for origins is the wrong place to look.
John Brockman assembles 25 of the sharpest minds across physics, robotics, and philosophy to debate AI's future.
Max Tegmark's balanced exploration of AI's future, from the three stages of life to superintelligence risks.
How I built a domain-specific language for runtime-configurable boolean expressions, with optimized .NET expression trees and variable binding.
A sweeping history of humankind that challenges how we think about cooperation, fiction, and the revolutions that shaped our species.
A survey of how philosophers, scientists, and transhumanists have tried to answer the oldest question.
Julian Baggini on why the meaning of life is not one profound answer, and why looking for origins is the wrong place to look.
Nothing matches.
