Don't Build Agents, Build Environments Instead

Channel: Hamel Husain
Published: August 25, 2026
URL: https://www.youtube.com/watch?v=JolFqvXj3BE

Join the AI Evals October 2026 cohort: https://maven.com/parlance-labs/evals?promoCode=yt-2026

You get a coding agent working on your laptop, then you want ten of them running at once. The thing that breaks is never the agent. It is the machine you handed it, and most teams spend months rebuilding that machine badly before they realize the environment was the actual product.

Transcript

Today we're going to talk about how to use modal as an agent sandbox, which is increasingly really important if you're building any kind of agentic workflow. Because of the things I mentioned—the fact that developer experience is so good, it's so fast, it's easy to use—it makes a really good sandbox and you can use it for more than sandboxes. It just happens to also be a really good sandbox. Adam's here to talk about it and show you how you can use modal. So let's kick it off and hand it over to him.

I'm super excited to talk a little bit about modal and really talk a little bit about the difficulty in building a system of letting agents build software for you. This presentation is going to be two parts. One is going to be talking about how does somebody design a system for agents to reliably use their environments to actually go and build software. And then towards the end, we'll actually talk about what the developer experience of building with this looks like, and we'll get into some code.

The Real Difficulty

I want to talk about the sort of real difficulty in building agents that can actually produce software. Before we get into this, my name is Adam and we're at a company called Modal. I've been at Modal for about two months now. I'm probably better known for the work I've done as a maintainer of Prefect and FastMCP. If you've built an MCP server, you've probably built it on FastMCP before.

This talk is about how there tends to be a lot of focus on agents, where the environments that those agents live in tend to be a more rich and interesting problem space, and arguably a more important one. I wanted to call this talk "Environmental Engineering 2011," or maybe the spicier take is that the hard part of background agents is always the background part, not the agent part. I called it "Don't Build Agents, Build Environments Instead" really as an homage to one of my favorite talks that I'd encourage everybody to go and watch. That talk was by Barry and Mahesh at Anthropic, and they gave a similar talk called "Don't Build Agents, Build Skills Instead." This talk did every single number you can imagine on YouTube.

The central argument of their talk, and why I appreciate this framing so much, is that agents have really converged on a pretty universal design. I'm sure that maybe this is a controversial take depending on the talks that have led up into this, but as folks here who have built with agents have felt, as you use something like the codeex harness or maybe Claude Code or use pi or Hermes, you'll notice that a lot of these agents have some pretty common primitives to them. It's how do I do compaction when a conversation gets too long—related to how do I do memory—but a lot of their tools are pretty much the same. It's how do I go and execute bash commands, how do I go read files, write files, edit files.

The argument that Anthropic made was that we've really coalesced on a common agent design, and so really the hardest part that we keep coming across is how to imbue these agents with expertise. Their argument was that skills were the best vehicle for imbuing an agent with that expertise. So that explains why, as you see a lot of folks design agents, a lot of it is: yeah, yeah, yeah, go buy whatever the agent harness is. The recipe for designing an agent always starts with going to pick a harness and then let's go obsess over the context that we give that agent.

There are obvious corner cases here that we could obsess over, but I would say that broadly, a lot of their predictions about how businesses should obsess over providing context to agents and maybe not redesigning every harness has largely borne out.

Background Agent Architecture

This talk, as an homage, follows the same feeling. I've worked with dozens of companies who are building their own background agents, and background agents are similarly converging on a pretty universal system architecture. This is kind of one layer up from a harness—a harness is really like what the individual agent is doing. But as you're trying to design a system where those agents operate, they all have a pretty similar architecture to them.

So what are the hardest and most reinvented parts here? What are the things that we really need to be obsessing over? Really, it's about how do you go and build what I call dev boxes for agents, and then how do you think about orchestrating them as they complete their work. This is of course from the last few weeks, and so I imagine there's some depreciating knowledge here, but I do expect this to continue to bear out.

I want to talk about these two problems. This is going to be a bit of a systems talk, and then we're going to go into what the actual code of writing with modal actually looks like.

A Brief History of Background Agents

To appreciate this, I wanted to give an abridged history of background agents. I'm going to skip over the creation which starts with Devon—I feel like Devon was just years ahead of all of this. I'm going to skip towards maybe the beginning of 2025.

I'm sure what a lot of folks in this room have felt pretty similarly. You can imagine yourself back in the day where a lot of this started with maybe goose by block—actually earlier than Claude Code. But it really started with all of us kind of pair programming with these harnesses because they weren't particularly smart, but it was still magic enough to where you could actually go a step up from tab complete from the IDE of the day.

With the release of Sonnet 4, I think we all collectively felt like these things were good enough to walk away from. And then, now that they were good enough to walk away from, there was this collective desire of: well great, now these things can execute asynchronously in the background. They were all sharing me and my laptop. So now I have to figure out how to check back in with them, how to keep myself in the loop when necessary, and I have to figure out how to give them their own machines.

We're going to talk about why giving them their own machines tends to be a hard problem in practice. It sounds very straightforward, but it tends to be hard in practice. When folks started to actually background agents, this is where you started seeing managed offerings from things like Claude Code or codeex that were basically like you were using Claude Code in your terminal, and now you can click around and try to get approximately a Claude Code experience in a UI.

I would say that where we really saw the real success of background agents is through folks like ramp, stripe, works, and others who really led open architectures showing how folks built their own background agents. I want to talk a little bit about what's in common between those architectures and where some of the actual difficulties go. So if you're somebody today who's using Claude Code on a laptop, how do you get that off your laptop? And how do you get many of these things to do work on your behalf in parallel?

Two Real Challenges

There are sort of like two real challenges here. One of them is designing a system that generates dev boxes on demand. I know that a lot of the setup for this talk is around sandboxes, and I'm using the word devbox here pretty intentionally.

A sandbox tends to be something that's pretty bare—let me go spin up a Linux machine on demand that has pretty much nothing attached to it. I'm going to go use that to run untrusted code. Where I want to upgrade our terminology here to dev boxes is: what do I do when I want to run or build untrusted software?

So a sandbox: if I give an agent a task to build me a front end like your hello world Next.js app, if I give it a sandbox that doesn't have any ports open or anything like this, it's going to be unable to actually spin up its own server if it doesn't actually have access to CLI tools to take screenshots. It's not going to be able to take screenshots of that app. It's not going to be able to see its own progress or visually inspect its own work. So there's a lot of things you're not able to do just on a bare Linux machine.

As you try to give agents more complex tasks that aren't just software engineering tasks—maybe they start resembling data or machine learning tasks—in the machine learning case, now you're going to want it to like, "I've got a model that's not performing very well. I might want to retrain this. Maybe I'm serving a model and I'm noticing that I'm not getting very good throughput. So now I actually want to go inspect it and profile it on a GPU."

If I go give it a sandbox that doesn't have access to that hardware, again, now I'm dooming an agent to try to solve a problem without a machine that's not prepped to actually solve it. So designing a system that actually generates these dev boxes on demand in a way that doesn't make you want to tear your hair out as an end user tends to be a really hard system problem.

The second piece is kind of that line between your control plane and your data plane. I think this second point was slightly more controversial. I would say the last 72 hours of Twitter makes me feel like this is now kind of a vanilla point, but we'll get into it nonetheless.

Why This Is Hard

I think it's fair for folks to point out that designing a system that generates dev boxes or sandboxes on demand isn't a particularly new problem. If you've been doing test-driven development for a while, the idea of generating machines to stand up environments feels like a thing that we've been working on for a long time, right? You have your GitHub runners, you're spinning up your Docker Compose, and then you're actually running tests on that. So I think it's a fair criticism. It's not necessarily a new problem. It's just something that requires a new solution because we have a lot of different constraints, and getting this to work for a whole new set of constraints without even that bare problem solved throws us back into the deep end.

I would say the first reason why it might not be obvious why this is a harder problem is that we like to think a lot about CI/CD in terms of jobs. We have this really happy path: first it's going to clone. It's going to assume that that is the state of the world, that that repo is exactly what you want to test against, which is fine. And then it's going to run your test, your deterministic test start to end, and then it's going to tell you whether or not things succeeded or failed.

For dev boxes, for agents, it's a totally different story. You have to go spin up a machine and then it's going to go do work for a while. It's going to come back to you and ask you for input. By the time that you respond to it—like somebody asks you for feedback on a PR and you don't get to it for a few days—we're not going to keep a machine running for a few days. So now we have this notion of how do we have long-running or resumable sessions that need to potentially even outlive the runtime of the box itself.

When you start thinking about sessions over jobs, it starts meaning that you have to think about how to persist the state of a machine. So that means thinking about how do you snapshot the memory of a process in a sandbox or a dev box so that when you resume it, you can resume it from its like mid-thought from an agent or maybe mid-process if your agent's executing something. Or maybe your system is best represented by its file system. So it's great, I created a Next.js app, it compiled it, and then it asked me for feedback on it a few days later. So what I'm going to do is snapshot that sandbox before I throw it away. And when I interact with it next, I need to be able to essentially spin up a new machine, mount that file system in a way that it feels like I'm interacting with the thing that I put to sleep a few days ago.

The one thing I hate about CI/CD also is the feedback loops take so long. Let's say in the fourth step of your CI/CD, something—some dependency is missing. Then you often times fiddle with that step. You're like, "Oh, let me add it," and you run the whole damn thing over again. It doesn't feel like this is a nightmare. You really want like a dev box where you can just do stuff. You might not know what the answer is perfectly.

Jobs Versus Sessions

Jobs versus sessions feels like maybe the first unintuitive thing when you're trying to think about environmental engineering. Like, why is this a new problem for agents? Sometimes when we talk about things for agents, it feels like we rebrand or invent new problems just so that they feel new. So I'm just very cautious sometimes of asking: from first principles, why is designing an environment for an agent actually distinct from a lot of the hard system engineering of designing environments for CI/CD?

I think one of these is the constraints of a job versus a session. The second one is really kind of related to the notion of a session: the environment really is alive. So often if you'll have an agent that checks out a repo and goes and does some work, by the time you come back to it, the underlying repo may have changed. So how do you actually give an agent the ability to resync its environment? Maybe it requires ffmpeg to be in your environment and it wasn't when you started. So there's this question of how do I programmatically spin up new environments or how do I shut down an environment, resume it with a snapshot so now it's actually imbued with the true environment it needs to do its job?

And it's not something that I just want to throw away and rerun. I think that's important. In CI/CD, I think we tend to treat these things as cattle—if you'll forgive the tired phrase—and so we're totally fine with throwing away CI/CD runs and resuming them later. Whereas an agent, it's really hard to throw away its progress because it could have been working for a day or so and you don't want to throw away all that progress.

The last bit is really that this is a totally different security boundary. When you're running CI, it often feels a lot more like running a CI platform, which is like, when you think about the threat or security model, what are you really afraid of?

(capture appears truncated)