5 Layers of AI Agent Architecture: Why Modal’s $750M Raise Matters

There is a problem with AI agents that cannot be resolved by better language models alone, namely, where exactly does all the work take place?

To write a script, a coding agent needs a computing environment to be provided, the code having to be run, the dependencies managed, and the script having to be kept from interfering with the other systems. If you apply that process to thousands of agents working at the same time, the infrastructure becomes just as important as the intelligence.

That helps explain Modal's $750 million Series D, announced on October 14 at a $15.75 billion valuation. The company reports more than $1 billion in annualized run rate, including $300 million from Sandboxes, its isolated execution environments. These figures refer to five architectural requirements that influence how production AI agents are built.

1. Model Inference: The Intelligence Layer

Large language models interpret instructions, reason when faced with problems, and decide what actions to take. In order to serve these models efficiently, it is necessary to have infrastructure that is capable of handling GPU capacity, latency, and varying demand. It is not necessary to use an expensive model for every agent operation, since simple classifications can be carried out using smaller models whereas more complex reasoning demands greater computing power. The decision as to when to use each model has now become an architectural one with direct consequences for cost and performance.

2. Orchestration: Keeping Complex Workflows Together

Unlike ordinary applications which stick to set instructions, agents can choose different tools and methods of execution according to the intermediate results. Orchestration is responsible for carrying out this process by scheduling the tasks, coordinating the tool calls, managing any failures, and determining when to restart the work. If a ten-step process is interrupted at some point in the middle because of a network failure, the agent will not know what has already been carried out. Moreover, it is particularly risky to retry a failed operation when that operation involves payments or changes to a database.

3. Sandboxes: Giving Agents Somewhere to Work

Agents capable of coding regularly produce programs which have to be executed before their results can be relied upon. It is risky to run such code directly in production environments. Sandboxes provide isolated computing environments with controlled access to files, dependencies, and networks. Modal states that it has launched almost three billion sandboxes and supports up to one million concurrent environments; its architectural approach has changed from providing servers for applications to creating disposable computing environments for individual tasks. Isolation still requires careful configuration since sandboxes cannot make up for having unrestricted credentials or for poorly controlled external connections.

4. Memory & State: Making Work Persistent

Agents often carry out their activities over several sessions, tools, and documents; if that context is lost it may result in redundant work or inconsistent decisions. With persistent storage, databases, retrieval systems, and execution checkpoints, workflows can continue after interruptions. Not all data should be included in the model's context window; store transaction records, completed actions, and official business data elsewhere.

5. Security & Observability: Knowing What Actually Happened

When agents access business systems, organizations need visibility into what they do. This involves authenticating agents, enforcing permissions when they carry out actions, keeping a record of tool calls, and documenting the results. In the case of sensitive operations, independent approval may also be required. The execution process is protected by an isolated environment, even though it does not automatically grant authorization for the actions carried out by the agent via connected services.

The Infrastructure Opportunity

The fact that Modal has secured funding indicates that investors believe there is considerable value present beyond the models themselves. Future competition will likely be about how cheaply, securely, and reliably agents can carry out work at scale. Producing an intelligent response is one technical problem, while building the infrastructure that enables millions of agents to act on those responses without losing control is another.