System Design: [Code execution sandbox name]

Reference design for a multi-tenant sandbox service that runs code written or chosen by AI agents and users: the isolation choice (microVMs, gVisor or hardened containers), egress control, the sandbox filesystem, resource quotas, snapshot and restore, warm pools and cold start, secrets handling and abuse prevention. It is the execution backend for the Agent Orchestration Platform System Design Template (system-design/design-agent-orchestration-platform) and for chat assistants with a code tool. The isolation model draws on the published Firecracker design (Agache et al., NSDI 2020). It follows the 12-section structure of the System Design Document Template (system-design/system-design-document) and is pre-filled with a worked design to adapt. Replace every [bracketed] value and every row marked Example; all numbers are illustrative, not benchmarks.

Document control

Approvals

Revision history

Contents

1. Overview
2. Requirements
3. Capacity estimation
4. API design
5. Data model
6. High-level design
7. Detailed design
8. Scalability and performance
9. Reliability, security and operations
10. Trade-offs and alternatives
11. Delivery plan and risks
12. Appendices

1. Overview

1.1 Problem statement

[Example: Agents and the assistant's code tool run Python in Docker containers on shared Kubernetes nodes. The containers share the host kernel, can reach the internal network and the cloud metadata endpoint, and a recent red-team exercise read another tenant's files through a misconfigured volume. Starting a container with the data-science image takes 6 to 10 seconds, so teams keep long-lived containers per user, which leak state between tasks. Two cryptocurrency-mining incidents ran for hours before anyone noticed.]

1.2 Proposed solution

[Example: A sandbox service that gives each session its own microVM with a minimal device model, started from a pre-booted snapshot or taken from a warm pool. Each microVM runs an execution agent that accepts commands and file operations over a vsock channel. The guest has no route to internal networks or metadata endpoints; outbound traffic goes through an egress proxy that applies the tenant's allow-list. Secrets never enter the guest: calls that need credentials go through a broker outside the VM. Every session has CPU, memory, disk, process, time and egress quotas, and abuse detectors watch for mining, scanning and fork bombs. Sessions are destroyed after use; users can persist a workspace as an encrypted snapshot.]

1.3 Decisions requested from reviewers

1.4 45-minute interview path

2. Requirements

2.1 Core functional requirements

2.2 Non-functional requirements

2.3 Below the line (out of scope)

• [Example: Deciding which code to run; that is the agent's job (see Agent Orchestration Platform System Design Template (system-design/design-agent-orchestration-platform))]
• [Example: GPU workloads and model training]
• [Example: Long-running services with inbound traffic (sandboxes accept no inbound connections from the internet)]
• [Example: Interactive desktop and browser automation sessions (separate design)]
• [Item deferred, with the release that will take it]

3. Capacity estimation

3.1 Assumptions

All values are Example planning inputs.

3.2 Concurrency and hosts

3.3 Warm pool

Pool size = arrival rate x replenish time = 87 x 5 s = about 435 ready sandboxes at peak, split across images by demand. Idle sandboxes use memory but almost no CPU: 435 x 1.1 GiB = about 479 GiB in total. Each host keeps 704 - 192 x 1.1 = about 493 GiB of memory unused by active slots, so the pool fits in spare memory spread over the fleet. Consequence: the pool needs no extra hosts if the scheduler counts idle pool members by memory, not by vCPU slots; it must be spread across hosts and zones so one host failure does not empty it.

3.4 Start time

3.5 Storage, egress and logs

3.6 Summary

4. API design

4.1 Endpoints

4.2 Create request example

The caller passes the run and step as the idempotency key, so a retried step gets the same session instead of a second one.

POST /v1/sessions
Authorization: Bearer [platform credential]
Idempotency-Key: run-7f3a:step-12
Content-Type: application/json

{"tenant": "acme", "image": "python-ds@2026-10", "size": "small", "timeout_s": 600, "network": {"mode": "allow_list", "allow": ["pypi.org:443", "files.pythonhosted.org:443"]}, "broker_scopes": ["tickets.read"]}

4.3 Errors

Errors use RFC 9457 problem details. Exit codes of user code are results, not API errors.

5. Data model

5.1 Entities

The control plane stores metadata only. Workspace content lives on the host for the session's lifetime, then in encrypted snapshots if requested.

5.2 Access patterns

5.3 Storage choices

6. High-level design

7. Detailed design

7.1 Session lifecycle

A pooled sandbox is assigned to exactly one session and never returns to the pool. Destroying and replacing is cheaper than proving a used sandbox is clean.

7.2 Deep dive 1: isolation boundary

Choice: C for all untrusted code. B where hosts lack virtualisation support (for example some nested environments), with a stricter egress policy. A only for first-party code that never runs model-chosen input.

Reason: the code is attacker-influenced by design (prompt injection can choose it), and tenants share hosts. A per-session guest kernel removes the shared-kernel failure mode of containers, and the microVM's small device model narrows what the guest can reach on the host.

Defence in depth around the VMM:

• Run each VMM process in its own jail: separate user, chroot, cgroups, seccomp filters and no capabilities beyond what it needs.
• Pin VMM and guest kernel versions; patch on a defined schedule with a fast path for virtualisation security advisories.
• Expose only virtio network, block storage and vsock to the guest.

How real systems do it: Firecracker (paraphrased)

The Firecracker paper describes the virtual machine monitor that AWS built for Lambda and Fargate after concluding that containers alone did not give the isolation needed to run many customers' code on shared hardware, while traditional VMs were too heavy. Firecracker uses Linux KVM but replaces the usual emulator with a small monitor written in Rust that offers only a few emulated devices, keeps the per-VM memory overhead and start time low enough for function workloads, and runs each monitor inside a further sandbox (the "jailer") using standard Linux isolation features, so a guest that escapes the VM still lands in a restricted process. Source: Agache et al., "Firecracker: Lightweight Virtualization for Serverless Applications", 17th USENIX Symposium on Networked Systems Design and Implementation (NSDI 2020). This box summarises; read the paper for design detail and figures.

7.3 Deep dive 2: warm pools, cold start and snapshot restore

Choice: C backed by D: pool members are themselves created by restoring the image's base snapshot; on a pool miss the scheduler restores directly. B is not allowed.

Reason: NFR-01 needs sub-second starts, and B trades isolation for speed, which defeats the purpose of the service. Pool size follows 3.3 and is adjusted per image from demand.

Snapshot uniqueness rules: the Firecracker snapshot documentation warns that resuming the same snapshot more than once is a security concern, because random seeds, identifiers and tokens captured in memory are duplicated across clones. Therefore:

Workspace snapshots (FR-06) store only the workspace disk, not memory, unless the tenant explicitly needs process state; this keeps them smaller and avoids the cloning risks above.

7.4 Deep dive 3: egress control and secrets

Choice: C. The guest's only route is to the egress proxy; the host firewall drops everything else, including private ranges and cloud metadata addresses.

Reason: egress is the exfiltration path for data the agent was given and the attack path for abuse. A proxy gives one place to enforce policy and one log to investigate.

Secrets: the guest never receives long-lived credentials. When code must call an approved service, the execution agent asks the credential broker (outside the VM) over vsock; the broker checks the session's broker_scopes, performs the call with a short-lived token and returns only the response. Environment variables, files and images are scanned so no secret is baked into a base snapshot. Permission scoping follows the Agent Tool and Permission Design Template (ai-data/agent-tool-permission-design).

7.5 Deep dive 4: resource quotas and abuse prevention

Detection must reach a decision within 60 s (NFR-04); responses are automatic and reviewed afterwards.

8. Scalability and performance

Start-time budget (NFR-01, p95 300 ms) on the restore path: admission and policy 20 ms, placement 20 ms, snapshot restore 150 ms, network and vsock setup 40 ms, health check 30 ms, headroom 40 ms.

9. Reliability, security and operations

9.1 Failure modes

9.2 Security

Mapping to the OWASP Top 10 for Agentic Applications 2026 (OWASP GenAI Security Project); confirm entry names against the current list.

Run the full analysis with the Agentic AI Threat Model Template (ai-data/agentic-threat-model), and align network rules with the Zero Trust Architecture Plan Template (security/zero-trust-architecture).

9.3 Observability

9.4 Cost estimate

10. Trade-offs and alternatives

11. Delivery plan and risks

11.1 Milestones

11.2 Rollout and rollback

[Example: Callers switch per tool behind a flag; the container path stays available for first-party code only. Image and kernel updates roll out to one zone first; rollback is a pool rebuild from the previous base snapshot.]

11.3 Risks

12. Appendices

12.1 Traceability

12.2 Open questions

12.3 Glossary

12.4 References

Standard and licence note

Based on the published Firecracker design (Agache et al., NSDI 2020) and the Firecracker and gVisor project documentation, RFC 9457, and the OWASP Top 10 for Agentic Applications 2026 by the OWASP GenAI Security Project (CC BY-SA 4.0; entry names are attributed to OWASP, and any adapted OWASP material you redistribute must carry the same licence under its ShareAlike term). The paper is paraphrased and cited as inspiration only; no figures are reproduced. The text is original; isolation technologies and their security advisories change often, so check the current official documentation before approval. Template licence: MIT.