# System Design: [Code execution sandbox name]

> Reference design for a multi-tenant sandbox service that runs code written
> or chosen by AI agents and users: the isolation choice (microVMs, gVisor or
> hardened containers), egress control, the sandbox filesystem, resource
> quotas, snapshot and restore, warm pools and cold start, secrets handling
> and abuse prevention. It is the execution backend for the Agent
> Orchestration Platform System Design Template
> (`system-design/design-agent-orchestration-platform`) and for chat
> assistants with a code tool. The isolation model draws on the published
> Firecracker design (Agache et al., NSDI 2020). It follows the 12-section
> structure of the System Design Document Template
> (`system-design/system-design-document`) and is pre-filled with a worked
> design to adapt. Replace every `[bracketed]` value and every row marked
> *Example*; all numbers are illustrative, not benchmarks.

<!--
Guidance: How to use this template
- Assume the code is hostile. Agents can be steered by prompt injection into
  running attacker-chosen code, so "our own agent wrote it" is not a trust
  boundary (OWASP ASI05, Unexpected Code Execution).
- Choose the isolation boundary first (7.2); quotas, networking, start time
  and cost all follow from it.
- Treat network egress and secrets as the main data-loss paths, not the
  filesystem (7.4).
- Measure start time, density and escape-hardening on your own hosts and
  kernel versions; vendor and paper figures do not transfer.
-->

## Document control

<!-- Guidance: A design is a proposal until approved. Record the isolation technology and the threat model it was approved against. -->

| Field | Value |
|---|---|
| Document ID | [SDD-SBX-001] |
| System / feature | [Agent code execution sandbox] |
| Version | [0.1] |
| Status | [Draft / In review / Approved / Implemented / Superseded] |
| Owner | [@owner] |
| Approvers / reviewers | [@reviewer-1], [@reviewer-2] |
| Last updated | YYYY-MM-DD |
| Classification | [Internal / Confidential] |
| Isolation technology | [microVM (VMM name and version) / gVisor / hardened containers] |
| Host kernel and hardware | [Kernel version, CPU family, virtualisation support] |
| Related | [Agentic threat model], [Agent tool permission design], [Zero trust plan], [Incident response plan] |

### Approvals

| Role | Name | Decision | Date |
|---|---|---|---|
| Tech lead, sandbox | [@name] | [Approved / Changes requested] | YYYY-MM-DD |
| Staff / principal engineer | [@name] | [Decision] | YYYY-MM-DD |
| Security architecture | [@name] | [Decision] | YYYY-MM-DD |
| SRE / operations | [@name] | [Decision] | YYYY-MM-DD |
| Trust and safety | [@name] | [Decision] | YYYY-MM-DD |

## Revision history

<!-- Guidance: Record design changes, not wording changes. -->

| Version | Date | Author | Change summary |
|---|---|---|---|
| 0.1 | YYYY-MM-DD | [@name] | First draft for design review |
| 0.2 | YYYY-MM-DD | [@name] | *Example:* Moved from containers to microVMs for untrusted code |
| 0.3 | YYYY-MM-DD | [@name] | *Example:* Added snapshot restore with post-restore reseeding |
| 1.0 | YYYY-MM-DD | [@name] | *Example:* Approved |

## Contents

<!-- Guidance: Keep aligned with the headings. -->

1. [Overview](#1-overview)
2. [Requirements](#2-requirements)
3. [Capacity estimation](#3-capacity-estimation)
4. [API design](#4-api-design)
5. [Data model](#5-data-model)
6. [High-level design](#6-high-level-design)
7. [Detailed design](#7-detailed-design)
8. [Scalability and performance](#8-scalability-and-performance)
9. [Reliability, security and operations](#9-reliability-security-and-operations)
10. [Trade-offs and alternatives](#10-trade-offs-and-alternatives)
11. [Delivery plan and risks](#11-delivery-plan-and-risks)
12. [Appendices](#12-appendices)

## 1. Overview

<!-- Guidance: One page a busy reviewer can read alone: problem, solution in a paragraph, decisions requested. -->

### 1.1 Problem statement

[*Example:* Agents and the assistant's code tool run Python in Docker
containers on shared Kubernetes nodes. The containers share the host kernel,
can reach the internal network and the cloud metadata endpoint, and a
recent red-team exercise read another tenant's files through a misconfigured
volume. Starting a container with the data-science image takes 6 to 10
seconds, so teams keep long-lived containers per user, which leak state
between tasks. Two cryptocurrency-mining incidents ran for hours before
anyone noticed.]

### 1.2 Proposed solution

[*Example:* A sandbox service that gives each session its own microVM with a
minimal device model, started from a pre-booted snapshot or taken from a warm
pool. Each microVM runs an execution agent that accepts commands and file
operations over a vsock channel. The guest has no route to internal networks
or metadata endpoints; outbound traffic goes through an egress proxy that
applies the tenant's allow-list. Secrets never enter the guest: calls that
need credentials go through a broker outside the VM. Every session has CPU,
memory, disk, process, time and egress quotas, and abuse detectors watch for
mining, scanning and fork bombs. Sessions are destroyed after use; users can
persist a workspace as an encrypted snapshot.]

### 1.3 Decisions requested from reviewers

| ID | Question | Recommended option | Section |
|---|---|---|---|
| Q-01 | *Example:* Isolation boundary for untrusted code? | microVM per session; gVisor as the fallback where nested virtualisation is unavailable | 7.2 |
| Q-02 | *Example:* Start-time strategy? | Warm pool per image, snapshot restore behind it, cold boot as last resort | 7.3 |
| Q-03 | *Example:* Network default? | No network; opt-in egress through an allow-list proxy | 7.4 |
| Q-04 | *Example:* Secrets in the sandbox? | Never; a credential broker outside the guest performs authenticated calls | 7.4 |
| Q-05 | *Example:* Reuse a sandbox across users or tasks? | Never across tenants or users; within one task only | 7.3 |
| [Q-nn] | [Question] | [Option] | [Section] |

### 1.4 45-minute interview path

<!-- Guidance: Use this order for an interview or a time-boxed review. Pick one deep dive. -->

| Minutes | Topic | What to show |
|---|---|---|
| 0 to 5 | Scope | Threat: hostile code chosen by an attacker through the agent |
| 5 to 10 | Requirements | Core list (2.1), start time and isolation (2.2), below the line (2.3) |
| 10 to 15 | Estimates | Concurrent sandboxes, hosts, warm pool size (3.2 to 3.4) with consequences |
| 15 to 20 | API | Create session, execute, files, snapshot, destroy (4) |
| 20 to 28 | High-level design | Diagram (6), session lifecycle (7.1) |
| 28 to 40 | One deep dive | Isolation choice (7.2) for security interviews; warm pools and snapshots (7.3) for performance interviews; egress and secrets (7.4) for data-loss questions |
| 40 to 45 | Wrap-up | Failure modes (9.1), 10x row (10) |

## 2. Requirements

<!-- Guidance: Split core requirements from what is explicitly below the line. Give every quality a number. -->

### 2.1 Core functional requirements

| ID | Requirement | Priority |
|---|---|---|
| FR-01 | *Example:* Create a session from an approved image (Python data science, Node.js, shell) | Must |
| FR-02 | *Example:* Execute commands and code; stream stdout and stderr; return exit codes | Must |
| FR-03 | *Example:* Upload and download files within a per-session workspace | Must |
| FR-04 | *Example:* Optional network egress restricted to an allow-list per tenant and per session | Must |
| FR-05 | *Example:* Quotas on CPU, memory, disk, processes, wall-clock time and egress bytes | Must |
| FR-06 | *Example:* Persist and restore a workspace snapshot on request | Should |
| FR-07 | *Example:* Authenticated calls to approved services without exposing credentials to the guest | Should |
| FR-08 | *Example:* Kill a session immediately on abuse detection or tenant request | Must |
| FR-09 | *Example:* Audit record of every session: who, image, commands, egress destinations, resource use | Must |
| [FR-nn] | [Requirement] | [Must / Should / Could] |

### 2.2 Non-functional requirements

| ID | Quality | Target | Measured as |
|---|---|---|---|
| NFR-01 | Session start | *Example:* p95 at or below 300 ms from request to ready | Service timing |
| NFR-02 | Isolation | *Example:* No guest-to-host or cross-session access in red-team and fuzzing campaigns | Security test results |
| NFR-03 | Egress | *Example:* Zero connections to destinations outside the allow-list | Proxy logs and flow logs |
| NFR-04 | Abuse detection | *Example:* Mining or scanning sessions killed within 60 s | Detector timing in drills |
| NFR-05 | Availability | *Example:* 99.9% monthly for session creation | SLO |
| NFR-06 | Density | *Example:* At least 150 active sessions per host at the default size | Capacity test |
| NFR-07 | Cleanup | *Example:* All session data destroyed within 5 min of session end unless snapshotted | Host audit |
| [NFR-nn] | [Quality] | [Target] | [Measurement] |

### 2.3 Below the line (out of scope)

- [*Example:* Deciding which code to run; that is the agent's job (see Agent Orchestration Platform System Design Template (`system-design/design-agent-orchestration-platform`))]
- [*Example:* GPU workloads and model training]
- [*Example:* Long-running services with inbound traffic (sandboxes accept no inbound connections from the internet)]
- [*Example:* Interactive desktop and browser automation sessions (separate design)]
- [Item deferred, with the release that will take it]

## 3. Capacity estimation

<!-- Guidance: Every number must end in a consequence. Show inputs, formulas and results. -->

### 3.1 Assumptions

All values are *Example* planning inputs.

| Input | Value | Source |
|---|---|---|
| Sessions per day | 3 million | Agent and assistant forecasts |
| Peak-to-average ratio | 2.5 | Hourly pattern |
| Mean active session lifetime | 90 s | Pilot |
| Default sandbox size | 2 vCPU, 1 GiB memory, 5 GiB disk | Product decision |
| Per-sandbox memory overhead (VMM, agent) | 0.1 GiB | Host measurement |
| Host | 96 vCPU, 768 GiB memory, 64 GiB reserved for the host | Hardware |
| CPU oversubscription | 4 to 1 (bursty workloads) | Pilot utilisation |
| Memory oversubscription | None | Policy |
| Pool replenish time (restore plus health check) | 5 s | Pilot |
| Sessions persisted as snapshots | 1%, 200 MB compressed each | Pilot |
| Sessions using network egress | 30%, 5 MB each | Pilot |

### 3.2 Concurrency and hosts

| Quantity | Calculation | Result | Consequence |
|---|---|---|---|
| Average arrivals per second | 3,000,000 / 86,400 | about 34.7 | |
| Peak arrivals per second | 34.7 x 2.5 | about 87 | Scheduler places about 87 sessions per second at peak |
| Peak concurrent sessions | 87 x 90 s | about 7,830 | |
| Slots per host by CPU | 96 x 4 / 2 | 192 | |
| Slots per host by memory | (768 - 64) / 1.1 | 640 | CPU, not memory, limits density, so memory can hold the warm pool (3.3) |
| Hosts at peak, full load | 7,830 / 192 | 40.8, so 41 | |
| Hosts with 80% target and 3 zones | 41 / 0.8 = 51.25, rounded to 54 (18 per zone) | 54 | Losing a zone leaves 36 hosts (6,912 slots, 88% of peak): shed the free tier during a zone loss, or buy 63 hosts for full N+1 |

### 3.3 Warm pool

Pool size = arrival rate x replenish time = 87 x 5 s = about 435 ready
sandboxes at peak, split across images by demand. Idle sandboxes use memory
but almost no CPU: 435 x 1.1 GiB = about 479 GiB in total. Each host keeps
704 - 192 x 1.1 = about 493 GiB of memory unused by active slots, so the pool
fits in spare memory spread over the fleet. **Consequence:** the pool needs no
extra hosts if the scheduler counts idle pool members by memory, not by vCPU
slots; it must be spread across hosts and zones so one host failure does not
empty it.

### 3.4 Start time

| Path | *Example* p95 | Share of starts | Consequence |
|---|---|---|---|
| Warm pool assignment | 30 ms | 95% target | Main path; meets NFR-01 |
| Snapshot restore | 150 ms | Pool misses | Fallback still meets NFR-01 |
| Cold boot of image | 1.5 s | Rare (new images) | Pre-build snapshots when an image is published |

### 3.5 Storage, egress and logs

| Quantity | Calculation | Result | Consequence |
|---|---|---|---|
| Snapshots per day | 3 million x 1% x 200 MB | 6 TB | |
| Snapshots, 7-day retention | 6 TB x 7 | 42 TB | Expire by default; deduplicate against the base image |
| Egress per day | 3 million x 30% x 5 MB | 4.5 TB | |
| Average egress rate | 4.5 TB / 86,400 s | about 52 MB/s (about 417 Mbit/s) | |
| Peak egress rate | 52 MB/s x 2.5 | about 130 MB/s (about 1 Gbit/s) | A small proxy fleet per region is enough; the hard part is policy, not bandwidth |
| Session output logs | 3 million x 50 KB | 150 GB per day | Truncate output per session; keep audit metadata longer than output |

### 3.6 Summary

| Quantity | Estimate | Design consequence |
|---|---|---|
| Peak concurrent sessions | about 7,830 | 54 hosts across 3 zones |
| Density | 192 per host (CPU-bound) | Oversubscribe CPU, never memory |
| Warm pool | about 435 sandboxes | Fits in spare memory; spread across hosts |
| Snapshots | 42 TB at 7-day retention | Opt-in, expiring, encrypted |
| Egress | about 1 Gbit/s at peak | Per-tenant allow-lists in a proxy |

## 4. API design

<!-- Guidance: Define the contract: resources, streaming output, limits, errors. The API is called by platforms, not end users. -->

### 4.1 Endpoints

| Method and path | Purpose |
|---|---|
| `POST /v1/sessions` | Create a session: image, size, network policy, time limit, optional snapshot to restore |
| `POST /v1/sessions/{id}/exec` | Run a command or code; output streamed as server-sent events |
| `PUT /v1/sessions/{id}/files/{path}` | Upload a file into the workspace |
| `GET /v1/sessions/{id}/files/{path}` | Download a file |
| `POST /v1/sessions/{id}/snapshot` | Persist the workspace (encrypted, expiring) |
| `DELETE /v1/sessions/{id}` | Destroy the session now |
| `GET /v1/sessions/{id}/usage` | CPU seconds, peak memory, egress bytes, destinations |

### 4.2 Create request example

The caller passes the run and step as the idempotency key, so a retried step
gets the same session instead of a second one.

```http
POST /v1/sessions
Authorization: Bearer [platform credential]
Idempotency-Key: run-7f3a:step-12
Content-Type: application/json

{"tenant": "acme", "image": "python-ds@2026-10", "size": "small", "timeout_s": 600, "network": {"mode": "allow_list", "allow": ["pypi.org:443", "files.pythonhosted.org:443"]}, "broker_scopes": ["tickets.read"]}
```

### 4.3 Errors

| Status | Problem type | Meaning |
|---|---|---|
| 400 | `image-not-approved` | Image not on the tenant's approved list |
| 403 | `egress-not-allowed` | Requested destination outside the tenant's policy |
| 429 | `tenant-concurrency-exceeded` | Tenant at its concurrent session limit |
| 503 | `capacity-unavailable` | No slot and no pool member; `Retry-After` set |
| 410 | `session-terminated` | Session ended by timeout, quota or abuse detector; reason given |

Errors use RFC 9457 problem details. Exit codes of user code are results, not
API errors.

## 5. Data model

<!-- Guidance: Choose storage by access pattern. List each query, the key that serves it and why. -->

### 5.1 Entities

The control plane stores metadata only. Workspace content lives on the host
for the session's lifetime, then in encrypted snapshots if requested.

```mermaid
erDiagram
  TENANT ||--o{ POLICY : has
  TENANT ||--o{ SESSION : runs
  IMAGE ||--o{ SESSION : "starts from"
  IMAGE ||--o{ BASE_SNAPSHOT : "pre-booted as"
  SESSION ||--o{ EXEC_RECORD : logs
  SESSION ||--o{ EGRESS_RECORD : logs
  SESSION ||--o| WORKSPACE_SNAPSHOT : "persisted as"
  HOST ||--o{ SESSION : hosts
  SESSION {
    string session_id PK
    string tenant_id FK
    string image_id FK
    string host_id FK
    string state
    datetime expires_at
  }
  POLICY {
    string tenant_id PK
    string allow_list
    int max_concurrent
    string sizes_allowed
  }
  EXEC_RECORD {
    string session_id PK
    int seq PK
    string command_hash
    int exit_code
    int cpu_ms
  }
  EGRESS_RECORD {
    string session_id PK
    string destination PK
    int bytes
    string decision
  }
  WORKSPACE_SNAPSHOT {
    string snapshot_id PK
    string session_id FK
    string object_key
    datetime expires_at
  }
```

### 5.2 Access patterns

| ID | Query | Key or index that serves it | Partition key and rationale |
|---|---|---|---|
| AP-01 *(Example)* | Find a free pool member for an image in a zone | Pool index `(image_id, zone)` in the scheduler's memory | Scheduler-local; rebuilt from host heartbeats |
| AP-02 *(Example)* | Route an exec call to the host of a session | `SESSION` by `session_id`, cached at the API tier | Session ID carries a host hint to avoid a lookup |
| AP-03 *(Example)* | Count a tenant's active sessions | Counter per `tenant_id` | Admission check on every create |
| AP-04 *(Example)* | Sessions past their deadline | Index `(host_id, expires_at)` | Each host reaper reads its own range |
| AP-05 *(Example)* | Everything one session did, for an investigation | `EXEC_RECORD` and `EGRESS_RECORD` by `session_id` | Append-only, partitioned by day for retention |
| AP-06 *(Example)* | Tenants whose sessions contacted a destination | Index on `destination` in the audit warehouse | Incident response after a malicious domain is found |

### 5.3 Storage choices

| Data | Store | Notes |
|---|---|---|
| Policies, images, tenants | Relational database | Changes audited |
| Session state | Replicated key-value store plus host heartbeats | Hosts are the source of truth for running sessions |
| Exec and egress records | Event stream to an audit warehouse | Command text hashed; output truncated |
| Base snapshots | Object storage, replicated to every host's local disk | Built when an image is published |
| Workspace snapshots | Object storage, encrypted per tenant, expiring | Opt-in only |

## 6. High-level design

<!-- Guidance: One diagram with every component on the session path and where state lives. -->

```mermaid
flowchart LR
  caller([Agent platform or assistant]) --> api[Sandbox API]
  api --> sched[Scheduler and pool manager]
  sched --> h1[Host agent]
  subgraph host [Sandbox host]
    h1 --> vm1[microVM: execution agent]
    h1 --> vm2[microVM: pool member]
    vm1 -. vsock .- h1
  end
  vm1 --> tap[Per-VM network device]
  tap --> egress[Egress proxy: allow-list, DNS, byte quotas]
  egress --> internet([Allowed destinations])
  h1 --> broker[Credential broker]
  broker --> svc([Approved internal services])
  h1 --> det[Abuse detectors]
  det --> sched
  api --> audit[[Audit events]]
  sched --> snaps[(Base and workspace snapshots)]
```

| Component | Responsibility | Scales by |
|---|---|---|
| Sandbox API | Auth, policy checks, idempotency, routing exec calls | Requests per second |
| Scheduler and pool manager | Placement, warm pools per image and zone, admission | Sessions per second |
| Host agent | Start, restore, snapshot and destroy microVMs; enforce cgroups and disk quotas | One per host |
| Execution agent (in guest) | Run commands, stream output, file operations over vsock | One per sandbox |
| Egress proxy | Allow-list, DNS control, byte quotas, logging | Egress bandwidth |
| Credential broker | Perform authenticated calls on behalf of the session with scoped tokens | Calls per second |
| Abuse detectors | CPU pattern, connection pattern and process-count signals | Hosts |

## 7. Detailed design

<!-- Guidance: Go deep on the hard problems; each deep dive lists options, the choice and the reason. -->

### 7.1 Session lifecycle

```mermaid
stateDiagram-v2
  [*] --> pooled: restored from base snapshot
  pooled --> assigned: create session
  assigned --> running: first exec
  running --> running: exec, file operations
  running --> snapshotting: snapshot requested
  snapshotting --> running
  running --> terminating: delete, timeout, quota or abuse
  assigned --> terminating: idle timeout
  terminating --> [*]: VM destroyed, disk and memory wiped
```

A pooled sandbox is assigned to exactly one session and never returns to the
pool. Destroying and replacing is cheaper than proving a used sandbox is clean.

### 7.2 Deep dive 1: isolation boundary

| Option | How it isolates | Strengths | Weaknesses |
|---|---|---|---|
| A. Hardened containers (namespaces, cgroups, seccomp, dropped capabilities, user namespaces) | Shared host kernel with a filtered system call surface | Fast start, high density, simple tooling | One kernel bug can break isolation for every tenant on the host |
| B. gVisor (user-space application kernel) | System calls served by the gVisor Sentry, which runs under a restricted seccomp profile; file access mediated by a separate Gofer process | Much smaller host kernel surface; OCI runtime (`runsc`) fits container tooling | Higher per-system-call overhead; some system calls and `/proc` or `/sys` files not implemented |
| C. microVM per session (for example Firecracker, or Kata Containers with a VMM) | Hardware virtualisation; guest kernel per sandbox; minimal device model | Strong boundary; per-session kernel | Needs virtualisation support on hosts; more memory per sandbox; snapshot and image tooling to build |
| D. WebAssembly runtime | Language-level sandbox | Very fast start | Cannot run arbitrary native packages that agents install |

**Choice:** C for all untrusted code. B where hosts lack virtualisation
support (for example some nested environments), with a stricter egress
policy. A only for first-party code that never runs model-chosen input.

**Reason:** the code is attacker-influenced by design (prompt injection can
choose it), and tenants share hosts. A per-session guest kernel removes the
shared-kernel failure mode of containers, and the microVM's small device model
narrows what the guest can reach on the host.

Defence in depth around the VMM:

- Run each VMM process in its own jail: separate user, chroot, cgroups,
  seccomp filters and no capabilities beyond what it needs.
- Pin VMM and guest kernel versions; patch on a defined schedule with a fast
  path for virtualisation security advisories.
- Expose only virtio network, block storage and vsock to the guest.

> **How real systems do it: Firecracker (paraphrased)**
>
> The Firecracker paper describes the virtual machine monitor that AWS built
> for Lambda and Fargate after concluding that containers alone did not give
> the isolation needed to run many customers' code on shared hardware, while
> traditional VMs were too heavy. Firecracker uses Linux KVM but replaces the
> usual emulator with a small monitor written in Rust that offers only a few
> emulated devices, keeps the per-VM memory overhead and start time low
> enough for function workloads, and runs each monitor inside a further
> sandbox (the "jailer") using standard Linux isolation features, so a guest
> that escapes the VM still lands in a restricted process. Source: Agache et
> al., "Firecracker: Lightweight Virtualization for Serverless Applications",
> 17th USENIX Symposium on Networked Systems Design and Implementation (NSDI
> 2020). This box summarises; read the paper for design detail and figures.

### 7.3 Deep dive 2: warm pools, cold start and snapshot restore

| Option | Strengths | Weaknesses |
|---|---|---|
| A. Cold boot per session | Clean state; simple | Too slow for interactive use (3.4) |
| B. Reuse a sandbox across sessions after cleanup | Fast | Cleanup must be perfect; cross-session leakage risk |
| C. Warm pool of pre-booted sandboxes, one use each | Fast; clean | Memory held by idle members; pool sizing |
| D. Restore from a pre-booted base snapshot | Fast enough for misses; cheap to keep | Restored clones share state captured at snapshot time |

**Choice:** C backed by D: pool members are themselves created by restoring
the image's base snapshot; on a pool miss the scheduler restores directly.
B is not allowed.

**Reason:** NFR-01 needs sub-second starts, and B trades isolation for speed,
which defeats the purpose of the service. Pool size follows 3.3 and is
adjusted per image from demand.

Snapshot uniqueness rules: the Firecracker snapshot documentation warns that
resuming the same snapshot more than once is a security concern, because
random seeds, identifiers and tokens captured in memory are duplicated across
clones. Therefore:

| Rule | Purpose |
|---|---|
| Base snapshots are taken before any secret, token or tenant data exists in the guest | Nothing sensitive is cloned |
| Use guest kernels that reseed their random generator after restore (VM generation ID support), and reseed user-space generators in the execution agent on resume | Distinct randomness per clone |
| Reset the guest wall clock after restore | Correct timestamps and certificate checks |
| Regenerate guest identifiers (hostname, machine ID) on resume | No duplicate identities |
| Encrypt and authenticate snapshot files; the VMM treats them as trusted input | Prevent tampered snapshots |

Workspace snapshots (FR-06) store only the workspace disk, not memory, unless
the tenant explicitly needs process state; this keeps them smaller and avoids
the cloning risks above.

### 7.4 Deep dive 3: egress control and secrets

| Option | Strengths | Weaknesses |
|---|---|---|
| A. Open internet, block internal ranges | Most packages install | Exfiltration, scanning, abuse from your IP ranges |
| B. No network at all | Strongest | Many agent tasks need package installs or APIs |
| C. No network by default; opt-in allow-list through a proxy that controls DNS and enforces byte quotas | Fits most tasks; every destination logged | Allow-lists need curation; some tools break behind proxies |

**Choice:** C. The guest's only route is to the egress proxy; the host
firewall drops everything else, including private ranges and cloud metadata
addresses.

**Reason:** egress is the exfiltration path for data the agent was given and
the attack path for abuse. A proxy gives one place to enforce policy and one
log to investigate.

| Rule | *Example* value |
|---|---|
| Default | No network |
| Package mirrors | Internal mirror of approved package indexes as the default allow-list entry |
| DNS | Resolved by the proxy only; guest resolver points to it and direct DNS is dropped, which also closes DNS tunnelling as an exfiltration channel |
| Byte quota | 100 MB per session egress (*Example*) |
| Blocked always | Private address ranges, metadata endpoints, mail ports, known mining pools |

Secrets: the guest never receives long-lived credentials. When code must call
an approved service, the execution agent asks the credential broker (outside
the VM) over vsock; the broker checks the session's `broker_scopes`, performs
the call with a short-lived token and returns only the response. Environment
variables, files and images are scanned so no secret is baked into a base
snapshot. Permission scoping follows the Agent Tool and Permission Design
Template (`ai-data/agent-tool-permission-design`).

### 7.5 Deep dive 4: resource quotas and abuse prevention

| Resource | Enforcement | *Example* default |
|---|---|---|
| CPU | vCPU count and cgroup CPU quota on the VMM | 2 vCPU, throttled above quota |
| Memory | Fixed guest memory; no balloon growth beyond the size | 1 GiB |
| Disk | Per-session block device of fixed size | 5 GiB |
| Processes | Guest limit on process count | 512 |
| Wall-clock time | Session timeout and per-exec timeout | 10 min session, 2 min per exec |
| Egress | Proxy byte quota and connection rate | 100 MB, 20 new connections per second |
| Tenant concurrency | Admission counter | Set per contract |

| Abuse pattern | Signal | Response |
|---|---|---|
| Cryptocurrency mining | Sustained full CPU, mining pool destinations, known binaries | Kill session, flag tenant, review |
| Network scanning | Many destinations or ports, denied connections | Kill session, block destination pattern |
| Fork bomb or resource exhaustion | Process count at limit, memory pressure | Contained by limits; kill on repeat |
| Hosting malicious content | Unexpected inbound attempts (none allowed), egress to file-sharing sites | Kill, report |

Detection must reach a decision within 60 s (NFR-04); responses are automatic
and reviewed afterwards.

## 8. Scalability and performance

<!-- Guidance: Show how the design meets section 3 and where it breaks first. -->

| Concern | Approach | Limit before redesign |
|---|---|---|
| Placement rate | Scheduler sharded by zone; pool assignment in memory | Scheduler throughput per zone |
| Density | CPU oversubscription, fixed memory | Noisy neighbours on CPU: lower ratio for paid tiers |
| Pool sizing | Per image and zone from arrival rate and replenish time | Many images: pool only the top images, restore the rest |
| Base snapshot distribution | Pre-staged to every host's local disk | Number of images times snapshot size per host |
| Egress | Regional proxy fleet | Policy evaluation cost, not bandwidth |

Start-time budget (NFR-01, p95 300 ms) on the restore path: admission and
policy 20 ms, placement 20 ms, snapshot restore 150 ms, network and vsock
setup 40 ms, health check 30 ms, headroom 40 ms.

## 9. Reliability, security and operations

<!-- Guidance: Name failure modes, threats and the signals operators watch. -->

### 9.1 Failure modes

| ID | Component | Failure | Detection | Blast radius | Mitigation | RTO |
|---|---|---|---|---|---|---|
| FM-01 | Host | Crash or kernel panic | Heartbeat loss | Up to 192 sessions | Sessions fail with `session-terminated`; callers retry on a new session; pools refill elsewhere | 1 min |
| FM-02 | Warm pool | Exhausted at a spike | Pool depth metric | Start time rises to restore path | Restore path; scale pool from arrival rate | Under 5 s |
| FM-03 | Egress proxy | Down | Health checks | Sessions with network | Proxies per zone; sessions without network unaffected; fail closed | 5 min |
| FM-04 | Credential broker | Down | Error rate | Sessions needing authenticated calls | Fail closed; retry from the caller | 10 min |
| FM-05 | VMM or guest kernel | Escape vulnerability disclosed | Security advisories | Potentially the whole fleet | Emergency patch path; drain and rebuild hosts; temporarily disable network | Per advisory, target 24 h |
| FM-06 | Base snapshot | Corrupt or tampered | Checksum and signature check on load | One image | Rebuild from image; keep the previous version | 30 min |
| [FM-nn] | [Component] | [Failure] | [Signal] | [Scope] | [Mitigation] | [Time] |

### 9.2 Security

Mapping to the OWASP Top 10 for Agentic Applications 2026 (OWASP GenAI
Security Project); confirm entry names against the current list.

| ID | Threat (OWASP entry where applicable) | Control |
|---|---|---|
| SEC-01 | Unexpected Code Execution (ASI05): agent runs attacker-chosen code | microVM per session; no secrets in guest; egress allow-list |
| SEC-02 | Tool Misuse and Exploitation (ASI02): sandbox used to attack others | Egress policy, abuse detectors, byte quotas |
| SEC-03 | Identity and Privilege Abuse (ASI03): credentials stolen from the sandbox | Credential broker; short-lived scoped tokens outside the guest |
| SEC-04 | Agentic Supply Chain Vulnerabilities (ASI04): malicious packages installed | Internal package mirror with scanning; approved images |
| SEC-05 | Cross-session data leakage | One-use sandboxes; wipe on destroy; per-tenant snapshot keys |
| SEC-06 | Host compromise via VMM bug | Jailed VMM, minimal devices, patch cadence, host isolation by tier |
| SEC-07 | Agent Goal Hijack (ASI01): command output or downloaded content carries instructions back to the calling agent | Output size-limited and returned as untrusted data; the caller marks it as data, never as instructions, in the model context |
| [SEC-nn] | [Threat] | [Control] |

Run the full analysis with the Agentic AI Threat Model Template
(`ai-data/agentic-threat-model`), and align network rules with the Zero Trust
Architecture Plan Template (`security/zero-trust-architecture`).

### 9.3 Observability

| Signal | Metric | Alert threshold |
|---|---|---|
| Start time | p95 by path (pool, restore, cold) | Above 300 ms for 10 min |
| Pool depth | Ready members per image and zone | Below 20% of target |
| Host density | Active sessions per host | Above 192 or CPU steal above threshold |
| Egress denials | Denied connections per tenant | Spike per tenant |
| Abuse kills | Count by pattern | Any mining or scanning kill pages security |
| Cleanup | Sessions ended but not wiped after 5 min | Any |

### 9.4 Cost estimate

| Item | Unit cost | Quantity | Monthly cost |
|---|---|---|---|
| Sandbox hosts | [USD per host-month] | 54 (or 63 for full N+1) | [USD] |
| Snapshot storage | [USD per TB-month] | about 42 TB | [USD] |
| Egress proxy fleet and bandwidth | [USD per TB] | about 135 TB per month | [USD] |
| Audit warehouse | [USD per TB-month] | [TB] | [USD] |
| **Total** | | | **[USD]** |

## 10. Trade-offs and alternatives

<!-- Guidance: For each major decision list options, the choice, why and what you give up; add what changes at 10x and 0.1x. -->

| Decision | Options | Chosen | Why | What we give up |
|---|---|---|---|---|
| Isolation | Containers; gVisor; microVM; WebAssembly | microVM, gVisor fallback | Hostile code on shared hosts | Density and tooling effort |
| Start time | Cold boot; reuse; warm pool; restore | Warm pool backed by restore | Fast and clean | Memory for idle members; snapshot hygiene |
| Network | Open; none; allow-list proxy | None by default, allow-list opt-in | Exfiltration and abuse control | Some tasks need allow-list work |
| Secrets | In guest; broker | Broker | Guest compromise yields no credentials | Broker must support each service |
| Build or buy | Managed sandbox vendor; self-hosted | [Decision] | [Data location, cost at volume, control] | [Operations or vendor dependency] |
| What changes at 10x (30 million sessions per day, about 78,000 concurrent) | | Bare-metal fleet per region; scheduler per zone with cell-based isolation; snapshot distribution by peer-to-peer or regional cache; dedicated security response rota | About 540 hosts make placement, snapshot distribution and patching the limits | More operational surface and a larger blast radius per advisory |
| What changes at 0.1x (300,000 sessions per day, under 1,000 concurrent) | | A managed sandbox provider or gVisor on a small node pool; egress and secrets rules unchanged | About 6 hosts do not justify an own microVM image and snapshot pipeline | Weaker isolation boundary or vendor dependency |
| [Decision] | [Options] | [Chosen] | [Why] | [Cost] |

## 11. Delivery plan and risks

<!-- Guidance: How the design reaches production safely, and what could stop it. -->

### 11.1 Milestones

| Milestone | Scope | Exit criterion | Target date |
|---|---|---|---|
| M1 | microVM hosts, host agent, cold boot, no network | Red-team and fuzzing campaign meets NFR-02 | YYYY-MM-DD |
| M2 | Base snapshots, warm pools | NFR-01 met at forecast peak | YYYY-MM-DD |
| M3 | Egress proxy, allow-lists, credential broker | NFR-03 audit passes; no secrets in images | YYYY-MM-DD |
| M4 | Quotas and abuse detectors | Mining and scanning drills meet NFR-04 | YYYY-MM-DD |
| M5 | Workspace snapshots, migration of all callers | Old container path switched off | YYYY-MM-DD |

### 11.2 Rollout and rollback

[*Example:* Callers switch per tool behind a flag; the container path stays
available for first-party code only. Image and kernel updates roll out to
one zone first; rollback is a pool rebuild from the previous base snapshot.]

### 11.3 Risks

| ID | Risk | Likelihood | Impact | Mitigation | Owner |
|---|---|---|---|---|---|
| R-01 | *Example:* Hosts without virtualisation support in some regions | Medium | Medium | gVisor fallback with stricter policy | [@name] |
| R-02 | *Example:* Snapshot cloning duplicates secrets or randomness | Medium | High | Rules in 7.3; tests that compare clones | [@name] |
| R-03 | *Example:* Allow-lists too strict, agents fail tasks | High | Medium | Curated mirrors; per-tenant requests with review | [@name] |
| R-04 | *Example:* Escape vulnerability in VMM or kernel | Low | High | Patch path, jail, host tiering | [@name] |
| [R-nn] | [Risk] | [L / M / H] | [L / M / H] | [Mitigation] | [@name] |

## 12. Appendices

<!-- Guidance: Traceability, open questions, glossary and references. -->

### 12.1 Traceability

| Requirement | Design element | Verification |
|---|---|---|
| NFR-01 | 7.3 pools and restore, 8 start budget | Load test at peak arrival rate |
| NFR-02, SEC-01 | 7.2 microVM and jail | Red-team and fuzzing campaign |
| FR-04, NFR-03 | 7.4 egress proxy | Proxy and flow log audit |
| FR-07, SEC-03 | 7.4 credential broker | Image and memory scans for secrets |
| FR-05, NFR-04 | 7.5 quotas and detectors | Abuse drills |
| [ID] | [Element] | [Test] |

### 12.2 Open questions

| ID | Question | Owner | Due |
|---|---|---|---|
| OQ-01 | *Example:* Do paid tenants get dedicated hosts? | [@name] | YYYY-MM-DD |
| OQ-02 | *Example:* Do we support GPU sandboxes, and with which isolation? | [@name] | YYYY-MM-DD |
| [OQ-nn] | [Question] | [@name] | YYYY-MM-DD |

### 12.3 Glossary

| Term | Meaning |
|---|---|
| microVM | Lightweight virtual machine with a minimal device model and its own guest kernel |
| VMM | Virtual machine monitor, the host process that runs a microVM |
| Jailer | Wrapper that confines the VMM process with Linux isolation features |
| Warm pool | Pre-started sandboxes waiting to be assigned, each used once |
| Base snapshot | Saved memory and device state of a freshly booted image, used to start clones quickly |
| vsock | Virtual socket between host and guest that needs no network |
| Egress proxy | Gateway that enforces outbound network policy for sandboxes |

### 12.4 References

| Reference | Use |
|---|---|
| Agache et al., "Firecracker: Lightweight Virtualization for Serverless Applications", NSDI 2020, https://www.usenix.org/conference/nsdi20/presentation/agache | Box in 7.2 (paraphrased) |
| Firecracker documentation, snapshot support, https://github.com/firecracker-microvm/firecracker/blob/main/docs/snapshotting/snapshot-support.md | Uniqueness rules in 7.3 |
| gVisor documentation, https://gvisor.dev/docs/ | Option B in 7.2 |
| OWASP Top 10 for Agentic Applications 2026, https://genai.owasp.org/ (CC BY-SA 4.0) | Threats in 9.2 |
| RFC 9457, Problem Details for HTTP APIs (2023) | Errors in 4.3 |
| Agent Orchestration Platform System Design Template (`system-design/design-agent-orchestration-platform`) | Main caller |
| Agent Tool and Permission Design Template (`ai-data/agent-tool-permission-design`) | Broker scopes |
| Multi-Agent System Design Template (`ai-data/multi-agent-system-design`) | Agent-level controls |

## Standard and licence note

Based on the published Firecracker design (Agache et al., NSDI 2020) and the
Firecracker and gVisor project documentation, RFC 9457, and the OWASP Top 10
for Agentic Applications 2026 by the OWASP GenAI Security Project (CC BY-SA
4.0; entry names are attributed to OWASP, and any adapted OWASP material you
redistribute must carry the same licence under its ShareAlike term). The
paper is paraphrased and cited as inspiration only; no figures are
reproduced. The text is original; isolation technologies and their security
advisories change often, so check the current official documentation before
approval. Template licence: MIT.