# Cloudex documentation > Cloudex is an open-source (MIT) CLI and API that gives coding agents Linux computers, called sandboxes, on Cloudflare Containers in the user's own Cloudflare account. Each sandbox is an Ubuntu machine with a shell, SSH, Docker, Node, Python, Claude Code and Codex. Stopping a sandbox saves its disk and resuming brings it back; sandboxes can be forked, share ports as HTTPS preview links, and show a desktop with Chrome. There is no hosted service: the user deploys the API to their own account and Cloudflare bills them directly. These docs describe Cloudex v0.2.3. The same pages are at https://cloudex.build/docs, and listed in https://cloudex.build/llms.txt. --- # Get started Source: https://cloudex.build/docs > What Cloudex is, what you need, and how to go from installing the CLI to a shell in your first sandbox. Cloudex gives coding agents Linux computers that run in your own Cloudflare account. Each one is called a sandbox: an Ubuntu machine with a shell, SSH, Docker, Node, Python, Claude Code, Codex and OpenCode. Stopping a sandbox saves its disk. Resuming it brings back your files, installed packages and agent logins. There is no Cloudex service in between. `cloudex deploy` installs the API in your Cloudflare account, your code and keys stay there, and Cloudflare bills you directly. Sandboxes can also run on a Linux machine of your own, called a [node](https://cloudex.build/docs/nodes.md), under the same deployment. Cloudex is open source under the MIT license, at [github.com/lucataco/cloudex](https://github.com/lucataco/cloudex). These docs describe version 0.2.3. v0.2.3 makes nodes faster: ssh goes [straight to a node](https://cloudex.build/docs/nodes.md#direct-connections) on your LAN or tailnet, and sandboxes on nodes can be [forked](https://cloudex.build/docs/forks.md#on-a-node). v0.2.2 adds [nodes](https://cloudex.build/docs/nodes.md): sandboxes on a home server or VPS of your own. v0.2 adds a web [dashboard](https://cloudex.build/docs/dashboard.md), OpenCode, keys for six more AI providers, and [access tokens](https://cloudex.build/docs/tokens.md) with sign-in by code. v0.1 commands, API routes and SDK calls work unchanged. ## What you need - A Cloudflare account on **Workers Paid** ($5 a month). Containers need it. The account also needs a workers.dev subdomain; if it has none, open Workers & Pages in the Cloudflare dashboard once to choose one. - An OpenSSH client. `cloudex ssh`, `prompt` and `attach` use it. - Optional: a domain in the same Cloudflare account, for [preview links](https://cloudex.build/docs/connect.md#preview-links). - Optional: R2 turned on for the account, for [backups](https://cloudex.build/docs/sandboxes.md#backups). The free tier is enough. - Optional: a Linux machine with Docker and root, to run sandboxes on as a [node](https://cloudex.build/docs/nodes.md). ## Quickstart 1. Install the CLI. On macOS and Linux: ```sh curl -fsSL https://cloudex.build/install | sh ``` On Windows, in PowerShell: ```powershell irm https://cloudex.build/install.ps1 | iex ``` 2. Deploy Cloudex to your Cloudflare account. It asks for a Cloudflare API token and prints a link that creates one with the right permissions. ```sh cloudex deploy --preview-domain example.com ``` Use a domain you have in the same account, or leave out `--preview-domain` if you don't want preview links. The first deploy takes a few minutes. See [Deploy and update](https://cloudex.build/docs/deploy.md). 3. Create a sandbox and open a shell in it: ```sh cloudex new --name web cloudex ssh ``` The new sandbox becomes the current one, so later commands act on it when you don't name a sandbox. 4. Stop it when you're done. Its disk is saved and its container stops. ```sh cloudex stop ``` `cloudex resume` starts it again from the save. A sandbox also stops by itself after 30 minutes without activity; see [Idle stop](https://cloudex.build/docs/sandboxes.md#idle-stop). ## Install options The macOS and Linux installer puts `cloudex` in `/usr/local/bin` when it can write there, and in `~/.local/bin` otherwise. It supports x86-64 and ARM64. Two environment variables change what it does: | Variable | Effect | |---|---| | `CLOUDEX_VERSION` | Install a given release, such as `v0.1.17`, instead of the latest | | `CLOUDEX_INSTALL_DIR` | Install into this directory | ```sh curl -fsSL https://cloudex.build/install | CLOUDEX_VERSION=v0.1.17 sh ``` The Windows installer also reads `CLOUDEX_VERSION`. To update an installed CLI, run `cloudex update`. ## What goes into your account `cloudex deploy` sets up these pieces in your Cloudflare account: | Piece | What it does | |---|---| | Worker `cloudex-api` | The API at `https://cloudex-api..workers.dev`, SSH and port connections, and preview links | | A Durable Object per sandbox | Starts, stops, saves and times out its container, and connects you to it | | A container per sandbox | The Ubuntu machine itself, from the `cloudex-base` image copied into your account's registry. A sandbox on a [node](https://cloudex.build/docs/nodes.md) runs on the node instead. | | A Durable Object per node | Keeps the connection to a node's agent, when you add nodes | | D1 database `cloudex` | The list of sandboxes | | R2 bucket `cloudex-backups` | Backups of each sandbox's `/home`, when R2 is on | | A proxied `*.` DNS record and route | Preview links, when you give a preview domain | ## What's in a sandbox The default image is Ubuntu 24.04 for x86-64 with: - Node.js 22, Python 3 with pip and venv, and build-essential - git, tmux, rsync, curl, wget, jq, zip, vim-tiny, htop and common network tools - Docker Engine with the buildx and compose plugins - Claude Code 2.1.318, Codex 0.166.0, OpenCode 2.2.5 and the GitHub CLI (`gh`). The agents stay at the versions this release was tested with, and Claude Code's and OpenCode's updaters are off. - A user named `cloudex` (uid 1000) with passwordless sudo, in the `docker` group, with its home in `/home/cloudex` You can install anything else with `apt`, `npm` or `pip`. A save keeps it. ## Next - [Deploy and update](https://cloudex.build/docs/deploy.md): tokens, preview domains, a second computer and updates - [Nodes](https://cloudex.build/docs/nodes.md): sandboxes on a Linux machine of your own - [Sandboxes](https://cloudex.build/docs/sandboxes.md): sizes, idle stop, saves and backups - [Settings](https://cloudex.build/docs/settings.md): environment variables, files and a repo for each sandbox - [Connect](https://cloudex.build/docs/connect.md): SSH, editors, `exec` and preview links - [Provider keys](https://cloudex.build/docs/keys.md): Anthropic and OpenAI keys for agents that stay outside sandboxes - [Limits](https://cloudex.build/docs/limits.md): what sandboxes can't do --- # Deploy and update Source: https://cloudex.build/docs/deploy > Install Cloudex in your Cloudflare account with cloudex deploy, use it from a second computer, check the setup, and update the CLI and the deployment. ## Deploy ```sh cloudex deploy --preview-domain example.com ``` `cloudex deploy` reads a Cloudflare API token from `$CLOUDFLARE_API_TOKEN`, or asks you to paste one. It prints a link that opens the Cloudflare dashboard with the token's permissions filled in; `cloudex deploy --help` prints the same link. The token needs these account permissions: | Permission | Used for | |---|---| | Workers Scripts: Edit | The API Worker | | Workers Containers: Edit | The sandbox image and containers | | D1: Edit | The list of sandboxes | | Account Settings: Read | Finding the account | | Workers Routes: Edit | Preview links | | DNS: Edit | Preview links | | Zone: Read | Preview links | | Workers R2 Storage: Edit | Backups | If the token lacks a permission that a step needs, deploy stops and names the permission to add. Then it works through these steps: 1. Checks the token, and that the account can run containers, which needs Workers Paid. If the token reaches more than one account, pick one with `--account-id` or `$CLOUDFLARE_ACCOUNT_ID`. 2. Copies the sandbox image of the CLI's version, such as `ghcr.io/lucataco/cloudex-base:v0.2.3`, into your account's container registry, and waits until Cloudflare has it ready to run. You don't need Docker or Node.js on your computer. `--image` installs a different image. [Nodes](https://cloudex.build/docs/nodes.md) pull the same image, by its digest, from `ghcr.io`. 3. Finds or creates the D1 database `cloudex` and the R2 bucket `cloudex-backups`. 4. Uploads the API Worker, `cloudex-api`, with the admin key as a Worker secret, and gives it its container application and its workers.dev URL. With a preview domain, it routes `*./*` to the Worker. The first deploy makes the key; later deploys from the same computer keep it. 5. With a preview domain, adds a proxied `*.` DNS record. 6. Waits for the API, then starts a test sandbox and deletes it, because a newly pushed image can take a while to reach every Cloudflare host. `--skip-check` skips this. As soon as the Worker is uploaded, deploy saves the API URL and the admin key on this computer, so a deploy that is interrupted or fails its test sandbox doesn't lose the key. At the end it prints the API URL, `https://cloudex-api..workers.dev`. If you later change the account's workers.dev subdomain, run `cloudex deploy` again: it finds the API at its new address, keeps the key, and saves the new URL. Other computers then need `cloudex login --url` with the new one. ## Preview domain `--preview-domain example.com` serves [preview links](https://cloudex.build/docs/connect.md#preview-links) on that domain. It must be a zone in the same Cloudflare account. Cloudex adds a proxied `*.example.com` DNS record and routes `*.example.com/*` to its Worker. Requests for your other subdomains, such as `www.example.com`, pass through to their origins unchanged. Preview links look like `https://3000-cx-7f3a2k-.example.com`. Everything sits in one label under the domain, so the zone's Universal SSL certificate covers it. Running `cloudex deploy` again keeps the preview domain. `--preview-domain ''` turns preview links off. When you turn them off, or move them to another domain, deploy removes the old domain's route and the `*.` record it added. ## Backups and R2 `cloudex deploy` creates the R2 bucket for [backups](https://cloudex.build/docs/sandboxes.md#backups) and [named saves](https://cloudex.build/docs/saves.md) when R2 is turned on for the account. The free tier is enough for backups. If R2 is off, or the token can't use it, deploy says so and carries on with backups and named saves off. Turn on R2 Object Storage in the Cloudflare dashboard, then run `cloudex deploy` again. `cloudex status` shows whether backups are on. ## Run it again to update Running `cloudex deploy` again updates the API and the sandbox image. It keeps your sandboxes, your API key and your preview domain. Sandboxes keep running during an update. Open connections, such as `cloudex ssh` sessions and desktops, move to the new version within about 5 minutes and may drop when they do; reconnect if one does. A sandbox that resumes from a save keeps the image it was saved on; see [Saves](https://cloudex.build/docs/sandboxes.md#saves). [Nodes](https://cloudex.build/docs/nodes.md) reconnect by themselves, and their sandboxes keep running. Their agents don't update themselves: after an update, run the installer again on each node with the new version, as [Updating the agent](https://cloudex.build/docs/nodes.md#updating-the-agent) says. `cloudex doctor` warns about a node whose agent's version differs from the deployment's. ## A second computer Don't run `cloudex deploy` on a second computer. It would give the deployment a new API key, and the first computer would stop working. Instead: 1. On the new computer, log in with the URL: ```sh cloudex login --url https://cloudex-api..workers.dev ``` It shows a code, such as `BCDF-GHJK`, and waits. 2. On the computer that already works, approve it: ```sh cloudex tokens approve BCDF-GHJK ``` The new computer gets an [access token](https://cloudex.build/docs/tokens.md) of its own, which you can revoke without touching the first one. The URL works without `https://` too. To use a key instead, `cloudex login --key` asks for one, and a key in `$CLOUDEX_API_KEY` or on standard input is used as it is. If `cloudex deploy` finds a running deployment whose key this computer doesn't have, it asks before replacing the key. `cloudex deploy --yes` replaces it without asking. ## Check the setup `cloudex status` checks the connection to your deployment: ```text API https://cloudex-api.example.workers.dev (version v0.2.3) Auth the admin key Key hidden (--show-key prints it; for another computer, cloudex login there and approve its code with cloudex tokens approve) Previews example.com Backups on: /home goes to the R2 bucket cloudex-backups on each stop Current cx-7f3a2k ``` `cloudex status` also checks the key. If the API rejects it, status says how to log in again and exits with status 1. If the deployment runs an older version than the CLI, an `Update` line says to run `cloudex deploy`. `cloudex doctor` looks for problems. It checks that ssh is installed, that a deployment is configured, that its API answers, that it isn't older than the CLI and that the key works. When `$CLOUDFLARE_API_TOKEN` is set, it also checks that the account can run containers. Each line starts with `ok` or `fix`, and it exits with status 1 if anything needs fixing. Lines that start with `warn` are about [nodes](https://cloudex.build/docs/nodes.md): one that never joined, is offline, runs another version's agent, fails its isolation self-test, is low on disk, or whose clock is off. ## Update ```sh cloudex update ``` `cloudex update` downloads the latest release of the CLI, checks it against the release's checksums and replaces the running binary. If your deployment runs an older version, it then offers to run `cloudex deploy`, which keeps your sandboxes. Sandboxes that are running keep the old version until they stop. `cloudex update --check` only says what's out of date. `cloudex --version` prints the CLI's version. A CLI or SDK newer than the deployment can ask for something the deployment doesn't have. The error then says the deployment is older and to update it with `cloudex deploy`. Once a day, the CLI checks for a new release and mentions it after a command. Set `CLOUDEX_NO_UPDATE_CHECK=1` to turn that off. ## Files on your computer The CLI keeps its settings in `~/.config/cloudex`, or in `$CLOUDEX_CONFIG_DIR` when that's set: | File | What it holds | |---|---| | `config.json` | The API URL, the admin key, the preview domain and the current sandbox. Only your user can read it. | | `id_ed25519` | The SSH key the CLI uses for sandboxes. It's added to a sandbox the first time you connect. | | `known_hosts` | The SSH host keys of your sandboxes | `$CLOUDEX_API_URL` and `$CLOUDEX_API_KEY` override the saved deployment, and `--api-url` overrides the URL for one command. The CLI and the SDKs send the saved key only to the saved URL. With any other URL, set `$CLOUDEX_API_KEY` as well. --- # Dashboard Source: https://cloudex.build/docs/dashboard > Your deployment's own web dashboard, with sandboxes, terminals, desktops, agents, commands, files, named saves, nodes and access, signed in from the CLI, with a code, or with a token. ```sh cloudex dashboard ``` Your deployment serves a dashboard at its API URL, `https://cloudex-api..workers.dev`. It needs no domain or setup of its own. `cloudex dashboard` opens it in your browser, already signed in. `cloudex dashboard my-box` opens it on that sandbox. It has these pages: - **Sandboxes:** each one's state, where it runs, size, idle stop and age, updated every 5 seconds. You can make new ones, from a fresh image or a [named save](https://cloudex.build/docs/saves.md), on Cloudflare or one of your [nodes](https://cloudex.build/docs/nodes.md), and stop, resume and delete them. - **A sandbox,** with these tabs: - **Overview:** details, settings names, [preview links](https://cloudex.build/docs/connect.md#preview-links), forks and saves. For a sandbox on a node, the node it runs on, its isolation and its machine's CPUs and memory. - **Terminal:** shells in the sandbox. - **Desktop:** the sandbox's [desktop](https://cloudex.build/docs/desktop.md). - **Agents:** Claude Code, Codex and OpenCode conversations; start one, follow it, send messages and interrupt. - **Commands:** [background commands](https://cloudex.build/docs/commands.md#background-commands) and their output. - **Files:** browse, download a file or a directory as tar.gz, upload by dropping files, and delete. - **Saves:** named saves, the node each was made on, and new sandboxes from one, on Cloudflare or a node. - **Nodes,** for admins: your [nodes](https://cloudex.build/docs/nodes.md), with their state, isolation, host and sandboxes. **Add node** shows the command that installs a node's agent, once. Removing a node that has sandboxes says what happens to them first. - **Access,** for admins: [tokens](https://cloudex.build/docs/tokens.md), dashboard sessions, sign-ins waiting for approval, and [provider keys](https://cloudex.build/docs/keys.md). A `read` session sees the pages but can't change anything, and has no terminal or desktop. An own-only session sees only its own sandboxes. ## Sign in | Way | How | |---|---| | From the CLI | `cloudex dashboard` makes a link that signs a browser in once, within 60 seconds, and opens it. `--no-open` prints the link instead, for a browser on another computer. A browser that's signed in already stays signed in as it was, and the link isn't used: sign out first to use it. | | With a code | On the login page, **Get a code** shows a code such as `BCDF-GHJK`. Approve it with `cloudex tokens approve BCDF-GHJK` on a computer that's signed in, or in a dashboard that's signed in. | | With a token | Paste a [token](https://cloudex.build/docs/tokens.md) or the admin key on the login page. The browser gets a session of its own; what you paste isn't kept. | `cloudex login` on a computer without a key prints a link to approve its code in the dashboard, `/login?code=BCDF-GHJK`. The page shows where the sign-in comes from and the role it asked for before you approve it. It starts at `user` (or `read`, if that was asked for); `admin` is given only if you choose it. **Access** lists the sign-ins waiting. Only approve a code you started yourself. ## Sessions A browser's sign-in is a dashboard session. It's a kind of token, listed by `cloudex tokens` and on **Access** with a name such as `dashboard (Firefox on Linux)`. - **Role:** it has the role of whatever signed it in. `cloudex dashboard` on a computer with a `read` token gives a `read` session. - **How long it lasts:** 30 days from its last use. A session signed in with a code never lasts past the expiry chosen when it was approved. - **Tokens:** a session signed in with a token acts for that token. It reaches the same own sandboxes, and it ends when the token is revoked or expires. - **Signing out:** **Sign out** ends the session, as `cloudex tokens revoke ` does from anywhere. Open terminals and desktops close within a minute and go to the login page. - **No sign-ins from a session:** a session can't make a sign-in link, and pasting one on the login page doesn't work. Use a token or the admin key. ## Terminal Each terminal tab is a login shell in the sandbox. Tabs run as the `cloudex` user, with `sudo` and Docker. Choose **+ root** for a root shell. - **Reconnecting:** each tab runs in a tmux session of its own. If the connection drops, the tab reconnects to the same shell, with what was running in it. Connections drop for a moment when the deployment is updated. - **Closing a tab** ends its shell. Leaving the page doesn't, and the tabs come back when you return. Shell 1 in two browsers is the same shell, and both see it. - **Selecting text:** hold Shift, or Option on a Mac, to select text with the mouse. The mouse wheel scrolls back. - **Idle stop:** while the browser tab is visible, each open terminal tells the sandbox once a minute that someone is there, so it doesn't [stop for being idle](https://cloudex.build/docs/sandboxes.md#idle-stop). A terminal left in a hidden browser tab doesn't keep it up. - **Stopped sandbox:** when a sandbox stops, its shells end. Resuming it starts new ones in the same tabs. ## Desktop **Desktop** starts the sandbox's [desktop](https://cloudex.build/docs/desktop.md) the first time it opens. Installing it the first time takes a minute or two. The dashboard shows it with a viewer of its own, over its own connection to the desktop, so the desktop needs no preview link. **Match this window's size** resizes the desktop to fit the browser window; otherwise the picture scales. **Clipboard** shows what's copied on the desktop, and sends text to the desktop's clipboard for pasting there. ## Security - **Session cookie:** the session lives in a cookie that page scripts can't read. It's sent only to your deployment's host, and never with a request another site starts. - **Cookie requests:** every API request that uses the cookie must also come from the dashboard's own pages, with its Origin and a header that another site can't add. A token or the admin key as the bearer works as before. - **Sandbox content:** nothing from a sandbox is ever served as a page of the dashboard's host: - agent output, command output and file names show as text; - downloads are attachments that can't run as pages; - the desktop uses the dashboard's own viewer; - preview links open on your preview domain, in a new tab. - **Links in the terminal** open in a new tab, `http` and `https` ones only. - **After signing in,** the login page goes only to the dashboard's own pages. - **Content Security Policy:** the dashboard's pages allow no inline scripts and no scripts from elsewhere, and can't be framed. ## Browsers These versions or newer: - Chrome or Edge 111; - Firefox 115; - Safari 16.4. Uploads in the dashboard go up to 100 MB a file; [`cloudex cp`](https://cloudex.build/docs/commands.md#copy-files) has no limit. --- # Nodes Source: https://cloudex.build/docs/nodes > Run sandboxes on a Linux machine of your own, a home server or a VPS, under the same deployment, CLI, API, SDKs and dashboard, with cloudex node add and cloudex new --on. What works there, ssh straight to it from your LAN or tailnet, how isolated it is, and what the machine can see. A node is a Linux machine of your own, such as a home server or a VPS, that runs sandboxes for your deployment. An agent on it, `cloudex-node`, dials out to your deployment and stays connected. The machine needs one inbound port, on its LAN and tailnet addresses only, so that ssh from a computer near it can go straight there; that port answers only with a ticket from your deployment, and `--direct off` closes it. Sandboxes on a node work with the same CLI, API, SDKs and dashboard as sandboxes on Cloudflare. You pay for the machine instead of Cloudflare's container time. ```sh cloudex node add box # prints the command to run on the machine cloudex node list # box online, with its CPUs, memory and isolation cloudex new --on box --cpus 4 --memory 16G --name n1 cloudex ssh n1 ``` Nodes are new in v0.2.2. The deployment and the CLI both need v0.2.2 or later. [Direct connections](#direct-connections) and [forks](#what-works-what-differs) on nodes are new in v0.2.3: they need v0.2.3 on the deployment and the node's agent, and direct connections need the v0.2.3 CLI too. ## When a node is worth it A node is worth it when you already have a machine with spare capacity, or when sandboxes would cost more on Cloudflare than a machine of your own: - **Long-running sandboxes.** A sandbox on a node has no 6-hour limit, and an idle time of 0 keeps it running until you stop it. - **Bigger sandboxes.** A sandbox on a node can have as many CPUs and as much memory as the machine has, not the 4 vCPU and 12 GiB of `standard-4`. - **Sandboxes that stay up.** Stopping and resuming take about a second, and the disk stays on the machine. - **Working nearby.** From the node's LAN or tailnet, ssh, scp and editors go straight to it, as fast as plain ssh to the machine. Each sandbox on a node still has its own Durable Object, and commands, previews and the dashboard reach it through your Worker, as for a sandbox on Cloudflare. Those count toward your Workers plan as usual. ssh from a computer on the node's LAN or tailnet goes [straight to the node](#direct-connections) instead. The container time is what you no longer pay. Don't use a node if: - **You want to run untrusted code on plain Docker.** Without Sysbox, code that escapes its container is root on the machine, and from there it reaches your LAN and tailnet. See [Isolation](#isolation). - **The machine holds things a sandbox mustn't reach,** and you can't give it Sysbox. The firewall keeps sandboxes off the host and the LAN, but under plain Docker it doesn't protect against a kernel or container escape. - **You need backups or `--size`.** They aren't available on nodes yet. See [What differs](#what-works-what-differs). - **The machine is arm64, has no systemd, or you have no root on it.** Synology and Unraid aren't supported yet. - **The machine is far from you, or on a slow line.** Unless you're on its LAN or tailnet, every command and connection goes through Cloudflare, and saves go up over the machine's own line either way. See [Performance](#performance). ## What you need The installer checks each of these and stops with a message naming what's missing: - **Linux on x86-64** (amd64). arm64 machines aren't supported yet. - **Root**, through `sudo`. The firewall needs it. - **systemd**, which loads the firewall at every boot. - **nftables** (`nft`) and **iproute2** (`ip`). On Debian and Ubuntu: `apt install nftables iproute2`. - **Docker Engine 28 or later**, running. Earlier versions ignore the network priority that keeps the agent's own route off sandbox networks, so the installer and the agent refuse them. It was tested with Docker 29 on Ubuntu 22.04. - **At least 10 GB free** in Docker's data directory. The agent refuses new sandboxes below that, and keeps 5 GB free while they run (see [Disk](#disk)). - **Outbound HTTPS** to your deployment's URL and to `ghcr.io`, where the agent and the sandbox image come from. - **One inbound port,** 7022, on the machine's private LAN and tailnet addresses only, for [direct connections](#direct-connections). Nothing needs to reach it from the internet. `--direct off` opens no port at all. - **A clock that keeps time**, such as with NTP. A node whose clock is more than 60 seconds off is refused, and `cloudex doctor` warns from 30. - **Optional: Sysbox,** installed before the agent. Machines then run under Sysbox, which is safer and runs Docker inside. See [Isolation](#isolation). Adding and removing nodes needs the admin key or an `admin` [token](https://cloudex.build/docs/tokens.md). ## Add a node ```sh cloudex node add box ``` `cloudex node add` adds the node and prints the command to run on the machine, as root, with Docker running: ```sh curl -fsSL https://cloudex.build/node | CLOUDEX_JOIN_TOKEN=cxj_... sudo --preserve-env=CLOUDEX_JOIN_TOKEN bash -s -- --api https://cloudex-api..workers.dev --version v0.2.3 --digest sha256:... ``` The join token goes in the environment, not on the command line, so it isn't in any process's arguments on the machine. It works once, for 30 minutes; `node add` prints the time it expires. `--digest` pins the agent's image: the installer pulls it by that digest, so Docker refuses an image that doesn't match, and runs that exact image, never a tag that can move. `cloudex deploy` looks the digest up when it installs your deployment; a deployment that isn't a release names the `edge` agent, without a digest. Adding the same name again before the node has joined gives a new token. `--version` is your deployment's version, so the agent matches it. `node add` also prints a warning about plain Docker, which applies when the machine has no Sysbox. Node names follow the rules for sandbox names: 2 to 32 lowercase letters, digits and hyphens, starting with a letter. They can't start with `nd-`, which marks node ids. With `--json`, `node add` prints the node, the token, its expiry and the command. In the [dashboard](https://cloudex.build/docs/dashboard.md), **Nodes** does the same for admins: **Add node** shows the command once. ### What the installer does The installer runs as root and changes the machine in these ways, in order. It says it worked only if every step did. 1. **Checks** the machine, as listed in [What you need](#what-you-need), and whether Docker has Sysbox. 2. **Picks a pool** for sandbox networks: a private /16 that collides with no route and no Docker network on the machine. `--pool` picks one yourself. It also picks the addresses for the [direct port](#direct-connections): the machine's tailnet (100.64/10) IPv4 address on Tailscale's interface, and its private (10/8, 172.16/12, 192.168/16) IPv4 addresses, leaving out Docker's and other bridges and VPN tunnels (WireGuard, OpenVPN, ZeroTier, NetBird and the like). It prints each with its interface. If one is a network you don't want reaching the port, such as a cloud provider's private network or a shared LAN, run it again with `--direct-addr` for each address you do want; `--direct off` picks none. 3. **Installs the firewall:** `/etc/cloudex-node/firewall.nft` and a systemd unit, `cloudex-node-firewall.service`, which loads it now, at every boot, and whenever `nftables.service` starts, restarts or reloads. A timer, `cloudex-node-firewall-check.timer`, loads it again within 30 seconds if anything else removed it. It's one nftables table, `inet cloudex`, which touches only traffic to and from the sandboxes' bridges (named `cx…`). Docker's restarts don't touch it. 4. **Joins your deployment** with the token, from a one-off container that reads it on its standard input, so the token is in no process's arguments and not in the agent's settings. Then it starts the agent as a container, `cloudex-node`, with `--restart unless-stopped`, its own Docker network and a volume, both also named `cloudex-node`. The agent has the Docker socket, a read-only file system, no capabilities, at most 1 GiB of memory and 1024 processes. Unless `--direct off`, the agent then runs a small relay, `cloudex-node-direct`, which Docker publishes the direct port on, on those addresses only. The relay has no Docker socket, no capabilities, a read-only file system and 64 MiB of memory; it passes connections to the agent and holds nothing. 5. **Waits** for the agent to come online and pass its isolation [self-test](#isolation). The agent pulls the sandbox image first, about 700 MB. 6. **Boots a throwaway sandbox**, checks from inside it that the internet answers and the host and metadata addresses don't, and removes it. It ends with `this node is installed and online`, then the plain-Docker warning if the machine has no Sysbox, the direct port's addresses, and the command that uninstalls it. On the test machine it took 23 to 29 seconds. `cloudex node list` then shows the node `online`. Each sandbox on the node is a container named `cloudex-` on a network of its own. Everything the agent makes carries `build.cloudex.*` labels. It reads, changes and deletes only containers, networks and volumes with those labels, and never prunes, so your own containers are safe. The installer keeps what it needs to update and uninstall in `/etc/cloudex-node`. | Option | Effect | |---|---| | `--version vX.Y.Z` | The agent's version; `node add` gives your deployment's | | `--digest sha256:…` | Pull the agent's image by this digest, and refuse any other; `node add` gives it for a release | | `--pool CIDR` | The pool for sandbox networks, such as `10.250.0.0/16`, inside `10.0.0.0/8` or `172.16.0.0/12` | | `--block CIDR` | Another IPv4 range sandboxes may not reach; repeatable | | `--machine-image REF` | Run sandboxes from this image instead of the deployment's | | `--allow-image REPO` | Another image repository sandboxes may run; repeatable. Without it, only `ghcr.io/lucataco/cloudex-base`. | | `--keep-free-gb N` | The floor for Docker's free disk, 5 GB by default. See [Disk](#disk). | | `--disk-gb N` | Cap each sandbox's disk at N GB, where Docker's storage driver can. See [Disk](#disk). | | `--direct auto` or `off` | The [direct port](#direct-connections). `auto`, the default, publishes it on the machine's tailnet IPv4 address and its private IPv4 addresses on LAN interfaces (not bridges or VPN tunnels), and prints them; `off` opens no port, and ssh goes through your deployment. | | `--direct-addr IP` | Publish the direct port on exactly this private or tailnet IPv4 address; repeatable. Use it on a VPS, with the tailnet address. | | `--direct-port N` | The direct port, 7022 by default | | `--unisolated` | Testing only, without root: no firewall. See [Isolation](#isolation). | | `--uninstall`, `--keep-images`, `--yes` | See [Uninstall](#uninstall) | ### Updating the agent After [`cloudex deploy`](https://cloudex.build/docs/deploy.md) updates your deployment, run the installer again on each node with the new version. It needs no token: it keeps the node's identity, its pool, its direct port's settings and its sandboxes, and replaces the running agent. To pin the image, add `--digest` with the one `cloudex node add` prints for that version. ```sh curl -fsSL https://cloudex.build/node | sudo bash -s -- --version v0.2.3 --digest sha256:... ``` Running sandboxes keep running while the agent is replaced, and the new agent joins their networks again as it starts. Open ssh sessions and commands end with the old agent. A node updated from v0.2.2 gets its [direct port](#direct-connections) this way; until then, ssh to it goes through your deployment. The agent doesn't update itself. `cloudex doctor` warns about a node whose agent's version differs from the deployment's, and the deployment refuses an agent whose protocol it doesn't speak, saying which version to install. ## Make sandboxes on a node ```sh cloudex new --on box --name n1 cloudex new --on box --cpus 4 --memory 16G --name big cloudex run --on box -- npm test ``` `--on` takes a node's name or id. `--cpus` (from 0.25) and `--memory` (such as `512M` or `16G`, from 512 MiB) set the sandbox's limits on the node, up to what the node has. Without them, a sandbox gets its size's CPUs and memory: 1 CPU and 6 GiB for `standard-2`, at most the node's CPUs and half its memory. `--cpus` and `--memory` work only with `--on`. On Cloudflare, pick a `--size` instead. `cloudex new --on` prints where the sandbox runs and its limits. On a node under plain Docker, it repeats the warning about trusted code. The other flags of `cloudex new` work as on Cloudflare. `cloudex list` has a WHERE column: `cloudflare`, the node's name, or the name with `(offline)` or `(removed)`. `cloudex info` shows where a sandbox runs, its machine's CPUs and memory, and its isolation. A sandbox stays on the node it was made on. To move it, see [Move between a node and Cloudflare](#move-between-a-node-and-cloudflare). ### In the dashboard **New sandbox** has a **Where** choice when the deployment has nodes, with each node's isolation, and CPUs and memory for a node. The sandbox list has a **Where** column, and a sandbox's page shows where it runs, its machine and its isolation. **Nodes**, for admins, lists the nodes with their state, isolation, host and sandboxes, adds and removes them. ### From code ```ts import { Cloudex } from "cloudex"; const cx = new Cloudex(); const nodes = await cx.nodes.list(); // name, state, runtime, isolation, cpus, memoryMb, ... const box = await cx.create({ name: "n1", node: "box", cpus: 4, memoryMb: 16384 }); console.log(box.placement); // { provider: "node", node: { name: "box", online: true, ... }, cpus: 4, memoryMb: 16384 } ``` ```python from cloudex import Cloudex cx = Cloudex() nodes = cx.nodes.list() box = cx.create(name="n1", node="box", cpus=4, memory_mb=16384) print(box.placement, box.isolation) ``` `nodes.get(name)` adds the image the agent has and, for admins, a round trip to it, and `nodes.add(name)` and `nodes.remove(name)`, with `force`, need an admin key. See [SDKs](https://cloudex.build/docs/sdk.md). ### When a node refuses a sandbox | Message starts with | Why | |---|---| | `node box hasn't joined yet` | The install command hasn't run on it, or failed | | `node box was removed` | `cloudex node rm` removed it; add it again with `cloudex node add box` | | `node box is offline (last seen …)` | Its agent isn't connected. See [Offline](#when-a-node-is-offline). | | `node box's isolation self-test fails (…)` | The firewall isn't working; it names the checks that failed. See [Troubleshooting](#troubleshooting). | | `node box is still pulling the sandbox image` | The agent is fetching the image after an install or an update; try again when `cloudex node info box` says it's present | | `an own-only token can't place sandboxes on node box` | Own-only tokens can't use plain-Docker nodes. See [Who may use a node](#who-may-use-a-node). | | `cpus: node box has 8 CPUs` | `--cpus` or `--memory` asked for more than the node has | | `Docker's disk on this node has … GB free` | Fewer than 10 GB are free for Docker on the machine for a new sandbox or a fork's copy, or fewer than 5 GB (the node's floor) for a resume. See [Disk](#disk). | | `a fork of n1 stays on node box` | A fork was asked for on Cloudflare or another node. Save the sandbox and start from the save instead; see [Move](#move-between-a-node-and-cloudflare). | | `a fork of n1 would have … image layers` | Forks of forks go about 100 levels deep at most. Make a named save of it and start from that. | ## What works, what differs | | On a node | |---|---| | `ssh`, `ssh-config`, `scp`, `rsync`, editors, port forwarding | The same. From the node's LAN or tailnet they go [straight to the node](#direct-connections), elsewhere through your deployment. | | `exec`, background commands, `cp`, the files API | The same. Uploads are at most 100 MB, as on Cloudflare. | | Preview links | The same, WebSockets included | | The dashboard's terminal and desktop | The same. The desktop's first start installs it over the machine's own line. | | Settings, `--repo`, `~/.cloudex/start` | The same | | Agents and [provider keys](https://cloudex.build/docs/keys.md) | The same. Provider calls go through the node's agent; see [What a node can see](#what-a-node-can-see). | | The `cloudex` command inside a sandbox | The same, `cloudex stop` inside included | | Idle stop | The same rules. `--idle 0` means never. While the node is offline, idle stop can't act. | | The 6-hour limit | None | | `stop` | Stops the machine in about half a second. The disk stays on the node: no save and no backup. | | `resume` | Starts the machine again with its disk, in 1 to 2 seconds | | [Named saves](https://cloudex.build/docs/saves.md) | Work. A save goes up over the machine's own line, and records the node it was made on. | | `new --from --on ` | Works, from a save made on Cloudflare or on any node | | Sizes | `--cpus` and `--memory` instead, for `new`, `fork` and `new --from`. `resume --size` is refused. | | [Forks](https://cloudex.build/docs/forks.md), `new --from ` | The same, on the same node, running or stopped. A running sandbox pauses while its disk is copied. See [Forks on a node](https://cloudex.build/docs/forks.md#on-a-node). | | [Backups](https://cloudex.build/docs/sandboxes.md#backups), `resume --from-backup` | None: the disk stays on the node. Make a named save for a copy off the node. | | Docker inside a sandbox | With Sysbox only. Under plain Docker, there's no Docker in the sandbox. | | Disk | Docker's disk on the machine, shared by its sandboxes. See [Disk](#disk). | | `delete` | Removes the machine and its network. While the node is offline, only with `--force`. | ### Disk A node's sandboxes share Docker's disk on the machine, usually its root file system. On most machines (overlayfs on ext4) Docker can't cap one sandbox's share, so the agent watches the free space every 5 seconds instead. When it falls under the node's floor, 5 GB unless the installer's `--keep-free-gb` says otherwise, the agent stops the running sandbox that has written the most, and that sandbox's notice says why. It stops another only while the space keeps falling. Resume a stopped sandbox once there's room again, and delete what filled it. The watchdog acts after the floor is crossed: a sandbox that writes fast can take the free space a little further down in the seconds before it stops. In testing, a sandbox writing about 135 MB/s was stopped with the free space 0.3 to 0.7 GB under the floor. Where Docker's storage driver has quotas (btrfs, zfs, or overlay2 on xfs mounted with `pquota`), the installer's `--disk-gb N` also caps each new sandbox's disk at N GB. Elsewhere the option does nothing, and the watchdog is the limit. **Fork images.** A [fork](https://cloudex.build/docs/forks.md#on-a-node) copies its original's disk into an image on the node, which the forks made by that request share. The image takes about as much space as the original had written, or about twice that on Docker's containerd image store, which Docker 29 uses on new installs. A fork is refused unless the copy leaves at least 10 GB free. The image goes with the last sandbox that uses it, and one that nothing uses is removed within the hour. On Docker's older overlay2 store, an image that a fork of a fork came from stays until that one's image goes too. Stopping sandboxes doesn't free fork images. When the watchdog stops a sandbox while fork images hold more than 1 GB, the notice says how much, and deleting forks frees it. `cloudex node info` shows how many fork images there are and their size. ### Connections to the agent A sandbox reaches its node's agent for `cloudex.internal` and its provider calls. Each sandbox may hold at most 256 connections to the agent at once, and have 64 requests in flight; more are closed as they come, so one sandbox can't use up the agent for the others. A preview's WebSocket message is at most 1 MiB. Your deployment holds at most 64 open streams (commands, preview requests and connections) to one sandbox on a node; more wait up to 15 seconds for one to end. ## Direct connections ssh to a sandbox on a node goes straight to the node when your computer can reach it, on the same LAN or over a tailnet such as Tailscale. Elsewhere it goes through your deployment, as before. You don't choose: the CLI tries both and takes the first that works. What goes direct is everything that runs over `cloudex ssh`'s connection: - `cloudex ssh`, with port forwarding (`-L`) and remote commands; - plain `ssh`, `scp`, `rsync` and editors, through the block that `cloudex ssh-config` prints; - `cloudex prompt` and `cloudex attach`; - `cloudex proxy`, for any TCP port. What doesn't: the dashboard, its terminal and desktop, preview links, `cloudex exec`, `run` and `cp`, agents without a terminal, and the API and SDKs. They go through your deployment. For a web server in the sandbox, forward its port over ssh, which goes direct: ```sh cloudex ssh n1 -- -L 3000:localhost:3000 ``` ### How it works The installer publishes one port on the node, 7022, on the machine's private LAN and tailnet IPv4 addresses. For each connection, the CLI asks your deployment where the sandbox is. For a sandbox on a node, the deployment answers with the node's addresses, its public key, and a ticket good for one connection to that sandbox and port, for 60 seconds. The CLI then tries the addresses, starting them 50 ms apart: first those on your own LAN subnet, then tailnet addresses if your computer is on a tailnet, then other private addresses. Each try is a TLS connection whose certificate must carry the node's key; anything else, such as another machine that has the address now, a VPN that swallows it, or a captive portal, fails that try. The first that proves to be the node gets the ticket, and the node checks it before it connects anything. If none has proved itself about 300 ms after the CLI asked, the CLI starts the path through your deployment too, and uses whichever connects first. When the node has no direct port, such as an older agent or `--direct off`, it goes through your deployment at once. When the path through your deployment wins on a network, for the next 10 minutes the CLI starts it at once there, and still tries the node beside it. The CLI keeps the node's addresses for 10 minutes too, so the next connection tries them before your deployment answers. Both paths reach the same sshd, so ssh checks the sandbox's host key either way, and `known_hosts` has one entry for both. Your deployment still decides who may connect: the ticket is minted only for a token that may use the sandbox, and only while it's running. A direct session counts as activity for idle stop, as on the other path. Revoking the token that opened it ends it within about 90 seconds (6 to 40 seconds in testing): each ticket names its token, and the node ends every session of a token that ended. `cloudex stop` ends it at once, and `cloudex node rm` ends it and closes the port. It doesn't drop when you run `cloudex deploy`, or when the node's own connection to your deployment does: in testing, a session lived through a 2-minute cut of the node's line to Cloudflare, and a new one opened during it. While the node's connection is down, though, a revoked token's session lasts until it's back. ### Checking it ```sh CLOUDEX_DEBUG=1 cloudex ssh n1 -- true ``` ```text cloudex proxy: [ 0.0ms] asking the API cloudex proxy: [374.2ms] cx-7f3a2k is on node box (nd-k2m4p6q8), direct [192.168.1.20 100.101.102.103] port 7022 cloudex proxy: [375.5ms] 192.168.1.20:7022 lan-onlink 1.3ms verified cloudex proxy: [375.9ms] chose direct 192.168.1.20:7022 (lan-onlink) ``` `CLOUDEX_DEBUG=1` prints each try, how long it took, why it failed, and which path was chosen. `CLOUDEX_DIRECT` sets the paths the CLI uses: | `CLOUDEX_DIRECT` | Effect | |---|---| | `auto` (the default) | Try the node and your deployment, as above | | `off` | Only through your deployment, as in v0.2.2 | | `only` | Only straight to the node, or fail. For checks and debugging. | `cloudex node info box` shows the direct port's addresses and whether it's `listening`, `off` or `failed`, with the reason, to admins; other tokens see the node without them, since a sandbox's own connection gets the addresses it needs. A node installed with `--direct off` shows `Direct off`. `cloudex doctor` tries a TLS connection to each online node's addresses from your computer, with no ticket, and says which answer; with a token that isn't an admin's, it says it skipped that check. ### Addresses and networks The installer picks the addresses once. If the machine's LAN address changes, such as with DHCP, ssh falls back to your deployment until you run the installer again, which picks them again. A DHCP reservation for the machine, or its tailnet address, avoids that. On a VPS, the private addresses are the provider's network, which your computer can't reach: give the tailnet address with `--direct-addr`. The relay starts on its own, so a missing address doesn't stop the agent or its sandboxes. If an address isn't there yet, such as a tailnet that comes up after Docker, the relay fails alone, `node info` says `failed` with Docker's reason, and the agent tries again every 30 seconds. If the port is published on a public address or on all addresses, such as by hand, the agent refuses it and says so in `node info`, until the installer runs again. | Your computer's network | What happens | |---|---| | The node's LAN | Direct, on the LAN address | | A tailnet the node is on | Direct, on the tailnet address | | A full-tunnel VPN | LAN addresses go into the tunnel and fail; through your deployment, or the tailnet address if it works | | A split-tunnel VPN | Direct, as on the LAN | | A Tailscale exit node | The LAN is blocked unless Tailscale's "Allow local network access" is on; the tailnet address still works | | Another VPN in 100.64/10, such as Cloudflare WARP | Its addresses can look like tailnet ones; the key check refuses any machine that isn't the node | | A captive portal | The tries fail. So does your deployment, until you sign in to the network. | | macOS Local Network privacy | When the terminal app isn't allowed to reach the local network, LAN tries can fail at once with `no route to host`, which `CLOUDEX_DEBUG` points out, and ssh goes through your deployment. Allow the app in System Settings, Privacy & Security, Local Network. | The CLI doesn't know whether Tailscale reaches the node directly or through one of its relays (DERP). A relayed tailnet path still wins when it answers first, and can be slower for big copies than your deployment. ## Move between a node and Cloudflare A sandbox stays where it was made. To move one, save it, then start a sandbox from the save where you want it: ```sh cloudex save n1 n1-copy # on the node cloudex new --from n1-copy --name n1-cf # on Cloudflare cloudex new --from n1-copy --on box --name n2 # or on a node, the same one or another ``` It works in both directions, and between nodes. A Cloudflare sandbox's save starts on a node the same way, with `--on`. The new sandbox has all of the saved disk, because both sides run the same image. It has a new id and new preview links, and the old sandbox keeps running until you stop or delete it. - **Speed.** A save of a small sandbox on the node took about 2 seconds on the test machine. Starting a node's save on Cloudflare took 6 to 13 seconds, and a Cloudflare save on the node 7 to 9. A large save is limited by the machine's upload speed. - **Device files** that Docker in a Cloudflare sandbox keeps under its data directory can't be made on a node, so they're left out of the restore, and the new sandbox's notice counts them. Docker's images in it may need pulling again. - **Trust.** A save made on a node holds whatever that node put in it. Before you start it on Cloudflare, treat it like code from that node. `cloudex saves` shows the node each save was made on in its ON column, and so do `new --from`, the dashboard and the SDKs (a save's `node`). ## When a node is offline A node is offline when its agent isn't connected: the machine is off, its network is down, or the agent or Docker stopped. `cloudex node list` shows it offline at once when the agent closes its connection, and within about 45 seconds when the connection just goes silent. While it's offline: - `cloudex list` shows its sandboxes as `box (offline)`, with the state they had. `cloudex info` still works. - Anything that needs the machine, such as `ssh`, `exec`, `stop` or `resume`, fails with `node box is offline (last seen 2026-11-11 14:02 UTC)`. When only the node's line to your deployment is down and your computer can still reach the node, ssh to a running sandbox can still go [direct](#direct-connections), if your CLI has connected to it since it last started. - The machines keep running on the host, if it's up. Idle stop can't stop them until the node is back. - A change to a running sandbox's settings is refused, and isn't kept. - `cloudex delete` is refused. `cloudex delete --force` deletes the sandbox at once, and the node removes its machine and network when it's back. `cloudex node info` lists those as pending deletes. When the agent connects again, which it does by itself, the node is online within a second or two. The agent reports its machines, and each sandbox settles with what really happened: - A machine that stopped while the node was offline marks its sandbox `stopped`, with a notice that its disk is kept. `cloudex resume` starts it. - A sandbox you'd stopped whose machine was started on the host by hand is stopped again. - A machine that's gone marks a running sandbox `error`. Resuming it starts over from the image, without its files, and its notice says so. - A container with the agent's labels that belongs to no sandbox is left alone and listed as an orphan in `cloudex node info`. ### Reboots and restarts The firewall loads at boot, before Docker, and Docker starts the agent again. Sandboxes don't start by themselves: each one shows `stopped`, with a notice that its machine stopped without being asked to and its disk is kept. Resume them. The same happens when Docker restarts and stops its containers. Restarting only the agent's container leaves the machines running. In testing, the node was online within 3 seconds of the machine being back up, and a resume took about 2 seconds, with the files. ### Deploys Nodes reconnect by themselves when you run `cloudex deploy`. As for sandboxes on Cloudflare, busy connections to a node's sandboxes through your deployment, such as ssh sessions, move to the new version within about 5 minutes and may drop then. [Direct](#direct-connections) ssh sessions don't go through your deployment, so a deploy doesn't touch them. ## Remove a node ```sh cloudex node rm box ``` `cloudex node rm` disconnects the node's agent at once, and its key stops working, so it can't connect again. It asks first; `--yes` skips the question. The node stays listed as `revoked` until you run `cloudex node rm` again, which forgets it. A node that still has sandboxes is refused unless you add `--force`. Then: - **If the node is online,** its machines are deleted first. The sandboxes stay listed, in state `error`, saying their machines went with the node, until you delete them. - **If the node is offline,** its machines can't be deleted then. The sandboxes stay listed in state `error` until you delete them. When the agent connects again, the deployment tells it the node was removed, and it stops every machine. Their disks stay on the host until you [uninstall](#uninstall) the agent. A sandbox on a removed node answers `410` to everything but delete, and deleting it needs nothing from the node. `node rm` doesn't uninstall anything on the host. The agent stops its machines once it hears it was removed, and then asks again only every 30 minutes; `cloudex-node status` on the machine says the node was removed. To add the same machine again, uninstall it first, then `cloudex node add` and run the new command: the installer refuses to run over a removed node. ## Isolation Every sandbox on a node runs in a Docker container on a network of its own, with the firewall around it. How far it's kept from the host depends on the runtime: | Isolation | When | Root in a sandbox is | Docker inside | Use it for | |---|---|---|---|---| | `sysbox` | Docker has Sysbox when a sandbox is made | An unprivileged user on the host | Works | Code you don't fully trust | | `docker` (plain Docker) | No Sysbox | Root on the host's kernel, held in by the container | Doesn't work | Code you trust only | | `none` | The node joined with `--unisolated` | As plain Docker, with no firewall | Doesn't work | Testing only | | `vm` | A sandbox on Cloudflare | Root in a VM of its own | Works (host network) | Anything | `cloudex node list`, `cloudex info`, the dashboard and the API show a node's and a sandbox's isolation. A plain-Docker node shows `docker (trusted code only)`. ### What the firewall does The firewall applies under both runtimes. From a sandbox: - The internet and DNS answer. - The host itself, your LAN (private address ranges), your tailnet (100.64.0.0/10), link-local and cloud metadata addresses such as 169.254.169.254, multicast and reserved ranges, and other sandboxes don't. Neither do the host's public on-link networks as they were at install, or anything you add with `--block`. - Nothing can open a connection into a sandbox, and sandbox networks have no IPv6. - A packet from a sandbox whose source address isn't on its own network is dropped, so a sandbox that takes another's address can't have answers sent there. A sandbox also reaches its node's agent, for `cloudex.internal` and its provider calls, and the agent answers it only from its own network. **The self-test** checks this from inside throwaway sandboxes, rather than reading the rules: when the agent starts, every hour after, and after an error. It probes the host, its gateway, its DNS servers and tailnet addresses, the direct port, metadata, another sandbox, and what root can do inside, and it checks that the internet and DNS answer. If anything that should be closed answers, the node's isolation shows as failing, it makes and starts no sandboxes until a self-test passes, and it stops the sandboxes that are running, each with a notice that says why. A pass doesn't count for ever: if no self-test has finished for about two hours, the node starts no sandboxes until one passes. The firewall is one nftables table. `systemctl restart nftables`, `nft flush ruleset`, or a firewall manager that rewrites the whole rule set removes it, and with it Docker's own rules: - **`nftables.service` starting, restarting or reloading** loads the table again right after, since its unit is tied to that service. - **Anything else** is caught within 30 seconds by `cloudex-node-firewall-check.timer`, which loads it again. - **Meanwhile,** the agent knocks on the host from a sandbox network every 5 seconds, a knock the table rejects. When the host answers instead, the agent marks the node failing, stops its running sandboxes, each with a notice that says why, and runs its self-test again. In testing, after `nft flush ruleset` a running sandbox could reach the host for 1 to 4 seconds before it was stopped, and the table was back 9 to 34 seconds later; after `systemctl restart nftables` the table was back within a quarter of a second, and the sandbox never reached the host. A flush also removes Docker's rules, so sandboxes lose the internet until Docker's rules are back (`systemctl restart docker` restores them). Once a self-test passes again, resume the sandboxes. ufw, firewalld and iptables-legacy weren't tested. ### The direct port The [direct port](#direct-connections) is the node's only inbound port. Docker publishes it on the addresses the installer chose, all private or tailnet IPv4, and the agent refuses to serve it when it finds it published on a public address or on all addresses. Behind it are the relay, which holds nothing, and the agent, which takes connections only from the relay. Anything on your LAN or tailnet can connect to it and complete a TLS handshake, and that's all. The agent reads one line after the handshake, at most 1 KiB within 5 seconds, and connects nothing without a valid ticket: one your deployment signed with this node's own secret, for a sandbox running on it, unused and less than 60 seconds old. Wrong, reused, expired and missing tickets get an error and the connection closes; sshd never sees them. The agent holds at most 64 handshakes at once and 16 from one address, so a flood from one machine can delay others' connections by a few seconds but not take the node down; in testing, 2000 connections at once left the agent at 10 MiB, and a real ssh right after went through. Sandboxes on the node can't reach the port: the firewall keeps them off the host's addresses, and the self-test checks that every hour. Under `--unisolated` they can, and get the same TLS handshake and nothing more. On a machine with Tailscale, Tailscale's own NAT makes tailnet clients reach the agent from Docker's gateway address rather than their own. That needs no setup. They share one budget, 64 of the agent's 128 handshakes at once, kept for them; other addresses share the other 64, at most 16 each. To narrow who can reach the port, use Tailscale's access rules, `--direct-addr` to publish it on fewer addresses, or `--direct off`. ### Plain Docker Under plain Docker, the container is all that keeps root in a sandbox off the host. Sandboxes drop some capabilities and never run privileged, with host paths, the Docker socket or the host's network. But a kernel bug or a container escape gives code in the sandbox root on the host, and from there your LAN and tailnet are in reach, whatever the firewall says. **Use a plain-Docker node only for code you trust.** An agent that reads web pages, issues or pull requests can be told what to run by them. That includes code from pull requests, unknown repos, and agents that fetch and run things from the internet. Use Sysbox, or Cloudflare, for those. ### Sysbox [Sysbox](https://github.com/nestybox/sysbox) maps root in a sandbox to an unprivileged user on the host, and lets Docker run inside. Install it before the agent, or run the installer again after installing it; sandboxes made from then on use it, and older ones keep the runtime they were made with. Sysbox's installer rewrites Docker's `daemon.json` and restarts Docker, and version 0.7.1 also needed the `fuse3` package, which it doesn't declare. Sysbox still shares the host's kernel. Root in a sandbox can change its own routes and send raw packets on its own network, where only the agent and the host's bridge address are. The firewall and the agent's checks hold against that (a forged source address, for one, is dropped), but a kernel bug can still reach the host. ### --unisolated `--unisolated` joins a node without the firewall, for testing on a machine where you have no root. Sandboxes on it reach the host, its LAN, its tailnet and its direct port. It shows as `none (testing)`. ### Who may use a node | Who | Can | |---|---| | The admin key, `admin` tokens | Add and remove nodes, and place sandboxes on any node | | `user` tokens | Place sandboxes on any node, and see the nodes | | Own-only `user` tokens | The same, except on plain-Docker and unisolated nodes, which they're refused: CI runs code from pull requests. They may use Sysbox nodes. | | `read` tokens | See the nodes | ## What a node can see A node runs your sandboxes, so it can see and change everything in them: their disks, processes and network traffic. It also gets their [settings](https://cloudex.build/docs/settings.md), which the deployment decrypts and sends at every start, as it does to a Cloudflare container. **Provider calls.** On a node, a sandbox's calls to an AI provider go to the node's agent, which passes them to your deployment, which adds the [provider key](https://cloudex.build/docs/keys.md) and sends them on. The sandbox's HTTPS connection ends at the agent, with a certificate from the node's own certificate authority, which can only sign the providers' host names. So the agent sees the agents' prompts and the providers' answers in plain text. It never sees the provider keys: requests leave it with the placeholder key, and the real key is added inside your Worker. **Saves.** A node writes the named saves its sandboxes make, and reads the saves they start from, while that runs. A save made on a node holds whatever that node put in it. A node can't: - read provider keys, the admin key, tokens or the deployment's database; - reach another node's sandboxes or a Cloudflare sandbox; - read or write other saves, or backups; - use your deployment as a proxy to anywhere but the providers' APIs and `cloudex.internal`; - open direct connections to another node's sandboxes: each node's tickets are signed with a secret of its own. Someone who takes over a node can still spend your provider keys for requests made in its sandboxes' names, for as long as it has a sandbox running. Set a spending limit with each provider if that matters, and remove a node you no longer trust with `cloudex node rm`. ## Performance Every command to a sandbox on a node goes through your deployment, as for a sandbox on Cloudflare, plus a round trip to the node and Docker's own time for each command. So does ssh, unless it goes [straight to the node](#direct-connections). ### Through your deployment These were measured with the CLI on a Mac in Florida on Wi-Fi, and a node on an AWS VM in us-east-1, which reached Cloudflare in Ashburn (IAD). The Cloudflare figures are from a `standard-2` sandbox in the same hour. | | Node | Cloudflare | |---|---|---| | `cloudex exec -- true`, start to end | 221 ms median, 241 ms p90 | 163 ms, 178 ms | | A command's round trip on a kept-alive connection | 99–141 ms median | 73 ms | | ssh connection set-up | 343 ms median | 315 ms | | Keystroke round trip in ssh | 63–68 ms median | 9 ms | | scp, 256 MiB, up / down | 23 / 28 MiB/s | 59 / 57 MiB/s | | `cloudex cp`, 90 MiB, up / down | 13–18 / 62–66 MiB/s | 22 / 19 MiB/s | | Create | 2.3 s | | | Stop | 0.45 s | | | Resume | 1.1–2.0 s | | A node that's far from you or from Cloudflare adds that distance to every round trip, unless ssh goes direct. `cloudex node info` shows where the node connects to Cloudflare and a round trip to its agent, and the deployment makes each of the node's sandboxes a Durable Object in the node's region, where Cloudflare has one. For long sessions, work inside tmux, with `cloudex attach` or the dashboard's terminal, so a dropped connection doesn't end what's running. ### Direct A direct connection is as fast as plain ssh to the machine: the TLS around it cost nothing measurable. These were measured with the CLI on the same Mac in Florida. The tailnet figures are v0.2.3's, against a VPS in New Jersey over Tailscale, with both peers connected directly, in the same runs as the figures through your deployment. The LAN figures are from the prototype of the same design, against a home server over Wi-Fi. | | LAN | Tailnet, far VPS | Through your deployment, to the same VPS | |---|---|---|---| | Keystroke round trip in ssh, median | 6.9 ms (plain ssh: 6.4) | 30 ms (plain ssh: 30) | 41 ms | | scp, up / down | 46 / 41 MiB/s, 1 GiB (plain scp: 45 / 42) | 39–43 / 35–41 MB/s, 256 MiB | 16–21 / 25–34 MB/s | | `cloudex ssh -- true`, start to end, median | | 550 ms | 736 ms | On a test VM on the same Mac, which is as close as a node gets, a keystroke took 1 ms and scp ran at 52–60 MB/s, against 18–44 through your deployment. When a direct try fails, falling back costs up to about 300 ms more than going through your deployment at once; in testing, between nothing and 330 ms, for a stale address, a closed port, another machine at the address, and a node that accepts and then stalls. Later connections on that network start through your deployment at once, for 10 minutes. ## Troubleshooting **From your computer:** - `cloudex node info box` shows the node's state and when it was last seen, its agent's version, its host (CPUs, memory, free disk), its runtime and isolation, when the self-test last ran and which checks failed, the sandbox image and whether it's present or still pulling, where it connects to Cloudflare, a round trip to it, its direct port's addresses and state (for admins), its fork images, its clock if it's off, pending deletes and orphans. - `cloudex doctor` warns about nodes that never joined, are offline, run an agent of another version, fail their self-test, have under 15 GB of Docker disk free, or whose clock is more than 30 seconds off. It also tries each online node's direct port from your computer, and says whether it answers. **On the machine:** ```sh sudo docker exec cloudex-node cloudex-node status # connected? exits 1 when not sudo docker logs --tail 50 cloudex-node # the agent's log sudo docker exec cloudex-node cloudex-node selftest # the self-test now, as JSON; exits 1 when it fails sudo docker exec cloudex-node cloudex-node selftest --machine # boot and check a throwaway sandbox sudo docker ps -a --filter name=cloudex-node-direct # the direct port's relay sudo systemctl status cloudex-node-firewall # the firewall's unit sudo nft list table inet cloudex # the firewall's rules ``` `status` says whether the agent is connected, since when, and if it isn't, the last error. `status --json` adds the node's id, its deployment and its last self-test. | Message | What it means | |---|---| | `that join token is unknown, used or expired` | A token works once, for 30 minutes. Run `cloudex node add box` again for a new one. | | `this node has joined already` | The machine has an agent's identity. To join again, [uninstall](#uninstall) first. | | `this deployment speaks node protocol … install the agent of the deployment's version` | Run the installer with the `--version` it names | | `node box's isolation self-test fails (…)` | The firewall isn't loaded, or something else answers from inside a sandbox. Check `systemctl status cloudex-node-firewall`, then the checks it names. | | `machines on this node may run only ghcr.io/lucataco/cloudex-base` | The deployment's image is from another repository. Run the installer with `--allow-image`. | | `every machine pool the installer knows collides with a route or a Docker network` | Give a free private /16 with `--pool` | | `Docker … is too old`, `nodes need nftables`, `run it as root` | A requirement in [What you need](#what-you-need) is missing | | `the node's clock is … s ahead` (or `behind`), in the agent's log | More than 60 seconds is refused. Set the machine's time with NTP. | | `this node was removed from the deployment`, in `status` and the agent's log | `cloudex node rm` removed it; the agent stopped its sandboxes. Uninstall, then add it again. | | `Direct … (failed): published on a public address` in `node info` | The direct port is published on a public address or all addresses, such as by hand. Run the installer again. | | `Direct … (failed)` with Docker's reason, such as `cannot assign requested address` | An address the installer chose isn't on the machine now. The agent tries again every 30 seconds; if the address changed, run the installer again. | | `Direct … (off): waiting for the deployment's secret` in `node info` | The agent has just started and hasn't heard from your deployment yet. It clears within seconds of the node coming online. | | `Direct off: turned off on this node (installed with --direct off)` | As it says. Run the installer again with `--direct auto` or `--direct-addr` to open the port. | | A sandbox's notice: `Node box stopped its machine: …` | The agent stopped it itself: Docker's disk fell under the floor, the firewall's table disappeared, or the self-test found the firewall open. The notice says which. | ## Uninstall ```sh curl -fsSL https://cloudex.build/node | sudo bash -s -- --uninstall ``` On the machine, `--uninstall` removes everything the installer made: the agent, the direct port's relay, every sandbox's container and network with their disks, fork images, the agent's volume, the firewall and its unit, `/etc/cloudex-node`, and the images it pulled, unless they were there before. It asks first, saying how many sandboxes it removes; `--yes` skips the question. `--keep-images` keeps the images it pulled, but not fork images: they're sandboxes' disks. If a fork's copy is under way, it waits for it to finish first, so that its image isn't left behind. Remove the node from your deployment too, with `cloudex node rm box`. Doing that first, while the node is online, deletes its sandboxes' machines through the agent (`--force` if it still has sandboxes), and the sandboxes stay listed in `error` until you delete them. If you uninstall first, the node shows offline until you remove it. --- # Sandboxes Source: https://cloudex.build/docs/sandboxes > Create, list, change, stop, resume and delete sandboxes, pick a size or a node, run commands at every start, manage a sandbox from inside it, and how idle stop, the 6-hour limit, saves and backups work. ## Create a sandbox ```sh cloudex new --name web cloudex new --name big --size standard-4 --idle 2h ``` `cloudex new` creates a sandbox, starts it and waits until it's running. The first start on a Cloudflare host can take a minute while the host fetches the image. | Flag | Default | Effect | |---|---|---| | `--name` | none | A name to use instead of the id | | `--size` | `standard-2` | `standard-1`, `standard-2`, `standard-3` or `standard-4`; see [Sizes](#sizes) | | `--idle` | `30m` | Stop after this long without activity; see [Idle stop](#idle-stop) | | `--no-use` | off | Don't make it the current sandbox | | `--from`, `--count` | none | Start from a copy of another sandbox's disk; see [Forks](https://cloudex.build/docs/forks.md) | | `--on`, `--cpus`, `--memory` | none | Run it on one of your [nodes](https://cloudex.build/docs/nodes.md) instead of Cloudflare, with these limits | | `--env`, `--env-file`, `--file`, `--repo` | none | Environment variables and files at every start, and a repo to clone; see [Settings](https://cloudex.build/docs/settings.md) | Every sandbox has an id like `cx-7f3a2k`. A name is optional and must be unique. Names are 2 to 32 characters of lowercase letters, digits and hyphens, start with a letter, don't end with a hyphen, and don't start with `cx-`, which marks ids. Commands take either the id or the name. `cloudex set --name` changes the name later; see [Change a sandbox](#change-a-sandbox). ## The current sandbox `cloudex new` makes the new sandbox the current one, and so does `cloudex fork` when it makes a single fork. Commands with an optional `[sandbox]` argument, such as `ssh`, `exec`, `stop` and `info`, act on it when you don't name one. `cloudex use ` picks another. ```sh cloudex list # all sandboxes; * marks the current one cloudex info # details of the current sandbox cloudex use api # make "api" the current sandbox ``` `cloudex list` (or `cloudex ls`) shows each sandbox's id, name, state, where it runs (`cloudflare` or a [node](https://cloudex.build/docs/nodes.md)), size, when it was last saved and when it was created. `cloudex info` adds the idle setting, how much the last save held, the last backup, what it was forked from or the [named save](https://cloudex.build/docs/saves.md) it started from, and notes such as a restore from backup. A sandbox is in one of five states: `starting`, `running`, `stopping`, `stopped` or `error`. `error` means its first start failed; the note in `cloudex info` says why, and `cloudex resume` tries again. A command the state doesn't allow says what to do instead. While the sandbox is `starting` or `stopping`, it says to try again in a moment. When the sandbox is `stopped`, it says to resume it. ## Sizes Sizes are Cloudflare's container instance types. The disk includes the system image. | Size | vCPU | Memory | Disk | |---|---|---|---| | `standard-1` | 1/2 | 4 GiB | 8 GB | | `standard-2` (default) | 1 | 6 GiB | 12 GB | | `standard-3` | 2 | 8 GiB | 16 GB | | `standard-4` | 4 | 12 GiB | 20 GB | The largest is 4 vCPU, 12 GiB and 20 GB. A sandbox on a [node](https://cloudex.build/docs/nodes.md) has the CPUs and memory you give it with `--cpus` and `--memory` instead, up to the node's own. A sandbox's size is set when you create it. `cloudex resume --size standard-4` starts a stopped sandbox on another size, which it keeps from then on; its disk has to fit the new size's. A [fork](https://cloudex.build/docs/forks.md) can also have a different size from its original. ## Stop and resume ```sh cloudex stop cloudex resume ``` `cloudex stop` saves the sandbox's disk, backs up `/home` to R2, and stops its container. A save holds only what changed since the previous one. Saving a full disk took about three minutes in testing; small changes take seconds. `cloudex resume` starts a stopped sandbox from its save, with its files, installed packages and agent logins. A sandbox with a few GB of files is usually back within seconds; one with 15 GB can take about two minutes. A save holds the disk, not running programs. After a resume, start your servers and agent sessions again, or let [start commands](#start-commands) do it. Agent history on disk is still there. On a [node](https://cloudex.build/docs/nodes.md), `cloudex stop` stops the machine in about a second and its disk stays on the node, with no save and no backup. `cloudex resume` starts it again in a second or two. ## Start commands ```sh cloudex exec web -- 'mkdir -p ~/.cloudex && echo "cd ~/app && npm run dev" > ~/.cloudex/start' ``` If the sandbox has a file `~/.cloudex/start`, it runs at every start: after `cloudex new` (and the repo clone, if any), `cloudex resume` and `cloudex fork`. It runs with bash as the `cloudex` user from a login shell, so it has the sandbox's [variables](https://cloudex.build/docs/settings.md) and [provider keys](https://cloudex.build/docs/keys.md), in the background, so the start doesn't wait for it. Its output goes to `~/.cloudex/start.log`. Use it to start dev servers, workers and agents' tools again after a resume. Since the file is on the sandbox's disk, saves, backups and forks keep it. To give a new sandbox one, use `cloudex new --file .cloudex/start=./start.sh`; a file from the [settings](https://cloudex.build/docs/settings.md) is written again at every start, replacing any change made inside the sandbox. ## Idle stop A sandbox stops by itself, saving its disk first, after `--idle` without activity. The default is 30 minutes. These count as activity: - An SSH connection that is sending data: `cloudex ssh`, plain `ssh` through [ssh-config](https://cloudex.build/docs/connect.md#scp-rsync-and-editors), `cloudex prompt` and `cloudex attach`. ssh sends a keepalive every 30 seconds, so an open session counts. A connection that has gone silent doesn't. - A command run with `cloudex exec`, or with an exec call through the API or an SDK, for as long as it runs, even when it prints nothing - `cloudex cp` and `cloudex run`, and command and file calls through the API or an SDK - A [background command](https://cloudex.build/docs/commands.md#background-commands) that has printed output in the last 10 minutes - Requests to a preview link - A browser showing the sandbox's [desktop](https://cloudex.build/docs/desktop.md) - An [agent session](https://cloudex.build/docs/agents.md) that has printed output in the last 10 minutes - Calls to AI providers that use [provider keys](https://cloudex.build/docs/keys.md) `--idle` takes a duration such as `45m` or `2h`, rounded down to whole minutes, up to 6 hours. `--idle 0` turns off Cloudex's own idle stop, which leaves Cloudflare's limit below; on a [node](https://cloudex.build/docs/nodes.md), which has no such limit, the sandbox then runs until you stop it. `cloudex set --idle` changes it later. ## Change a sandbox ```sh cloudex set --idle 2h # the current sandbox cloudex set web --name site # rename ``` `cloudex set` changes a sandbox's name or its idle time, running or stopped. A new idle time applies at once, counting from now. To change the size, stop the sandbox and run `cloudex resume --size`. ## Inside the sandbox Every sandbox has its own `cloudex` command, which manages that sandbox and nothing else. It needs no key: the deployment knows which sandbox each request comes from. ```sh cloudex host 3000 # a preview link to port 3000 cloudex host --list # the links that work; --revoke and --revoke-all as outside cloudex info # this sandbox cloudex set --idle 2h # stay up longer without activity cloudex stop # save the disk and stop ``` It prints JSON when its output isn't a terminal, like the CLI on your computer. It can't create, fork or delete sandboxes. `cloudex stop` answers at once, and the sandbox saves and stops a few seconds later. It works the same on a [node](https://cloudex.build/docs/nodes.md), where its requests go through the node's agent. Claude Code, Codex and OpenCode in a sandbox are told about it, in `/etc/claude-code/CLAUDE.md` and a marked block in `~/.codex/AGENTS.md` and `~/.config/opencode/AGENTS.md`, so an agent that starts a server can give you its link. A `cloudex` you install in the sandbox yourself is left alone. ## The 6-hour limit Whatever `--idle` says, Cloudflare stops a container after at most 6 hours of inactivity. Sandboxes on a [node](https://cloudex.build/docs/nodes.md) have no such limit. Its timer counts connections from outside, such as an open `cloudex ssh` session; work inside the sandbox, such as an agent running on its own, doesn't reset it. To keep a long task going past 6 hours, keep a connection open, for example `cloudex attach` in a terminal. When Cloudflare stops a sandbox, the sandbox saves its disk and backs up `/home` first. If that save fails, `cloudex info` says so, and `cloudex resume` starts from the previous save. ## Saves A save is a copy of the sandbox's whole disk, stored in your account's container registry. A sandbox on a [node](https://cloudex.build/docs/nodes.md) has none: its disk stays on the node when it stops. - A save expires 30 days after it was made or last restored. After that, `cloudex resume` restores `/home` from the newest [backup](#backups) onto a fresh copy of the image instead, and anything installed outside `/home` needs installing again. `cloudex resume` and `cloudex info` say when this happened. - A save keeps the system image it was made on. Resuming from it brings back the same system, even after `cloudex deploy` has installed a newer image. To move a sandbox onto the current image, use `cloudex resume --from-backup`. To keep a copy of a sandbox's disk for longer, and start new sandboxes from it, make a [named save](https://cloudex.build/docs/saves.md) with `cloudex save`. ## Backups Each stop also backs up `/home`, without `~/.cache`, and the SSH host keys to the R2 bucket `cloudex-backups` in your account. Cloudex keeps the newest backup for each sandbox. Backups don't expire, and they restore onto any image. ```sh cloudex resume --from-backup ``` `--from-backup` restores `/home` from the newest backup onto a fresh copy of the current image instead of using the save. Anything installed outside `/home`, such as apt packages and Docker images, needs installing again. - If a backup fails, the stop still succeeds and the save has your files. `cloudex stop` prints the error, and `cloudex info` shows it. - Backups need R2 turned on for the account. `cloudex status` shows whether they're on; see [Backups and R2](https://cloudex.build/docs/deploy.md#backups-and-r2). - In testing, backups moved about 110 MiB/s each way: a 5 GB home directory backed up in 43 seconds and restored in 54. - Sandboxes on a [node](https://cloudex.build/docs/nodes.md) have no backups. For a copy off the node, make a [named save](https://cloudex.build/docs/saves.md). ## Delete ```sh cloudex delete web ``` `cloudex delete` (or `cloudex rm`) deletes a sandbox and its backups. It asks first; `-y` or `--yes` skips the question, and is required when the command isn't run in a terminal. The files can't be recovered. The sandbox's last save stays in your account's container registry until it expires, within 30 days. On a [node](https://cloudex.build/docs/nodes.md), delete removes the sandbox's machine there. While the node is offline it's refused, unless you add `--force`: then the sandbox goes at once, and the node removes the machine when it's back. --- # Variables, files and repos Source: https://cloudex.build/docs/settings > Give a sandbox environment variables and files at every start, and a repo cloned when it's created, with cloudex new, cloudex env and the SDKs. GitHub tokens let git and gh work with private repos. A sandbox's settings are environment variables and files it gets at every start, and a repo cloned into it when it's created. Set them when you make the sandbox: ```sh cloudex new --name web \ --repo https://github.com/you/app \ --env GH_TOKEN=$(gh auth token) \ --env-file .env \ --file .npmrc=$HOME/.npmrc ``` Then change them on a running or stopped sandbox: ```sh cloudex env # names, files and the repo cloudex env set DATABASE_URL=postgres://... # the current sandbox cloudex env set web --env-file .env.test cloudex env unset web DATABASE_URL --file .npmrc ``` The [SDKs](https://cloudex.build/docs/sdk.md) take the same settings in `create()`, and change them with `updateSettings()`. ## Environment variables Every shell, ssh session, `cloudex exec`, agent and desktop terminal in the sandbox gets them. Values can hold anything, including spaces, quotes, `#` and newlines. | Flag | Effect | |---|---| | `--env NAME=VALUE` | One variable. Repeat it for more. | | `--env-file PATH` | `NAME=VALUE` lines from a `.env` file. Lines starting with `#` are skipped, `export ` is allowed, and quotes around a whole value are removed. | Names are letters, digits and underscores, not starting with a digit. `PATH`, `HOME`, `USER`, `LOGNAME`, `SHELL`, `PWD` and names starting with `CLOUDEX_` are reserved; set `PATH` in `~/.profile` instead. A sandbox can have up to 100 variables. A variable you set wins over a [provider key](https://cloudex.build/docs/keys.md) placeholder: `ANTHROPIC_API_KEY` in the settings is used as is, and goes to the provider unchanged. ## Files `--file DEST=SRC` copies the file `SRC` from your computer to `DEST` in the sandbox's home directory: `.npmrc`, `~/.npmrc` and `/home/cloudex/.npmrc` are the same place. The CLI turns a `DEST` your shell expanded, such as `/Users/you/.npmrc`, back into `~/.npmrc`. Files are written as the `cloudex` user, readable only by it, and their directories are created as needed. They're written again at every start, so an edit made inside the sandbox lasts until the next one. A file removed from the settings is deleted from the sandbox. A sandbox can have up to 20 files, and variables and files together can take up to 1 MB. ## Repos `--repo URL` clones a repo into the home directory when the sandbox is created, into a directory named after the repo. Add `#branch` to the URL to check out a branch or tag: ```sh cloudex new --repo https://github.com/you/app#dev ``` The SDKs and the API also take `{ url, dir, branch }`, to choose the directory. Only `https://` URLs work, and a URL with a user name or token in it is refused, because Cloudex shows the URL as is. For a private repo, set `GH_TOKEN`; see [GitHub](#github). The repo is cloned once; after that it's an ordinary directory, saved with the disk and copied to [forks](https://cloudex.build/docs/forks.md). If the clone fails, the sandbox starts anyway, with the reason in its `notice` (`cloudex info` shows it). ## GitHub With `GH_TOKEN` or `GITHUB_TOKEN` among a sandbox's variables: - `git` uses it for `https://github.com`, so cloning a private repo, and pushing, work without a login. That includes `--repo` when the sandbox is created. - `gh`, the GitHub CLI, uses it too, so an agent can open a pull request. A fine-grained token limited to the repos the sandbox needs is the safer choice: anything in the sandbox can read the variable. ## When changes take effect `cloudex env set` and `unset` change a running sandbox at once. New shells, `exec` calls and agent turns see the change; programs already running keep the variables they started with, until they're restarted. That includes an agent session that `cloudex prompt` started: its tmux server started with the old values. A stopped sandbox gets the change when it next starts. Forks get a copy of the original's settings, then change separately. ## Where they're kept The deployment keeps each sandbox's variables and files in its Durable Object, encrypted under a key derived from the admin key. `cloudex env` and the API show only names and paths; nothing returns values or contents. After `cloudex deploy --yes` replaces the admin key, the settings can't be read: the sandbox starts without them, with a notice, and the next `cloudex env set` starts over. Inside the sandbox, variables live in `/etc/cloudex/env.sh` and files in the home directory, so saves contain them like any file, and [backups](https://cloudex.build/docs/sandboxes.md#backups) contain the files. Anything that runs in the sandbox, including an agent, can read them. For AI provider keys, use [`cloudex keys`](https://cloudex.build/docs/keys.md) instead: those never enter the sandbox. --- # Connect to a sandbox Source: https://cloudex.build/docs/connect > Open a shell with cloudex ssh, reach sandboxes from scp, rsync and editors, run commands with cloudex exec, share ports as HTTPS preview links and revoke them, and forward ports. Sandboxes have no public address. Every connection goes through your deployment's API: SSH and port forwarding over a WebSocket, commands through `exec`, and web servers through preview links. That's the same for a sandbox on one of your [nodes](https://cloudex.build/docs/nodes.md): its agent keeps a connection to your deployment, and everything below reaches the sandbox through it, a little slower than on Cloudflare. A sandbox on a node that's offline can't be reached that way until it's back. The exception is SSH and what runs over it: from a computer on the node's LAN or tailnet, it goes [straight to the node](https://cloudex.build/docs/nodes.md#direct-connections). ## SSH ```sh cloudex ssh # the current sandbox cloudex ssh web # a sandbox by name or id cloudex ssh web -- uname -a # run one command ``` `cloudex ssh` runs your system's `ssh` with a proxy command that carries the connection over a WebSocket to your API, which connects it to port 22 in the sandbox. For a sandbox on a node, the proxy command also tries the node itself, and takes the [direct path](https://cloudex.build/docs/nodes.md#direct-connections) when it answers first. You log in as the `cloudex` user. Anything after `--` goes to `ssh`: options, or a command to run. The CLI uses its own SSH key, `~/.config/cloudex/id_ed25519`, and adds it to the sandbox the first time you connect. The sandbox must be running; if it's stopped, `cloudex ssh` tells you to run `cloudex resume`. ## Port forwarding Forward a port with ssh's `-L` option after `--`: ```sh cloudex ssh web -- -L 3000:localhost:3000 ``` While that session is open, `http://localhost:3000` on your computer reaches port 3000 in the sandbox. Only you can use it. To share a port with someone else, use a [preview link](#preview-links). ## scp, rsync and editors ```sh cloudex ssh-config >> ~/.ssh/config ``` This adds a block to your SSH config so plain `ssh`, `scp`, `rsync` and editors reach a sandbox as `.cloudex`: ```sh ssh web.cloudex scp notes.txt web.cloudex: rsync -a ./app/ web.cloudex:app/ ssh -L 5173:localhost:5173 web.cloudex ``` Editors that open folders over SSH can use the same host name, such as `web.cloudex`. The block looks like this, with your own paths: ```text # Cloudex sandboxes: ssh .cloudex Host *.cloudex User cloudex ProxyCommand '/usr/local/bin/cloudex' proxy %n 22 IdentityFile /Users/you/.config/cloudex/id_ed25519 IdentitiesOnly yes UserKnownHostsFile /Users/you/.config/cloudex/known_hosts StrictHostKeyChecking accept-new ServerAliveInterval 30 ``` It names the path of the `cloudex` binary, so run `cloudex ssh-config` again if you move it. These connections don't resume a stopped sandbox; run `cloudex resume` first. ## Run a command: exec ```sh cloudex exec -- npm test cloudex exec web -- 'cd app && git status' cloudex exec --root -- apt-get install -y ripgrep ``` `cloudex exec` runs a command with `bash -lc` as the `cloudex` user in its home directory, prints its output as it comes, and exits with the command's exit code. `exec -d` leaves a command running in the background. [Commands and files](https://cloudex.build/docs/commands.md) covers both, and copying files with `cloudex cp`. ## Preview links ```sh cloudex host 3000 cloudex host web 8080 ``` `cloudex host` prints an HTTPS link that serves a port in the sandbox on the preview domain you chose with `cloudex deploy --preview-domain`: ```text https://3000-cx-7f3a2k-a1b2c3d4e5.example.com ``` - The link holds a secret token made from your deployment's API key, the sandbox and the port. Anyone who has the link can open it, and nobody can guess it. There is no other sign-in. - The same sandbox and port give the same link until you [revoke it](#revoke-a-link), or the deployment's API key changes. - Links work only while the sandbox is running. A request to a stopped sandbox gets an error that says to run `cloudex resume`; it doesn't start the sandbox. - Requests to a link count as activity for [idle stop](https://cloudex.build/docs/sandboxes.md#idle-stop). - Servers that listen only on `localhost` work, such as Vite, Django, Flask, Rails and uvicorn with their default settings. The first request starts a small relay in the sandbox, which takes a moment; it stops when the server does, and never holds the server's port. A first request with a body, such as a form POST, goes through too when the body is at most 1 MB. One with a bigger body gets a 503 that asks you to send it again. - The server sees the link's host name in `Host` and `X-Forwarded-Host`, `X-Forwarded-Proto: https`, and the visitor's address in `X-Forwarded-For`. Visitors never appear to come from localhost, so tools that trust local requests, such as Rails' web console, don't open up to them. Vite's dev server accepts the preview domain without any configuration. Other dev servers that check the host name, such as webpack's or Angular's, need the preview domain in their allowed hosts. - WebSockets work, so hot reload does too. When the dev server itself restarts, reload the page by hand: through a preview link, Vite's page doesn't notice the restart. - Preview links need a preview domain. Without one, `cloudex host` says to run `cloudex deploy --preview-domain `. - Inside the sandbox, `cloudex host 3000` prints the same link; see [Inside the sandbox](https://cloudex.build/docs/sandboxes.md#inside-the-sandbox). Fetching a preview link from a script with Python's `urllib` needs a `User-Agent` header: Cloudflare's Browser Integrity Check, on by default for your domain, answers urllib's default one with error 1010. ### Revoke a link ```sh cloudex host web 3000 --revoke # this port's link stops working cloudex host web 3000 # a new link for the port cloudex host web --list # the links made since v0.1.14 that still work cloudex host web --revoke-all # every link to the sandbox, the desktop's too ``` A revoked link stops working at once, for everyone, and connections open through it, such as a desktop someone is watching, close. Asking for the port's link again gives a new one, which works until you revoke it in turn. `--list` shows the links made since Cloudex v0.1.14; `--revoke-all` also ends older ones. --- # Commands, files and CI Source: https://cloudex.build/docs/commands > Run commands with their output as it comes, leave long jobs and servers running in the background, copy files in and out with cloudex cp, and run CI jobs in a fresh sandbox with cloudex run. Four commands cover scripts, CI jobs and agents that drive a sandbox from outside: ```sh cloudex exec -- npm test # run a command; output as it comes cloudex exec -d -- npm run dev # leave one running in the background cloudex cp ./data :data # copy files in, or out cloudex run -- npm test # a fresh sandbox with this directory, then gone ``` The [API](https://cloudex.build/docs/api.md#commands) and the [SDKs](https://cloudex.build/docs/sdk.md) do the same. ## Run a command ```sh cloudex exec -- npm test cloudex exec web -- 'cd app && git status' cloudex exec -- python3 -c 'print(42)' cloudex exec --root -- apt-get install -y ripgrep cloudex exec --cwd app --timeout 30m -- make ``` `cloudex exec` runs a command with `bash -lc` in the sandbox, as the `cloudex` user in its home directory, as an SSH login would. Its standard output and standard error come back as they're printed, and `cloudex exec` exits with the command's exit code. One argument after `--` is a shell command line, so `&&`, pipes and globs work in it. Several arguments are one command and its arguments, each kept whole, the way `python3 -c 'print(42)'` reads. Without `--`, every argument is the command and the current sandbox runs it. | Flag | Default | Effect | |---|---|---| | `--root` | off | Run as root | | `--cwd` | the home directory | Working directory; a relative one starts in the home directory | | `--timeout` | `10m` | Stop the command after this long, at most `1h` | | `-d`, `--detach` | off | Start the command in the background; see below | A timeout, or Ctrl-C, stops the command and everything it started. `exec` has no terminal, and it doesn't send standard input, so a command that reads it sees it end at once. Use `cloudex ssh` for interactive programs. Its output is text, and bytes that aren't valid UTF-8 come back as U+FFFD. To put a file in a sandbox, or to get a binary file out, use [`cloudex cp`](#copy-files). With `--json`, it waits and prints `exitCode`, `stdout`, `stderr`, `timedOut` and `ms`, with each stream cut at 1 MiB. ## Background commands ```sh cloudex exec -d -- npm run dev # start; prints its id cloudex exec -d --timeout 2h -- make all # stop it after 2 hours cloudex commands # list them cloudex commands logs -f cm-1a2b3c4d # follow its output to the end cloudex commands kill cm-1a2b3c4d # stop it and what it started cloudex commands rm cm-1a2b3c4d # remove its record and output ``` `cloudex exec -d` starts a command and returns. The command keeps running in the sandbox with no time limit, unless `--timeout` sets one, and a dropped connection doesn't stop it. Its standard output and standard error go to one log on the sandbox's disk, which `cloudex commands logs` prints; with `-f` it keeps printing until the command ends and then exits with its exit code, or, for a command stopped by a signal, 128 plus the signal's number, as bash does. Ctrl-C stops following, not the command. `cloudex commands` shows each command's state: | State | Means | |---|---| | `running` | Still running | | `exited N` | Ended by itself with exit code N | | `killed (TERM)` | Stopped by a signal, from `cloudex commands kill` or anything else | | `timed out` | Stopped by its `--timeout` | | `lost` | The sandbox stopped while it ran | A stop ends every running command, and a resume doesn't start them again; put servers you want after every start in [`~/.cloudex/start`](https://cloudex.build/docs/sandboxes.md#start-commands). The records and logs stay until `cloudex commands rm` removes them, and [forks](https://cloudex.build/docs/forks.md) get a copy. [Named saves](https://cloudex.build/docs/saves.md) leave them out. A background command that has printed something in the last 10 minutes counts as activity for [idle stop](https://cloudex.build/docs/sandboxes.md#idle-stop), like an agent; one that stays quiet longer lets the sandbox stop. Cloudflare's [6-hour limit](https://cloudex.build/docs/sandboxes.md#the-6-hour-limit) still applies to jobs that run longer, except on a [node](https://cloudex.build/docs/nodes.md), which has none. ## Copy files ```sh cloudex cp report.csv :data/ # into the current sandbox's ~/data cloudex cp ./site web:/srv/site # a directory cloudex cp :app/dist ./dist # out of the current sandbox cloudex cp web:/var/log/app.log - # to standard output echo hello | cloudex cp - :hello.txt # from standard input ``` Name a path in a sandbox as `SANDBOX:PATH`, or `:PATH` for the current sandbox. Relative paths there start in the home directory. It works like `cp -r`: when the destination is a directory that exists, or ends with `/`, what you copy goes inside it; otherwise it's created with that name. Files keep their permissions, and directories keep their symlinks. A local path that is a symlink to a directory copies that directory. A download makes the local directories it needs. `-r`, `-R` and `-a` are accepted and change nothing, since directories always copy whole. A download can't reach outside where you put it: it never writes through a symlink, leaves out symlinks that point outside it, and leaves out `.git` directories, whose hooks and settings could run commands on your computer. To bring a sandbox's commits home, push them from the sandbox. A download also leaves out hard links, devices, and files the `cloudex` user can't read, which `--root` reads. The summary that `cloudex cp` prints on standard error counts what it left out, except the files it couldn't read. Files are read and written as the `cloudex` user, so the sandbox's permissions apply; `--root` reads and writes as root. A file is replaced whole, so nothing reads half of it. Cloudflare limits an upload to 100 MB at a time, compressed. For more, or to sync a folder repeatedly, use `rsync` through [ssh-config](https://cloudex.build/docs/connect.md#scp-rsync-and-editors). ## CI jobs: cloudex run ```sh cloudex run -- npm test cloudex run --from node-base --out dist -- 'npm ci && npm run build' cloudex run --env CI=1 --keep-on-failure -- make check ``` `cloudex run` does a whole job in one command. It: 1. Starts a sandbox. 2. Copies the current directory into it. 3. Runs the command there, printing its output as it comes. 4. Copies back what `--out` names. 5. Deletes the sandbox. It exits with the command's exit code, so a CI step fails when the command does. What `--out` copies back leaves out the same things as a [download](#copy-files). In a git repository it copies the files git would commit: tracked files, and new ones that `.gitignore` doesn't exclude, but not `.git` itself. Elsewhere it copies everything under `--dir`. They go to `~/`, where the command runs. `--from` starts from a [named save](https://cloudex.build/docs/saves.md), or a fork of a sandbox, instead of the plain image. Set up a sandbox once with your tools and dependencies, save it, and each run starts from that save with your latest files copied over it: ```sh cloudex new --name node-base cloudex exec node-base -- 'git clone https://github.com/you/app && cd app && npm ci' cloudex save node-base node-base cloudex run --from node-base --workdir app -- npm test # in ~/app, where node_modules is ``` The command runs as a [background command](#background-commands), so it has no time limit unless you set `--timeout`, and a dropped connection doesn't stop it; `run` reconnects for up to two minutes. Ctrl-C stops the command and deletes the sandbox; a second Ctrl-C quits at once and leaves the sandbox. A command stopped by its `--timeout` makes `run` exit with 124. | Flag | Default | Effect | |---|---|---| | `--from` | the image | Start from this named save, or a fork of this sandbox | | `--dir` | `.` | The directory to copy | | `--workdir` | `~/` | Where the files go, and the command runs | | `--no-upload` | off | Copy nothing | | `--out` | none | A file or directory to copy back after, relative to the workdir; it lands at the same place under `--dir`. Repeatable. | | `--env` | none | `NAME=VALUE`, repeatable. `--env-file` reads a file of them. | | `--timeout` | none | Stop the command after this long | | `--keep` | off | Keep the sandbox after | | `--keep-on-failure` | off | Keep it when the command fails or is stopped, to look around with `cloudex ssh` | | `--size`, `--name`, `--idle` | as `cloudex new` | The sandbox's size, name and idle stop | | `--on` | Cloudflare | Run it on one of your [nodes](https://cloudex.build/docs/nodes.md). An own-only token can use only nodes with Sysbox. | In GitHub Actions, give the job your deployment's URL and a [token](https://cloudex.build/docs/tokens.md) limited to its own sandboxes as secrets: `cloudex tokens create github-actions --own --expires 90d` makes one, and `cloudex status` prints the URL. ```yaml - run: curl -fsSL https://cloudex.build/install | CLOUDEX_INSTALL_DIR="$HOME/.local/bin" sh - run: ~/.local/bin/cloudex run --from node-base -- npm test env: CLOUDEX_API_URL: ${{ secrets.CLOUDEX_API_URL }} CLOUDEX_API_KEY: ${{ secrets.CLOUDEX_TOKEN }} ``` --- # Forks Source: https://cloudex.build/docs/forks > Make up to ten new sandboxes that start from a copy of one sandbox's whole disk, with cloudex fork or cloudex new --from --count. A fork is a new sandbox that starts from a copy of another sandbox's whole disk: files, installed packages, settings and agent logins. Set up a sandbox once, then fork it to try several approaches at once, or to give each agent its own copy. ```sh cloudex fork # one fork of the current sandbox cloudex fork web --name web-try cloudex fork web --count 5 --name agent # agent-1 to agent-5 cloudex new --from web --count 5 --name agent # the same thing ``` `cloudex new --from ` does the same as `cloudex fork ` and takes the same flags. | Flag | Default | Effect | |---|---|---| | `--count` | `1` | How many forks to make, up to 10 | | `--name` | none | The fork's name. With `--count`, forks are named `-1`, `-2` and so on. | | `--size` | the original's | `standard-1` to `standard-4`. Not for a sandbox on a node. | | `--cpus`, `--memory` | the original's | For a sandbox on a [node](https://cloudex.build/docs/nodes.md): the forks' CPUs and memory there | | `--idle` | the original's | Stop after this long without activity | | `--no-use` | off | Don't make the fork the current sandbox | ## What gets copied On Cloudflare: - A running sandbox is saved first. That takes a few seconds, and it keeps running; its file system pauses for a moment. - A stopped sandbox is copied from its last save. - Every fork starts from the same save, at the same time. Ten forks took about 20 seconds in testing. - The copy is of the disk, not of running programs. Start servers and agents again in each fork. - Anything stored in files comes along, including agent logins and any API keys you've saved on the disk. Keys stored with [`cloudex keys`](https://cloudex.build/docs/keys.md) aren't on the disk; forks use them like any new sandbox. Forks also get a copy of the original's [settings](https://cloudex.build/docs/settings.md), its environment variables and files, which then change separately. A fork on a node is copied differently; see [On a node](#on-a-node). A fork copies a sandbox as it is now. To keep a copy for later, and start sandboxes from it when you need them, make a [named save](https://cloudex.build/docs/saves.md) instead: it stays until you delete it. ## After forking Each fork is a separate sandbox from then on, with its own id, saves and backups. Changes in a fork don't reach the original, or the other way round. `cloudex info` shows what a sandbox was forked from. A single fork becomes the current sandbox, unless you pass `--no-use`. With `--count`, the CLI prints a table of the forks and leaves the current sandbox as it was. With `--json`, it prints the list of forks. A fork that fails to start is kept in the `error` state with the reason. The CLI says how many failed and exits with an error. Run `cloudex resume` on a failed fork to try again, or delete it. ## When a sandbox can't be forked - A stopped sandbox with no save can't be forked. Resume it, then fork it. - A stopped sandbox whose save has [expired](https://cloudex.build/docs/sandboxes.md#saves) can't be forked either. Resume it, which restores its backup, then fork it. - A sandbox that is starting or stopping can't be forked until it has finished. - A fork stays where its original is: a sandbox on Cloudflare can't be forked onto a node, nor one on a node onto Cloudflare or another node. Save the sandbox with `cloudex save`, then start sandboxes from the save with `cloudex new --from `, adding `--on ` to start them on a node. - A sandbox on a node that's offline can't be forked until the node is back. ## On a node A sandbox on a [node](https://cloudex.build/docs/nodes.md) forks on that node, where its disk is. Forks are new on nodes in v0.2.3, and need it on the deployment and the node's agent. ```sh cloudex fork n1 --count 3 # n1-1 to n1-3, on n1's node cloudex fork n1 --cpus 8 --memory 32G # more than n1 has ``` - **The copy.** The node copies the original's disk into an image, running or stopped, with no save and no 30-day limit. The forks of one request share that image, so the node holds one copy however many forks there are. Each fork then writes to its own layer on top. - **A running original pauses** while its disk is copied, then carries on. That takes about 2 to 3 seconds per GB the sandbox has written on Docker's overlay2 image store, or 12 to 13 seconds on its containerd image store, which Docker 29 uses on new installs. A sandbox that has written 5 GB is paused for about a minute on the containerd store. `cloudex fork` says before it starts about how long it is per GB on that node, and `cloudex node info` shows it too. Stop the sandbox first to avoid the pause. Starting the forks after the copy takes a few seconds. - **Disk.** The copy takes about as much space as the original had written, twice that on the containerd store, and is removed with the last sandbox that uses it, once each fork of the request has started or been deleted (a fork that hasn't keeps it for up to 7 days). Copies under way count against the node's free space, so a second fork, or a new sandbox, that would leave the node under 10 GB free once the copy is done is refused until it finishes. See [Disk](https://cloudex.build/docs/nodes.md#disk). - **Limits.** `--cpus` and `--memory` default to the original's, up to what the node has. `--size` is refused. Own-only tokens can't fork on plain-Docker nodes, as they can't make sandboxes there. Each fork of a fork adds a layer to its image, so forks of forks go about 100 levels deep; past that, make a named save and start from it, which starts flat. - **Docker inside.** On a Sysbox node, the forks have the original's Docker images and containers, whether it was running or stopped. - **What's new in each fork:** its id, host name, preview links and run state, as on Cloudflare. Like a Cloudflare fork, it shares the original's SSH host keys. ## From code The [SDKs](https://cloudex.build/docs/sdk.md) fork too. This runs a test suite in ten forks at once: ```ts import { Cloudex } from "cloudex"; const box = await new Cloudex().get("web"); const forks = await box.fork({ count: 10, name: "try" }); const results = await Promise.all(forks.map((f, i) => f.exec(["npm", "test", "--", `--shard=${i + 1}/10`]))); await Promise.all(forks.map((f) => f.delete())); ``` --- # Named saves Source: https://cloudex.build/docs/saves > Keep a copy of a sandbox's disk under a name until you delete it, start new sandboxes from it with cloudex save and cloudex new --from, and move sandboxes between Cloudflare and your nodes. A sandbox's own [save](https://cloudex.build/docs/sandboxes.md#saves) goes 30 days after it was last made or resumed from. A named save is a copy of a sandbox's disk that stays in your account's R2 bucket until you delete it. Set a sandbox up once, with its packages, Docker images, repos and agent logins, save it, and start new sandboxes from it whenever you need one. ```sh cloudex save base # save the current sandbox as base cloudex save web node-22 # save the sandbox web as node-22 cloudex new --from base --name task-1 # a new sandbox from the save cloudex new --from base --count 5 --name agent # agent-1 to agent-5 cloudex saves # list the saves cloudex saves rm node-22 # delete one ``` ## Making a save `cloudex save [sandbox] ` saves a running sandbox, which keeps running. To save a stopped sandbox, resume it first. Names follow the rules for sandbox names: lowercase letters, digits and dashes, 2 to 32 characters. Saving under a name that's taken replaces that save once the new one has finished; until then, the old one keeps working. In testing on `standard-2`, saving and restoring each took about 15 seconds per GB: a sandbox with 2 GB of new files saved in 26 seconds, and a new sandbox started from that save in 30. ## What a save holds The image lists its own files, so a save holds only what the sandbox changed: - Files you made or changed anywhere on the disk, and the image's files you deleted. - Packages installed with apt, npm or pip, and Docker's images, containers and volumes. - The sandbox's [settings](https://cloudex.build/docs/settings.md): its environment variables and files. Sandboxes started from the save get a copy, which then changes separately. - Everything else stored in files, including agent logins and any keys you've saved on the disk. Keys stored with [`cloudex keys`](https://cloudex.build/docs/keys.md) aren't on the disk; sandboxes started from a save use them like any new sandbox. A save leaves out: - The SSH host keys, so each new sandbox makes its own. - `/tmp`, `/run` and Cloudex's own logs. - Running programs. Commands in [`~/.cloudex/start`](https://cloudex.build/docs/sandboxes.md#start-commands) run when a sandbox starts from the save, and Docker containers with a restart policy start again. A file that's being written while the save is made, such as a busy database, may be saved halfway through a write. Stop such programs first for an exact copy. ## Starting sandboxes from a save `cloudex new --from ` takes the same flags as [a fork](https://cloudex.build/docs/forks.md): `--count`, `--name`, `--size`, `--idle` and `--no-use`. The size and idle time default to those of the sandbox that was saved. With `--on `, and `--cpus` and `--memory` if you like, the new sandboxes start on one of your [nodes](https://cloudex.build/docs/nodes.md). Each new sandbox starts from a fresh copy of the image, and the save is restored before anything in it runs, so services start with the restored files. `cloudex info` shows the save a sandbox started from. If a sandbox and a save have the same name, `--from` picks the sandbox; `--from save:` picks the save. ## Saves and nodes A sandbox on a [node](https://cloudex.build/docs/nodes.md) makes named saves as one on Cloudflare does. The save goes up to your R2 bucket over the node's own line. A save is how a sandbox moves between Cloudflare and a node, or between nodes. Start a sandbox from the save where you want it: ```sh cloudex save n1 n1-copy # n1 runs on the node box cloudex new --from n1-copy --name n1-cf # the same files, on Cloudflare cloudex new --from base --on box --name n2 # a Cloudflare save, on the node ``` The new sandbox has a new id and new preview links. Both sides run the same image, so the whole disk comes back, apart from a few device files that Docker in a Cloudflare sandbox keeps and a node can't make; the notice counts them. A save made on a node holds whatever that node put in it. `cloudex saves` shows the node each save was made on, in its ON column, and `cloudex new --from` says so when it starts one. Treat such a save like code from that node, especially before you start it on Cloudflare. ## After an update A save restores in full only onto the image it was made on. After [`cloudex deploy`](https://cloudex.build/docs/deploy.md) installs a newer image, a sandbox started from an older save gets only `/home` back: your files, anything installed in the home directory (`pip install --user`, `~/.local`), agent logins and `~/.cloudex/start`. Anything installed elsewhere, such as apt packages and Docker images, needs installing again, and the new sandbox's notice says so. `cloudex new` prints it, and `cloudex info` shows it. To bring a save onto the new image, start a sandbox from it, install what's missing, and save that sandbox under the same name. Sandboxes that run an image from before v0.1.15 make saves of `/home` only, because those images don't list their files. An image of your own built `FROM ghcr.io/lucataco/cloudex-base` (see `cloudex deploy --image`) should end with `RUN cloudex-files baseline`, which writes that list, so that saves hold only what sandboxes change. ## Deleting a save `cloudex saves rm ` deletes a save for good. It asks first; `-y` or `--yes` skips the question, and is required when the command isn't run in a terminal. Sandboxes already started from the save keep their disks. A sandbox that started from a save, but stopped before it made a save of its own, starts from that save again when you resume it. If the save has been replaced since, it uses the newest save of that name, and if the save has been deleted, a fresh copy of the image. Its notice says which. ## Storage Saves are kept, compressed, in the R2 bucket `cloudex-backups` in your account. Like [backups](https://cloudex.build/docs/sandboxes.md#backups), they need R2 turned on; see [Backups and R2](https://cloudex.build/docs/deploy.md#backups-and-r2). R2 charges for storage beyond its free tier, so delete the saves you no longer need. `cloudex saves` shows how much each one holds and how much it takes up. ## From code The [SDKs](https://cloudex.build/docs/sdk.md) make and use saves too: ```ts import { Cloudex } from "cloudex"; const cx = new Cloudex(); await (await cx.get("setup")).save("base"); const boxes = await cx.startFromSave("base", { count: 3, name: "job" }); ``` ```python from cloudex import Cloudex cx = Cloudex() cx.get("setup").save("base") boxes = cx.start_from_save("base", count=3, name="job") ``` --- # Desktop Source: https://cloudex.build/docs/desktop > Open a Linux desktop with Chrome and a terminal from a sandbox in your browser, share it with a link, and automate its Chrome on localhost:9222. ```sh cloudex desktop ``` `cloudex desktop` starts a desktop in the sandbox and opens it in your browser. The desktop has Google Chrome, a terminal (xterm), the openbox window manager and a tint2 panel. It runs on TigerVNC, and your browser shows it through noVNC. The first start installs the desktop, about 950 MB, which takes a minute or two. On a [node](https://cloudex.build/docs/nodes.md), it comes over the node's own line. A save keeps it, so later starts take seconds. A [restore from backup](https://cloudex.build/docs/sandboxes.md#backups) doesn't keep it, because it's installed outside `/home`; the next start installs it again. ## On your computer, or by link By default, the CLI forwards a local port, opens your browser on it, and keeps the connection until you press Ctrl-C. The desktop keeps running in the sandbox after that, and `cloudex desktop` connects to it again. ```sh cloudex desktop --link ``` `--link` prints an HTTPS link on your [preview domain](https://cloudex.build/docs/deploy.md#preview-domain) instead. It works in any browser while the sandbox runs, for anyone who has the link. The link is the only protection; the desktop has no password. To make a shared link stop working, run `cloudex host 6080 --revoke`; see [Revoke a link](https://cloudex.build/docs/connect.md#revoke-a-link). ## Options | Flag | Default | Effect | |---|---|---| | `--size` | `1280x720` for a new desktop | Screen size as WIDTHxHEIGHT. A running desktop is resized to it; without `--size`, it keeps its size. | | `--quality` | `3` | Picture quality from 0 to 9. Lower uses less bandwidth on moving images; text stays sharp. | | `--link` | off | Print an HTTPS link on your preview domain instead of using a local port | | `--port` | `6080`, or a free one | The local port to use | | `--no-open` | off | Don't open a browser | | `--stop` | off | Stop the desktop and the programs on it | ```sh cloudex desktop web --size 1920x1080 cloudex desktop --quality 0 cloudex desktop --stop ``` A still page uses little bandwidth. Full-screen motion can use tens of Mbit/s; on a slow connection, `--quality 0` cuts that by more than half. ## Windows from the sandbox While the desktop runs, programs started from a shell in the sandbox show their windows on it, as long as the shell was opened after the desktop started. That includes a browser an agent opens to test a page. The desktop's Chrome listens for automation, the Chrome DevTools Protocol, on `localhost:9222` inside the sandbox. Tools in the sandbox, such as Playwright or Puppeteer, can connect to `http://localhost:9222`. To reach it from your computer, forward the port: ```sh cloudex ssh -- -L 9222:localhost:9222 ``` Chrome keeps its profile in `~/.config/cloudex-chrome`, so sign-ins and history stay with the sandbox. ## Idle stop and updates A browser showing the desktop counts as activity, so the sandbox stays up while you watch it. When you close the browser, [idle stop](https://cloudex.build/docs/sandboxes.md#idle-stop) counts from then. Running `cloudex deploy` can drop desktop connections; the viewer reconnects by itself. --- # Coding agents Source: https://cloudex.build/docs/agents > Run Claude Code, Codex or OpenCode on a task in a sandbox with cloudex prompt and reattach with cloudex attach, or without a terminal with cloudex agents, the API and the SDKs. Every sandbox has Claude Code, Codex and OpenCode installed. `cloudex prompt` gives one of them a task in a tmux session inside the sandbox, so it keeps working after you disconnect. [`cloudex agents`](#without-a-terminal) and the [SDKs](#from-code) start agents with no terminal instead: you follow what they do, and send them follow-ups. ## Sign in once Sign in to each agent once in a sandbox: ```sh cloudex ssh claude # or: codex, opencode ``` Follow the agent's own sign-in steps. The login is saved in your home directory with the sandbox's disk, so it survives stop and resume, it's in backups, and [forks](https://cloudex.build/docs/forks.md) copy it. A new sandbox made with `cloudex new` needs signing in again. To skip signing in, store a provider's API key with `cloudex keys set`: Anthropic for Claude Code, OpenAI for Codex, and any of them for OpenCode, which also works with OpenRouter, Google, Groq, Mistral, xAI and DeepSeek. Every sandbox then uses it, and the key itself stays outside them. While a provider's key is stored, its agent uses the key even where someone has signed in with a subscription. See [Provider keys](https://cloudex.build/docs/keys.md). For an agent to clone, push and open pull requests on GitHub, give the sandbox a token with `cloudex env set GH_TOKEN=...`; `git` and `gh` use it. See [Settings](https://cloudex.build/docs/settings.md#github). ## Give an agent a task ```sh cloudex prompt "make the tests pass" cloudex prompt web --agent codex "add a dark mode" cloudex prompt --agent opencode "write a README" cloudex prompt -d "upgrade the dependencies" ``` `cloudex prompt` starts the agent in a new tmux session in the home directory, then attaches your terminal to it. Name the directory to work in as part of the task if the code isn't in the home directory. | Flag | Default | Effect | |---|---|---| | `--agent` | `claude` | `claude`, `codex` or `opencode` | | `-d`, `--detach` | off | Start the agent without attaching | Press Ctrl-b d to detach. The agent keeps working. When it finishes, the session drops to a shell and stays open until you exit it. Agents run without their own permission prompts: Claude Code with `--dangerously-skip-permissions`, Codex with `--dangerously-bypass-approvals-and-sandbox` and OpenCode with `--auto`. The sandbox is the boundary. On a [node](https://cloudex.build/docs/nodes.md) under plain Docker, that boundary is weaker: give agents there only work you'd trust them with on the machine itself. Claude Code's warning about this mode is accepted for you, by `"skipDangerousModePermissionPrompt": true` in `~/.claude/settings.json`; set it to `false` to be asked. Don't put anything in a sandbox that the agent shouldn't be able to read or change. ## Reattach ```sh cloudex attach # the newest agent session cloudex attach web agent-1029-142501 # a given session in a given sandbox ``` Sessions are named `agent--