Run an agent node

Adding an app

Your HTTP service behind the connector at its own price, confined, with its variables and secrets.

toon add puts a new app behind the connector of a TOON app you already run. The app gets its own route, g.toon.<segment>.<app>, and its own price, and shares the connector's identity, channels and peerings.

Add an app to an existing connector unless it needs its own identity, prices for the whole connector, or its own peerings. Then create a TOON app instead.

What an app is

An app is a plain HTTP service. The connector unseals each paid packet and makes the HTTP request inside it (method, path and body) to the app, then returns the app's answer to the payer. The app never sees a payment, so any HTTP service can be one. gas-station and anytoon are real payment-oblivious services, worth reading to see what a full app looks like. Neither can be started with toon add --image: each listens on its own ports and needs configuration and secrets toon add does not pass. Each is deployed with its own compose file, behind its own connector. The minimal app below is the one image this guide shows running under toon add --image.

Given as a container image, an app must:

• listen on port 3100 inside the container (TOON_BLS_PORT says so too),
• answer GET /health with 200 within two minutes of starting,
• keep anything it must not lose under /data (TOON_DATA_DIR), which is a directory in the agent node's home that survives restarts.

The app is given TOON_BLS_PORT and TOON_DATA_DIR, and whatever variables and secrets you add it with (Variables and secrets). An image that needs another variable to start fails with app_failed; toon logs <app> shows what it wrote before it stopped.

toon runs it with docker, confined, on a network of its own that publishes no port: nothing reaches the app except through its connector, and the app reaches nothing. See What an app may and may not do.

Any answer is a paid answer. A 404 or a 400 is real work the payer asked for, so the packet is still fulfilled; status codes are for the payer to read, not a refund.

A minimal app

# app.py: answers every request with what it was sent
from http.server import BaseHTTPRequestHandler, HTTPServer

class App(BaseHTTPRequestHandler):
    protocol_version = "HTTP/1.1"

    def answer(self):
        size = int(self.headers.get("content-length") or 0)
        body = b'{"path":"%s","received":%d}' % (self.path.encode(), len(self.rfile.read(size)))
        self.send_response(200)
        self.send_header("content-type", "application/json")
        self.send_header("content-length", str(len(body)))
        self.end_headers()
        self.wfile.write(body)

    do_GET = do_POST = answer

HTTPServer(("0.0.0.0", 3100), App).serve_forever()

FROM python:3.13-alpine
COPY app.py /app.py
EXPOSE 3100
CMD ["python", "-u", "/app.py"]

docker build -t echo-app .

The same app is used by the end-to-end run, in docs/end-to-end/app.

Walkthrough: add an app

1. Find the TOON app to add it to.

toon status
# TOON app relay (ILP address g.toon.fb0e007c71750599): connector running ...

2. Add it, with a price. --price is what a client pays the connector for one packet to the app, in base units; it defaults to 0.

toon add echo --to relay --image echo-app --price 2 --yes --json > add.json
jq '{address, restarted}' add.json
# { "address": "g.toon.fb0e007c71750599.echo", "restarted": true }

A connector reads its routes only when it starts, so add restarts it, which drops the packets it holds in flight. While the connector is running, that needs --yes; without it add fails with confirmation_required and changes nothing. On a stopped agent node no --yes is needed and the app starts with the next toon up.

3. Check it is running and pay it.

toon status
toon logs echo
toon send "$(jq -r .address add.json)" --amount 2 --yes
echo '{"text":"hello"}' > body.json
toon send "$(jq -r .address add.json)" --amount 2 \
  --method POST --path '/say?loud=1' --body body.json --yes

toon logs echo shows what the app wrote to its standard output and standard error, kept in its data directory once its container stops, so it also answers after a failed start or while the agent node is stopped. toon logs echo --connector shows the log of the connector it is behind, where its paid requests pass. An app added with --url is not run here and has no log.

--method and --path are the request the app receives, and --body is a file sent as its application/json body; without them the request is an empty POST /. This, not the app's loopback address, is how you use it: a call to the loopback address is not paid.

An app that takes a signed event as its body is sent one made by toon event sign, which prints an event signed with the agent identity and publishes nothing:

toon event sign --kind 5301 --tags '[["i","https://example.com","url"]]' > body.json

A packet to an app behind your own connector needs no --seal-to. Others reach it over a peering toward your connector, with --seal-to set to your connector's /ilp URL.

An app you already serve

Give --url instead of --image and toon runs nothing; the connector delivers to that URL. status shows it as served there, since its process is yours to keep running:

toon add search --to relay --url http://127.0.0.1:8080/ --price 10 --yes

Keep such an app on loopback. Anything that can reach its URL directly can use it without paying.

Choose the address

By default the route is g.toon.<segment>.<app>. --address sets another prefix. Keep it under g.toon.<segment>: that is where every address the connector answers to sits.

toon add search-v2 --to relay --image search:2 --address g.toon.fb0e007c71750599.search.v2 --yes

Say what a client should send

--request <file> takes a file holding one JSON object that states what a client should send the app. The connector publishes it on the route, in routes[] of GET <connector>/ilp and in its terms, and never reads it, so its content is yours to define. A client reads it with toon describe, which shows each route's request under the route. A probe is not how to read it: the reject toon probe reports states what the path costs and carries no request.

cat > request.json <<'EOF'
{ "protocol": "http", "method": "POST", "path": "/echo", "params": { "text": "a string" } }
EOF
toon add echo --to relay --image echo-app --request request.json --yes
toon route list --json | jq '.routes[] | {prefix, request}'

toon checks only that the file is one JSON object a TOML table can carry: a file that is anything else, or holds a null, is refused with usage before anything changes. An app added without --request is published with no request. The request stays with the app across toon down and toon up and across restarts. To change it, remove the app and add it again.

Change the price

toon route price g.toon.fb0e007c71750599.echo 5 --yes
toon route list

This restarts the connector too, so it needs --yes while the connector runs.

Take an app away

toon remove echo --yes

It removes the app and its route and restarts the connector.

Variables and secrets

An app that calls a chain or signs anything needs a key, and an image may read its settings from its environment. Give them when adding the app, with toon add or toon create and only with --image (any of them with --url is refused with usage). Each flag is repeatable.

toon add minter --to relay --image minter:1 --env LOG_LEVEL=debug \
  --secret SIGNING_KEY=./signing.key --secret-file /run/secrets/minter.key=./minter.key --yes

A variable is kept in the app's record in the agent node's home, and toon status shows its name and toon status --json its value. A secret is copied into the app's secrets directory in the home, outside /data, in a file only your account can read (mode 0600, in a directory of mode 0700). The record holds its name or its path in the container, never its contents, and no output, log line or error message of toon carries them. The copy is what the app is started with every time, across toon down, toon up and restarts, although you delete your own file. Removing the app removes the copies.

toon status --json shows, for each app:

"variables": { "LOG_LEVEL": "debug" },
"secrets": { "variables": ["SIGNING_KEY"], "files": ["/run/secrets/minter.key"] }

with empty values for an app that has none, and null for both for an app at a URL.

Which form to use for a secret: one given with --secret is a variable of the container, and anyone who can use docker on the machine can read it (docker inspect). One given with --secret-file is a file the app can only read, and docker does not show it. Use the file where the image can read one. Whoever can read the agent node's home can read both, as they can already sign as the agent.

These are refused with usage, before anything is written, pulled or restarted:

• a name that is not a letter or _ followed by letters, digits and _, a name that begins TOON_ (those belong to the contract between toon and an app), and a name given twice across --env and --secret;
• a file that does not exist, cannot be read, is empty or is larger than 64 KiB;
• for --secret, contents that are not UTF-8 or that hold a NUL byte or a newline other than the trailing one;
• for --secret-file, a path that is not absolute or that holds :, , or .., a path at or under /data, /tmp or one of the app's --scratch paths, and a path given twice, in any spelling.

A variable or a secret is not an allowance: it tells the app something and lets it reach nothing more. They are set when the app is added, and toon change changes them (below). The relay takes none: what it is started with comes from its settings and the agent identity. An agent key cannot supply a secret. It can name one an app needs, with --secret NAME or --secret-file <path> and no file; the add is then held as a pending app, and the operator supplies each file on toon approve <app> --secret NAME=<file> --secret-file <path>=<file>. The approval must give exactly the secrets named, no fewer and no more, or it is refused with usage.

What an app may and may not do

Every app runs confined (ADR 0011), and so does the relay, except that the relay stays on the host's network. An app has:

• a docker network of its own, made when the app starts and removed when it stops. It is an internal network: the app opens no connection out and reaches no other app. No port is published; toon reaches the app at its container's address on that network;
• no capabilities and no way to gain any;
• the user and group of the process that runs toon, whatever the image names;
• a read-only root filesystem. It can write only /data, a 64 MB /tmp and its scratch paths;
• 512 MB of memory with no swap, 1 CPU and 256 processes.

toon status --json reports this as confinement for each app.

An operator can make an allowance for one app when adding it, with toon add or toon create (and only with --image; any of them with --url is refused with usage):

A value that cannot be used (a relative scratch path, /data or /tmp as one, a size that does not parse, zero) is refused with usage and nothing changes. The allowances are kept with the app across toon down, toon up and restarts. To change them, use toon change (below).

There is no allowance for a capability, for another user or for running unconfined. An image that needs root or a capability is not supported, and fails with app_failed. Rootless docker is not supported: there the host cannot reach a container's address. Each app takes one docker network, and docker's default address pools hold about thirty.

An app that was added before apps were confined has no allowances and is confined with the defaults from the next toon up. If it no longer starts, or its calls out now fail, it may need an allowance (most often --allow-egress: toon change <app> --allow-egress --yes) or a new image; an image is changed by removing the app and adding it again. When an app does not start or does not answer its health check, app_failed says it runs confined; toon logs <app> shows what it wrote.

A runtime for an app

A confined app still shares the machine's kernel. A runtime starts its container under a sandbox such as gVisor's runsc, so that the kernel the app talks to is not the machine's. A runtime is not an allowance: it lets the app do nothing more, and an app with a runtime and nothing else has no allowances.

toon add oracle --to relay --image example.com/team/oracle --runtime runsc --yes

--runtime <name> on toon add --image and toon create --image starts that app's container with docker run --runtime <name>. It is kept with the app, so it holds across toon down, toon up and restarts, and toon status --json reports it as confinement.runtime (null for docker's own); plain toon status names it. --runtime default means docker's own runtime: no --runtime reaches docker. A name holds letters, digits, ., _ and - and does not start with -; anything else, and --runtime with --url, is refused with usage and nothing changes.

The agent node can have a runtime too. If the home holds untrusted-runtime, a text file you edit by hand (toon init does not write it), the first line that is not blank and does not start with # is its name:

# gVisor, for every image off the list of trusted images
runsc

When you add or create an app from an image that is not on the list of trusted images (trusted-images, see agent-keys.md) and give no --runtime, the app gets the node's runtime. An explicit --runtime, default included, always wins. The list never refuses your add; this is the only thing it does to it. An unusable name in the file refuses the add with usage.

The runtime is settled when the app is added and recorded with it. Editing or removing untrusted-runtime, or editing trusted-images, changes no app already added, and no app in a home made before runtimes. To change one app's, run toon change <app> --runtime <name> (or --runtime default); the app restarts alone, as for a change of a limit, and keeps its allowances, variables and secrets. The relay has no runtime: toon change relay --runtime is refused with usage.

toon does not check that the runtime is installed. Where it is not, docker refuses to start the container and the add fails with app_failed, whose message names the runtime beside what docker said. Installing gVisor and registering runsc with docker is yours to do. An agent key cannot name a runtime and have the app added at once (see agent-keys.md).

Change an app

toon change <app> gives an app that is already added other variables, secrets or allowances and keeps everything else about it: its image, address, price, request and whatever no flag names. It takes the flags of toon add (--runtime, --allow-egress, --scratch, --memory, --cpus, --pids, --env, --secret, --secret-file), with the same meaning and the same checks, and flags that take something away: --no-allow-egress, --unset-scratch <path>, --unset-env <name>, --unset-secret <name>, --unset-secret-file <path>. --memory, --cpus and --pids each accept default, which returns the limit to its default.

toon change oracle --secret API_KEY=./new-key --unset-env DEBUG --memory 2g --yes

A name is one thing the app is started with: --secret NAME=<file> over a variable NAME replaces the variable with the secret, and --env NAME=value over a secret NAME replaces the secret. A name given twice in one command is refused. A secret given again is always a change.

It restarts the app, so it refuses with confirmation_required without --yes. Packets to the app are rejected until it is back; packets to the other apps behind its connector are delivered. The connector restarts too, and drops the packets it holds in flight, only if the app comes back somewhere else: the app keeps its docker network across the restart (granting or withdrawing --allow-egress remakes it), and the relay keeps its ports if they are still free. Otherwise the connector is left running, with its pid and its count of restarts as they were. --json and the sentence it prints say which happened (restarted). If the agent node is not running, the change is checked and recorded, and the app starts with it at the next toon up. If the app does not start with the change, the change is put back, the app starts again as it was (and its connector restarts only if the app comes back somewhere else), and the command fails with app_failed; the copy of a secret that was replaced stays in the home until the app has started with the change, and is deleted then. --json prints the app, its TOON app, what was set and what was taken away by name (never a secret's contents) and whether a connector restarted.

It is refused with usage, with nothing recorded and nothing restarted, when no flag is given, when the change leaves the app as it is, when it takes away a variable, secret, secret file or scratch path the app does not have (the message names what it has), and when the app is at a --url, which is not run. The relay takes only --memory, --cpus and --pids. An unknown app is unknown_name.

What it refuses

Written from toon_cli/docs/guide/adding-an-app.md at commit cc23333f90c4.