The SDK wire protocol
shadowlap's SDKs are thin clients of one small HTTP API on the sidecar. This page specifies it completely, so that an SDK can be written for a language shadowlap does not ship.
Model API calls need none of this: they are captured by pointing the provider SDK at the sidecar through its base URL, which works in any language already. This protocol is only for the things a provider SDK cannot see: tool calls, and reads of the clock and randomness.
A client that implements this page should pass the conformance suite, which runs against a real sidecar and needs no API keys.
Normative language. MUST, SHOULD and MAY carry their usual weight. Where this page settles a question the shipped SDKs answered differently, it says so.
On this page
- A complete exchange
- Discovery
- The URL is opaque
- Outside shadowlap
- Transport
- Never through a proxy
- Timeouts
- Failure is fatal
- Endpoints
POST /v1/ calls/ resolve POST /v1/ calls/ {callNo}/ complete - User input
- MCP tool calls
POST /v1/ guard POST /v1/ hello - Errors
- Error types
- Injected failures
- Scopes
- Scope names
- Call identity
- Checklist for a new client
- Versioning
- See also
A complete exchange
#A tool call is two requests: resolve before anything runs, and complete once the tool has run. A
read tool during a recording:
→ POST /t/<token>/v1/calls/resolve
x-shadowlap-run: 01K5CQ4N7G
{ "kind": "tool", "name": "search_web", "effect": "read", "args": {"q":"shadowlap"}, "implHash": "9f2…" }
← 200 { "origin": "live", "callNo": 7 }
… the client runs the real implementation …
→ POST /t/<token>/v1/calls/7/complete
{ "outcome": "ok", "result": {"hits": 3}, "latencyMs": 412 }
← 200 { "ok": true }The same tool during a clean replay is one request, and the tool never runs:
→ POST /t/<token>/v1/calls/resolve
{ "kind": "tool", "name": "search_web", "effect": "read", "args": {"q":"shadowlap"}, "implHash": "9f2…" }
← 200 { "origin": "replayed", "callNo": 7, "outcome": "ok", "result": {"hits": 3} }
… the client returns {"hits": 3} without calling the implementation …Discovery
#shadowlap run and shadowlap replay set these environment variables for the agent process:
| Variable | What it does | Default |
|---|---|---|
SHADOWLAP_URL | Base URL of the sidecar, including a session token path | Absent: shadowlap is not running |
SHADOWLAP_RUN_ID | The run these calls belong to | MAY be absent when the sidecar serves a single run |
SHADOWLAP_SCOPE | Optional. The scope this process starts in | — |
SHADOWLAP_STDIN | Optional. The input shadowlap was given, as dev:ino (unsigned) | — |
SHADOWLAP_RUN_ID: send it when it is present.SHADOWLAP_SCOPEis for an agent whose branches are separate processes. A client reads it as the initial scope, and scopes entered in code nest underneath.SHADOWLAP_STDIN: a client that records what a person types reads it from that input only, a terminal, or the null device (see User input). On Windows only a file has one:devis its volume's 32-bit serial number andinoits 64-bit file id.
The URL is opaque
#A client MUST treat SHADOWLAP_URL as opaque. It already contains the session token as a path
segment (http://127.<token>), and the sidecar rejects any request that does not carry
it. So a client:
- strips a trailing
/; - appends the endpoint paths below;
- changes nothing else.
Because the token is a credential, a client MUST NOT log the URL or include it in an error message.
Outside shadowlap
#When SHADOWLAP_URL is unset, the client MUST do nothing: call the real tool, read the real
clock, return the real random value. A wrapped tool outside shadowlap must behave exactly like the
function it wraps. This is what makes it safe to leave the wrappers in production code.
Transport
#Every request is POST with a JSON body and content-type: application/json. Responses are JSON.
A client sends these headers:
| Header | When |
|---|---|
content-type: application/json | Always |
x-shadowlap-run: <run id> | Whenever SHADOWLAP_RUN_ID is set |
x-shadowlap-scope: <scope> | When the calling context is inside a scope |
user-agent | SHOULD name the language, e.g. shadowlap-go/0.1 |
x-shadowlap-sdk: <language>/<version> | SHOULD name the client and its release, e.g. go/0.1.0 |
x-shadowlap-scope:%and anything outside printable ASCII percent-encoded. See Scopes.user-agent: the sidecar buckets clients by this for telemetry only; it never affects behavior.x-shadowlap-sdk: the official SDKs sendnode/<version>andpython/<version>on every call. A run's summary names an SDK whose major.minor differs from the sidecar's, once. The header never changes how a call is answered.
There is no authentication header. The token in the path is the whole of it.
Never through a proxy
#The sidecar is on the machine's loopback. A client MUST reach it directly, whatever proxy the
environment names (HTTP_PROXY, a system proxy). Through a proxy, the call would cross the network
in clear text with the session token in its URL, and the proxy can't reach the loopback anyway.
Both SDKs open the sidecar with proxies turned off, whatever the agent sets up for its own traffic:
- The Python SDK uses a urllib opener that has no proxy handler.
- The TypeScript SDK uses
node:httpand an agent of its own rather thanfetch, whose dispatcher an agent may route through a proxy (undici'sProxyAgentignoresNO_PROXY).
shadowlap run adds 127.0.0.1, localhost and ::1 to NO_PROXY and no_proxy for the agent,
for the clients that read them.
Timeouts
#A client SHOULD apply a request timeout of about 30 seconds. The sidecar answers resolve and
complete locally and does not wait on the network, so a slow response means it is wedged, not
busy.
Failure is fatal
#If the sidecar is unreachable, times out, or answers with a status outside 2xx, the client MUST propagate the failure to the caller. It MUST NOT fall back to executing the tool. Falling back would run a side effect the run was meant to replay, and would leave a hole in the recording: the two failures shadowlap exists to prevent.
On a non-2xx response the body is {"error": {"message": "..."}}. A client SHOULD surface that
message.
Endpoints
#The sidecar serves four endpoints:
| Endpoint | What it is for |
|---|---|
POST /v1/calls/resolve | What to do about a call that is about to happen |
POST /v1/ | What a live or stubbed tool call did |
POST /v1/guard | Optional. An outbound connection that bypassed shadowlap |
POST /v1/hello | Optional. A preload says it loaded |
POST /v1/calls/resolve
#Asks the sidecar what to do about a call that is about to happen. Send it before running anything.
A tool call:
{ "kind": "tool", "name": "search_web", "effect": "read", "args": { "query": "…" }, "implHash": "…" }Its fields, required unless the type says optional:
| Field | Type | Meaning |
|---|---|---|
kind | string | "tool" |
name | string | Non-empty. Part of the call's identity |
effect | string | "pure", "read" or "write". The sidecar rejects anything else with 400 |
args | JSON | The arguments as JSON, null if there are none. Part of the call's identity |
implHash | string, optional | Any stable hash of the implementation's source |
trust | string, optional | "untrusted" for a read that brings outside text in (a page, an email) |
input | boolean, optional | true for what a person typed (User input) |
mcp | object, optional | Where a call through shadowlap mcp came from (MCP tool calls) |
effecthas no default, by design. The only safe default would bewrite, and making that implicit would hide a decision the developer has to make. A client MUST require it and SHOULD reject a bad value locally, before any request.implHashis never part of matching. It only powers the "this tool changed since the recording" warning.trustis never part of matching either."untrusted"marks whereshadowlap injectionplants payloads, stored as the call'smeta.trust, withmeta.trustFromsdk(mcpfor a call throughshadowlap mcp)."trusted"(the default) is stored as nothing.- A tool the config's
toolsnames is stored with the config's word instead, either way, andmeta.trustFromconfig.
A clock or randomness read:
{ "kind": "clock", "name": "now", "label": "" }
{ "kind": "random", "name": "random", "label": "" }
{ "kind": "random", "name": "uuid", "label": "" }| Field | Type | Meaning |
|---|---|---|
kind | string | "clock" or "random" |
name | string | "now" for the clock; "random" or "uuid" for randomness |
label | string | Disambiguates concurrent reads of the same kind. Part of the call's identity |
Send "" as the label when there is none. These reads resolve in one round trip: the sidecar
generates the value and records it, so there is no complete for them.
The response is one of seven shapes. For example, a replayed tool call:
{ "origin": "replayed", "callNo": 7, "outcome": "ok", "result": { "hits": 3 } }Each shape tells the client what to do:
origin | Shape | What the client does |
|---|---|---|
replayed | { origin, callNo, outcome: "ok", result } | Return result. Do not run the implementation. |
replayed | { origin, callNo, outcome: "error", error } | Raise the recorded error (see below). Do not run the implementation. |
replayed | { origin, callNo, outcome: "error", error, injected } | Raise the failure --fail-at asked for, as the kind injected names (see Injected failures). Do not run the implementation. No complete. |
live | { origin, callNo, outcome: "ok", result } | Clock and randomness only: return result. No complete. |
live | { origin, callNo } | Tools: run the implementation, then complete. |
blocked | { origin, callNo, outcome: "error", error } | Raise a write-blocked error. Do not run the implementation. No complete. |
stubbed | { origin, callNo } | Do not run the implementation. Produce the stub value (see below) and complete with it. |
The response's fields:
| Field | Type | Meaning |
|---|---|---|
origin | string | replayed, live, blocked or stubbed |
callNo | integer | Identifies the call for complete |
outcome | string | "ok" or "error", in the shapes that have one |
result | JSON | The value to return |
error | object | A serialized error to raise |
injected | object | The failure --fail-at asked for (Injected failures) |
A client MUST treat an unknown origin as an error rather than guessing.
blocked and stubbed happen only after a fork, and only for write tools. blocked is the
default; --writes stub produces stubbed instead.
POST /v1/calls/{callNo}/complete
#Reports what a live or stubbed tool call did. Send it exactly once per such call:
{ "outcome": "ok", "result": …, "latencyMs": 12 }
{ "outcome": "error", "error": { … }, "latencyMs": 12 }| Field | Type | Meaning |
|---|---|---|
outcome | string | Anything other than "error" is read as "ok" |
result | JSON, on ok | Send null rather than omitting it |
error | object, on error | A serialized error (below) |
latencyMs | number, optional | How long the implementation took. Send 0 for a stub |
The sidecar answers 200 with { "ok": true }. It answers 409 if the call was already completed,
and 404 if callNo is unknown.
Numbers. Numbers in result (and in a resolve call's args) are kept as the client wrote them:
a replay serves 20.0 and 1234567890123456789 back with the same digits. In args, an integer
past 2^53 matches by its exact digits, and a divergence report shows it as its digits in a string.
Any other number matches by the value it reads as, so 20.0 matches 20.
A live tool call MUST be completed on both paths: when it returns and when it throws.
- A call left pending is a hole in the recording: the replay has no answer for it.
- A result that can't be turned into JSON: the client MUST complete the call with
outcome: "error"describing that, and then raise. Both shipped SDKs do this, and theunserializable-resultconformance scenario checks it. - A completion the sidecar refuses (a
400) is the same hole. So a client that fails to send theokcompletion SHOULD complete the call with an error before raising.
A call the caller cuts off (Python's asyncio.wait_for deadline, a cancelled task) has not
failed: the tool was stopped. The Python SDK completes it with outcome: "error", the error named
CancelledError and data: { "cancelled": true } (false when the tool raised CancelledError
from an awaitable of its own).
Replaying it, a client MUST NOT raise a replayed error. It waits until the caller cancels the call
again, as its deadline or cancel did when it was recorded, so the agent takes the same path. An
interrupt or an exit inside a tool (KeyboardInterrupt, SystemExit) is not the tool's answer
either, and leaves the call as the run left it.
NaN and Infinity. JSON has no NaN or Infinity, and the sidecar refuses a body that has them.
Both shipped SDKs record them as null, in arguments and results, as JSON.stringify does. The
caller still gets the tool's own value while recording, and the null on replay (the
non-finite-result scenario).
User input
#What a person types is input from outside the agent, like a tool's result, and a replay has no one
to type it. A client records it as a read tool named user_input, with the prompt as its argument
and "input": true on resolve:
{ "kind": "tool", "name": "user_input", "effect": "read", "args": { "prompt": "How can I help? " }, "input": true }It completes the call with the answer as { "text": "…" }, without the line ending. When the input
ended (Ctrl-D, a closed pipe), it completes with outcome: "error" and an error named EOFError,
and raises whatever the language raises at the end of its input. The sidecar stores the call with
meta.input = true.
How the sidecar answers an input call on replay:
| When | The sidecar answers |
|---|---|
| Before a fork | As it answers any read |
| After a fork | replayed, with the recorded answers in the order they were given, until they run out; then live |
After a fork, with --live-input | live |
After a fork there is no one to type, so the recorded answers are served whatever the prompt says by
then. A replayed result is { "text": … }, or a bare string when the run overrides the answer
(--override n='"…"'). A client accepts both.
Every run the CLI starts an agent for is marked meta.inputs = true: what a person typed is part of
what it holds. So an input call a replay of it doesn't find is a change, and the run forks there. A
recording without the mark and without any input call was made before input was recorded. A replay
of it answers its input calls live and goes on replaying, rather than forking at the first one.
The shipped clients record input without code changes. shadowlap's preloads record:
- in Python,
input()and reads ofsys.stdin; - in Node, readline's
questionandfor await.
They record it on the input shadowlap gave the agent only: a terminal, the null device, or the pipe
or file named by SHADOWLAP_STDIN as dev:ino. userInput() / user_input() in the SDKs record
input that arrives any other way.
MCP tool calls
#shadowlap mcp resolves each tools/call it forwards like any tool call, with the server's tool
annotations as its effect, and adds where the call came from:
{ "kind": "tool", "name": "send_note", "effect": "write", "args": { "to": "billing" },
"mcp": { "transport": "http", "server": "mcp.example.com", "tools": [{ "name": "send_note", "effect": "write" }] } }The mcp object's fields:
| Field | Type | Meaning |
|---|---|---|
transport | string | stdio or http |
server | string | The remote server's host, or the local server's command, never its arguments or a URL's path |
tools | array, optional | The server's tools with their effects |
server, for a local server, includes the package a runner such asnpxstarts.toolscomes on the first call to that server in a run and whenever the list changes. It marks a read whose server setsopenWorldHintwith"trust": "untrusted", and the call itself then carriestrusttoo.
The sidecar checks and bounds the object and stores it as meta.mcp.
POST /v1/guard
#Optional. Reports an outbound connection that bypassed shadowlap, for the network guard's summary:
{ "target": "api.example.com:443", "action": "blocked" }| Field | Type | Meaning |
|---|---|---|
target | string | Where the connection went; a UDP target is named udp://host:port |
action | string | "blocked" or "warned" |
It answers 202 with { "ok": true }. target is truncated to 300 characters. A client that does
not intercept the network can ignore this endpoint entirely.
POST /v1/hello
#Optional. A preload says it loaded in one process of the agent, once, when it has set itself up:
{ "runtime": "node", "runtimeVersion": "22.16.0", "pid": 4242, "ppid": 4241, "guard": "warn", "capture": true }| Field | Type | Meaning |
|---|---|---|
runtime | string | The process's runtime, such as node; claude-code from the fence |
runtimeVersion | string | The runtime's version |
pid, ppid | integer | The process's id and its parent's |
guard | string | "warn", "block" or "off" |
capture | boolean | Whether in-process capture is on |
reach | array of strings | The fence's only: what the CLI can still connect to unwatched |
The Claude Agent SDK's fence says hello for the Claude Code CLI, which loads no preload:
"runtime": "claude-code", "guard": "off", and reach, a few words each (20 kept), [] when
nothing is left open.
It answers 202, or 409 when the run has ended.
- One entry per process. The sidecar keeps one entry per runtime and
pid. A later hello from that process (a worker thread's, a second query's fence) updates it. Itsreachholds what both said, and is unknown once one says nothing of it. - A run with no hello was never watched. A run the CLI started its agent for that made model or
tool calls with no hello from any process was never watched by the network guard: its summary
says so, and under
--guard blockit fails. - A hello whose
guardisoffdoes the same, except the Claude Code CLI's when itsreachis empty orguard.accepttakes it.
A client that loads no preload sends none. The sidecar also reads x-shadowlap-sdk
(python/0.1.0) on a hello, as on every /v1/calls/* request. It names an SDK whose major.minor
differs from its own once per run.
Errors
#A serialized error is:
{ "name": "TimeoutError", "message": "upstream timed out", "code": "ETIMEDOUT", "data": null }| Field | Type | Meaning |
|---|---|---|
name | string | The error's name |
message | string | Its message |
code | string, optional | Its code, such as ETIMEDOUT |
data | JSON, optional | Anything else it carries |
module | string, optional | The module that defined the error's class, left out for a builtin |
args | array, optional | The arguments the error was made with |
The original class is not preserved across a replay, only these fields. So a client SHOULD raise a dedicated "replayed error" type carrying them, rather than trying to reconstruct the original.
module and args let a replayed error print as the original did, where the language prints more
than the message:
module: a traceback names a class by it (myapp.errors. OrderMissing).argsis left out when it is the message alone or isn't JSON as it is (a tuple, say). Python'srepr()prints them (KeyError('A-17'),RuntimeError()).
The Python SDK records and replays both; a client that doesn't use them ignores them:
{ "name": "FileNotFoundError", "message": "[Errno 2] No such file or directory: 'orders/A-17'", "args": [2, "No such file or directory"] }Error types
#Every client SHOULD expose two error types, because agent code catches them by name:
| Error type | Raised for | Carries |
|---|---|---|
| replayed error | replayed with outcome: "error" | name, message, code, data |
| write-blocked error | blocked | The tool name and the arguments it was called with |
A write-blocked error carries those so a handler can report what was prevented.
A replayed error SHOULD read as the recorded error did: the same text when turned into a string or
printed (Python's str() and repr(), TypeScript's String() and util.inspect()). Agents hand a
tool's error to the model as text, so a replayed error that reads differently changes the next
request, and a replay of unchanged code forks. Both SDKs raise an instance of a subclass of their
replayed-error type named after the recorded error.
Injected failures
#shadowlap replay --fail-at <call>= fails a tool call instead of
serving or running it, so the agent's handling of a failing tool runs. The sidecar answers resolve
with a replayed error that also carries injected, and has already recorded the call:
{ "origin": "replayed", "callNo": 19, "outcome": "error",
"error": { "name": "TimeoutError", "message": "search_web timed out", "code": "ETIMEDOUT" },
"injected": { "kind": "timeout" } }The error is what a real failure of that kind would carry, and nothing more. Agents hand errors to
the model (an MCP client passes data on too), and a model told it is being tested behaves
differently. Only injected says otherwise, and only to the client.
A client SHOULD raise an injected failure that looks like the failure it stands for, because the agent's handling of the real one is what is being tested:
| Kind | injected | What the client raises |
|---|---|---|
timeout | { "kind": "timeout" } | The language's timeout error: TimeoutError in Python, an error named TimeoutError with code ETIMEDOUT in TypeScript |
error | { "kind": "error", "message": "…" } | An error with error.message; the message is optional |
status | { "kind": "status", "status": 503 } | An error that carries the status the way the language's HTTP clients do (status_code, status) |
The injected failure SHOULD carry injected = true, the tool's name and its arguments, so a test can
tell it from a real failure. The client MUST NOT run the implementation and MUST NOT call complete.
A proxy for another protocol answers in that protocol's shape for a failing tool: shadowlap mcp
sends the MCP client a tool result with isError: true whose text is the message.
A client that does not know injected raises its replayed-error type with the same name and message.
That still never runs the tool, which is why the failure travels in this shape: an older client keeps
working, and gets the less specific error.
Scopes
#A scope names the branch of the agent a call was made in: one sub-agent of several running at once. Repeats are counted per scope, so two branches that send the same request at the same time each get their own recorded answer on replay, in whatever order they arrive.
A client implements this with whatever its language uses for implicit context: AsyncLocalStorage
in Node, contextvars in Python, a context.Context in Go. It sends the current value as
x-shadowlap-scope on every request made inside it, for tools, clock, randomness and captured HTTP
alike.
- Scopes nest by joining with
/:researcherinsideteamisteam/researcher. - The sidecar ignores a scope that is empty, longer than 200 characters, or contains control characters. A client SHOULD apply the same limits rather than relying on that.
- Outside shadowlap, entering a scope MUST still work: it just sets the variable.
Warning: A client that skips scopes is not merely less featureful: it is wrong for parallel agents, which will swap each other's recorded answers on replay. Implement scopes or document loudly that the client is single-branch only.
Scope names
#A scope can have any name, and a header holds Latin-1 at most. So a client MUST percent-encode % and
every character outside printable ASCII (as UTF-8 bytes) before sending it:
| Scope | Sent as |
|---|---|
東京 | %E6%9D%B1%E4%BA%AC |
50% off | 50%25 off |
/ and the rest of printable ASCII are sent as they are, so an ASCII name without % is unchanged.
The sidecar decodes the header, and records the name as it was given; a value that doesn't decode is
taken as it was sent. The scope-names conformance scenario checks it.
Call identity
#The client does not compute identity; the sidecar does. It is worth knowing what feeds it, because it determines what the client must send faithfully:
| Call | What feeds its identity |
|---|---|
| Tools | name and args, canonicalized and passed through the run's normalizers |
| Clock and randomness | kind, name and label: random() and uuid() are two reads, labelled or not |
Arrival order is never part of identity. A call is matched by (kind, fingerprint, occurrence),
where occurrence is n for the n-th identical call in that scope.
The recording format specifies how a fingerprint is computed.
This is why args must be serialized consistently. A client that reorders object keys between runs
still matches (the sidecar canonicalizes), but one that sometimes omits a null field and sometimes
includes it does not.
Checklist for a new client
#- Read
SHADOWLAP_URLandSHADOWLAP_RUN_ID; do nothing at all when the first is unset. - A tool wrapper that takes a required effect and rejects a bad one locally.
resolvebefore running anything, withname,effect,argsand ideallyimplHash.- Handle all seven response shapes, an injected failure included; treat an unknown
originas an error. completeeveryliveandstubbedcall, on both the success and the failure path.- One round trip for clock and randomness, with no
complete. - Scope propagation through the language's implicit context, as
x-shadowlap-scope. - A replayed-error type, a write-blocked type, and an injected-failure type.
- Fail closed: never run the implementation because the sidecar was unreachable.
- Run the conformance suite.
Versioning
#This protocol is pre-1.0 and may change. It is the protocol of the release that runs the sidecar,
which shadowlap --version reports: a client sends no protocol version, and the sidecar asks for
none.
A client names its own release in x-shadowlap-sdk. A run's summary says when an SDK's major.minor
differs from the sidecar's, since an SDK and the CLI work together at the same one.
See also
#- The recording format: how calls are stored, and how a fingerprint is computed.
- SDKs: the shipped TypeScript and Python clients.
- The engine as a library: reading and writing recordings directly.