# Cadre Agent API — Open Spec v1

The Cadre Agent API is an **open, local integration surface** that lets any AI
agent drive the *editing* of a screen recording: inspect recording metadata and
private time-aligned speech and token-bounded visual context, inspect the timeline, add and
adjust zoom keyframes, cuts, speed segments and captions, set style, and trigger
and monitor an export. Codex and Claude Code have first-class, one-click
integrations, while the protocol remains a plain
[Model Context Protocol](https://modelcontextprotocol.io) (MCP) server so any
MCP-capable agent can connect.

An agent can **record as well as edit** (v1.2, 7 recording tools). Recording
uses the same start path as the app's own Record button: Cadre creates the
floating recording-controls window for the take, the countdown defaults on but
can be disabled, and screen/area capture attempts its highlight and display
dimming cues. Recording and editing are free to try; export retains the same
licence gate as the UI. See
[`tools.md` § Recording](./tools.md#recording).

- **Spec version:** `v1`
- **Transport:** MCP streamable HTTP over `127.0.0.1` (loopback only)
- **Auth:** `Authorization: Bearer <token>` on every request
- **Tools:** 57 (see [`tools.md`](./tools.md)) — 47 editing/perception + 7 recording + 3 background-job lifecycle
- **Offline alternative:** direct `project.json` editing (see
  [`project-format.md`](./project-format.md))

---

## 1. How it works

Cadre embeds an MCP server inside its Electron **main process**. When the app
launches it:

1. Mints a fresh bearer token (32 bytes of entropy, base64url).
2. Binds an HTTP server to `127.0.0.1` on an **ephemeral port** (`listen(0)`).
3. Writes the port + token to a **connection-info file** (see §2), `chmod 600`.
4. Registers the 57 v1 tools on `POST /mcp`.

The MCP **transport is stateless**: a fresh MCP server + transport is created
per POST and torn down when the response closes. Long imports, transcriptions,
and model downloads use a bounded in-memory job registry owned by the current
boot token. Each start response includes a second unguessable `jobToken` needed
to poll or cancel that job. Jobs never survive an app restart: active work is
aborted on shutdown, results expire after 15 minutes, at most four jobs run at
once, and at most 32 terminal results are retained.

Two classes of tool sit behind the same surface:

- **Main-owned tools** (project listing/open, license, export, caption
  generation) call main-process services directly.
- **Bridge-backed tools** (everything that mutates the live timeline or style)
  forward to the renderer's editor stores over an in-process request/response
  bridge, because **the renderer's in-memory state is the source of truth while
  a project is open in the editor** — not the file on disk. This is why edit
  tools require a project to be open (they return `NO_PROJECT_OPEN` otherwise),
  and why offline `project.json` editing is only safe when the editor is
  *closed* (see [`project-format.md`](./project-format.md)).

---

## 2. Discovery — the connection-info file

On boot the server writes:

```
~/Library/Application Support/Cadre/agent-api.json
```

> The directory is Electron's `userData` path, which resolves to
> `<appData>/<productName>`. The product name is **Cadre**, so on macOS the
> file lives under `Cadre/`. (Do not hardcode this — read `agent-api.json`
> from the path above; on other builds the parent folder follows the app's
> product name.)

Shape:

```json
{
  "port": 51734,
  "token": "u2Yk8s...base64url...9Qp",
  "version": "<current-app-version>",
  "pid": 48211,
  "startedAtMs": 1787693000123
}
```

| Field     | Type     | Meaning                                                                 |
|-----------|----------|-------------------------------------------------------------------------|
| `port`    | number   | Ephemeral TCP port bound on `127.0.0.1`. Changes every launch.          |
| `token`   | string   | Bearer token required on every request. **Changes every launch.**       |
| `version` | string   | Agent-API / app version.                                                |
| `pid`     | number   | Main-process PID — lets a client detect a stale file after a crash.     |
| `startedAtMs` | number | Estimated main-process start time — lets a client detect PID reuse.  |

The MCP endpoint is therefore:

```
http://127.0.0.1:<port>/mcp
```

**Lifecycle guarantees:**

- The file is written **fresh on every boot** — `port` and `token` both rotate,
  so a client must re-read it after the app restarts.
- The shutdown path attempts to delete the file before closing the server.
- After a crash or force-quit the file may survive. Shipped bridges check the
  recorded `pid` and, when present, `startedAtMs` before trusting it.
- Writes request permissions of `0600` (owner read/write only) and follow with a
  `chmod(0600)` attempt. The token is plaintext and this mode does not isolate
  it from every process running as the same user.

---

## 3. Connecting

### 3a. Codex / ChatGPT (recommended: one click)

In Cadre, open **Preferences → AI Editing** and click **Connect** in the
**Codex / ChatGPT** card. Cadre
installs a local `cadre` plugin containing both the current MCP connection and
the `cadre-editor` workflow skill. Whenever Cadre relaunches and rotates its
private port/token, it materialises and reinstalls a new credential-specific
plugin version automatically. Open a new Codex task and ask:

> Use Cadre. Open my most recent project, inspect it, tighten the dead air, add
> cinematic zooms, then export it.

No repository checkout, `config.toml` edit, terminal command, port, or token is
required. See the public [connection guide](https://cadre.cam/agents.html#connect)
for the supported setup.

### 3b. Claude Code (recommended: one click)

In Cadre, open **Preferences → AI Editing** and click **Connect** in the
**Claude Code** card.
Cadre registers the current endpoint at user scope, then refreshes it
automatically whenever Cadre relaunches and rotates its private port/token.
Open a new Claude Code session and ask:

> Use Cadre. Open my most recent project, inspect it, tighten the dead air, add
> cinematic zooms, then export it.

No repository checkout, terminal command, port, or token is required.

### 3c. Claude Desktop and Cowork (local sessions)

In **Preferences → AI Editing**, click **Prepare plugin** in the **Claude
Cowork** card. Cadre opens Claude and reveals a token-free plugin archive.
Upload it once from Claude's **Customize → Plugins** screen. The plugin's local
bridge reads Cadre's rotating connection file at call time, so it continues to
work after Cadre restarts without storing a token in the archive.

This path works in Claude Desktop and local Cowork sessions. Remote Cowork
sessions run outside this Mac and cannot reach Cadre's localhost-only service.

### 3d. Manual registration (developers / generic clients)

Read the port and token from `agent-api.json`, then register the server:

```bash
INFO=~/Library/Application\ Support/Cadre/agent-api.json
PORT=$(jq -r .port "$INFO")
TOKEN=$(jq -r .token "$INFO")

claude mcp add --transport http cadre \
  "http://127.0.0.1:${PORT}/mcp" \
  --header "Authorization: Bearer ${TOKEN}"
```

Cadre's one-click integrations wrap this registration and refresh the private
connection whenever the app restarts. The public
[connection guide](https://cadre.cam/agents.html#connect) covers the supported
customer flows; generic clients can use the protocol directly as described below.

### 3e. Generic MCP client

Any MCP client that speaks streamable HTTP works. Point it at
`http://127.0.0.1:<port>/mcp` and send the bearer token on every request. For a
project-scoped Claude Code config, a `.mcp.json` looks like:

```json
{
  "mcpServers": {
    "cadre": {
      "type": "http",
      "url": "http://127.0.0.1:51734/mcp",
      "headers": {
        "Authorization": "Bearer u2Yk8s...9Qp"
      }
    }
  }
}
```

Because the port and token rotate on every launch, a hand-written static config
goes stale after the next app restart. Codex and Claude Code users should
prefer Cadre's one-click integrations, which refresh their managed connection
on launch.

### 3f. Verify the connection

Once registered, list the tools. From Codex or Claude Code:

```
> use the cadre mcp server to call get_app_state
```

`get_app_state` is the recommended first call — it reports the version, whether
a project is open, license status, and current export status, so an agent can
orient before editing.

The server publishes the inspect-first sequence as MCP `instructions` and
annotates every tool with accurate read-only, destructive, idempotent, and
open-world hints. Cadre's generated Codex plugin uses
`default_tools_approval_mode: "writes"`: perception calls run quietly while
mutations continue to follow Codex's write-approval policy.

---

## 4. Security posture

The Agent API exposes only the documented recording and editing surface, and
only through a loopback-bound connection:

- **Loopback bind only.** The server binds `127.0.0.1`; it is never reachable
  from another host.
- **The connected AI client is a separate data boundary.** Loopback keeps the
  transport between Cadre and the client on the Mac. When the client requests
  transcript, interaction or frame context, that result enters the client's
  process, and a cloud-backed client may send it to its model provider. The
  client's service, account settings and data policy apply to that transfer;
  review them separately before connecting an assistant to sensitive material.
  Cadre's [privacy page](https://cadre.cam/privacy.html) explains this boundary
  in user-facing terms.
- **Bearer token on every request.** Both values are SHA-256 hashed, then the
  fixed-width digests are compared with `timingSafeEqual`; this avoids a direct
  variable-length buffer comparison. An unauthenticated request gets `401`.
- **Long-job capability tokens.** `import_video`, `generate_captions`,
  `generate_transcript`, and `download_caption_model` return immediately with
  a random `jobId` + `jobToken`. Polling and cancellation require the exact
  pair, unknown and mismatched pairs return the same `NOT_FOUND` response, and
  the registry is destroyed when the app's bearer token rotates.
- **DNS-rebinding defence.** Requests whose `Host` header is not a loopback
  name are rejected with `403`.
- **Shared recording UI and TCC gates.** An agent *can* start a capture (v1.2),
  but only through the same main-process path the Record button uses. Cadre
  creates the floating recording-controls window. The countdown defaults on but
  is optional; the blue target highlight and non-recorded-display dimming are
  conditional, best-effort cues for screen/area capture. macOS TCC still
  governs desktop capture — the automation path cannot grant Screen Recording
  access.
- **The export paywall is enforced server-side.** Recording and editing are
  free to try; `export_video` requires an active license in the main process
  (`LICENSE_REQUIRED`). An agent cannot bypass that gate and follows the same
  policy as the UI (D-059).
- **No agent-owned auto-stop.** There is deliberately no server-side duration
  timer. An agent that dies mid-recording leaves the capture running until the
  user stops it or the disk monitor force-stops it at the critical threshold.
- **Connection file requests `0600`.** Other Unix users should not be able to
  read it; same-user process isolation is outside this file-mode control.

**Known v1 risk (documented, not yet mitigated):** `export_video`'s
`outputPath` is a caller-selected absolute `.mp4` or `.mov` destination subject
to the validation documented in the tool reference. There is no allow-list of
writable directories in v1 — the one destination that *is*
blocked is the open project's `recording/` directory (see
[`tools.md`](./tools.md#export_video)), because writing there would overwrite
the only copy of the source capture, not because of a general access-control
policy. Treat the agent as you would any local process writing files on your
behalf. A future version may restrict `outputPath` to user-chosen directories.

---

## 5. Errors

Every tool returns either a success result (a single JSON block carrying the
post-mutation entity or snapshot, so an agent can verify its effect) or a
structured error result with `isError: true` and this shape:

```json
{ "error": { "code": "NO_PROJECT_OPEN", "message": "...", "hint": "..." } }
```

| Code                  | When                                                                 |
|-----------------------|----------------------------------------------------------------------|
| `EDITOR_NOT_AVAILABLE`| No editor window is open to service a live-edit command.             |
| `NO_PROJECT_OPEN`     | An editor exists but no project is loaded. Call `open_project`.      |
| `LICENSE_REQUIRED`    | `export_video` without an active subscription. Recording remains available so the user can try the complete editing workflow. |
| `EXPORT_IN_PROGRESS`  | An export is already running. Poll `get_export_status` or cancel.    |
| `INVALID_ARGS`        | Payload failed validation — cross-field (e.g. `endTime <= startTime`), a sanity ceiling (e.g. a time value past 24 h, a caption over 5,000 characters), a path-safety rule (relative, `..`, NUL byte, leading-dash filename), a `set_style` structural limit, or a partial-update `updates` object with no recognised field. See [`tools.md`](./tools.md) for the exact bound on each parameter. |
| `NOT_FOUND`           | Referenced entity id (zoom/cut/caption) does not exist, a recording source is not currently enumerable, or a background `jobId` + `jobToken` pair is unknown/expired. |
| `BUSY`                | A conflicting single-flight background operation is already running, or all four Agent job slots are occupied. Poll/cancel the job you own, then retry. |
| `PERMISSION_REQUIRED` | A macOS TCC grant the requested capture needs (Screen Recording, Accessibility, Input Monitoring) is missing. The `hint` names the System Settings pane and the parameter that avoids needing it. |
| `INVALID_STATE`       | The call is illegal from the current recording state machine position (e.g. `pause_recording` when nothing is recording). The `message` names the actual state. |
| `TIMEOUT`             | A recording transition exceeded its bound, or a background job exceeded its operation runtime and was aborted. Inspect the relevant status tool before retrying. |
| `INTERNAL`            | Unexpected failure, transport timeout, or unknown command.          |

`INVALID_ARGS` also covers MCP-level input-schema validation: a bad argument is
returned to the client as an `isError` tool result (MCP `-32602` text), not a
thrown call.

See [`tools.md`](./tools.md) for the per-tool error codes and examples.

---

## 6. Files in this spec

| File                                       | Contents                                                            |
|--------------------------------------------|---------------------------------------------------------------------|
| [`README.md`](./README.md)                 | This overview: discovery, connection, lifecycle, security.          |
| [`tools.md`](./tools.md)                    | Complete reference for all 57 tools (params, types, errors, examples). |
| [`project-format.md`](./project-format.md) | The `.screencraft` directory layout and the offline `project.json` editing contract. |

The reference implementation lives in
`src/main/services/agent-api/` in the Cadre repository.