# Tiltmeter

> An early-warning system for the third-party APIs your AI agents depend on, with the fix as a pull request. Open source under the MIT license. https://tiltmeter.dev

Tiltmeter learns the shape of JSON responses from real traffic, notices when a vendor renames a field, starts sending `null` or turns a number into a string, and tells you, your CI and your agents before anything breaks. Then `tiltmeter fix` hands the change to an agent that adapts the code reading that API and opens the pull request, leaving what it cannot know for a human.

Typed code crashes loudly when an API changes. Agents don't: they read the new response, fill the gaps with guesses, or retry until the budget is gone.

## Set it up for a project

If you are an agent setting Tiltmeter up for your user:

1. Find the external HTTP APIs the codebase calls that return JSON.
2. Create `tiltmeter.yaml` in the project root with one endpoint per API. Use `${ENV_VAR}` placeholders for keys, never literal secrets.
3. Add `.tiltmeter/` to `.gitignore` and run `uvx tiltmeter check` once to record the baselines.
4. Register the MCP server for yourself with the command `uvx tiltmeter serve --config tiltmeter.yaml`.
5. Add to CLAUDE.md or AGENTS.md: "Before relying on a third-party API, call the tiltmeter has_drifted tool for it."

Show the user the endpoints you chose before writing the file.

## Install and first check

```bash
pip install tiltmeter        # or: uv tool install tiltmeter
tiltmeter init               # writes tiltmeter.yaml with a public example API
tiltmeter check              # the first check records the baseline
```

From a web app that already calls the API, export a HAR from the browser's DevTools (Network tab, "Export HAR") and let Tiltmeter write the config from the JSON calls the page made. Tokens and cookies become `${VAR}` placeholders:

```bash
tiltmeter init --from-har session.har --host api.example.com
```

Run `tiltmeter check` again from cron or CI, or leave `tiltmeter watch` running. When a vendor renames `phone` and turns `balance` into a string:

```console
$ tiltmeter check
✗ billing_customers  200 · 13 ms  BREAKING · 3 changes
    breaking  type_changed   $.data[*].balance       integer → string
    breaking  field_removed  $.data[*].phone         string → (absent)  possibly renamed to $.data[*].phone_number
    info      field_added    $.data[*].phone_number  (absent) → string  possibly replaces $.data[*].phone
1 endpoint checked · 1 breaking
$ echo $?
2
```

With an Anthropic API key, Claude explains what changed, the likely cause and what to do about it.

## Where it fits

Tiltmeter is built for APIs that change without notice: smaller vendors and partner APIs with no changelog you can trust, internal APIs owned by another team, undocumented endpoints behind a web app, and anything an agent calls that has no spec. APIs like Stripe that version every change, let you pin a version and publish a changelog don't need it: pin the version and read the changelog.

It does not replace typed models (DTOs). A DTO checks the fields you modeled when your code reads them, in production, and coercion can hide type changes: Pydantic turns "1200" into 1200 without an error. Tiltmeter checks the whole response on a schedule or in CI, reports a removed field next to a similar new one as a probable rename, and reports integer → string as breaking. Agents read raw JSON and have no DTOs, so has_drifted gives them that check.

## Configuration

```yaml
endpoints:
  - id: partner_orders
    url: https://api.partner.example/v1/orders
    params:
      limit: 10
    headers:
      Authorization: Bearer ${PARTNER_API_KEY}   # expanded from the environment at request time
    ignore_paths:
      - $.data[*].metadata                      # free-form, changes all the time

notify:
  - type: slack
    webhook_url: ${SLACK_WEBHOOK_URL}
```

Every option: https://raw.githubusercontent.com/tiltmeter-dev/tiltmeter/main/docs/configuration.md

## What counts as drift

| Change | Example | Default severity |
|---|---|---|
| `field_removed` | `phone` was in every response and is gone | breaking |
| `field_now_optional` | `email` was always there and is now missing from some objects | breaking |
| `type_changed` | `balance` went from integer to string | breaking |
| `nullable_introduced` | `phone` can now be `null` | breaking |
| `content_type_changed` | the JSON body became an HTML error page | breaking |
| `status_changed` | 200 to 503 is breaking, 404 to 200 is info, 200 to 202 is a warning | depends |
| `enum_value_added` | `status` returned `suspended` besides `active` and `past_due` | warning |
| `latency_regression` | median latency over 3x the baseline and at least 250 ms slower | warning |
| `field_added` | a new `phone_number` field | info, or warning with `strict: true` |

A removed field next to a similar new one is flagged as a probable rename. Override any severity globally or per endpoint with `severity_overrides`. Provider notices and spec diffs also report `endpoint_removed` (an endpoint is gone, with `moved_to` when it moved) and `value_retired` (requests can no longer send a value, such as a retired model id, with `replaced_by`), both breaking.

How it decides: a check turns the JSON body into a shape (the JSON types at every path, how many objects contained each field and, for categorical fields, which values were seen). The baseline is the merge of the shapes from the last 20 successful checks, and only contradictions are reported. The new shape then joins the baseline, so each change is reported once.

## MCP server for agents

The server is `tiltmeter serve`. It speaks MCP over stdio and reads the same `tiltmeter.yaml` and `.tiltmeter/` history as the CLI. Agents can only check the endpoints in the config.

| Tool | What it answers |
|---|---|
| `has_drifted(endpoint_id, since?)` | Did the responses change at warning level or worse recently? Returns a summary and the changes. |
| `get_schema(endpoint_id)` | Which fields, types and enum values to expect, as JSON Schema. |
| `get_changes(endpoint_id?, since?, min_severity?, limit?)` | The change history, newest first. |
| `list_endpoints()` | Which APIs are watched and how their last check went. |
| `check_api(endpoint_id, explain?)` | Calls one API now and records the result. |
| `check_all(explain?)` | Calls every API now. |

Claude Code:

```bash
claude mcp add tiltmeter -- uvx tiltmeter serve --config tiltmeter.yaml
```

Codex:

```bash
codex mcp add tiltmeter -- uvx tiltmeter serve --config tiltmeter.yaml
```

Cursor (`.cursor/mcp.json`):

```json
{
  "mcpServers": {
    "tiltmeter": {
      "command": "uvx",
      "args": ["tiltmeter", "serve", "--config", "tiltmeter.yaml"]
    }
  }
}
```

OpenCode (`opencode.json`):

```json
{
  "$schema": "https://opencode.ai/config.json",
  "mcp": {
    "tiltmeter": {
      "type": "local",
      "command": ["uvx", "tiltmeter", "serve", "--config", "tiltmeter.yaml"],
      "enabled": true
    }
  }
}
```

Explanations need `ANTHROPIC_API_KEY` in the server's environment. Claude Desktop and LangChain setups: https://raw.githubusercontent.com/tiltmeter-dev/tiltmeter/main/docs/agents.md

Suggested instruction for the agent: "Before you rely on a third-party API, call `has_drifted` for it. If it drifted, read the summary and adapt before trusting the response."

## Fixing code

```bash
pip install "tiltmeter[fix]"            # the built-in LangChain Deep Agent, needs ANTHROPIC_API_KEY
tiltmeter fix 2 --repo ~/code/billing-service
tiltmeter fix 2 --agent claude-code     # or --agent codex, on your own login, no API key
```

`tiltmeter fix <check id>` creates a new branch in a temporary git worktree, lets the agent adapt the code that reads the API, shows the diff and the agent's report, and asks "Commit, push and open a pull request? [y/N]". Answer `n` and everything is discarded. Pull requests are opened with the GitHub CLI (`gh`).

- The checkout is never touched, and nothing is committed until you approve or pass `--yes`.
- The agent stays inside the worktree: no network access, only tracked files; `.env` files, keys and `.git` are blocked.
- Tiltmeter sees types, not values. When a number turns into a string, the agent updates the types and leaves the format question to a human in the pull request.
- Flags: `--dry-run` shows the diff and discards it, `--no-pr` commits to a local branch, `--json` prints the outcome as JSON.

Details: https://raw.githubusercontent.com/tiltmeter-dev/tiltmeter/main/docs/fix.md

## Provider notices

An API provider describes a change once, in a signed JSON notice served from its own domain, and every customer that follows the provider gets a pull request that adapts its code. When a provider publishes no notices, a customer can follow its OpenAPI spec instead.

Provider side:

```bash
tiltmeter notice keygen key.pem --public site/.well-known/api-change-key
tiltmeter notice create --from-openapi v2.yaml v3.yaml --effective 2026-12-01 -o notice.json   # or --from-check <id>
tiltmeter notice sign notice.json --key key.pem
tiltmeter notice publish notice.json --feed site/.well-known/api-changes.json
```

Customer side, in `tiltmeter.yaml`:

```yaml
subscriptions:
  - provider: api.partner.example        # reads https://api.partner.example/.well-known/api-changes.json
  - openapi: https://raw.githubusercontent.com/acme/api/main/openapi.json   # follows a spec instead
```

Then `tiltmeter notices check --repo .` lists what is new (read-only) and `tiltmeter notices apply --repo . --yes` opens one pull request per change that affects the repository. A plain text search for the API's hosts, paths, SDKs and changed field names decides first, without an LLM; each notice is handled once. Only notices signed with the key at `https://<provider>/.well-known/api-change-key` act on their own; `tiltmeter fix --notice <file> --allow-unsigned` handles one by hand. The Action runs it on a schedule with `mode: notices`, and providers draft and publish notices from CI with `mode: notice-draft` and `mode: notice-publish`.

Format: https://raw.githubusercontent.com/tiltmeter-dev/tiltmeter/main/docs/notice-spec.md · Guide: https://raw.githubusercontent.com/tiltmeter-dev/tiltmeter/main/docs/notices.md · Schemas: https://tiltmeter.dev/schemas/api-change-notice/v1.json

## CI

| Exit code | Meaning |
|---|---|
| 0 | No drift. Info-only changes and new baselines count as no drift. |
| 1 | Warning-level drift. |
| 2 | Breaking drift. |
| 3 | A check could not run (network error, unexpected status, invalid JSON) and nothing broke. |
| 4 | Usage or configuration error. |

On GitHub, the Tiltmeter Action keeps the baseline in the Actions cache, writes the changes to the job summary and fails the job on drift:

```yaml
- uses: tiltmeter-dev/tiltmeter-action@v1
  env:
    PARTNER_API_KEY: ${{ secrets.PARTNER_API_KEY }}
```

Anywhere else, keep `.tiltmeter/` between runs. Jobs for GitHub Actions, GitLab CI and cron: https://raw.githubusercontent.com/tiltmeter-dev/tiltmeter/main/docs/ci.md

## How it compares

- OpenAPI diff tools such as oasdiff compare two versions of a spec. They are the right tool when the vendor publishes and maintains one, and they cannot see changes the vendor did not document.
- Contract testing such as Pact needs the provider to run your contracts, which third-party vendors don't do.
- Uptime and synthetic monitors tell you an API is up. You write and maintain the response assertions yourself.
- Tiltmeter learns the contract from real responses, so it needs nothing from the vendor and catches undocumented changes. It also hands what it learns to your agents.

## Privacy and security

- `${VAR}` placeholders are expanded right before a request or notification is sent. The values are never written to disk, sent to the LLM or kept in the LangGraph state, and they are scrubbed from error messages.
- The history stores shapes, status codes, latencies and changes, never response bodies unless you set `store_samples: true`.
- Agents can only check the endpoints in your config. They cannot make Tiltmeter call an arbitrary URL.
- `tiltmeter fix` sends the change list and the files the agent opens to the model provider. It never sees response values, `${VAR}` values or untracked files such as `.env`.

Report vulnerabilities privately: https://github.com/tiltmeter-dev/tiltmeter/security/advisories/new

## Links

- Source: https://github.com/tiltmeter-dev/tiltmeter
- GitHub Action: https://github.com/tiltmeter-dev/tiltmeter-action
- PyPI: https://pypi.org/project/tiltmeter/