Open source · MIT · Python 3.12+

Know when the APIs
under your agents move.

Tiltmeter learns the shape of every third-party API you depend on from real responses, warns you, your CI and your agents the moment it changes, and opens the pull request that adapts your code.

Field report · October 2026

Retired APIs are still
in production code.

We searched public GitHub code for API changes that already shipped, then let Tiltmeter fix the most-starred active repositories, one reviewed pull request at a time. Every pull request and its status →

files on GitHub still name llama-3.3-70b-versatile, ten weeks after Groq shut it down for free and developer keys
314k
pull requests Tiltmeter opened for changes at 10 APIs
84
landed so far: 21 merged as Tiltmeter wrote them, 5 adopted by their maintainers
26
GitHub stars across the repositories that got a fix
1.34M

From the reviews

“Verified and merging — thanks, this is a real break rather than a cosmetic endpoint bump.”
@beenuar, merging AiSOC's Jira fix after checking each change in the new search API's contract
“Fixes like this keep StaffML and SocratiQ working for everyone, so thank you for jumping in and contributing.”
Vijay Janapa Reddi, Harvard CS249r, merging the Groq fix to the Machine Learning Systems book
“Verified against Anthropic's current docs […]: the premise checks out.”
@willem-bd, reviewing deer-flow's Anthropic fix
“Independently checked Anthropic's model overview, retirement table and Sonnet 5.5 migration guide. […] This conclusion does not rely on the unsigned Tiltmeter notice.”
@CrazyBoyM, approving learn-claude-code's Anthropic fix
“Thanks — Cloud-only switch, token paging and the early-end guard look right. Merging.”
@WaylandYang, merging utopia's Jira Cloud fix
“Only two references to the old URL exist in the repo, and both files are covered by this PR. […] Thanks for the fix!”
@boy-hack, approving Tencent AI-Infra-Guard's Hugging Face fix

Maintainers check each fix against the provider's own documentation before they merge it. A notice signed by the provider is what would let them skip that step.

billing_customers GET /v1/customers shape stable breaking drift

BREAKING · 3 changes

balance integer → string

phone removed, possibly renamed to phone_number

Reported once. Then it's the new normal.

$ tiltmeter check
✗ billing_customers  200 · 13 ms  BREAKING · 3 changes
    breaking  type_changed   $.data[*].balance       integer → string
    breaking  field_removed  $.data[*].phone         string → (absent)  possibly renamed to $.data[*].phone_number
    info      field_added    $.data[*].phone_number  (absent) → string  possibly replaces $.data[*].phone
1 endpoint checked · 1 breaking
$ echo $?
2

ClaudeThe vendor most likely renamed phone to phone_number. Code doing math on balance will break now that it is a string. Read phone_number and parse balance as a number.

Typed code crashes loudly when an API changes. Agents don't. They read the new response, fill the gaps with guesses, or retry until the budget is gone.

How it works

No spec. No noise.
No guessing.

  1. 01

    Observe

    A check calls the endpoint and turns the JSON body into a shape: types, how often each field appears and the values of fields that look categorical.

    balance   integer   20/20
    phone     string    20/20
    status    active | past_due
  2. 02

    Compare

    The baseline merges the last 20 successful checks. Only contradictions count: a missing optional field is fine, a field that was always there and is gone is not.

    balance   integer → string
    phone     20/20 → 0/1
  3. 03

    Warn, once

    Fail CI, ping Slack or a webhook, explain it with Claude and tell your agents over MCP. The new shape joins the baseline, so you hear about it once.

    exit 2   breaking
    slack    #api-alerts
    mcp      has_drifted → true
  4. 04

    Fix

    An agent finds the code that reads the API, adapts it and opens a pull request. You review the diff.

    ✓ opened acme/billing-service#41
      3 files  +5 -5

Already calling the API from a web app? Export a HAR from your browser and let Tiltmeter write the config. Tokens and cookies become ${VAR} placeholders.tiltmeter init --from-har session.har --host api.example.com

What counts as drift

Breaking or harmless,
at a glance.

Every change is classified, and a removed field next to a similar new one is flagged as a probable rename. Override any severity globally or per endpoint.

ChangeExampleSeverity
field_removedphone was in every response and is gonebreaking
field_now_optionalemail was always there and is now missing from some objectsbreaking
type_changedbalance went from integer to stringbreaking
nullable_introducedphone can now be nullbreaking
content_type_changedthe JSON body became an HTML error pagebreaking
status_changed200 to 503 is breaking, 404 to 200 is info, 200 to 202 is a warningdepends
enum_value_addedstatus returned suspended besides active and past_duewarning
latency_regressionmedian latency over 3x the baseline and at least 250 ms slowerwarning
field_addeda new phone_number fieldinfo

Why not just a DTO

Your models find out
in production.

Keep your DTOs. They check the fields you modeled, at the moment your code reads them. Tiltmeter checks the whole response, before your code runs into it, and tells you when to change them.

The vendor renames phone and sends balance as a string

class Customer(BaseModel):
    name: str
    phone: str
    balance: int

Customer.model_validate(response)
ValidationError: 1 validation error for Customer
phone
  Field required [type=missing]

balance came back as "1200" and Pydantic turned it into 1200 without a word. Was it cents or dollars?

  • When it notices. A DTO fails in the request that breaks, in production. Tiltmeter notices on a schedule or in CI, before any user does.
  • What it sees. A DTO only knows the fields you modeled. Tiltmeter sees the new phone_number next to the missing phone and calls it a probable rename.
  • What slips through. Coercion hides type changes. Tiltmeter reports integer → string as breaking, because the format may have changed with it.
  • Who is covered. Agents don't have DTOs; they read raw JSON. has_drifted gives them the check your typed code already has.

Built for APIs that change without notice

  • Smaller vendors and partner APIs with no changelog you can trust
  • Internal APIs owned by another team
  • Undocumented endpoints behind a web app, starting from a HAR
  • Anything your agents call that has no spec

Not where it earns its keep

APIs like Stripe that version every change, let you pin a version and publish a changelog. Pin the version and read the changelog: Tiltmeter adds little there.

For agents

Agents ask
before they act.

An MCP server lets Claude Code, Cursor, LangChain or any MCP client call has_drifted before trusting an API, and get_schema to see which fields to expect. Agents can only check the endpoints in your config.

Before you rely on a third-party API, call has_drifted for it. If it drifted, read the summary and adapt before trusting the response.

> Text customer 42 that their invoice is overdue

● tiltmeter · has_drifted("billing_customers")

drifted       true
max_severity  breaking
changed       phone → phone_number
checked       2 min ago

● The billing API changed: phone is now phone_number. I'll read the new field before texting, instead of sending to an empty number.

has_driftedDid responses change at warning level or worse recently?
get_schemaWhich fields, types and enum values to expect, as JSON Schema.
get_changesThe change history, newest first.
list_endpointsWhich APIs are watched and how their last check went.
check_apiCalls one API now and records the result.
check_allCalls every API now.
claude mcp add tiltmeter -- uvx tiltmeter serve --config tiltmeter.yaml
Run it in your project folder.
{
  "mcpServers": {
    "tiltmeter": {
      "command": "uvx",
      "args": ["tiltmeter", "serve", "--config", "tiltmeter.yaml"]
    }
  }
}
Or paste this into .cursor/mcp.json.Add to Cursor
codex mcp add tiltmeter -- uvx tiltmeter serve --config tiltmeter.yaml
Or add [mcp_servers.tiltmeter] to ~/.codex/config.toml.
{
  "$schema": "https://opencode.ai/config.json",
  "mcp": {
    "tiltmeter": {
      "type": "local",
      "command": ["uvx", "tiltmeter", "serve", "--config", "tiltmeter.yaml"],
      "enabled": true
    }
  }
}
Add this to opencode.json in your project.
Set up Tiltmeter (https://github.com/tiltmeter-dev/tiltmeter) in this project so you can check whether the third-party APIs we use have changed before relying on them.

1. Find the external HTTP APIs this codebase calls that return JSON.
2. Create tiltmeter.yaml in the project root with one endpoint per API. Use ${ENV_VAR} placeholders for keys, never literal secrets. Docs: https://github.com/tiltmeter-dev/tiltmeter/blob/main/docs/configuration.md
3. Add .tiltmeter/ to .gitignore and run `uvx tiltmeter check` once to record the baselines.
4. Register the MCP server for yourself, using the command: uvx tiltmeter serve --config tiltmeter.yaml
5. Add to CLAUDE.md or AGENTS.md: "Before relying on a third-party API, call the tiltmeter has_drifted tool for it."

Show me the endpoints you chose before writing the file.
Paste it into any coding agent. It finds your APIs, writes the config and installs the server.

Fixes, too

From drift to
pull request.

tiltmeter fix hands the change to an agent that finds the code reading that API, shows you the diff and, once you agree, opens the pull request. Run the built-in LangChain agent, or your own Claude Code or Codex login with no API key.

  1. Drift found
  2. Clean worktree
  3. Agent edits
  4. You approve
  5. Pull request

$ tiltmeter fix 26 --repo ~/code/billing-service

✗ billing_customers #26 · 200 · 13 ms BREAKING · 3 changes

Working on a new branch, tiltmeter/fix-billing-customers-26, from main at 162dadd.

agent is editing the worktree

3 files changed +5 -5

diff --git a/billing/client.py b/billing/client.py

- return f"{customer['name']} <{customer['phone']}>"

+ return f"{customer['name']} <{customer['phone_number']}>"

...

╭─ agent ─────────────────────────────────╮

Billing customer code now reads phone_number instead of phone, in the client, the Customer TypedDict and the test fixture.

Left for a human:

- The format of the string balance is unknown ("1200" or "12.00"). Confirm it with the billing provider.

confidence medium

╰─────────────────────────────────────────╯

Commit, push and open a pull request? [y/N]: y

✓ opened https://github.com/acme/billing-service/pull/41

What the code has to follow

  • $.data[*].phone removed, possibly renamed to phone_number
  • $.data[*].balance integer → string

New fields, status and latency changes are left out.

Worktree · tiltmeter/fix-billing-customers-26

  • billing/client.py
  • billing/models.py
  • tests/conftest.py
  • .envblocked
  • .gitblocked

No network. Your checkout is never touched.

Open Follow billing_customers drift from check #26
tiltmeter/fix-billing-customers-26 → main · acme/billing-service#41
3 files changed+5-5
billing/client.py
-    return f"{customer['name']} <{customer['phone']}>"
+    return f"{customer['name']} <{customer['phone_number']}>"
billing/models.py
-    phone: str
+    phone_number: str
Left for a human The format of balance is unknown: "1200" or "12.00"? Confirm it with the billing provider before merging.

For CI

An exit code that
knows what broke.

The GitHub Action keeps the baseline in the Actions cache, writes the changes to the job summary and fails the job on drift. GitLab CI and cron work too.

0no drift 1warning 2breaking 3check couldn't run 4config error
# .github/workflows/api-drift.yml
- uses: tiltmeter-dev/tiltmeter-action@v1
  env:
    PARTNER_API_KEY: ${{ secrets.PARTNER_API_KEY }}
# tiltmeter.yaml
notify:
  - type: slack
    webhook_url: ${SLACK_WEBHOOK_URL}

For API providers

Changing your API?
Ship the fix with it.

Describe the change once, in a signed notice on your own domain, and every customer that follows you gets a pull request that adapts their code before the old responses stop. No notices yet? Customers can follow your OpenAPI spec directly. Provider mode →

Field report · October 2026

Stripe's own sample
broke on Stripe's API.

We searched public code for fields that Stripe and Spotify removed. 78 repositories still read Stripe's subscription.current_period_end, gone since API version 2025-03-31, and 228 use Spotify fields and endpoints that changed for new apps in February 2026. One of them was Stripe's official subscription sample, which no longer type-checks against Stripe's own SDK. Tiltmeter fixed it and 14 others we checked by hand, and the pull requests are open. The fix for Stripe's sample →

Since then Tiltmeter has opened 84 pull requests for changes at Stripe, Jira, Spotify, OpenAI, Anthropic, Groq, Slack, OpenWeatherMap, Hugging Face and HubSpot. So far 26 have landed: 21 were merged as Tiltmeter wrote them, and 5 were adopted by maintainers who landed the fix themselves. inbox-zero and QOwnNotes carried Tiltmeter's commit into changes of their own, and aicommits made the same change in its own commit. OpenMetadata picked up Tiltmeter's commit in its own pull request for its 2.1 release, merged on 5 November. Seven were closed: anything-llm asks for a linked issue first, three touched code their projects no longer use, PraisonAI kept Groq's models because enterprise contracts still have them, LeanType found that the replacement, a reasoning model, spent its token budget before answering, and Irene Voice Assistant gave no reason. One repository was deleted. Serenity BDD, AiSOC, utopia and KodeRover's zadig moved their Jira calls off /rest/api/3/search, an endpoint Jira Cloud removed. ruflo, radare2's r2ai, mcp-chat, ByteDance's deer-flow and learn-claude-code moved their defaults off Claude models Anthropic has retired. Harvard's Machine Learning Systems book and KindleEar moved off the Groq models Groq shut down. Share-The-Link and nestjs-saas-starter read subscription periods from subscription items, as Stripe's API has required since 2025-03-31. Tencent's AI-Infra-Guard and SciSharp's BotSharp moved their Hugging Face calls to the router that replaced the retired Inference API.

Every pull request and its status, by provider →

How it compares

Needs nothing
from the vendor.

OpenAPI diff toolsCompare two versions of a spec. Right when the vendor maintains one, blind to what they didn't document.
Contract testingNeeds the provider to run your contracts, which third-party vendors don't do.
Uptime monitorsTell you the API is up. You write and maintain the response assertions yourself.
TiltmeterLearns the contract from real responses, catches undocumented changes and hands what it learns to your agents.

Nothing leaves
your machine.

Put a tiltmeter
under your agents.

View on GitHub