Open source · MIT · Python 3.12+
Know when the APIs
under your agents move.
Tiltmeter learns the shape of every third-party API you depend on from real responses, warns you, your CI and your agents the moment it changes, and opens the pull request that adapts your code.
uvx tiltmeter scan →
Try it on a fake API in five minutes →
Are you an agent? Read /llms.txt and set it up yourself.
Field report · October 2026
Retired APIs are still
in production code.
We searched public GitHub code for API changes that already shipped, then let Tiltmeter fix the most-starred active repositories, one reviewed pull request at a time. Every pull request and its status →
- files on GitHub still name
llama-3.3-70b-versatile, ten weeks after Groq shut it down for free and developer keys - 314k
- pull requests Tiltmeter opened for changes at 10 APIs
- 84
- landed so far: 21 merged as Tiltmeter wrote them, 5 adopted by their maintainers
- 26
- GitHub stars across the repositories that got a fix
- 1.34M
- Groq. 260 of 1,619 popular, active AI repositories still name a model Groq has retired, often as the default. 15 fixes went out. Harvard's Machine Learning Systems book and KindleEar merged theirs. inbox-zero and QOwnNotes carried Tiltmeter's commit into changes of their own, and aicommits landed the same change in its own commit. Groq →
- Anthropic. Swapping the model id isn't enough: Claude Sonnet 5.5, the recommended replacement, rejects
temperatureand thinking budgets. Two maintainers caught our first fixes on it, so Tiltmeter now adapts how requests are built. Anthropic → - Stripe. 78 repositories still read
subscription.current_period_end, removed in API version 2025-03-31, including Stripe's own subscription sample, which no longer type-checks against Stripe's SDK. Share-The-Link and nestjs-saas-starter merged fixes that read the period from the subscription's items instead. Stripe → - Slack.
files.uploadstopped on November 12, 2025, and openreplay, reconftw and intuit/auto still called it. Their fixes use the three-step upload that replaced it. openreplay's fix → - OpenWeatherMap. One Call API 2.5 closed in June 2024, and Raycast's script commands and Irene Voice Assistant still called it. Three other matches, in Aerial, sol and an old build of Linux Mint's weather applet, were in code those projects no longer use, so their fixes were closed. OpenWeatherMap →
- Hugging Face. The serverless Inference API's domain, api-inference.huggingface.co, no longer resolves, yet Microsoft's Semantic Kernel, langchaingo and Obsidian's web clipper still defaulted to it. Tencent's AI-Infra-Guard and SciSharp's BotSharp merged theirs, and OpenMetadata merged Tiltmeter's commit in its own pull request for its 2.1 release. Hugging Face →
- HubSpot. Pipelines API v1 stops on December 4, 2026, and Airbyte's HubSpot connector still read it. Its fix keeps the connector's output the same, before the shutdown instead of after. Airbyte's fix →
From the reviews
“Verified and merging — thanks, this is a real break rather than a cosmetic endpoint bump.”
“Fixes like this keep StaffML and SocratiQ working for everyone, so thank you for jumping in and contributing.”
“Verified against Anthropic's current docs […]: the premise checks out.”
“Independently checked Anthropic's model overview, retirement table and Sonnet 5.5 migration guide. […] This conclusion does not rely on the unsigned Tiltmeter notice.”
“Thanks — Cloud-only switch, token paging and the early-end guard look right. Merging.”
“Only two references to the old URL exist in the repo, and both files are covered by this PR. […] Thanks for the fix!”
Maintainers check each fix against the provider's own documentation before they merge it. A notice signed by the provider is what would let them skip that step.
BREAKING · 3 changes
balance integer → string
phone removed, possibly renamed to phone_number
Reported once. Then it's the new normal.
$ tiltmeter check ✗ billing_customers 200 · 13 ms BREAKING · 3 changes breaking type_changed $.data[*].balance integer → string breaking field_removed $.data[*].phone string → (absent) possibly renamed to $.data[*].phone_number info field_added $.data[*].phone_number (absent) → string possibly replaces $.data[*].phone 1 endpoint checked · 1 breaking $ echo $? 2
ClaudeThe vendor most likely renamed phone to phone_number. Code doing math on balance will break now that it is a string. Read phone_number and parse balance as a number.
Typed code crashes loudly when an API changes. Agents don't. They read the new response, fill the gaps with guesses, or retry until the budget is gone.
How it works
No spec. No noise.
No guessing.
-
01
Observe
A check calls the endpoint and turns the JSON body into a shape: types, how often each field appears and the values of fields that look categorical.
balance integer 20/20 phone string 20/20 status active | past_due
-
02
Compare
The baseline merges the last 20 successful checks. Only contradictions count: a missing optional field is fine, a field that was always there and is gone is not.
balance integer → string phone 20/20 → 0/1
-
03
Warn, once
Fail CI, ping Slack or a webhook, explain it with Claude and tell your agents over MCP. The new shape joins the baseline, so you hear about it once.
exit 2 breaking slack #api-alerts mcp has_drifted → true -
04
Fix
An agent finds the code that reads the API, adapts it and opens a pull request. You review the diff.
✓ opened acme/billing-service#41 3 files +5 -5
Already calling the API from a web app? Export a HAR from your browser and let Tiltmeter write the config. Tokens and cookies become ${VAR} placeholders.tiltmeter init --from-har session.har --host api.example.com
What counts as drift
Breaking or harmless,
at a glance.
Every change is classified, and a removed field next to a similar new one is flagged as a probable rename. Override any severity globally or per endpoint.
field_removedphone was in every response and is gonebreakingfield_now_optionalemail was always there and is now missing from some objectsbreakingtype_changedbalance went from integer to stringbreakingnullable_introducedphone can now be nullbreakingcontent_type_changedthe JSON body became an HTML error pagebreakingstatus_changed200 to 503 is breaking, 404 to 200 is info, 200 to 202 is a warningdependsenum_value_addedstatus returned suspended besides active and past_duewarninglatency_regressionmedian latency over 3x the baseline and at least 250 ms slowerwarningfield_addeda new phone_number fieldinfoWhy not just a DTO
Your models find out
in production.
Keep your DTOs. They check the fields you modeled, at the moment your code reads them. Tiltmeter checks the whole response, before your code runs into it, and tells you when to change them.
The vendor renames phone and sends balance as a string
class Customer(BaseModel):
name: str
phone: str
balance: int
Customer.model_validate(response)
ValidationError: 1 validation error for Customer
phone
Field required [type=missing]
balance came back as "1200" and Pydantic turned it into 1200 without a word. Was it cents or dollars?
- When it notices. A DTO fails in the request that breaks, in production. Tiltmeter notices on a schedule or in CI, before any user does.
- What it sees. A DTO only knows the fields you modeled. Tiltmeter sees the new
phone_numbernext to the missingphoneand calls it a probable rename. - What slips through. Coercion hides type changes. Tiltmeter reports integer → string as breaking, because the format may have changed with it.
- Who is covered. Agents don't have DTOs; they read raw JSON.
has_driftedgives them the check your typed code already has.
Built for APIs that change without notice
- Smaller vendors and partner APIs with no changelog you can trust
- Internal APIs owned by another team
- Undocumented endpoints behind a web app, starting from a HAR
- Anything your agents call that has no spec
Not where it earns its keep
APIs like Stripe that version every change, let you pin a version and publish a changelog. Pin the version and read the changelog: Tiltmeter adds little there.
For agents
Agents ask
before they act.
An MCP server lets Claude Code, Cursor, LangChain or any MCP client call has_drifted before trusting an API, and get_schema to see which fields to expect. Agents can only check the endpoints in your config.
Before you rely on a third-party API, call has_drifted for it. If it drifted, read the summary and adapt before trusting the response.
> Text customer 42 that their invoice is overdue
● tiltmeter · has_drifted("billing_customers")
drifted true max_severity breaking changed phone → phone_number checked 2 min ago
● The billing API changed: phone is now phone_number. I'll read the new field before texting, instead of sending to an empty number.
has_driftedDid responses change at warning level or worse recently?get_schemaWhich fields, types and enum values to expect, as JSON Schema.get_changesThe change history, newest first.list_endpointsWhich APIs are watched and how their last check went.check_apiCalls one API now and records the result.check_allCalls every API now.claude mcp add tiltmeter -- uvx tiltmeter serve --config tiltmeter.yaml
{
"mcpServers": {
"tiltmeter": {
"command": "uvx",
"args": ["tiltmeter", "serve", "--config", "tiltmeter.yaml"]
}
}
}
codex mcp add tiltmeter -- uvx tiltmeter serve --config tiltmeter.yaml
{
"$schema": "https://opencode.ai/config.json",
"mcp": {
"tiltmeter": {
"type": "local",
"command": ["uvx", "tiltmeter", "serve", "--config", "tiltmeter.yaml"],
"enabled": true
}
}
}
Set up Tiltmeter (https://github.com/tiltmeter-dev/tiltmeter) in this project so you can check whether the third-party APIs we use have changed before relying on them.
1. Find the external HTTP APIs this codebase calls that return JSON.
2. Create tiltmeter.yaml in the project root with one endpoint per API. Use ${ENV_VAR} placeholders for keys, never literal secrets. Docs: https://github.com/tiltmeter-dev/tiltmeter/blob/main/docs/configuration.md
3. Add .tiltmeter/ to .gitignore and run `uvx tiltmeter check` once to record the baselines.
4. Register the MCP server for yourself, using the command: uvx tiltmeter serve --config tiltmeter.yaml
5. Add to CLAUDE.md or AGENTS.md: "Before relying on a third-party API, call the tiltmeter has_drifted tool for it."
Show me the endpoints you chose before writing the file.
Fixes, too
From drift to
pull request.
tiltmeter fix hands the change to an agent that finds the code reading that API, shows you the diff and, once you agree, opens the pull request. Run the built-in LangChain agent, or your own Claude Code or Codex login with no API key.
-
Claude Code
Your Claude plan
--agent claude-code -
Codex
Your ChatGPT plan
--agent codex -
Anthropic API
Your key, on LangChain Deep Agents
ANTHROPIC_API_KEY - More labs coming soon Tell us which one
- Drift found
- Clean worktree
- Agent edits
- You approve
- Pull request
$ tiltmeter fix 26 --repo ~/code/billing-service
✗ billing_customers #26 · 200 · 13 ms BREAKING · 3 changes
Working on a new branch, tiltmeter/fix-billing-customers-26, from main at 162dadd.
agent is editing the worktree
3 files changed +5 -5
diff --git a/billing/client.py b/billing/client.py
- return f"{customer['name']} <{customer['phone']}>"
+ return f"{customer['name']} <{customer['phone_number']}>"
...
╭─ agent ─────────────────────────────────╮
Billing customer code now reads phone_number instead of phone, in the client, the Customer TypedDict and the test fixture.
Left for a human:
- The format of the string balance is unknown ("1200" or "12.00"). Confirm it with the billing provider.
confidence medium
╰─────────────────────────────────────────╯
Commit, push and open a pull request? [y/N]: y
✓ opened https://github.com/acme/billing-service/pull/41
What the code has to follow
$.data[*].phoneremoved, possibly renamed tophone_number$.data[*].balanceinteger → string
New fields, status and latency changes are left out.
Worktree · tiltmeter/fix-billing-customers-26
- billing/client.py
- billing/models.py
- tests/conftest.py
- .envblocked
- .gitblocked
No network. Your checkout is never touched.
tiltmeter/fix-billing-customers-26 → main · acme/billing-service#41
billing/client.py - return f"{customer['name']} <{customer['phone']}>" + return f"{customer['name']} <{customer['phone_number']}>" billing/models.py - phone: str + phone_number: str
balance is unknown: "1200" or "12.00"? Confirm it with the billing provider before merging.- Your checkout is never touched. The agent works in a temporary git worktree on a new branch. Answer
nand it is all discarded. - It stays inside the worktree. No network, only tracked files;
.envfiles, keys and.gitare blocked. - It says what it could not do. Tiltmeter sees types, not values, so format questions go to a human.
For CI
An exit code that
knows what broke.
The GitHub Action keeps the baseline in the Actions cache, writes the changes to the job summary and fails the job on drift. GitLab CI and cron work too.
# .github/workflows/api-drift.yml - uses: tiltmeter-dev/tiltmeter-action@v1 env: PARTNER_API_KEY: ${{ secrets.PARTNER_API_KEY }}
# tiltmeter.yaml notify: - type: slack webhook_url: ${SLACK_WEBHOOK_URL}
For API providers
Changing your API?
Ship the fix with it.
Describe the change once, in a signed notice on your own domain, and every customer that follows you gets a pull request that adapts their code before the old responses stop. No notices yet? Customers can follow your OpenAPI spec directly. Provider mode →
Field report · October 2026
Stripe's own sample
broke on Stripe's API.
We searched public code for fields that Stripe and Spotify removed. 78 repositories still read Stripe's subscription.current_period_end, gone since API version 2025-03-31, and 228 use Spotify fields and endpoints that changed for new apps in February 2026. One of them was Stripe's official subscription sample, which no longer type-checks against Stripe's own SDK. Tiltmeter fixed it and 14 others we checked by hand, and the pull requests are open. The fix for Stripe's sample →
Since then Tiltmeter has opened 84 pull requests for changes at Stripe, Jira, Spotify, OpenAI, Anthropic, Groq, Slack, OpenWeatherMap, Hugging Face and HubSpot. So far 26 have landed: 21 were merged as Tiltmeter wrote them, and 5 were adopted by maintainers who landed the fix themselves. inbox-zero and QOwnNotes carried Tiltmeter's commit into changes of their own, and aicommits made the same change in its own commit. OpenMetadata picked up Tiltmeter's commit in its own pull request for its 2.1 release, merged on 5 November. Seven were closed: anything-llm asks for a linked issue first, three touched code their projects no longer use, PraisonAI kept Groq's models because enterprise contracts still have them, LeanType found that the replacement, a reasoning model, spent its token budget before answering, and Irene Voice Assistant gave no reason. One repository was deleted. Serenity BDD, AiSOC, utopia and KodeRover's zadig moved their Jira calls off /rest/api/3/search, an endpoint Jira Cloud removed. ruflo, radare2's r2ai, mcp-chat, ByteDance's deer-flow and learn-claude-code moved their defaults off Claude models Anthropic has retired. Harvard's Machine Learning Systems book and KindleEar moved off the Groq models Groq shut down. Share-The-Link and nestjs-saas-starter read subscription periods from subscription items, as Stripe's API has required since 2025-03-31. Tencent's AI-Infra-Guard and SciSharp's BotSharp moved their Hugging Face calls to the router that replaced the retired Inference API.
How it compares
Needs nothing
from the vendor.
Nothing leaves
your machine.
- One SQLite file, no account. Shapes, status codes, latencies and changes. Response bodies only if you turn samples on.
- Secrets stay in your environment.
${VAR}values are expanded at request time and never written to disk or sent to the LLM. - Models see only what you opt into. Explanations and fixes get the change list and the files the agent opens, never response values.