The model.cards API

Four steps from nothing to a routed completion. Everything below assumes your key is in MODELCARDS_API_KEY.

export MODELCARDS_API_KEY="mc-v1-…"

curl https://model.cards/api/v1/chat/completions \
  -H "Authorization: Bearer $MODELCARDS_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "modelcards/auto",
    "messages": [
      {"role": "user", "content": "Write a haiku about idempotency."}
    ]
  }'

model.cards is bring-your-own-key by design: the gateway authenticates to each provider with your credentials, so your negotiated rates, credits and rate limits apply unchanged and the provider's invoice goes to you. There is no balance to top up here and no per-token fee.

How stored keys are handled

Keys are encrypted at rest with AES-256-GCM and decrypted only in memory at the moment a request needs them. They are never logged, never sent anywhere except the provider they belong to, and never shown again after you save them — the dashboard displays a short hint like sk-ant-…x4Kq so you can recognise a key without exposing it. Deleting your account deletes the stored keys with it.

Which key serves a request

Your connected key for a provider always wins. If you have no key for the provider a model routes to, the candidate is skipped; if no candidate is reachable at all, the free simulator answers, clearly flagged. If a provider rejects your key (revoked, out of quota), the error message says so explicitly — a 401 from your own provider is yours to fix, and the response tells you where.

Every request carries a bearer token in the Authorization header. There are no cookies and no signing; your provider keys are connected once in the dashboard and never appear in requests.

POST /api/v1/chat/completions
Authorization: Bearer mc-v1-4f3c…9a21
Content-Type: application/json

Key format

Keys are issued as mc-v1- followed by 48 hexadecimal characters. Only a SHA-256 hash is stored, so a lost key cannot be recovered — revoke it and issue another. The dashboard shows a truncated hint such as mc-v1-4f3c…9a21 to help you match a key to a service.

Per-key spend guardrails

Each key can carry an optional guardrail on estimated upstream spend — what your own providers will bill for the requests this key routed, priced from the catalog. Requests that would push the estimate past the limit fail with 403 key_spend_limit_exceeded while the rest of the account keeps running — useful for capping a staging environment or a third-party integration before it runs up your provider bill. Disabling a key takes effect on the next request.

The guardrail is enforced before generation against the most a request could cost: prompt tokens plus max_tokens at the output price of the most expensive endpoint the request could fail over to. If you omit max_tokens, the projection assumes a 4,096-token completion rather than the model's full ceiling, and the request is capped there. Ask for more by passing max_tokens explicitly. Zero-cost requests (free models) always pass.

The reservation is taken atomically and trued up when the request settles, so concurrent requests cannot each authorize against the same headroom and collectively overshoot the limit.

POST/api/v1/chat/completions

The one endpoint you need. Request and response bodies follow the OpenAI chat completions schema, with two additions: model ids are namespaced by author, and the response reports which provider served the call plus what it cost.

Request fields

Example request

{
  "model": "anthropic/claude-sonnet-4.5",
  "messages": [
    {"role": "system", "content": "You answer in one sentence."},
    {"role": "user", "content": "Why is exactly-once delivery hard?"}
  ],
  "max_tokens": 200,
  "temperature": 0.3
}

Example response

{
  "id": "chatcmpl-2f9c41a7b3e05d9c8a4f16b2",
  "object": "chat.completion",
  "created": 1761500000,
  "model": "anthropic/claude-sonnet-4.5",
  "provider": "anthropic",
  "choices": [
    {
      "index": 0,
      "message": {
        "role": "assistant",
        "content": "Because the network can always fail between the act and the acknowledgement."
      },
      "finish_reason": "stop"
    }
  ],
  "usage": {
    "prompt_tokens": 27,
    "completion_tokens": 18,
    "total_tokens": 45,
    "cost": 0.000351
  },
  "x_modelcards": {
    "provider": "anthropic",
    "simulated": false,
    "byok": true,
    "cost_nano": 351000,
    "latency_ms": 842
  }
}

cost is the estimated upstream cost in US dollars — what the provider key that served the request will be billed, computed from the serving endpoint's per-token prices. model.cards deducts nothing; the same figure appears on the usage event in your dashboard. x_modelcards.byok reports whether the request ran on your own connected key, and auto-routed requests additionally carry requested_model, route_tier, complexity, decision_source, trajectory_confidence and signals (see routing).

Set stream: true and the response becomes text/event-stream. Each frame is a data: line carrying a chat.completion.chunk object whose delta holds the newly generated text. The stream ends with a literal data: [DONE].

data: {"id":"chatcmpl-2f9c41a7b3e05d9c8a4f16b2","object":"chat.completion.chunk","choices":[{"delta":{"role":"assistant"}}]}

data: {"id":"chatcmpl-2f9c41a7b3e05d9c8a4f16b2","object":"chat.completion.chunk","choices":[{"delta":{"content":"Because"}}]}

data: {"id":"chatcmpl-2f9c41a7b3e05d9c8a4f16b2","object":"chat.completion.chunk","choices":[{"delta":{"content":" the network"}}]}

data: {"id":"chatcmpl-2f9c41a7b3e05d9c8a4f16b2","object":"chat.completion.chunk","choices":[{"finish_reason":"stop","delta":{}}],"usage":{"prompt_tokens":27,"completion_tokens":18,"cost":0.000351}}

data: [DONE]

Usage and the cost estimate arrive on the final chunk, once the completion is complete enough to meter. The usage event is written when the stream closes, including when a client disconnects early — your provider bills for what it generated. Disconnecting also cancels the upstream request, so nothing keeps generating on your key after you stop reading.

If a stream fails part-way it emits a single data: frame carrying an error object and then ends without [DONE]. That is deliberate: SDK stream iterators treat [DONE] as a clean finish, so sending it after a failure would present a truncated answer as a complete one. Tokens already delivered still appear in the usage log, because the provider produced — and will bill for — them.

curl -N https://model.cards/api/v1/chat/completions \
  -H "Authorization: Bearer $MODELCARDS_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"model": "openai/gpt-4o-mini", "messages": [{"role":"user","content":"hi"}], "stream": true}'

GET /api/v1/models

Returns the full catalog. Prices are strings in US dollars per token so you can render them without floating-point surprises. This endpoint does not require authentication.

{
  "object": "list",
  "data": [
    {
      "id": "anthropic/claude-sonnet-4.5",
      "object": "model",
      "created": 1759104000,
      "name": "Claude Sonnet 4.5",
      "owned_by": "anthropic",
      "context_length": 200000,
      "architecture": {"input_modalities": ["text", "image"], "output_modalities": ["text"]},
      "supported_parameters": ["max_tokens", "temperature", "top_p", "stop", "seed", "user"],
      "pricing": {"prompt": "0.000003", "completion": "0.000015"}
    }
  ]
}

supported_parameters lists what the gateway actually honours, not what the underlying model can do. Anything absent is refused with unsupported_parameter rather than dropped, so a client is never billed for a response that ignored half its request. Free models price at "0".

GET /api/v1/key

Reports what the calling key is allowed to do. Handy for a start-up check or a billing widget in your own product.

{
  "data": {
    "label": "production-api",
    "usage_usd": 41.87,
    "limit_usd": 250.0,
    "disabled": false
  }
}

usage_usd and limit_usd are per key — limit_usd is null when the key carries no guardrail. Both are estimates of upstream cost at catalog prices; the money itself moves between you and your providers.

Errors use conventional HTTP status codes and always carry the same JSON envelope, whether they originate at the gateway or upstream.

{
  "error": {
    "message": "API key spend limit would be exceeded. This request's estimated upstream cost is up to $0.400000 (worst case at max_tokens=4000), and this key has used $9.80 of its $10.00 limit. Pass a lower max_tokens or raise the key's limit.",
    "type": "permission_error",
    "code": "key_spend_limit_exceeded"
  }
}

The envelope matches OpenAI's, so an existing SDK surfaces error.message unchanged. Branch on error.code — error.type is the coarse OpenAI family (invalid_request_error, authentication_error, permission_error, api_error) and several codes share one type.

Retrying a 500 or 502 is safe — a request that never produced tokens costs your provider account nothing. On 403 the fix is on your side: raise the key's guardrail or pass a lower max_tokens. If the message says your own provider key was rejected, update it on the Providers page.

modelcards/auto — the closed loop

Ask for modelcards/auto and the gateway picks the concrete model per request, in three moves:

1. Score. Deterministic heuristics rate the request's complexity in [0,1]: prompt length, presence of code, reasoning demands (“prove”, “debug”, “optimize”…), math, multi-step structure and conversation depth. No extra model call, no added latency.
2. Route. The score maps to a tier — light, standard or heavy — and the tier picks a model from the providers you have connected: light requests land on fast, cheap models, heavy ones on frontier reasoning models.
3. Learn. Every outcome updates two running statistics for the route that served it: how long it took, and whether it failed. Both decay, so a bad afternoon fades instead of condemning a provider forever. Latency is pooled across accounts — provider capacity is shared, and your request is slow for the same reason everyone else's is. Failures are counted per account, because a rate limit is your quota being spent, not the provider being unwell.

Auto-routed responses carry the decision in x_modelcards: requested_model is modelcards/auto, route_tier and complexity explain the tier, deadline_s and p_bad show the budget the choice was made against and the risk it was judged to carry, and the top-level model field names the model that actually served it. Prefer a specific model? Address it by full catalog id and the router stays out of the way.

Three more fields say why your request landed where it did. decision_source names the rule that actually chose the tier — content when the prompt decided it, override_error or override_compaction when a hard rule forced the top tier, settled when a passing suite over recent edits dropped it, trajectory when the tool history moved it, and ambiguous when there was tool history but it did not agree with itself. signals lists the heuristics that fired, and trajectory_confidence is how strongly the tool-history signals corroborated each other, in [0,1] — zero whenever there was no tool history to read.

Within one model: endpoint selection and failover

A model id can be served by several providers. The gateway collects every active endpoint for the model and keeps the ones you can reach with a connected key. Choosing between them is not price-first: the cheapest endpoint is a poor deal if it times out half the time, because you pay your provider for the attempt and then wait for the retry.

So each candidate is scored on what it should cost plus what going wrong would cost — cost + C × P(late or failed). The price half is exact and comes from the catalog, counting prompt and completion together, so an endpoint that is cheap on input and expensive on output cannot win by default. The risk half is drawn from that endpoint's own recorded history, sampled rather than averaged: a route we know little about is tried optimistically instead of ignored, and one that has been failing is passed over without being banned outright. A brand-new endpoint inherits from its nearest sibling — the same model at another provider, or the closest-priced model at the same one — so it starts with a reasonable guess rather than no opinion at all.

C is a multiple of the request's own reference cost rather than a fixed sum, so one setting means the same thing whether the request costs a cent or a thousandth of one. It defaults to 10 — a late or failed response is treated as ten times worse than paying for a good one — and is configurable per key.

“Late” is whatever you say it is. Send x-router-deadline-ms on a request, or set a default on the key; the value is clamped to between 250 ms and 10 minutes, and falls back to 60 s. This is the single most useful knob here. An interactive key with a 5 s deadline and a batch key with a 5 minute one will make genuinely different choices from the same catalog — the batch key happily takes a slow cheap endpoint that the interactive key rejects on sight.

If it fails with a server error, a timeout, a rate limit, or a credential rejection, the next endpoint in the ordering is tried. A 400-class refusal is not retried: the request itself is wrong and every other provider would reject it identically. Failover happens before any token is emitted, so a streamed response never switches provider mid-flight. The response's provider field always names whichever endpoint actually produced tokens.

Endpoints that cannot hold the prompt are filtered out before ranking. If no endpoint can hold it you get a context_length_exceeded 400 naming the largest context available. When every candidate fails you get a 502 and your provider account is charged nothing. When no candidate is reachable at all — you have no key connected for any provider serving the model, and the deployment holds no fallback — the built-in simulator answers instead, always flagged with "simulated": true and a modelcards provider. Simulated output is never mistaken for inference and never costs anything.

Tool calling

Standard OpenAI shape: send tools, get tool_calls back with finish_reason: "tool_calls", reply with a role: "tool" message carrying the matching tool_call_id. Streaming works, with call fragments arriving as they are generated rather than buffered to the end.

Two behaviours are worth knowing because they are deliberate refusals rather than gaps.

Tools never fall back to a model that cannot call them. A request naming a text-only model returns 400 tools_unsupported instead of answering with prose, and the built-in simulator — which is the friendly fallback everywhere else — is excluded for the same reason. Getting a paragraph where your framework expected a function call is the single most expensive way this API could fail you. supported_parameters on GET /api/v1/models tells you which models qualify — it lists tools only when every endpoint serving that model qualifies, since failover could otherwise land you on one that does not.

Arguments are never repaired. arguments is a JSON string, and we forward exactly what the model produced — we never re-serialise it, and never patch malformed JSON into something valid. If you echo back a call whose arguments do not parse you get 400 invalid_tool_arguments, because the alternative is running your tool with empty arguments while the round trip looks successful.

{
  "model": "modelcards/auto",
  "messages": [{ "role": "user", "content": "weather in Oslo?" }],
  "tools": [
    {
      "type": "function",
      "function": {
        "name": "get_weather",
        "parameters": {
          "type": "object",
          "properties": { "city": { "type": "string" } },
          "required": ["city"]
        }
      }
    }
  ]
}

The legacy functions / function_call API is not supported — use tools and tool_choice. parallel_tool_calls: false is refused rather than ignored, since it cannot be honoured on every dialect a model routes to and a caller who sets it is usually protecting a tool that is unsafe to run concurrently.

Routing control

Four constraints, and it is worth knowing which are absolute and which are preferences, because the difference decides what happens during an outage.

Model allowlist (per key) and provider only / ignore (per key or per request) are hard. A key scoped to two models reaches exactly those two, whether it names one directly or lets the router choose — an allowlist that only bound auto-routing would be no boundary at all.

Provider order is soft. Listing a provider first means “all else equal, prefer this one”: it wins when its expected loss is within 5% of the best available, and loses when it is materially worse. A hard pin that survived an outage would be a footgun — the request fails, and the router had a working alternative it was not allowed to use. Use only when you genuinely mean it.

A key-level preference is a boundary; a request-level one is a hint from whoever holds the key. The body may narrow what the key permits and can never widen it, so a leaked key scoped to one provider does not become access to all of them.

{
  "model": "modelcards/auto",
  "messages": [{ "role": "user", "content": "hi" }],
  "provider": {
    "order": ["anthropic", "openai"],
    "ignore": ["some-flaky-provider"]
  }
}

Minimum tier (per key) is a floor on the quality layer: heavy means this key never routes below the frontier band, while a request the classifier independently scores as heavy is never dragged down by a lower override. It can only raise quality, which is what makes it safe to set and forget.

Finally, two slug suffixes on a direct model id: :floor makes cost dominate — the cheapest workable endpoint wins — and :nitro tightens the deadline to 10 s and prices a miss steeply. Both are shorthand for settings you can express yourself; they are applied as policy adjustments rather than a separate selection path, so they compose with everything above and cannot disagree with the ranker.

Rate limits

600 requests per minute per key by default, configurable per key. This is a runaway guardrail, not a quota — you are paying your own providers, so metering your capacity would serve nobody. It exists so a loop cannot bill your provider account thousands of times before you notice.

The bucket refills continuously rather than resetting on a boundary, so a client that has been idle keeps a full minute of burst and one pacing itself evenly is never refused for an accident of alignment. A refused request does not consume budget, so a brief overage cannot become a lockout. Refusals carry Retry-After and X-RateLimit-Limit.

The whole gateway is free during beta — you pay your providers, never us. And one model works before you connect any key at all:

modelcards/simulator-v1

simulator-v1 is a test model built into the gateway. It does no inference — it reads your last message and answers with a structured acknowledgement of what was asked — so it is useful for wiring up a client, checking your SSE parsing, or exercising the whole loop before you connect a provider key. It costs nothing, always.

One thing it will not do is pretend to call a tool. A request carrying tools is refused here with 400 tools_unsupported rather than answered with prose — which is exactly the point, since wiring an agent framework against a stand-in that silently returns paragraphs is a debugging session nobody enjoys. Point it at a tool-capable model once you have a key connected.

It is also the only model available without an account: the playground will run it for signed-out visitors, and every other model asks you to sign in first. Responses from it always carry "simulated": true, which is your signal never to treat the text as a real model's answer.

Org accounts with shared provider keys, per-member quotas, SSO, DPAs and security review are handled as an enterprise agreement while the platform is in beta. Details are on the pricing page, or email sales@model.cards.