Health

Check the real-time health state of every model in the gateway, including success rates, average latency, cooldown status, and daily usage.

Overview

The health endpoint exposes the live health state tracked by the gateway for each configured model. It is public — no authentication is required.

Endpoint: GET /health

Authentication: None

Request

GET https://your-gateway.workers.dev/health

Response

{
  "ok": true,
  "models": [
    {
      "key": "groq:llama-3.3-70b-versatile",
      "attempts": 100,
      "success_rate": 0.994,
      "avg_latency_ms": 820,
      "p90_latency_ms": 1400,
      "p99_latency_ms": 4200,
      "cooldown_until": 0,
      "headroom": 0.69,
      "daily_used": 3100,
      "daily_limit": 10000
    }
  ]
}

Response Fields

Field	Type	Description
`ok`	boolean	Whether the health endpoint responded successfully
`models`	array	Rolling health state per tracked model

Per-Model Fields

Field	Type	Description
`key`	string	Provider and model identifier
`attempts`	number	Number of rolling health samples retained for this model, capped at 100
`success_rate`	number	Rolling success rate (0–1)
`avg_latency_ms`	number	Rolling average response latency in milliseconds
`p90_latency_ms`	number	Rolling p90 response latency in milliseconds
`p99_latency_ms`	number	Rolling p99 response latency in milliseconds
`cooldown_until`	number	Unix timestamp in milliseconds when cooldown expires, or 0
`headroom`	number	Remaining configured daily headroom from 0 to 1
`daily_used`	number	Successful requests routed to this model today
`daily_limit`	number \| null	Configured daily limit for this model, or null if unlimited

When a model’s error rate exceeds the configured threshold, the gateway places it in cooldown for a short window. During cooldown, the router skips that model and falls back to healthy alternatives. Once cooldown_until passes, the model is re-evaluated on the next request.

A model marked healthy: false with in_cooldown: true will recover automatically — no operator action is needed.

Examples

curl https://your-gateway.workers.dev/health

const response = await fetch('https://your-gateway.workers.dev/health');
const health = await response.json();

console.log(`Gateway status: ${health.status}`);

for (const [model, state] of Object.entries(health.models)) {
  const status = state.healthy ? 'healthy' : state.in_cooldown ? 'cooldown' : 'unhealthy';
  console.log(`${model} (${state.provider}): ${status} — ${(state.success_rate * 100).toFixed(1)}% success`);
}