# Free AI - full content > OpenAI-compatible gateway across free-tier LLM providers with health-aware routing, automatic failover, and no credit card required. Generated: 2026-10-27 --- # Free AI — 80+ chat models through one gateway key > OpenAI-compatible gateway across free-tier LLM providers with health-aware routing, automatic failover, and no credit card required. Canonical: https://ai-gateway.sassmaker.com/ sass-maker module — v1.0 # Route to free-tier models with one gateway key One OpenAI-compatible gateway across 12 free-tier providers. Routing depends on configured credentials, model health and remaining quota. Operator-provisioned access; no self-serve signup. [Check access requirements](https://ai-gateway.sassmaker.com/authentication/) [See live provider status](https://ai-gateway.sassmaker.com/v1/routing/status) 12 Provider integrations 80+ Catalog models, not live capacity $0 Per Request SSE Streaming Build-time snapshot: live request count unavailable. Free requests routed $0 in provider API costs 12 Provider fallback chain Auto-skips throttled or failed providers Build-time snapshot: live savings estimate unavailable. vs. GPT-4o mini pricing Paid APIs charge ~$0.30/1K requests Capabilities One endpoint. 12 providers. Auto-failover when one breaks. ### Multi-Provider Routing Routes across 12 providers, including Workers AI, Groq, Gemini, Cerebras, SambaNova, NVIDIA, OpenRouter, and Z.ai, with health-aware fallback. ### Health-Aware Selection Tracks success rates, latency, and cooldowns per model. Unhealthy providers are automatically skipped. ### Streaming SSE Full Server-Sent Events streaming for chat completions and responses API. Drop-in OpenAI SDK compatible. ### Embeddings Generate embeddings via Workers AI, Gemini, or Voyage AI. Standard OpenAI embeddings format. ### Built-in Analytics Request logging, per-provider stats, and a live usage dashboard. Monitor everything from /v1/analytics. ### Rate Limiting IP-based rate limiting on public endpoints. Per-provider daily limits with configurable thresholds. Quick Start ## Up and running in one curl Point your existing OpenAI SDK or curl commands at the gateway. Set `model: "auto"` and the gateway handles provider selection, retries, and failover. Heads up: this aggregates free provider tiers — best-effort, no SLA. Per-IP limit is ~10 burst / ~20 rpm, each model has a daily cap, and the gateway returns `503` only when every capable model is rate-limited — rate-limit windows reset within minutes. For production throughput, pin a model and bring your own keys. [Full API reference](https://ai-gateway.sassmaker.com/getting-started/) terminal openai sdk ``` # 1. Check which providers are live right now (no key needed) curl https://ai-gateway.sassmaker.com/v1/routing/status # 2. Send your first chat completion curl https://ai-gateway.sassmaker.com/v1/chat/completions \ -H "Authorization: Bearer $KEY" \ -H "Content-Type: application/json" \ -d '{ "model": "auto", "project_id": "my_project", "messages": [{ "role": "user", "content": "Hello!" }] }' # Embeddings curl https://ai-gateway.sassmaker.com/v1/embeddings \ -H "Authorization: Bearer $KEY" \ -H "Content-Type: application/json" \ -d '{ "model": "@cf/baai/bge-base-en-v1.5", "project_id": "my_project", "input": "Hello world" }' ``` ``` // npm install openai (already installed if you use OpenAI SDK) import OpenAI from 'openai'; const client = new OpenAI({ baseURL: 'https://ai-gateway.sassmaker.com/v1', apiKey: process.env.GATEWAY_API_KEY, }); // Chat completion — same params as OpenAI const chat = await client.chat.completions.create({ model: 'auto', project_id: 'my_project', messages: [{ role: 'user', content: 'Hello!' }], }); console.log(chat.choices[0].message.content); // Embeddings const emb = await client.embeddings.create({ model: 'gemini-embedding-001', project_id: 'my_project', input: 'Hello world', }); console.log(emb.data[0].embedding.length + ' dims'); ``` --- # About — free-ai gateway > Why it exists, how it routes, and what the SLOs are. Canonical: https://ai-gateway.sassmaker.com/about/ [← free-ai](https://ai-gateway.sassmaker.com/) # About free-ai is an OpenAI-compatible gateway on top of a fleet of free LLM providers. Send one request; the gateway scores 30+ candidate models on per-IP rate-limit headroom, recent success rate, latency, and capability fit, then routes to the best one with retry on failure. ## Why Free tiers exist on Groq, Cerebras, SambaNova, OpenRouter, Workers AI, Gemini, NVIDIA, and a few more. Each has different rate limits, model coverage, and reliability. Routing across all of them gives you meaningful free-tier throughput plus automatic fallback when one is down — without you having to maintain the matrix. ## Capability filtering - Requests with `tools` route only to tool-capable models. - `response_format: json_object` routes to JSON-mode models. - Image content routes to vision-capable models. - If no candidate matches, the gateway returns `503` with a clear reason. ## Cost guardrails Committed Cloudflare config is intentionally free-first: `WORKERS_AI_ENABLED="false"` by default, CPU capped at the Workers Free limit, Workers Logs sampling off, no paid Rate Limiting binding. A `pnpm audit:cloudflare-costs` step runs in `pnpm check` and the deploy pipeline, so a cost regression fails the build before it ships. ## API shape Use it like the OpenAI SDK. Set `baseURL` to `https://ai-gateway.sassmaker.com/v1` and pass a `project_id` in the body so usage gets logged against your project for analytics. --- # Analytics > OpenAI-compatible AI gateway routing across free-tier providers with health-aware selection. Canonical: https://ai-gateway.sassmaker.com/analytics/ ## Overview The analytics endpoint returns aggregated usage statistics for your gateway — request volume, success/failure rates, and breakdowns by provider, model, project, and day. Aggregates are stored in Cloudflare D1. It exposes operational load data, so **all requests require a valid `GATEWAY_API_KEY` Bearer token**; without one the gateway returns `401`. **Endpoint:** `GET /v1/analytics` **Authentication:** Required (`GATEWAY_API_KEY`) ## Request ``` GET https://your-gateway.workers.dev/v1/analytics?project_id=proj_abc123 Authorization: Bearer ``` ### Query Parameters | Parameter | Type | Required | Description | |-----------|------|----------|-------------| | `project_id` | string | No | Filter analytics to a single project. Omit for all-project totals. | | `days` | number | No | Restrict to the last N days (1–365). Omit for all time. | ## Response ```json { "total_requests": 14823, "successful_requests": 14201, "failed_requests": 622, "success_rate": 0.958, "providers": { "openai": { "requests": 6400, "successful": 6350, "failed": 50 }, "anthropic": { "requests": 5200, "successful": 4980, "failed": 220 } }, "models": { "gpt-4o": { "requests": 3100, "successful": 3080, "failed": 20 } } } ``` ### Response Fields | Field | Type | Description | |-------|------|-------------| | `total_requests` | number | Total number of requests handled by the gateway for this project | | `successful_requests` | number | Requests that returned a 2xx response | | `failed_requests` | number | Requests that returned a non-2xx response | | `success_rate` | number | Ratio of successful to total requests (0–1) | | `providers` | object | Per-provider breakdown of request counts | | `models` | object | Per-model breakdown of request counts | ## Examples ### curl ```bash curl "https://your-gateway.workers.dev/v1/analytics?project_id=proj_abc123" \ -H "Authorization: Bearer " ``` ### JavaScript ```js const response = await fetch('https://your-gateway.workers.dev/v1/analytics?project_id=proj_abc123', { headers: { 'Authorization': 'Bearer ', }, }); const stats = await response.json(); console.log(`Total requests: ${stats.total_requests}`); console.log(`Success rate: ${(stats.success_rate * 100).toFixed(1)}%`); for (const [provider, data] of Object.entries(stats.providers)) { console.log(`${provider}: ${data.requests} requests`); } ``` --- # Authentication > OpenAI-compatible AI gateway routing across free-tier providers with health-aware selection. Canonical: https://ai-gateway.sassmaker.com/authentication/ ## Bearer Token Auth All requests to protected endpoints must include an `Authorization` header with a Bearer token: ``` Authorization: Bearer ``` ## Obtaining an API Key Production keys are currently provisioned by the operator. The legacy primary key is stored as the `GATEWAY_API_KEY` Worker secret. Additional keys can be added without exposing plaintext values by storing SHA-256 hashes in the `GATEWAY_API_KEY_HASHES` Worker secret. `GATEWAY_API_KEY_HASHES` accepts comma-separated or newline-separated entries. Each entry can be either a raw SHA-256 hex digest or `label:sha256hex`: ``` testing:9f86d081884c7d659a2feaa0c55ad015a3bf4f1b2b0b822cd15d6c15b0f00a08 customer-a:... ``` The documented self-serve `POST /access/request-key` flow is not live yet; do not build clients that depend on it until that endpoint ships. Operators should use `scripts/gateway-key-ring.mjs` to generate or import hash entries and upload the complete hash ring. Cloudflare Worker secrets are write-only, so the local operator manifest must contain every hash that should remain valid before uploading `GATEWAY_API_KEY_HASHES`. Store this key securely. Treat it like a password: do not commit it to source control, expose it in browser code, or log it from Workers. ## Using the Key Include the key in every request to a protected endpoint: ```bash curl https://your-gateway.workers.dev/v1/chat/completions \ -H "Authorization: Bearer gw_..." \ -H "Content-Type: application/json" \ -d '{ "model": "auto", "project_id": "my_project", "messages": [{ "role": "user", "content": "Hi" }] }' ``` ## Environment Variable Pattern Store your key in an environment variable and reference it in requests: ```bash export GATEWAY_API_KEY="gw_..." curl https://your-gateway.workers.dev/v1/chat/completions \ -H "Authorization: Bearer $GATEWAY_API_KEY" \ -H "Content-Type: application/json" \ -d '{ "model": "auto", "project_id": "my_project", "messages": [{ "role": "user", "content": "Hi" }] }' ``` ## Public Endpoints The following endpoints do **not** require authentication: | Endpoint | Method | Description | |---|---|---| | `/health` | `GET` | Provider health status | | `/v1/analytics` | `GET` | Aggregate usage analytics | | `/v1/stats/providers` | `GET` | Provider throttle statistics | | `/v1/models` | `GET` | Available model list | | `/docs` | `GET` | This documentation site | | `/openapi.json` | `GET` | OpenAPI specification | Token-spending generation endpoints and `/usage` require a valid Bearer token. --- # AI Gateway changelog — Verified updates > A first-party record of meaningful releases to the sass-maker AI Gateway. Canonical: https://ai-gateway.sassmaker.com/changelog/ Meaningful, verified updates to routing, model support, documentation, and operating safety. Planned work stays in GitHub Issues. ## July 18, 2026: Documentation became one connected system Product, architecture, development, operations, and knowledge documentation were consolidated into a single maintained tree. The public reference remains generated from the same source material. Documentation, Maintenance ## July 17, 2026: Public API reference and agent indexes The gateway gained a searchable public API reference plus llms.txt, full-text agent documentation, an agent catalog endpoint, and crawler guidance. API, Agent access ## June 28, 2026: Embedding catalog released Production model discovery began reporting six enabled embedding models with dimensions and aliases. The release was smoke-tested against the live gateway before downstream RAG consumers adopted it. Embeddings, Production ## May 28, 2026: Free-compute routing audited Provider allowances and failure modes were reviewed against the gateway’s cost rules. Hugging Face remained a manual probe rather than an automatic production route because its free allowance was too small for the pool. Routing, Cost --- # Chat Completions > OpenAI-compatible AI gateway routing across free-tier providers with health-aware selection. Canonical: https://ai-gateway.sassmaker.com/chat-completions/ ## Endpoint ``` POST /v1/chat/completions ``` ## Request Body | Field | Type | Default | Description | |---|---|---|---| | `model` | string | `"auto"` | Model identifier. Use `"auto"` for health-aware round-robin across all providers, or specify a model from [`GET /v1/models`](https://ai-gateway.sassmaker.com/models). | | `messages` | array | required | Conversation history. Each item has `role`, `content`, and optional `name`. | | `stream` | boolean | `false` | When `true`, responses are streamed as server-sent events. | | `temperature` | number | — | Sampling temperature (0–2). Higher values produce more varied output. | | `max_tokens` | number | — | Maximum number of tokens to generate. | | `min_reasoning_level` | string | — | Minimum reasoning tier for auto-routing. One of `"low"`, `"medium"`, `"high"`. | | `tools` | array | — | List of tool/function definitions. When present, the gateway only routes to models that support tool calling. | | `tool_choice` | string \| object | — | Controls tool use: `"none"`, `"auto"`, `"required"`, or `{ type: "function", function: { name: "..." } }`. | | `response_format` | object | — | `{ type: "json_object" }` requests JSON mode. `{ type: "json_schema", json_schema: { name, strict: true, schema } }` requests schema output from qualified models. | | `project_id` | string | required* | Project tag for analytics and rate accounting. You can also send `x-gateway-project-id`; one of the two is required. | ### Message Object Messages support both text and multimodal (vision) content: ```json { "role": "user", "content": "What is the capital of France?", "name": "alice" } ``` For vision requests, use the array content format with image URLs: ```json { "role": "user", "content": [ { "type": "text", "text": "What's in this image?" }, { "type": "image_url", "image_url": { "url": "https://example.com/photo.jpg" } } ] } ``` When the gateway detects `image_url` content parts, it automatically filters to vision-capable models (e.g. Gemini, Groq Llama 4). | Field | Type | Description | |---|---|---| | `role` | string | One of `"system"`, `"user"`, `"assistant"`, `"tool"`. | | `content` | string \| array | Message text, or array of content parts for multimodal input. | | `name` | string | Optional display name for the message author. | ### Capability-Based Routing The gateway automatically detects what capabilities your request needs and filters models accordingly: | Request Feature | Detected Capability | Effect | |---|---|---| | `tools` array present | Tool calling | Only routes to models that support function calling | | `response_format: { type: "json_object" }` | JSON mode | Only routes to models with structured output | | `image_url` in message content | Vision | Only routes to vision-capable models | | Large prompt | Context window | Excludes models whose context window is too small | This is fully automatic — no extra headers or configuration needed. ## Non-Streaming Response A standard OpenAI-compatible response object is returned. The gateway adds extra diagnostic headers: | Header | Description | |---|---| | `x-gateway-provider` | The backend provider that served the request (e.g. `groq`, `gemini`). | | `x-gateway-model` | The exact model used by the provider. | | `x-gateway-attempts` | Number of provider attempts before a successful response. | | `x-gateway-request-id` | Unique request identifier for support and log correlation. | | `x-gateway-reasoning-effort` | The reasoning effort level applied to the request. | ### Example Response ```json { "id": "chatcmpl-abc123", "object": "chat.completion", "created": 1714000000, "model": "llama-3.3-70b-versatile", "choices": [ { "index": 0, "message": { "role": "assistant", "content": "The capital of France is Paris." }, "finish_reason": "stop" } ], "usage": { "prompt_tokens": 15, "completion_tokens": 9, "total_tokens": 24 } } ``` ## Streaming Response When `stream: true`, the response is a stream of `data:` lines in SSE format, each containing a JSON delta object. The stream is terminated by `data: [DONE]`. ``` data: {"id":"chatcmpl-abc123","object":"chat.completion.chunk","created":1714000000,"model":"llama-3.3-70b-versatile","choices":[{"index":0,"delta":{"role":"assistant","content":"The"},"finish_reason":null}]} data: {"id":"chatcmpl-abc123","object":"chat.completion.chunk","created":1714000000,"model":"llama-3.3-70b-versatile","choices":[{"index":0,"delta":{"content":" capital"},"finish_reason":null}]} data: [DONE] ``` ## Examples ### Non-Streaming ### curl ```bash curl https://your-gateway.workers.dev/v1/chat/completions \ -H "Authorization: Bearer " \ -H "Content-Type: application/json" \ -d '{ "model": "auto", "project_id": "my_project", "messages": [ { "role": "system", "content": "You are a helpful assistant." }, { "role": "user", "content": "What is the capital of France?" } ], "temperature": 0.7, "max_tokens": 256 }' ``` ### JavaScript ```js const response = await fetch('https://your-gateway.workers.dev/v1/chat/completions', { method: 'POST', headers: { 'Authorization': 'Bearer ', 'Content-Type': 'application/json', }, body: JSON.stringify({ model: 'auto', project_id: 'my_project', messages: [ { role: 'system', content: 'You are a helpful assistant.' }, { role: 'user', content: 'What is the capital of France?' }, ], temperature: 0.7, max_tokens: 256, }), }); const data = await response.json(); console.log(data.choices[0].message.content); // Inspect gateway headers console.log('Provider:', response.headers.get('x-gateway-provider')); console.log('Model:', response.headers.get('x-gateway-model')); ``` ### Streaming ### curl ```bash curl https://your-gateway.workers.dev/v1/chat/completions \ -H "Authorization: Bearer " \ -H "Content-Type: application/json" \ -d '{ "model": "auto", "project_id": "my_project", "messages": [{ "role": "user", "content": "Tell me a short story." }], "stream": true }' ``` ### JavaScript ```js const response = await fetch('https://your-gateway.workers.dev/v1/chat/completions', { method: 'POST', headers: { 'Authorization': 'Bearer ', 'Content-Type': 'application/json', }, body: JSON.stringify({ model: 'auto', project_id: 'my_project', messages: [{ role: 'user', content: 'Tell me a short story.' }], stream: true, }), }); const reader = response.body.getReader(); const decoder = new TextDecoder(); while (true) { const { done, value } = await reader.read(); if (done) break; const chunk = decoder.decode(value); for (const line of chunk.split('\n')) { if (!line.startsWith('data: ')) continue; const payload = line.slice(6).trim(); if (payload === '[DONE]') break; const delta = JSON.parse(payload); const text = delta.choices?.[0]?.delta?.content ?? ''; process.stdout.write(text); } } ``` ### With Reasoning Effort ### curl ```bash curl https://your-gateway.workers.dev/v1/chat/completions \ -H "Authorization: Bearer " \ -H "Content-Type: application/json" \ -d '{ "model": "auto", "project_id": "my_project", "messages": [{ "role": "user", "content": "Solve: x^2 - 5x + 6 = 0" }], "min_reasoning_level": "high" }' ``` ### JavaScript ```js const response = await fetch('https://your-gateway.workers.dev/v1/chat/completions', { method: 'POST', headers: { 'Authorization': 'Bearer ', 'Content-Type': 'application/json', }, body: JSON.stringify({ model: 'auto', project_id: 'my_project', messages: [{ role: 'user', content: 'Solve: x^2 - 5x + 6 = 0' }], min_reasoning_level: 'high', }), }); const data = await response.json(); console.log(data.choices[0].message.content); ``` ### Tool Calling (Agentic) When you include `tools`, the gateway automatically routes to a model that supports function calling (Groq, Gemini, SambaNova, NVIDIA, Cerebras, or OpenRouter models with tool support). ### curl ```bash curl https://your-gateway.workers.dev/v1/chat/completions \ -H "Authorization: Bearer " \ -H "Content-Type: application/json" \ -d '{ "model": "auto", "project_id": "my_project", "messages": [{ "role": "user", "content": "What is the weather in San Francisco?" }], "tools": [{ "type": "function", "function": { "name": "get_weather", "description": "Get current weather for a location", "parameters": { "type": "object", "properties": { "location": { "type": "string" } }, "required": ["location"] } } }], "tool_choice": "auto" }' ``` ### OpenAI SDK ```js const client = new OpenAI({ baseURL: 'https://your-gateway.workers.dev/v1', apiKey: '', }); const completion = await client.chat.completions.create({ model: 'auto', project_id: 'my_project', messages: [{ role: 'user', content: 'What is the weather in San Francisco?' }], tools: [{ type: 'function', function: { name: 'get_weather', description: 'Get current weather for a location', parameters: { type: 'object', properties: { location: { type: 'string' } }, required: ['location'], }, }, }], tool_choice: 'auto', }); const toolCall = completion.choices[0].message.tool_calls?.[0]; console.log(toolCall?.function.name, toolCall?.function.arguments); ``` ### JSON Mode (Structured Output) When you set `response_format`, the gateway only picks models that support JSON mode. ### curl ```bash curl https://your-gateway.workers.dev/v1/chat/completions \ -H "Authorization: Bearer " \ -H "Content-Type: application/json" \ -d '{ "model": "auto", "project_id": "my_project", "messages": [{ "role": "user", "content": "List 3 programming languages with their year of creation as JSON" }], "response_format": { "type": "json_object" } }' ``` ### OpenAI SDK ```js const completion = await client.chat.completions.create({ model: 'auto', project_id: 'my_project', messages: [{ role: 'user', content: 'List 3 programming languages with their year of creation as JSON' }], response_format: { type: 'json_object' }, }); const parsed = JSON.parse(completion.choices[0].message.content); console.log(parsed); ``` ### Vision (Image Input) Send images using the multimodal content format. The gateway auto-detects images and routes to vision-capable models (Gemini, Groq Llama 4). ### curl ```bash curl https://your-gateway.workers.dev/v1/chat/completions \ -H "Authorization: Bearer " \ -H "Content-Type: application/json" \ -d '{ "model": "auto", "project_id": "my_project", "messages": [{ "role": "user", "content": [ { "type": "text", "text": "What is in this image?" }, { "type": "image_url", "image_url": { "url": "https://upload.wikimedia.org/wikipedia/commons/a/a7/Camponotus_flavomarginatus_ant.jpg" } } ] }] }' ``` ### OpenAI SDK ```js const completion = await client.chat.completions.create({ model: 'auto', project_id: 'my_project', messages: [{ role: 'user', content: [ { type: 'text', text: 'Describe this image in detail.' }, { type: 'image_url', image_url: { url: 'https://example.com/photo.jpg' } }, ], }], }); console.log(completion.choices[0].message.content); ``` ### Schema-constrained output For non-streaming chat without tools, use `response_format.type: "json_schema"` with `json_schema: { name: "classifier", strict: true, schema: { ... } }`. The root schema must be an object with `properties`, `required`, and `additionalProperties: false`. Nested objects should also require every field and forbid additional properties. Keep schemas within the providers' supported JSON Schema subset. Validate semantic values in the consumer as well. Automatic routing retains health and cost ranking among qualified schema models: Gemini chat models and Groq GPT-OSS 20B/120B or Qwen 3.8 27B. Other models' JSON-mode support alone does not qualify them. Token caps and the two-attempt gateway limit are unchanged. Generated `json_validate_failed` errors can use normal fallback; other upstream 400 caller errors remain non-retriable. --- # Decision APIs > OpenAI-compatible AI gateway routing across free-tier providers with health-aware selection. Canonical: https://ai-gateway.sassmaker.com/decisions/ ## Endpoints `POST /v1/decide` answers typed questions using a supplied JSON state. `POST /v1/compare` runs selected engines on the same state and questions. Both require gateway authentication and a nonempty `project_id`. Private Fleet service bindings support these operations with the same project allowlist. Public callers send `Authorization: Bearer ` or `x-api-key`. Project IDs contain 1–64 letters, digits, dots, underscores, colons, or hyphens; if `x-gateway-project-id` is present it must match the body. ```json { "project_id": "starboard", "engine": "free_ai", "model": "auto", "state": {"requirement": "persistent caching", "tool": "in-memory cache"}, "questions": [ {"id": "fit", "type": "choice", "prompt": "Does the tool meet the requirement?", "options": ["yes", "no"]}, {"id": "priority", "type": "score", "prompt": "How urgent is the gap?", "min": 0, "max": 10}, {"id": "proven", "type": "bool", "prompt": "Is the gap supported by evidence?"} ], "limits": {"deadline_ms": 30000, "max_output_tokens": 2048, "max_cost_usd": 0} } ``` `free_ai` uses existing health-aware routing, provider quotas, and the unchanged Workers AI fallback neuron budget. `auto` is the default. Explicit registered model selections are pinned for comparisons; unknown models are unavailable. `max_cost_usd` currently accepts only zero: this API does not authorize paid providers or increase any existing budget. `jev` is a reserved engine selector and always returns `provider_disabled`. Its adapter, paid approval, and priced budget integration are future work. It never calls Jev or silently substitutes a chat model. For comparison, replace `engine` and `model` with: ```json {"engines": [{"engine": "free_ai", "model": "auto"}, {"engine": "jev"}]} ``` The request must select 2–4 distinct engine/model pairs. The response contains `input_hash`, `project_id`, and `results` in request order. No winner or accuracy claim is inferred from agreement. ## Results and limits Each result includes `input_hash`, `engine`, `requested_model`, actual `model` and `provider`, `status`, `answers`, `usage`, and `latency_ms`. Each answer has its question `id`, `status: "answered" | "abstained"`, typed `value`, and a brief `reason`. Abstention requires `value: null`. Result status is `ok`, `error`, or `unavailable`; a successful single decision also includes `project_id`. Reasons contain 1–2,000 characters and question IDs contain 1–128 characters. Successful outputs must contain exactly every question ID once; values must match the question type, choice options, and score bounds. Invalid outputs return `status: "error"` without usable answers. Reasons are model-generated; callers remain responsible for checking citations and semantic correctness. The SHA-256 `input_hash` covers only `{state, questions}`, using recursively sorted object keys (JavaScript codepoint sort) and preserving array order. Project, model, engine, and limits are excluded. Comparison freezes this input once and supplies the same serialization to every selected engine. - 1–60 unique questions; each prompt up to 2,000 characters. - Choice: 1–64 unique nonempty strings, each up to 256 characters. A sole explicit `none` option is valid; unsupported evidence can still abstain. - Score: finite `min` and `max`, with `max > min`. Bool: a boolean value. - Request and buffered gateway response: at most 65,536 bytes each. - Final decision response: at most 65,536 bytes. Final comparison response: at most 270,336 bytes across all engines. Larger outputs fail closed with `response_limit_exceeded` and empty answers. - Canonical frozen state/questions: at most 32,768 UTF-8 bytes and depth 32. - Deadline: 1,000–60,000 ms; default 30,000 ms. Cancellation propagates to active provider inference and stops later attempts. - Output: 128–4,096 tokens per engine; default 2,048. Valid requests return HTTP 200 even if an engine is unavailable or errors, with a safe `error.code` and message. Unavailable/error results have empty answers; disabled engines have null actual model/provider/usage. Malformed, oversized, or mismatched project requests return HTTP 400. Authentication and rate-limit failures retain existing HTTP 401/503/429 behavior. Providers do not receive caller routing override headers through these decision endpoints. --- # API documentation > OpenAI-compatible API documentation for the Free AI Gateway. Canonical: https://ai-gateway.sassmaker.com/docs/ An OpenAI-compatible gateway across free-tier providers. Start with [`/docs/getting-started`](https://ai-gateway.sassmaker.com/docs/getting-started/) or jump directly to an API reference below. ## [Getting started](https://ai-gateway.sassmaker.com/docs/getting-started/) Connect an OpenAI-compatible client to the gateway. ## [Chat completions](https://ai-gateway.sassmaker.com/docs/chat-completions/) Generate text with streaming, tools, JSON mode, and vision. ## [Responses API](https://ai-gateway.sassmaker.com/docs/responses-api/) Use the OpenAI Responses format through the gateway. ## [Decision APIs](https://ai-gateway.sassmaker.com/docs/decisions/) Answer typed questions and compare engines on one frozen input. ## [Models](https://ai-gateway.sassmaker.com/docs/models/) Browse routing capabilities and available model families. ## [Authentication](https://ai-gateway.sassmaker.com/docs/authentication/) Authenticate protected requests with a gateway key. OpenAI-compatible endpoints live under `/v1`. --- # Analytics > Retrieve aggregated usage statistics for a specific project — request counts, success/failure rates, and breakdowns by provider and model. Canonical: https://ai-gateway.sassmaker.com/docs/analytics ## Overview The analytics endpoint returns aggregated usage statistics for your gateway — request volume, success/failure rates, and breakdowns by provider, model, project, and day. Aggregates are stored in Cloudflare D1. It exposes operational load data, so **all requests require a valid `GATEWAY_API_KEY` Bearer token**; without one the gateway returns `401`. **Endpoint:** `GET /v1/analytics` **Authentication:** Required (`GATEWAY_API_KEY`) ## Request ``` GET https://your-gateway.workers.dev/v1/analytics?project_id=proj_abc123 Authorization: Bearer ``` ### Query Parameters | Parameter | Type | Required | Description | |-----------|------|----------|-------------| | `project_id` | string | No | Filter analytics to a single project. Omit for all-project totals. | | `days` | number | No | Restrict to the last N days (1–365). Omit for all time. | ## Response ```json { "total_requests": 14823, "successful_requests": 14201, "failed_requests": 622, "success_rate": 0.958, "providers": { "openai": { "requests": 6400, "successful": 6350, "failed": 50 }, "anthropic": { "requests": 5200, "successful": 4980, "failed": 220 } }, "models": { "gpt-4o": { "requests": 3100, "successful": 3080, "failed": 20 } } } ``` ### Response Fields | Field | Type | Description | |-------|------|-------------| | `total_requests` | number | Total number of requests handled by the gateway for this project | | `successful_requests` | number | Requests that returned a 2xx response | | `failed_requests` | number | Requests that returned a non-2xx response | | `success_rate` | number | Ratio of successful to total requests (0–1) | | `providers` | object | Per-provider breakdown of request counts | | `models` | object | Per-model breakdown of request counts | ## Examples ### curl ```bash curl "https://your-gateway.workers.dev/v1/analytics?project_id=proj_abc123" \ -H "Authorization: Bearer " ``` ### JavaScript ```js const response = await fetch('https://your-gateway.workers.dev/v1/analytics?project_id=proj_abc123', { headers: { 'Authorization': 'Bearer ', }, }); const stats = await response.json(); console.log(`Total requests: ${stats.total_requests}`); console.log(`Success rate: ${(stats.success_rate * 100).toFixed(1)}%`); for (const [provider, data] of Object.entries(stats.providers)) { console.log(`${provider}: ${data.requests} requests`); } ``` --- # Authentication > How to obtain and use API keys with the AI Gateway. Canonical: https://ai-gateway.sassmaker.com/docs/authentication ## Bearer Token Auth All requests to protected endpoints must include an `Authorization` header with a Bearer token: ``` Authorization: Bearer ``` ## Obtaining an API Key Production keys are currently provisioned by the operator. The legacy primary key is stored as the `GATEWAY_API_KEY` Worker secret. Additional keys can be added without exposing plaintext values by storing SHA-256 hashes in the `GATEWAY_API_KEY_HASHES` Worker secret. `GATEWAY_API_KEY_HASHES` accepts comma-separated or newline-separated entries. Each entry can be either a raw SHA-256 hex digest or `label:sha256hex`: ``` testing:9f86d081884c7d659a2feaa0c55ad015a3bf4f1b2b0b822cd15d6c15b0f00a08 customer-a:... ``` The documented self-serve `POST /access/request-key` flow is not live yet; do not build clients that depend on it until that endpoint ships. Operators should use `scripts/gateway-key-ring.mjs` to generate or import hash entries and upload the complete hash ring. Cloudflare Worker secrets are write-only, so the local operator manifest must contain every hash that should remain valid before uploading `GATEWAY_API_KEY_HASHES`. Store this key securely. Treat it like a password: do not commit it to source control, expose it in browser code, or log it from Workers. ## Using the Key Include the key in every request to a protected endpoint: ```bash curl https://your-gateway.workers.dev/v1/chat/completions \ -H "Authorization: Bearer gw_..." \ -H "Content-Type: application/json" \ -d '{ "model": "auto", "project_id": "my_project", "messages": [{ "role": "user", "content": "Hi" }] }' ``` ## Environment Variable Pattern Store your key in an environment variable and reference it in requests: ```bash export GATEWAY_API_KEY="gw_..." curl https://your-gateway.workers.dev/v1/chat/completions \ -H "Authorization: Bearer $GATEWAY_API_KEY" \ -H "Content-Type: application/json" \ -d '{ "model": "auto", "project_id": "my_project", "messages": [{ "role": "user", "content": "Hi" }] }' ``` ## Public Endpoints The following endpoints do **not** require authentication: | Endpoint | Method | Description | |---|---|---| | `/health` | `GET` | Provider health status | | `/v1/analytics` | `GET` | Aggregate usage analytics | | `/v1/stats/providers` | `GET` | Provider throttle statistics | | `/v1/models` | `GET` | Available model list | | `/docs` | `GET` | This documentation site | | `/openapi.json` | `GET` | OpenAPI specification | Token-spending generation endpoints and `/usage` require a valid Bearer token. --- # Chat Completions > Generate chat completions using the OpenAI-compatible /v1/chat/completions endpoint. Canonical: https://ai-gateway.sassmaker.com/docs/chat-completions ## Endpoint ``` POST /v1/chat/completions ``` ## Request Body | Field | Type | Default | Description | |---|---|---|---| | `model` | string | `"auto"` | Model identifier. Use `"auto"` for health-aware round-robin across all providers, or specify a model from [`GET /v1/models`](https://ai-gateway.sassmaker.com/models). | | `messages` | array | required | Conversation history. Each item has `role`, `content`, and optional `name`. | | `stream` | boolean | `false` | When `true`, responses are streamed as server-sent events. | | `temperature` | number | — | Sampling temperature (0–2). Higher values produce more varied output. | | `max_tokens` | number | — | Maximum number of tokens to generate. | | `min_reasoning_level` | string | — | Minimum reasoning tier for auto-routing. One of `"low"`, `"medium"`, `"high"`. | | `tools` | array | — | List of tool/function definitions. When present, the gateway only routes to models that support tool calling. | | `tool_choice` | string \| object | — | Controls tool use: `"none"`, `"auto"`, `"required"`, or `{ type: "function", function: { name: "..." } }`. | | `response_format` | object | — | `{ type: "json_object" }` requests JSON mode. `{ type: "json_schema", json_schema: { name, strict: true, schema } }` requests schema output from qualified models. | | `project_id` | string | required* | Project tag for analytics and rate accounting. You can also send `x-gateway-project-id`; one of the two is required. | ### Message Object Messages support both text and multimodal (vision) content: ```json { "role": "user", "content": "What is the capital of France?", "name": "alice" } ``` For vision requests, use the array content format with image URLs: ```json { "role": "user", "content": [ { "type": "text", "text": "What's in this image?" }, { "type": "image_url", "image_url": { "url": "https://example.com/photo.jpg" } } ] } ``` When the gateway detects `image_url` content parts, it automatically filters to vision-capable models (e.g. Gemini, Groq Llama 4). | Field | Type | Description | |---|---|---| | `role` | string | One of `"system"`, `"user"`, `"assistant"`, `"tool"`. | | `content` | string \| array | Message text, or array of content parts for multimodal input. | | `name` | string | Optional display name for the message author. | ### Capability-Based Routing The gateway automatically detects what capabilities your request needs and filters models accordingly: | Request Feature | Detected Capability | Effect | |---|---|---| | `tools` array present | Tool calling | Only routes to models that support function calling | | `response_format: { type: "json_object" }` | JSON mode | Only routes to models with structured output | | `image_url` in message content | Vision | Only routes to vision-capable models | | Large prompt | Context window | Excludes models whose context window is too small | This is fully automatic — no extra headers or configuration needed. ## Non-Streaming Response A standard OpenAI-compatible response object is returned. The gateway adds extra diagnostic headers: | Header | Description | |---|---| | `x-gateway-provider` | The backend provider that served the request (e.g. `groq`, `gemini`). | | `x-gateway-model` | The exact model used by the provider. | | `x-gateway-attempts` | Number of provider attempts before a successful response. | | `x-gateway-request-id` | Unique request identifier for support and log correlation. | | `x-gateway-reasoning-effort` | The reasoning effort level applied to the request. | ### Example Response ```json { "id": "chatcmpl-abc123", "object": "chat.completion", "created": 1714000000, "model": "llama-3.3-70b-versatile", "choices": [ { "index": 0, "message": { "role": "assistant", "content": "The capital of France is Paris." }, "finish_reason": "stop" } ], "usage": { "prompt_tokens": 15, "completion_tokens": 9, "total_tokens": 24 } } ``` ## Streaming Response When `stream: true`, the response is a stream of `data:` lines in SSE format, each containing a JSON delta object. The stream is terminated by `data: [DONE]`. ``` data: {"id":"chatcmpl-abc123","object":"chat.completion.chunk","created":1714000000,"model":"llama-3.3-70b-versatile","choices":[{"index":0,"delta":{"role":"assistant","content":"The"},"finish_reason":null}]} data: {"id":"chatcmpl-abc123","object":"chat.completion.chunk","created":1714000000,"model":"llama-3.3-70b-versatile","choices":[{"index":0,"delta":{"content":" capital"},"finish_reason":null}]} data: [DONE] ``` ## Examples ### Non-Streaming ### curl ```bash curl https://your-gateway.workers.dev/v1/chat/completions \ -H "Authorization: Bearer " \ -H "Content-Type: application/json" \ -d '{ "model": "auto", "project_id": "my_project", "messages": [ { "role": "system", "content": "You are a helpful assistant." }, { "role": "user", "content": "What is the capital of France?" } ], "temperature": 0.7, "max_tokens": 256 }' ``` ### JavaScript ```js const response = await fetch('https://your-gateway.workers.dev/v1/chat/completions', { method: 'POST', headers: { 'Authorization': 'Bearer ', 'Content-Type': 'application/json', }, body: JSON.stringify({ model: 'auto', project_id: 'my_project', messages: [ { role: 'system', content: 'You are a helpful assistant.' }, { role: 'user', content: 'What is the capital of France?' }, ], temperature: 0.7, max_tokens: 256, }), }); const data = await response.json(); console.log(data.choices[0].message.content); // Inspect gateway headers console.log('Provider:', response.headers.get('x-gateway-provider')); console.log('Model:', response.headers.get('x-gateway-model')); ``` ### Streaming ### curl ```bash curl https://your-gateway.workers.dev/v1/chat/completions \ -H "Authorization: Bearer " \ -H "Content-Type: application/json" \ -d '{ "model": "auto", "project_id": "my_project", "messages": [{ "role": "user", "content": "Tell me a short story." }], "stream": true }' ``` ### JavaScript ```js const response = await fetch('https://your-gateway.workers.dev/v1/chat/completions', { method: 'POST', headers: { 'Authorization': 'Bearer ', 'Content-Type': 'application/json', }, body: JSON.stringify({ model: 'auto', project_id: 'my_project', messages: [{ role: 'user', content: 'Tell me a short story.' }], stream: true, }), }); const reader = response.body.getReader(); const decoder = new TextDecoder(); while (true) { const { done, value } = await reader.read(); if (done) break; const chunk = decoder.decode(value); for (const line of chunk.split('\n')) { if (!line.startsWith('data: ')) continue; const payload = line.slice(6).trim(); if (payload === '[DONE]') break; const delta = JSON.parse(payload); const text = delta.choices?.[0]?.delta?.content ?? ''; process.stdout.write(text); } } ``` ### With Reasoning Effort ### curl ```bash curl https://your-gateway.workers.dev/v1/chat/completions \ -H "Authorization: Bearer " \ -H "Content-Type: application/json" \ -d '{ "model": "auto", "project_id": "my_project", "messages": [{ "role": "user", "content": "Solve: x^2 - 5x + 6 = 0" }], "min_reasoning_level": "high" }' ``` ### JavaScript ```js const response = await fetch('https://your-gateway.workers.dev/v1/chat/completions', { method: 'POST', headers: { 'Authorization': 'Bearer ', 'Content-Type': 'application/json', }, body: JSON.stringify({ model: 'auto', project_id: 'my_project', messages: [{ role: 'user', content: 'Solve: x^2 - 5x + 6 = 0' }], min_reasoning_level: 'high', }), }); const data = await response.json(); console.log(data.choices[0].message.content); ``` ### Tool Calling (Agentic) When you include `tools`, the gateway automatically routes to a model that supports function calling (Groq, Gemini, SambaNova, NVIDIA, Cerebras, or OpenRouter models with tool support). ### curl ```bash curl https://your-gateway.workers.dev/v1/chat/completions \ -H "Authorization: Bearer " \ -H "Content-Type: application/json" \ -d '{ "model": "auto", "project_id": "my_project", "messages": [{ "role": "user", "content": "What is the weather in San Francisco?" }], "tools": [{ "type": "function", "function": { "name": "get_weather", "description": "Get current weather for a location", "parameters": { "type": "object", "properties": { "location": { "type": "string" } }, "required": ["location"] } } }], "tool_choice": "auto" }' ``` ### OpenAI SDK ```js const client = new OpenAI({ baseURL: 'https://your-gateway.workers.dev/v1', apiKey: '', }); const completion = await client.chat.completions.create({ model: 'auto', project_id: 'my_project', messages: [{ role: 'user', content: 'What is the weather in San Francisco?' }], tools: [{ type: 'function', function: { name: 'get_weather', description: 'Get current weather for a location', parameters: { type: 'object', properties: { location: { type: 'string' } }, required: ['location'], }, }, }], tool_choice: 'auto', }); const toolCall = completion.choices[0].message.tool_calls?.[0]; console.log(toolCall?.function.name, toolCall?.function.arguments); ``` ### JSON Mode (Structured Output) When you set `response_format`, the gateway only picks models that support JSON mode. ### curl ```bash curl https://your-gateway.workers.dev/v1/chat/completions \ -H "Authorization: Bearer " \ -H "Content-Type: application/json" \ -d '{ "model": "auto", "project_id": "my_project", "messages": [{ "role": "user", "content": "List 3 programming languages with their year of creation as JSON" }], "response_format": { "type": "json_object" } }' ``` ### OpenAI SDK ```js const completion = await client.chat.completions.create({ model: 'auto', project_id: 'my_project', messages: [{ role: 'user', content: 'List 3 programming languages with their year of creation as JSON' }], response_format: { type: 'json_object' }, }); const parsed = JSON.parse(completion.choices[0].message.content); console.log(parsed); ``` ### Vision (Image Input) Send images using the multimodal content format. The gateway auto-detects images and routes to vision-capable models (Gemini, Groq Llama 4). ### curl ```bash curl https://your-gateway.workers.dev/v1/chat/completions \ -H "Authorization: Bearer " \ -H "Content-Type: application/json" \ -d '{ "model": "auto", "project_id": "my_project", "messages": [{ "role": "user", "content": [ { "type": "text", "text": "What is in this image?" }, { "type": "image_url", "image_url": { "url": "https://upload.wikimedia.org/wikipedia/commons/a/a7/Camponotus_flavomarginatus_ant.jpg" } } ] }] }' ``` ### OpenAI SDK ```js const completion = await client.chat.completions.create({ model: 'auto', project_id: 'my_project', messages: [{ role: 'user', content: [ { type: 'text', text: 'Describe this image in detail.' }, { type: 'image_url', image_url: { url: 'https://example.com/photo.jpg' } }, ], }], }); console.log(completion.choices[0].message.content); ``` ### Schema-constrained output For non-streaming chat without tools, use `response_format.type: "json_schema"` with `json_schema: { name: "classifier", strict: true, schema: { ... } }`. The root schema must be an object with `properties`, `required`, and `additionalProperties: false`. Nested objects should also require every field and forbid additional properties. Keep schemas within the providers' supported JSON Schema subset. Validate semantic values in the consumer as well. Automatic routing retains health and cost ranking among qualified schema models: Gemini chat models and Groq GPT-OSS 20B/120B or Qwen 3.8 27B. Other models' JSON-mode support alone does not qualify them. Token caps and the two-attempt gateway limit are unchanged. Generated `json_validate_failed` errors can use normal fallback; other upstream 400 caller errors remain non-retriable. --- # Live Dashboard > Public HTML dashboard at /dashboard showing real-time KPIs, throttle heatmap, model health, and project usage for the gateway. Canonical: https://ai-gateway.sassmaker.com/docs/dashboard ## Endpoint ``` GET /dashboard # canonical GET /live # alias GET /v1/dashboard # alias ``` **Authentication:** None — public. A single self-contained HTML page that visualizes the gateway's live health and usage. No build step, no framework — just `