Responses API and conversations
POST /v1/responses implements the OpenAI Responses API shape, so the OpenAI SDKs work unchanged. Unlike chat completions, which store nothing, a response is stored on Zabio’s side together with the conversation it belongs to. You send the new message; Zabio assembles the history, routes it, stores the answer and returns it.
For the exact shape of every request and response, with curl, Python and Node examples, see the API reference.
The key needs the conversations scope (keys have it by default). A key without it receives 403 insufficient_scope.
Create a response
Section titled “Create a response”from openai import OpenAI
client = OpenAI(base_url="https://api.zabio.ai/v1", api_key="insert your zabio.ai key here")
first = client.responses.create(model="auto", input="My name is Ana. Remember it.")second = client.responses.create(model="auto", input="What is my name?", previous_response_id=first.id)print(second.output_text)| Field | Type | Notes |
|---|---|---|
model |
string | a catalog id or auto, auto/best, auto/balanced, auto/cheap. Omitted means auto |
input |
string or list of items | a string is one user message. Items: {"role": "user" or "assistant" or "system" or "developer", "content": string or parts}, {"type": "function_call_output", "call_id", "output"}, {"type": "function_call", "call_id", "name", "arguments"}. Content parts: input_text, input_image (image_url), input_audio (input_audio with data as base64 and format wav or mp3, as on chat completions; only models with audio input take it), input_video (video_url as a data URL, an https URL or a YouTube URL, or file_id, as on chat completions; only models with video input take it) |
instructions |
string | system instructions for this turn (stored with the turn) |
previous_response_id |
resp_... |
continue from that response. Pointing at an older response branches: the conversation keeps its own thread and this response starts a new one |
conversation |
conv_... or {"id": ...} |
continue from the conversation’s latest turn. With previous_response_id, both must belong to the same conversation |
store |
bool, default true | false computes the turn from the stored history and stores nothing. The response id is not retrievable afterwards |
tools, tool_choice |
function tools: {"type": "function", "name", "description", "parameters"}, and {"type": "web_search"}, see Web search; tool_choice is auto, none, required, {"type": "function", "name"} or {"type": "web_search"} |
|
max_output_tokens, temperature, top_p, parallel_tool_calls |
sampling; max_output_tokens has no default: with none set, Zabio adds no output limit. A value above the served model’s maximum output is lowered to that limit, on every attempt |
|
text.format |
{"type": "json_object"} or {"type": "json_schema", "name", "schema", "strict"}. The Anthropic json_object caveat from Chat completions applies |
|
reasoning.effort |
none, minimal, low, medium, high or xhigh, mapped onto the served model’s own control; zabio.reasoning reports what was applied |
|
metadata |
object | up to 16 keys, 64 character keys, 512 character values |
stream |
bool | server sent events, see below |
background |
bool, default false | run the turn later and return at once with status: "queued"; poll GET /v1/responses/{id} for the result. Needs store true and no stream. See Background responses |
modalities, audio |
the Responses API answers in text: any modalities other than ["text"], or an audio object, answers 400 unsupported_parameter. For a spoken answer use chat completions, as OpenAI’s own audio models do |
|
zabio |
object | session_id, vendors, fallback, cache, split, deadline as in chat completions, plus retention_days (0 to 365, for a conversation created by this call) and compaction (auto or off) |
Without previous_response_id or conversation a new conversation is created for the response when store is true.
The response object
Section titled “The response object”{ "id": "resp_...", "object": "response", "created_at": 1758283200, "status": "completed", "model": "claude-haiku-4-5", "output": [ {"type": "message", "id": "msg_...", "status": "completed", "role": "assistant", "content": [{"type": "output_text", "text": "...", "annotations": []}]}, {"type": "function_call", "id": "fc_...", "call_id": "call_...", "name": "get_weather", "arguments": "{\"city\":\"Lima\"}", "status": "completed"} ], "usage": {"input_tokens": 120, "input_tokens_details": {"cached_tokens": 100}, "output_tokens": 20, "output_tokens_details": {"reasoning_tokens": 0}, "total_tokens": 140}, "previous_response_id": null, "conversation": {"id": "conv_..."}, "metadata": {}, "store": true, "zabio": {"vendor": "anthropic", "model_served": "claude-haiku-4-5", "routing": {...}, "cost_micros": 63, "cost": {...}, "byok": false, "escalated_from": "", "compacted": false, "split": {...}}}status is completed, incomplete (the output hit max_output_tokens; incomplete_details.reason says so; a reasoning model that spent its whole budget thinking ends this way with no text, so raise max_output_tokens so it leaves room for the answer above the thinking, or lower the reasoning effort) or failed; a background response is also queued or cancelled, and carries background: true. usage.output_tokens_details.reasoning_tokens is the vendor’s thinking count when it reports one, else 0. The zabio object is the one described in Chat completions, and here three more fields matter:
escalated_from: set when the first model chosen byautoasked for a stronger one, or when its answer failed Zabio’s check, and the answer came from another model. The escalation itself never appears inoutput.compacted: set when the stored history was summarized before this turn because it no longer fit the model’s context window.split: what split and route did to this turn (see Split and route), stored with the response likecompacted. Only the turn’s own last user message can be split, never the stored history;zabio.split: "off"sends the turn whole.tier: the service tier of this turn (see Flex tier):requested,appliedandfallback, stored with the response likesplit. Azabio.deadlineof 15 minutes or more makes the turn eligible for the vendor’s flex tier; one of 24 hours or more asks for the batch tier, which only a background response can wait for. On a synchronous turn with such a deadlineappliedisflexorstandardandfallbackissynchronous.plan: present on a turn served under the Free limits (see Billing, “The Free plan”):tier(free),reason(free_planorzero_balance),paid_would_use(onauto, the model a paid account would have been given, when it differs) andtopup_url. A background response carries it on the queued object and on the final one; a turn admitted with credit is billed paid to its end even if the balance crosses zero while it runs.
Streaming
Section titled “Streaming”With stream: true the response is a stream of server sent events: response.created, response.in_progress, response.output_item.added, response.content_part.added, response.output_text.delta, response.output_text.done, response.content_part.done, response.output_item.done, response.function_call_arguments.delta, response.function_call_arguments.done, and finally response.completed, response.incomplete or response.failed with the full response object. Each event carries sequence_number. On a vendor failure an error event is sent.
const stream = await client.responses.create({ model: "auto", input: "Hello", stream: true });for await (const event of stream) { if (event.type === "response.output_text.delta") process.stdout.write(event.delta); if (event.type === "response.completed") console.log("\n", event.response.conversation.id, event.response.zabio);}Web search
Section titled “Web search”Send tools: [{"type": "web_search"}] and the model’s own vendor searches the web for the turn: Anthropic’s web search on a Claude model, OpenAI’s on an OpenAI model. The model decides whether to search and what for. tool_choice: {"type": "web_search"}, or required with no function tool, makes an OpenAI model search before it answers; on a Claude model the choice stays the model’s.
const response = await client.responses.create({ model: "claude-sonnet-5", input: "What is the latest stable Go release? Cite the source.", tools: [{ type: "web_search" }],});It works on a model named in model whose zabio.pricing.per_search is set in GET /v1/models. auto does not search yet, and neither do the other vendors’ models: such a request answers 400 unsupported_for_model.
| Field | |
|---|---|
tools[].search_context_size |
low, medium or high on OpenAI models. Claude models have no such setting, so it is listed in zabio.ignored_parameters there |
tools[].user_location |
{"type": "approximate", "country", "city", "region", "timezone"} |
tools[].filters |
allowed_domains and blocked_domains, up to 100 each. A Claude model takes one of the two, not both |
max_tool_calls, zabio.search.max_uses |
the most searches the turn may run, 1 to 100, 3 when neither is sent; zabio.search.max_uses wins when both are |
include: ["web_search_call.action.sources"] |
lists the addresses each search returned in the search’s action.sources |
The answer carries one web_search_call output item per search, before the message (id, status and action with its type and query), and the message’s output_text carries annotations of type url_citation (url, title, start_index, end_index, counted in characters of the text). On a stream the events are OpenAI’s: response.web_search_call.in_progress, response.web_search_call.searching and response.web_search_call.completed for each search, and response.output_text.annotation.added for each citation.
Each search the vendor counts is billed on its own line beside the call, at the vendor’s price per search with Zabio’s fee. zabio.search says who searched (engine), how many searches the vendor counted (searches), the most the turn could run (max_uses) and what they cost (cost_micros); zabio.cost.search_micros and zabio.cost.search_fee_micros are that line’s two parts, and zabio.cost_micros includes it. The header x-zabio-searches carries the count on a response that is not streamed. The results count as input tokens of the call, as the vendor counts them. A request sets its searches aside on the balance when it is admitted, so a balance that cannot cover them answers 402 before any search runs.
What does not search:
- a
backgroundturn and a batch line (400unsupported_parameter) - a call on your organization’s own OpenAI key (400
unsupported_for_model): OpenAI’s search runs on Zabio’s OpenAI key only. A Claude model on your own Anthropic key searches, and Anthropic bills you the searches itself - a key whose web search is turned off (403
web_search_disabled), and a call on the Free plan’s limits (402insufficient_credits) - on OpenAI models, a request with
stop, with an output limit under 16 tokens, or with audio or video web_search_previewandexternal_web_access: false: sendweb_search
A stored conversation keeps the searches (their queries and state) and the citations (address and title), never the text of the results, which only the vendor holds. A later turn’s model reads the earlier answer, not the earlier search.
The vendor runs the search and reads the pages. Zabio does not see or screen the results, so treat what a cited page says as untrusted content.
Background responses
Section titled “Background responses”Send background: true and the call returns at once with the response object in status: "queued", background: true and an empty output. The turn runs later; poll GET /v1/responses/{id} until the status is completed, incomplete, failed or cancelled, then read output, usage and zabio as on any response. Everything else about the turn is the same: the conversation, previous_response_id, tools, compaction, the zabio options. background needs store true (the default; the response is stored so it can be polled) and cannot be combined with stream. A turn with an input_video part cannot run in the background (400 unsupported_parameter on background): Zabio does not keep videos until later.
Where the turn runs depends on zabio.deadline:
- A deadline of 24 hours or more on
autoor on a model with a batch tier puts the turn in your organization’s next batch: single background requests are gathered for up to 15 minutes, or until 500 have arrived, and sent together as one vendor batch per model at the vendors’ batch prices, about half of the list price. The result arrives within the vendors’ 24-hour window from that moment, usually much sooner. The turn is billed exactly like a line of a batch: the batch price for the tokens it used, plus the usual fees.zabio.tieris{"requested": "batch", "applied": "batch"}. Before the turn is queued, its estimated cost is set aside on your balance like a batch’s; if the estimate exceeds your available balance the call answers 402insufficient_creditsand nothing is stored. - Any other background request runs as soon as a worker picks it up, through the same routing as a synchronous call: at the flex tier from a deadline of 15 minutes where the model has one, at standard otherwise. A deadline of 24 hours or more on a model whose vendor has no batch tier runs this way too, and says so:
zabio.tieris{"requested": "batch", "applied": "flex"(or"standard"), "fallback": "no_batch_tier"}. Nothing is silently moved to a dearer tier. The call has the time limit of a synchronous call that is not streamed: after 20 minutes without an answer the turn endsfailedwithrequest_timeout(see Errors), or with the error of an attempt that failed before the limit. The 20 minutes count from when the call starts, not from when the turn was queued.
The queued object already carries zabio.tier with the tier the request got; the final object carries what served it.
Cancel. POST /v1/responses/{id}/cancel ends a queued turn, or one still waiting for its batch to be sent, with status: "cancelled" and nothing billed; its inputs stay in the conversation with that status, like a failed turn’s. A turn already running as a job is cut and ends cancelled. A turn already sent to the model vendor inside a batch cannot be pulled back: the call answers 409 not_cancellable and the answer arrives and is billed. Cancelling a turn that has ended returns the object as it is; cancelling a response that was not created with background answers 400 not_background.
Python
r = client.responses.create(model="auto", input="Summarize this report: ...", background=True, extra_body={"zabio": {"deadline": "24h"}})print(r.status) # queuedwhile r.status in ("queued", "in_progress"): time.sleep(60) r = client.responses.retrieve(r.id)print(r.status, r.output_text, r.model_extra["zabio"]["tier"])Node
let r = await client.responses.create({ model: "auto", input: "Summarize this report: ...", background: true, zabio: { deadline: "24h" } });while (r.status === "queued" || r.status === "in_progress") { await new Promise((ok) => setTimeout(ok, 60_000)); r = await client.responses.retrieve(r.id);}console.log(r.status, r.output_text, r.zabio.tier);Errors specific to this endpoint
Section titled “Errors specific to this endpoint”| Status | code | When |
|---|---|---|
| 400 | invalid_value on background |
background with stream, or with store: false |
| 400 | background_unavailable |
background responses are not available on this server |
| 400 | not_background |
cancel on a response that was not created with background |
| 409 | not_cancellable |
cancel on a turn already sent to the model vendor inside a batch; the answer arrives and is billed |
| 400 | conversation_mismatch |
previous_response_id belongs to another conversation |
| 400 | conversation_too_large |
the conversation reached 4,000 entries or 2,000,000 estimated tokens on its thread; start a new one |
| 400 | context_length_exceeded |
the history plus this turn does not fit the model’s context window and compaction is off or cannot help; the message carries the numbers |
| 403 | conversations_closed |
stored conversations are closed for this organization |
| 404 | previous_response_not_found, conversation_not_found, response_not_found |
unknown or deleted |
| 503 | conversations_unavailable |
stored conversations are not available on this server |
Everything in Errors applies as well, including 402 insufficient_credits and the free_tier_* codes of the Free plan.
Read and delete responses
Section titled “Read and delete responses”GET /v1/responses/{id}returns a stored response with its output, or its current status when it is a background response still running.POST /v1/responses/{id}/cancelcancels a background response.DELETE /v1/responses/{id}removes the response with its turn (the inputs it stored and the outputs) and everything stored after it in the same thread. The conversation continues from where that turn started. Deleting a queued background response drops it from the queue.
Conversations
Section titled “Conversations”POST /v1/conversationswith{"metadata": {...}, "items": [...], "zabio": {"retention_days": 30}}creates an empty (or seeded) conversation.GET /v1/conversations/{id}returns{"id", "object": "conversation", "created_at", "metadata", "zabio": {"entries", "token_estimate", "expires_at", "last_turn_at", "retention_days", "leaf_id"}}.leaf_idis the id of the last item of the conversation’s own thread (null while it is empty).POST /v1/conversations/{id}with{"metadata": {...}}replaces the metadata.DELETE /v1/conversations/{id}deletes the conversation and everything in it, immediately and permanently.GET /v1/conversations/{id}/items?limit=20&after=msg_...&order=asclists every stored item in creation order (all threads), including compaction summaries as{"type": "compaction", "summary": ...}. Each item carrieszabio: {"parent_id", "response_id"}: the item before it on its thread and the response that stored it (each null when there is none), so you can rebuild a thread: fromleaf_idfor the conversation’s own thread, or from the newest item for the latest branch made withprevious_response_id.POST /v1/conversations/{id}/itemswith{"items": [...]}appends items without a model turn, for example to seed history.
Which key sees what
Section titled “Which key sees what”A key that belongs to a project reads, continues, changes and deletes only the responses and conversations made with that project’s keys; a key at the organization’s level reaches the whole organization. A response or conversation outside answers 404 not_found, as if it did not exist. The project is the one the key that made it is in now. See Organizations, members and keys.
Escalation, stickiness and compaction
Section titled “Escalation, stickiness and compaction”On auto turns the model may signal that the task exceeds its capability. Zabio then re-runs the same turn on the next model up the curve, one its knowledge says is probably better, with the full history and returns that answer; later turns stay on the stronger model. Inside a conversation auto classifies each new user message and reuses that job across the tool-result turns that follow, until a signal (a run of failed tool calls, a long pause, a low confidence classification, user pushback) triggers a new choice.
When the stored history plus a new turn would not fit the context window of the model the request is served by, Zabio summarizes the oldest part of the history with a fast model, stores the summary as a compaction item, and continues. The summary call runs on Zabio’s key, is billed like any other call and appears in the conversation’s responses; a turn that needs compaction therefore needs credit even when it names a model served on your own key, and at zero balance it runs under the Free terms like any auto call, or gets 402 insufficient_credits before any summary is made when the organization is set to stop. zabio.compaction: "off" disables it; the request then fails with context_length_exceeded and the numbers.
Long calls
Section titled “Long calls”A model can think or write for minutes. Zabio receives the answer from the model provider as a stream, also when you did not ask for one (calls on the flex tier and decision models are sent in one piece, with their own time limit), and keeps the connection alive meanwhile. An attempt ends only when the provider stops sending data for 2 minutes (10 minutes before the first data on those OpenAI calls), or after 30 minutes in all; until any part of the answer has reached you, a failed attempt is tried again on another model.
- Not streamed. As soon as a model provider accepts the request (after 90 seconds at the latest), Zabio sends the
200status and headers, then one space every 15 seconds until the response object is ready; JSON parsers skip the spaces. On OpenAI’s models that reason, a call on your own OpenAI key, or one that setsstopon o1 or o3-mini, is answered only once the model has finished thinking, so its200goes out at the 90-second mark. A failure after that point is delivered as the response object withstatus: "failed"and anerrorobject (code,message), the shape a failed response has in any case. - Streamed. Whenever nothing else has been sent for 15 seconds, before
response.createdif need be, Zabio sends an SSE comment line (: ZABIO PROCESSING), which SSE clients ignore. A failure comes as aresponse.failedevent carrying the object withstatus: "failed"and itserror. - Your own timeout. Stream long answers, or set your client’s request timeout to at least 30 minutes.
- If you disconnect. Closing the connection cancels the vendor call, and you pay for the work done: the input, and the output generated before you disconnected, streamed or not. A stored turn that was cut is marked
failedorincomplete. - Headers on a long call. The
200and its headers go out before the answer; the vendor, the model served and the routing are in the object’szabiofield.
Responses carry the Server-Timing header that chat completions do, with the same stages; see Server timing.
Retention and deletion
Section titled “Retention and deletion”Each organization has a default retention (30 days, adjustable from 1 to 365 in Settings). A conversation may set a shorter or longer retention_days (0 to 365) when it is created; 0 deletes it as soon as the response is returned. The retention clock restarts on every turn. Expired conversations are deleted nightly. Deleting through the API is immediate and permanent. Bodies are encrypted at rest with a key that belongs to your organization; closing the organization deletes the conversations and destroys the key. A stored turn keeps its images and audio clips with its text, so later turns can still see and hear them; the items listing shows an audio clip’s format and size, never its bytes. Because every later turn resends the stored clips, a conversation that holds one needs a model that takes audio on each turn (auto keeps choosing among them; a named model without audio input answers 400 unsupported_for_model), and the audio limit of the model (100 MB on Gemini models, 20 MB on the others, and 20 MB on every model with background: true) counts the stored clips with the new ones. Stored images are resent the same way: a named model without vision answers 400 unsupported_for_model on a conversation that holds one. A video is the exception: Zabio never keeps one, so a stored turn keeps only the video’s format, size and length, the items listing shows that reference, and on later turns the model gets a short note in its place (“A video was attached here…”) instead of the video. To ask about it again, attach it again; a later turn without a new video needs no model with video input. With store: false nothing of the turn is kept. See Privacy.
Body logging for stateless calls
Section titled “Body logging for stateless calls”/v1/chat/completions and /v1/embeddings store nothing unless the API key has body logging on (a key setting, off by default). With it on, the request and response of each call are stored encrypted for the organization’s retention period, and owners and admins can read them while signed in to the web app. The call id is the response id without its chatcmpl- prefix, written as call_....