Docs/Text Models

Text Models

Query chat models via the OpenAI-compatible API

Token Hub serves chat models through a single OpenAI-compatible endpoint. If you have used the OpenAI SDK before, you already know the shape — point your client at Token Hub, swap the API key, and change the model id. Everything else stays the same.

Chat Completions API

Send a POST to /v1/chat/completions with a bearer token and a messages array. Each message has a role (system, user, assistant, or tool) and content.

curl https://tokenhub-staging.coality.io/v1/chat/completions \
  -H "Authorization: Bearer $TOKEN_HUB_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "glm-5-3",
    "messages": [{"role": "user", "content": "Explain vector databases in one paragraph"}]
  }'

Request parameters

Parameter	Type	Description
`model`	string	Model id from the catalog, e.g. glm-5-3. Required.
`messages`	array	Conversation so far, oldest first. Required.
`max_tokens`	integer	Upper bound on generated tokens. Recommended — unbounded responses bill accordingly.
`temperature`	number	Sampling randomness, 0–2. Lower is more deterministic.
`top_p`	number	Nucleus sampling cutoff.
`stream`	boolean	Token-by-token responses. Coming soonStreaming is on the roadmap; omit for now.
`tools`	array	Function definitions for tool-capable models. See Tool Calling.
`response_format`	object	Constrain output to JSON. See Structured Outputs.

Multi-turn conversations

Models are stateless — send the whole conversation each time. Append the assistant's reply as a role: "assistant" message and continue. Long conversations bill per input token, so trim history you don't need; prompt caching (see Prompt Caching) discounts repeated prefixes automatically.

System prompts

Put stable instructions (persona, constraints, output rules) in a firstsystem message. Keeping the system prompt byte-identical across requests maximizes cache hits.

Reading the response

json
{
  "id": "chatcmpl-8f3a1c",
  "model": "glm-5-3",
  "choices": [{
    "index": 0,
    "message": { "role": "assistant", "content": "..." },
    "finish_reason": "stop"
  }],
  "usage": {
    "prompt_tokens": 52,
    "completion_tokens": 23,
    "total_tokens": 75
  }
}

usage is what you are billed for: input tokens at the model's input rate, output tokens at its output rate. Cached input tokens are billed at the cache-read rate — see Prompt Caching.

Listing available models

GET /v1/models (authenticated) returns the models your key can call. The public catalog at GET /api/public/models needs no auth and includes pricing — handy for building model pickers.

Migrating from the OpenAI SDK

python
from openai import OpenAI

client = OpenAI(
    base_url="https://tokenhub-staging.coality.io/v1",
    api_key="YOUR_TOKEN_HUB_API_KEY",
)
# from here, client.chat.completions.create(...) works unchanged

There is nothing else to install. Any SDK that lets you override the base URL — OpenAI (Python, Node, Go), LangChain, LlamaIndex, Vercel AI SDK — works out of the box.