API
Parameters
What the gateway touches, what it passes through, what it drops.
ThaiRouter is a thin proxy. It validates a handful of fields, rewrites model and max_tokens, and forwards the rest of the JSON body to vLLM exactly as you sent it. That makes the rules simple.
Handled by ThaiRouter
| Field | Type | Description |
|---|---|---|
| modelrequired | string | Validated against the live catalog, rewritten to the upstream weight name before forwarding. Responses echo your id. |
| messagesrequired | array | Must be a non-empty array. Byte length is used to estimate prompt tokens for the reservation. |
| max_tokens | integer | Missing or invalid → the model's max_output_tokens. Always clamped to that cap and to context_length minus the estimated prompt. Sizes the credit reservation. |
| stream | boolean | Selects SSE. When true, stream_options.include_usage is forced on so the final chunk carries usage. |
| stream_options | object | Overwritten to { "include_usage": true } on streams. You cannot turn usage off. |
| reasoning_effort | string | Validated (none … max), snapped to a level the model supports, translated for the chat template. Invalid values → 400. Not forwarded as-is. |
| reasoning | object | { effort, enabled, exclude }. Same handling; exclude strips reasoning from the response. Not forwarded as-is. |
| chat_template_kwargs | object | Forwarded, merged under the keys ThaiRouter sets for reasoning (ours win). |
Forwarded to the model
These are standard OpenAI fields vLLM understands. ThaiRouter neither validates nor changes them; a value the model server rejects comes back as its 400 with the reservation refunded.
| Field | Type | Description |
|---|---|---|
| temperature | number | 0 to 2. Higher is more random.Default 1. |
| top_p | number | Nucleus sampling. Use this or temperature, not both.Default 1. |
| stop | string | string[] | Up to a few sequences that end generation. |
| frequency_penalty | number | -2 to 2. |
| presence_penalty | number | -2 to 2. |
| seed | integer | Best-effort determinism for the same seed and parameters. |
| n | integer | Number of choices. All choices count as completion tokens and are billed. |
| logprobs / top_logprobs | boolean / integer | Token log probabilities in the response. |
| user | string | Opaque end-user id. Forwarded, never stored by ThaiRouter. |
Example
{
"model": "thairouter/glm-5.3-flash",
"messages": [{ "role": "user", "content": "สรุปข่าวนี้ใน 3 บรรทัด: …" }],
"temperature": 0.3,
"top_p": 0.9,
"max_tokens": 300,
"stop": ["\n\n\n"],
"seed": 42
}Experimental
These fields are forwarded too, and vLLM implements them, but we have not certified them per model yet. Test against the model you plan to use and treat the behaviour as subject to change.
| Field | Type | Description |
|---|---|---|
| response_format | object | { "type": "json_object" } or a json_schema. Constrains output to valid JSON. Verified on GLM 5.3 Flash. See Structured output. |
| structured_outputs | object | vLLM-native { choice | regex | grammar | json }. Verified. The legacy top-level guided_* fields are ignored by the current server. |
| tools / tool_choice | array / string | object | Function calling. Whether a model emits tool_calls depends on the model and server configuration. |
Unsupported
- Image, audio or file parts inside
messages[].content. Text only today. - Streaming without a usage chunk. Usage is always on.
- Turning thinking off on models whose template can't (GLM 5.3 Flash).
effort: "none"becomes the lowest supported level. - Endpoints other than chat completions: no
/v1/completions,/v1/embeddings,/v1/images,/v1/audio, batch or fine-tuning. - Provider routing, fallback or price-based model selection. One model id, one server.
Unknown top-level fields are forwarded, not rejected. If vLLM ignores them, so do we. See Errors for how upstream rejections surface.