Skip to main content

​
Quickstart

Terminal exposes an OpenAI-compatible Chat Completions endpoint.

​
1. Set your API key

export TERMINAL_API_KEY="your_api_key"
To request an API key, email shubham@terminal.fyi.

​
2. Choose latency and cost per request

Different agent steps have different requirements. Use Standard when a user is waiting, and use Flex when a background step can trade time for lower cost. Both windows use the same endpoint. Add service_tier: "flex" when the request can be patient. Terminal charges per token and stays fully serverless, so you do not need to manage deployments or plan around rate limits. For finer request-level control, Terminal is adding SLA headers for latency and cost preferences. See Custom SLAs.

​
3. Make a Standard window request

Standard is the default window. It is designed for fast responses. 
from openai import OpenAI

client = OpenAI(
    base_url="https://api.terminal.fyi/v1",
    api_key="TERMINAL_API_KEY",
)

response = client.chat.completions.create(
    model="gemma-4-31b",
    messages=[
        {
            "role": "user",
            "content": "Write a short release note for a lower-cost inference window.",
        }
    ],
    max_tokens=8000,
)

print(response.choices[0].message.content)

​
4. Make a Gemma Flex window request

Flex uses the same endpoint and waits for completion. It is designed for async-style workloads where cost matters more than latency. Allow long client timeouts. 
from openai import OpenAI

client = OpenAI(
    base_url="https://api.terminal.fyi/v1",
    api_key="TERMINAL_API_KEY",
    timeout=1800.0,
)

response = client.chat.completions.create(
    model="gemma-4-31b",
    service_tier="flex",
    messages=[
        {
            "role": "user",
            "content": "Summarize this overnight evaluation run for the platform team.",
        }
    ],
    max_tokens=8000,
)

print(response.choices[0].message.content)

​
Request shape

{
  "model": "gemma-4-31b",
  "service_tier": "standard",
  "messages": [
    {
      "role": "user",
      "content": "Classify this support ticket by urgency and next owner."
    }
  ],
  "max_tokens": 8000
}
service_tier is optional. If omitted, Terminal uses standard.