Quickstart

Terminal exposes an OpenAI-compatible Chat Completions endpoint.

1. Set your API key

export TERMINAL_API_KEY="your_api_key"

Contact the Terminal team to request access.

2. Choose latency and cost per request

Different agent steps have different requirements. Use Standard when a user is waiting, and use Flex when a background step can trade time for lower cost. Both windows use the same endpoint. Add service_tier: "flex" when the request can be patient. Terminal charges per token and stays fully serverless, so you do not need to manage deployments or plan around rate limits. For finer request-level control, Terminal is adding SLA headers for latency and cost preferences. See Custom SLAs.

3. Make a Standard window request

Standard is the default window. It is designed for fast responses.

from openai import OpenAI

client = OpenAI(
    base_url="https://api.terminal.fyi/v1",
    api_key="TERMINAL_API_KEY",
)

response = client.chat.completions.create(
    model="gemma-4-31b",
    messages=[
        {
            "role": "user",
            "content": "Write a short release note for a lower-cost inference window.",
        }
    ],
    max_tokens=8000,
)

print(response.choices[0].message.content)

import OpenAI from "openai";

const client = new OpenAI({
  baseURL: "https://api.terminal.fyi/v1",
  apiKey: process.env.TERMINAL_API_KEY,
});

const response = await client.chat.completions.create({
  model: "gemma-4-31b",
  messages: [
    {
      role: "user",
      content: "Write a short release note for a lower-cost inference window.",
    },
  ],
  max_tokens: 8000,
});

console.log(response.choices[0].message.content);

import requests

response = requests.post(
    "https://api.terminal.fyi/v1/chat/completions",
    headers={
        "Authorization": "Bearer TERMINAL_API_KEY",
        "Content-Type": "application/json",
    },
    json={
        "model": "gemma-4-31b",
        "messages": [
            {
                "role": "user",
                "content": "Write a short release note for a lower-cost inference window.",
            }
        ],
        "max_tokens": 8000,
    },
    timeout=60,
)
response.raise_for_status()

completion = response.json()
print(completion["choices"][0]["message"]["content"])

const response = await fetch("https://api.terminal.fyi/v1/chat/completions", {
  method: "POST",
  headers: {
    Authorization: `Bearer ${process.env.TERMINAL_API_KEY}`,
    "Content-Type": "application/json",
  },
  body: JSON.stringify({
    model: "gemma-4-31b",
    messages: [
      {
        role: "user",
        content: "Write a short release note for a lower-cost inference window.",
      },
    ],
    max_tokens: 8000,
  }),
});
if (!response.ok) throw new Error(await response.text());

const completion = await response.json();
console.log(completion.choices[0].message.content);

curl -sS https://api.terminal.fyi/v1/chat/completions \
  -H "Authorization: Bearer $TERMINAL_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "gemma-4-31b",
    "messages": [
      {
        "role": "user",
        "content": "Write a short release note for a lower-cost inference window."
      }
    ],
    "max_tokens": 8000
  }' | jq

4. Make a Gemma Flex window request

Flex uses the same endpoint and waits for completion. It is designed for async-style workloads where cost matters more than latency. Allow long client timeouts.

from openai import OpenAI

client = OpenAI(
    base_url="https://api.terminal.fyi/v1",
    api_key="TERMINAL_API_KEY",
    timeout=1800.0,
)

response = client.chat.completions.create(
    model="gemma-4-31b",
    service_tier="flex",
    messages=[
        {
            "role": "user",
            "content": "Summarize this overnight evaluation run for the platform team.",
        }
    ],
    max_tokens=8000,
)

print(response.choices[0].message.content)

import OpenAI from "openai";

const client = new OpenAI({
  baseURL: "https://api.terminal.fyi/v1",
  apiKey: process.env.TERMINAL_API_KEY,
  timeout: 1_800_000,
});

const response = await client.chat.completions.create({
  model: "gemma-4-31b",
  service_tier: "flex",
  messages: [
    {
      role: "user",
      content: "Summarize this overnight evaluation run for the platform team.",
    },
  ],
  max_tokens: 8000,
});

console.log(response.choices[0].message.content);

import requests

response = requests.post(
    "https://api.terminal.fyi/v1/chat/completions",
    headers={
        "Authorization": "Bearer TERMINAL_API_KEY",
        "Content-Type": "application/json",
    },
    json={
        "model": "gemma-4-31b",
        "service_tier": "flex",
        "messages": [
            {
                "role": "user",
                "content": "Summarize this overnight evaluation run for the platform team.",
            }
        ],
        "max_tokens": 8000,
    },
    timeout=1800,
)
response.raise_for_status()

completion = response.json()
print(completion["choices"][0]["message"]["content"])

const response = await fetch("https://api.terminal.fyi/v1/chat/completions", {
  method: "POST",
  headers: {
    Authorization: `Bearer ${process.env.TERMINAL_API_KEY}`,
    "Content-Type": "application/json",
  },
  body: JSON.stringify({
    model: "gemma-4-31b",
    service_tier: "flex",
    messages: [
      {
        role: "user",
        content: "Summarize this overnight evaluation run for the platform team.",
      },
    ],
    max_tokens: 8000,
  }),
  signal: AbortSignal.timeout(1_800_000),
});
if (!response.ok) throw new Error(await response.text());

const completion = await response.json();
console.log(completion.choices[0].message.content);

curl -sS --max-time 1800 https://api.terminal.fyi/v1/chat/completions \
  -H "Authorization: Bearer $TERMINAL_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "gemma-4-31b",
    "service_tier": "flex",
    "messages": [
      {
        "role": "user",
        "content": "Summarize this overnight evaluation run for the platform team."
      }
    ],
    "max_tokens": 8000
  }' | jq

Request shape

{
  "model": "gemma-4-31b",
  "service_tier": "standard",
  "messages": [
    {
      "role": "user",
      "content": "Classify this support ticket by urgency and next owner."
    }
  ],
  "max_tokens": 8000
}

service_tier is optional. If omitted, Terminal uses standard.

Get started

Inference

Roadmap

Quickstart

Quickstart

1. Set your API key

2. Choose latency and cost per request

3. Make a Standard window request

4. Make a Gemma Flex window request

Request shape

​Quickstart

​1. Set your API key

​2. Choose latency and cost per request

​3. Make a Standard window request

​4. Make a Gemma Flex window request

​Request shape

Quickstart

1. Set your API key

2. Choose latency and cost per request

3. Make a Standard window request

4. Make a Gemma Flex window request

Request shape