Skip to main content

​
Overview

Use Terminal to run OSS models through a fully serverless, OpenAI-compatible API built to scale to trillions of tokens every day.

​
Why Terminal for agents

Agent workloads are not one-size-fits-all. An interactive coding step may need fast tokens now, while a background review, eval, or research job can wait if that makes it cheaper. Terminal lets you choose that tradeoff per request. Use the Standard window for interactive work, use Flex when a job can be patient, and use request-level SLA headers as they become available for finer latency and cost control. You only pay per token. Terminal is serverless by default, with no deployments to manage and no rate limits to size around.

​
Base URL

https://api.terminal.fyi/v1

​
Endpoint

POST /chat/completions

​
Authentication

Terminal is currently invite-based. To request an API key, email shubham@terminal.fyi. All examples in these docs use the placeholder TERMINAL_API_KEY.

​
First request

from openai import OpenAI

client = OpenAI(
    base_url="https://api.terminal.fyi/v1",
    api_key="TERMINAL_API_KEY",
)

completion = client.chat.completions.create(
    model="gemma-4-31b",
    messages=[
        {
            "role": "user",
            "content": "Draft a short launch note for a new usage dashboard.",
        }
    ],
    max_tokens=8000,
)

print(completion.choices[0].message.content)

​
Current models

ModelIDGood for
Gemma 4 31Bgemma-4-31bCheap multimodal/general chat workloads
MiniMax M3minimax-m3Long-context, agentic, and multimodal workloads
Additional serverless OSS models can be requested. See Model requests.

​
Service windows

WindowUse whenBehavior
StandardYou need interactive latencyFast responses, 100+ tokens/sec on average
FlexYou can wait to reduce costSame endpoint waits for completion, usually 5-20 minutes

​
Next steps