ChatGPT API Guide in 2026 (Developers' Practical Reference)

A practical guide — pricing, models, function calling, structured outputs, streaming, and prompt caching — for developers actually shipping with the OpenAI API.

If you are paying $300/month on stacked ChatGPT Plus seats so a small team can churn through long-context tasks, you are funding the wrong meter. A weekend project — a 200-line Python script wired to the OpenAI API with prompt caching, structured outputs, and a basic rate limiter — usually replaces that bill with a $20–$40 monthly invoice and gives you everything Plus users complain about losing: longer effective context, no message caps, no random throttling at 4 PM, and full control over which model handles which task.

This guide is the version a senior engineer would write for a junior teammate on day one — the actual mechanics of shipping with the OpenAI API in 2026. Models and pricing, the request shapes you will use ninety percent of the time, function calling, structured outputs, streaming, prompt caching, the Realtime API for voice, and the four mistakes every team makes on their first production deploy. No "AI is transforming software." Just the stuff you wish someone had told you before you billed your first dollar of tokens.

The 2026 context — what changed since GPT-4

Three shifts matter if your last hands-on with the API was during the GPT-4 Turbo era. First, GPT-5 is the default capable model and it is dramatically cheaper than the model it replaced — input tokens dropped from $5 per million on GPT-4o to $1.25 on GPT-5, and the model is materially better at multi-step reasoning and tool use. The old advice of "use GPT-3.5 for cheap stuff, GPT-4 for hard stuff" is obsolete; GPT-5 is now both the cheap default and the hard-stuff default.

Second, prompt caching went from beta curiosity to the single biggest cost-control lever in the API. Any prefix of 1,024 tokens or more that you reuse across requests is automatically cached at 50% off input price for cache hits. This means a RAG pipeline that crams 30,000 tokens of context into every call now bills the context once per five-minute window instead of once per call — which moves serious workloads from "we cannot afford this" to "this is cheaper than the database query."

Third, the Realtime API and Assistants v2 closed the gap with bespoke voice and agent stacks. You can stream bidirectional audio with sub-300ms latency over a single WebSocket, and Assistants v2 manages threads, file search, and code interpreter so you do not have to reinvent retrieval and tool orchestration on every project. For ninety percent of agent use cases, build-on-API now beats build-from-scratch.

Setup — keys, billing, and rate limits

You will not build anything until the boring infrastructure is correct. The setup story has three concrete steps that always trip first-time developers, so handle them in order before you write a single line of code.

Models and pricing — picking the right one

OpenAI publishes a dozen models but you will use three or four in practice. The table below cuts through the catalog and shows the price-per-quality picture as of 2026.

The rule of thumb in 2026: start with GPT-5 for everything, profile your costs after two weeks, and only then route specific high-volume routes to mini or nano. The temptation to over-optimize for the cheapest model on day one always backfires — you ship a worse product and lose more in user friction than you save in tokens.

Basic completion — the request shape you will use most

Ninety percent of the API calls you write will look almost identical: a system message setting context, a user message with the request, and a response. Master this shape and the rest of the API is variations on the theme.

from openai import OpenAI
client = OpenAI()

response = client.chat.completions.create(
    model="gpt-5",
    messages=[
        {"role": "system", "content": "You are a concise technical writer."},
        {"role": "user", "content": "Explain prompt caching in two sentences."}
    ],
    temperature=0.3,
    max_tokens=200,
)

print(response.choices[0].message.content)
print(f"Tokens used: {response.usage.total_tokens}")

Three details worth internalizing. temperature at 0.0 is deterministic and best for extraction, classification, and code generation; 0.7 is the right default for prose. max_tokens is a safety belt — set it even when you think you do not need to, because a runaway model continuing past your expected output is the most common cause of surprise bills. And always read response.usage; it is the only honest record of what the call cost.

Function calling (tools) — giving the model real capabilities

A naked language model is a fancy autocomplete. Function calling — now branded "tools" in the API — is what turns it into something useful. You declare the functions your code can run (with JSON Schema for arguments), the model decides when to call them and with what arguments, your code executes them, and the model uses the results to compose its final answer.

tools = [{
    "type": "function",
    "function": {
        "name": "get_weather",
        "description": "Get current weather for a city",
        "parameters": {
            "type": "object",
            "properties": {
                "city": {"type": "string"},
                "unit": {"type": "string", "enum": ["celsius", "fahrenheit"]}
            },
            "required": ["city"]
        }
    }
}]

response = client.chat.completions.create(
    model="gpt-5",
    messages=[{"role": "user", "content": "What's the weather in Tokyo?"}],
    tools=tools,
)

tool_call = response.choices[0].message.tool_calls[0]
args = json.loads(tool_call.function.arguments)
result = get_weather(args["city"], args.get("unit", "celsius"))

The trap nearly every team falls into is treating tools as one-shot calls. They are not — the natural shape is a loop. The model calls a tool, you append the result to messages as a tool-role message, you call the API again, and the model either calls another tool or produces the final answer. Most production agent code is fifty lines of loop and four hundred lines of tool definitions. Start with one tool, prove the loop, then add the rest.

Structured outputs — guaranteed JSON that matches your schema

For years, getting reliable JSON out of an LLM meant pleading with it in the system prompt and parsing defensively. Structured outputs solved this with a constraint-decoding feature: the model is forced at the token level to emit only output valid against your JSON Schema. No more retry loops, no more "remove the markdown fence around the JSON" hacks.

from pydantic import BaseModel

class Invoice(BaseModel):
    invoice_number: str
    total_usd: float
    line_items: list[str]
    due_date: str

response = client.chat.completions.parse(
    model="gpt-5",
    messages=[
        {"role": "system", "content": "Extract invoice data from the user message."},
        {"role": "user", "content": invoice_text}
    ],
    response_format=Invoice,
)

invoice = response.choices[0].message.parsed
print(invoice.total_usd)

The Python SDK auto-generates the schema from a Pydantic model, which is why this snippet is six lines instead of forty. In TypeScript, use Zod with the helper from the official SDK and you get the same ergonomics. Once you have used structured outputs you will never write a JSON-extraction prompt again.

Streaming — making the UI feel fast

A user staring at a blank screen for eight seconds while the model finishes thinks your product is broken. Streaming fixes this by sending tokens as they are generated, so words appear in 200ms and the total wait is the same but the perceived wait is half. Pass stream=True and iterate over the response — the SDK handles the server-sent-events plumbing.

stream = client.chat.completions.create(
    model="gpt-5",
    messages=[{"role": "user", "content": "Write a short poem about caching."}],
    stream=True,
)

for chunk in stream:
    delta = chunk.choices[0].delta.content
    if delta:
        print(delta, end="", flush=True)

The catch nobody warns you about: when streaming, you do not get a final usage object by default. Pass stream_options={"include_usage": True} to get token counts in the last chunk, otherwise your cost telemetry will silently drop streaming requests and you will spend a week wondering why your dashboard says nothing happened.

Prompt caching — the 50% discount nobody activates

This is the biggest "free money" feature in the API and most teams ignore it because it is invisible. Any prefix of your messages array that is at least 1,024 tokens long and identical to a previous request within five minutes hits the cache and bills at 50% of the input price. The cache is automatic — you do not call any "cache" API. You simply put your stable content first.

Concretely: if your system prompt is 800 tokens of instructions plus 5,000 tokens of retrieved RAG context plus 200 tokens of few-shot examples, that 6,000-token prefix is identical across requests in a session. Send the user message last, and from the second request onward the 6,000 tokens cost half. For a chatbot with 10,000 sessions a day each making five turns, that is the difference between a $300/day bill and a $170/day bill — same product, same model, same answers, no code change beyond message order.

Realtime API — voice in production

The Realtime API is a WebSocket-based duplex audio channel that takes user speech in and emits model speech out with sub-300ms latency. It runs on a dedicated voice-tuned variant of GPT-5 and replaces the brittle three-stage pipeline (Whisper → GPT → TTS) most teams ran in 2024. You connect, you stream audio frames, you receive audio frames, and the model handles barge-in, interruption, and turn-taking natively.

It is priced higher than text — roughly $32 per million input audio tokens and $64 per million output — but the latency and quality are not comparable to anything you can build yourself. Use it for voice agents, accessibility features, and live translation. Do not use it for non-realtime transcription; Whisper is twenty times cheaper for that.

Common mistakes — what burns every team in production

FAQ

The bottom line

The OpenAI API in 2026 is mature in a way it was not in 2024. Pricing is low enough that hobby projects are essentially free, structured outputs eliminate the JSON-parsing tax, prompt caching makes long-context applications affordable, and the Realtime API closes the voice gap. The remaining differentiator is engineering discipline — the teams shipping faster are not the ones with the cleverest prompts but the ones with retries, token counting, structured outputs, and per-route model selection wired in from day one.

If you are still on stacked Plus subscriptions for things that should be code, or running 2023-era prompt engineering rituals to coerce JSON out of the model, you are leaving real money and real reliability on the table. The week you spend porting to the API pays back forever.