Better prompt caching for GPT-6

摘要: OpenAI

September 22, 2026

Better prompt caching for GPT‑6

Higher cache hit rates and new tools to help persistent agents run faster and co

正文

September 22, 2026

Better prompt caching for GPT‑6

Higher cache hit rates and new tools to help persistent agents run faster and cost less.

Loading…

GPT‑6 enables persistent agents to work for hours on complex tasks, from refactoring codebases to producing well-researched documents and presentations. The applications behind these agents make a series of API requests that build on one another, often carrying forward the same instructions, tool definitions, and context from earlier turns. OpenAI caches that shared context to reuse computation across requests, reducing response times and giving developers discounts of up to 90% on cached input tokens.

With the GPT‑6 family, we launched an improved prompt caching system that delivers higher cache hit rates by default. We now give cache discounts for eligible shared prefixes reused within a 30-minute window. We’re also introducing new tools to help developers monitor cache performance, diagnose misses, and choose how much of a prompt to cache.

OpenAI’s prompt caching plays a critical role in helping GitHub Copilot deliver fast, efficient experiences at scale. Over the past several months, we’ve reduced by more than 50% the share of prompt tokens requiring fresh processing across billions of requests to OpenAI models, relative to our previous baseline. The result is a more efficient inference stack and faster time to first response for developers.

—Mario Rodriguez, Chief Product Officer

Monitor caching and diagnose cache misses

Prompt Caching Dashboard

(opens in a new window)

shows how much of your application’s input is served from cache. Track hit rates over time and use the input composition chart to compare cached and uncached tokens. These views help you spot drops in cache hits and evaluate how changes to your application impact caching performance.

When you see an unexpected cache miss, use the

prompt caching diagnostics tool

(opens in a new window)

来源: OpenAI Blog