Context Window

The context window is the maximum number of tokens a language model can consider within one processing step.

Depending on the model and provider, input and potential output may share this budget or have additional separate limits. If a limit is exceeded, the application must trim, divide or reject the request.

Why the context window is limited

Supported context length depends on model architecture, training, runtime environment and the limits of the particular service.

More context generally requires additional compute and memory. The effect on processing effort, latency and cost varies by model and technical implementation.

Supported token counts vary considerably between models and may change with new versions. The documentation for the specific model version in use is therefore authoritative.

Converting tokens to words provides only a rough estimate because language, spelling and tokenisation affect the ratio.

Why bigger is not automatically better

A larger context window allows more documents or a longer history within one request. The available capacity does not establish how reliably the model will use every included piece of information.

Relevant, well-structured selection therefore remains more important than maximising context volume.

What this means for architecture decisions

For agents, document analysis or long conversations, context length is a concrete architecture constraint. More processed context also affects inference costs, latency and potentially output quality.

Define which information each step requires, how older material is condensed and when external knowledge or state storage is more appropriate.