What is a context window? How much text can AI use at once?

The model has a limited working page

Imagine answering a question while looking at one page of notes. Anything left off that page cannot help you answer. A language model also receives a limited amount of information for each request.

That limit is called its context window. The amount is counted in tokens, the small text pieces from lesson 5. A limit of 4,000 tokens is not a promise of 4,000 words.

The notes page is an analogy, not a picture of how the model thinks. The real system performs calculations using the text it receives.

Your question is only part of the text

A chat app may send instructions, earlier messages, document passages, and your latest question together. All of those can take up space. The model also needs room to produce a reply.

For an ordinary text chat, think about the request and reply together. Some models have separate reply limits too. Check the exact model and app rather than assuming every product uses the same limit.

Saved history is not the same as active context

Your app can save a conversation on disk. That does not mean it sends every saved message back to the model for every answer.

When a conversation is too long, an app may shorten it, leave out messages, or refuse the request. The exact behaviour depends on the software. Check its documentation and any warning it shows.

If you add a fact to a chat, the model may use that fact in a later reply while it is included in the request. This does not, by itself, change the model’s learned values.

A larger window does not promise perfect recall

In the 2024 study Lost in the Middle, researchers tested questions about long collections of text. For the models and tasks they studied, moving the useful information into the middle could reduce answer accuracy.

That finding does not tell us the accuracy of every current model. It tells us why a published capacity limit should not be treated as a guarantee that every detail will be used correctly.

For a school notice, ask the model to quote the line containing the opening time. Then compare that line with the original notice yourself.

Longer text can need more computer memory

Ollama’s documentation says that increasing the context length increases the memory needed to run a model. More available text space can therefore make a local setup harder to fit.

The size printed on a model page and the amount your app actually uses may differ. Do not turn the setting up to its maximum just because the model advertises a large limit.

Begin with the text you need for the task. Remove repeated notes. For important facts, keep the original document and check the answer against it. This is practical guidance, not a measured memory result for your computer.

Try it: keep the important facts in view

Use this made-up note: “The library opens at 10 am on Tuesday. Returns go in the blue box.” Ask: “When does the library open? Quote the line you used.”

Now add a few harmless paragraphs around the note and try again. Compare the answer with the note. A correct answer on this small exercise is not proof that the model can handle a whole book.

If the app shows a context setting or usage meter, record it. Otherwise, do not guess the number. The purpose is to practise checking the source, not to measure a model’s maximum capacity.