A language model reads neither letters nor whole words: it reads tokens. A token is a fragment of text, often a short word, a piece of a long word, a punctuation mark or a space. It is the unit in which models measure what they read, what they write and what they bill.
What does a token look like?
How text is split depends on the model, but the principle is always the same. Frequent words often form a single token. Rare, long or compound words are cut into several pieces. A word like "assistant" may fit in one or two tokens, while "incomprehensibilities" needs several. Numbers, proper names and e-mail addresses are often split into many fragments.
OpenAI gives as a rule of thumb that one token corresponds on average to about four characters of English text, roughly three quarters of a word. French and Dutch usually need slightly more tokens for the same content, because tokenizers were designed with a lot of English text. The gap is modest for these languages, but it exists.
Why models use tokens
Working letter by letter would make texts very long and computation slow. Working word by word would require knowing every word in every language, including typos and made-up words. Tokens are a compromise: a vocabulary of a few tens of thousands of fragments is enough to represent any text, in any language, by combining the pieces.
The model also produces its answer token by token. At each step, it chooses the most likely next fragment given everything that came before. This mechanism explains both the fluency of the answers and the risk of hallucination: the model extends a plausible text, fragment after fragment.
Tokens and the bill
Model providers bill usage by the number of tokens, usually distinguishing between:
- Input tokens: everything sent to the model, that is the prompt, the history, the document passages and the question
- Output tokens: the answer written by the model, often billed at a higher price per token than input
Prices are usually expressed in dollars per million tokens and vary widely with the power of the model. The same assistant can cost much more or much less to run depending on the chosen model and the amount of text sent with each message.
For an SME, these calculations usually remain invisible. Assistant tools convert them into more readable units, such as conversations, credits or resolutions, or include them in a flat fee. Understanding tokens still helps when reading an offer: a "credit" can be worth a different number of tokens depending on the model used. Our article on the cost of an AI assistant compares these units.
Tokens and the context window
The context window, the amount of text a model can read in one go, is also measured in tokens. A window of 100,000 tokens represents several hundred pages of text. Everything sent to the model, including the answer being written, must fit within that limit.
This is one reason an assistant with a large knowledge base does not send all the documents with every question. It selects a few relevant passages through RAG, which reduces the number of tokens sent, and therefore the cost and the response time. For a modest base, sending the documents in full remains possible and avoids missing a passage.
What drives the token count up
Some habits increase consumption without improving the answers:
- A very long prompt, resent with every message of every conversation
- Too many or too long passages attached to each question
- Endless conversations, whose whole history is reread with every answer
- Wordy answers, when a short, sourced answer is often enough
Conversely, a concise prompt, well-structured documents and answers that get to the point reduce the number of tokens and often improve clarity for the visitor.
Tokens in KernChat
KernChat tracks AI usage for each organization, including the tokens consumed and their estimated cost per model, visible in the workspace. This usage is included in the subscription: the organization buys no credits and does not pay per token. Tracking is there to check that usage stays consistent with activity and to spot, for example, an unusually long conversation.