Tool · o200k_base tokenizer
Token counter
Paste a prompt, a document or code to see how many tokens it is and how the tokenizer splits it. The count is exact for GPT-4o, GPT-4.1, the o-series and GPT-5 models, and a close estimate for the rest.
The tokenizer (about 2 MB) loads when you start typing.
- Tokens
- 0
- Characters
- 0
- Words
- 0
- Chars / token
- —
- Words / token
- —
- Input cost
- Calculate →
How the text splits into tokens
What a token is
Language models do not read letters or words. A tokenizer first cuts text into tokens: common words become a single token, rare words are split into pieces, and spaces are usually attached to the start of the next word. The model reads and writes token IDs, and API vendors bill per token, which is why the count decides both what fits in the context window and what a request costs.
This counter uses o200k_base, a byte-pair encoding with about 200,000 entries. It is the encoding OpenAI's open-source tiktoken library assigns to GPT-4o, GPT-4.1, o1, o3, o4-mini and the GPT-5 family. Colour bands above show each token; a · marks a leading space.
Why the same text costs a different number of tokens on different models
Every vendor trains its own tokenizer, so token counts for Claude, Gemini, Grok and DeepSeek differ from this one. Differences are small for plain English prose and larger for code, numbers and other languages. Anthropic says the tokenizer in Claude 4.7 and later produces about 30% more tokens for the same text than its previous one. When exact numbers matter, use the vendor's token-counting API before you commit to a budget, then compare prices on the LLM API pricing page.
Questions
Which models does this token count match?
The count uses o200k_base, the encoding OpenAI's tiktoken library assigns to GPT-4o, GPT-4.1, o1, o3, o4-mini and every model whose name starts with gpt-5, including GPT-5.6 Sol, Terra and Luna. tiktoken does not yet list an encoding for GPT-6 Astra.
Is the count exact for Claude, Gemini, Grok or DeepSeek?
No. Each vendor uses its own tokenizer, so the same text produces a different number of tokens. Treat this count as a ballpark for those models. Anthropic says the tokenizer in Claude 4.7 and later models produces about 30% more tokens than its previous one for the same text. For an exact figure, use the vendor's own token-counting endpoint: Anthropic and Google both offer one in their APIs.
Why does the API report more tokens than this counter?
The counter measures plain text. A chat request also carries message formatting, the system prompt, tool definitions and images, all of which add tokens. Reasoning models also bill hidden thinking tokens as output.
Is my text sent anywhere?
No. The tokenizer and its vocabulary file are downloaded from ai-magazine.com once, and the counting runs in your browser.
How many tokens is a word?
For English, OpenAI's rule of thumb is about 0.75 words per token, or roughly 4 characters per token. Code, numbers, rare words and most non-English languages need more tokens per word. Paste your own text above to see the ratio for it.