See how a prompt splits into tokens.

A token is the unit a language model counts when it reads or writes text. It is not always a word. For GPT-family models the common rule of thumb is that one token is about four characters of English text, so the word “tokenization” becomes three tokens and the word “cat” becomes one. Tokens are produced by an algorithm called BPE (byte-pair encoding) or a relative like SentencePiece, which learns which character sequences to merge into single units from a large training corpus.

Every model family trains its own tokenizer. The set of merges is different, so the same text produces different token counts. OpenAI’s published rule of thumb for English is about four characters per token for GPT-4 and GPT-4o. Anthropic’s Claude tokenizer runs a little tighter at roughly three and a half characters per token. Llama 3 and Gemini land near four. Non-English text, code, numbers, and emoji push every ratio up.

No. This page is a teaching aid, not a reference implementation. The heuristic mode splits long word runs into four-character chunks and keeps whitespace and punctuation as single tokens. That approximates the order of magnitude of GPT-family counts for English prose, but it will disagree with tiktoken and Claude’s real tokenizer on specific cases, especially rare words, numbers, URLs, and non-ASCII characters. For exact counts, call your model’s official counter: OpenAI’s tiktoken library or Anthropic’s count-tokens API.

Three modes make the concept visible. Word-boundary mode shows what older whitespace-based tokenizers did and gives you the lowest token count. Heuristic BPE mode chunks long words to approximate modern subword tokenizers and lands roughly where GPT-4 would for English. Raw byte mode shows how many UTF-8 bytes the prompt is — the lower bound if a model ran at the byte level. Comparing the three numbers for the same prompt is the whole lesson.

Because this is an approximation. Real BPE tokenizers have vocabularies of 50,000 to 200,000 merges learned from training data, and they produce different counts for the same text. For a precise number, paste into OpenAI’s tokenizer tool or call Anthropic’s messages/count-tokens endpoint. Use this page to build intuition, not to estimate billing.

Because a prompt can contain any character — newlines, quotes, emoji, unicode. Base64 (URL-safe variant) encodes any UTF-8 bytes as ASCII letters, digits, minus, and underscore, which every URL parser and link handler treats the same. The trade is size: base64 makes the URL about one-third larger than the source prompt, so this page caps share URLs at a length most browsers and servers handle without truncation. The URL shape is ?q=<base64url>.

The input textarea itself has no hard limit, but the share URL does. Long URLs get truncated by mail clients, chat apps, and some servers. Vercel edge routes accept roughly 14 KB of URL. Safe browser-side limits are nearer 2,000 characters for reliable copy-paste. When a prompt produces a share URL longer than about 1,800 characters the page warns you. Shorter prompts round-trip cleanly.

Almost nothing. Tokenization, byte counting, base64 encoding, and copy-to-clipboard all run client-side. The only outbound requests are to fonts.googleapis.com and fonts.gstatic.com for the font files, which leak your IP, User-Agent, and referrer to Google. Block those domains in your browser if that matters and the page falls back to system fonts. The prompt text you paste never leaves the tab unless you copy the share URL yourself.