LLM Tokenizer Playground

See how language models split your text into tokens — count tokens, inspect token IDs, and estimate API costs

Your text
Tokens (GPT-4/cl100k-style BPE)
The quick brown fox jumps over the lazy dog.🦊 Tokenization splits text into subword units common words are single tokens, while rarer words get split into pieces. Numbers like1234567 and code like console.log("hi") tokenize in interesting ways!
57
tokens
249
characters
40
words
Rough input-cost comparison (per this text, indicative pricing)
Claude Sonnet
$0.000171
$3/1M tokens
Claude Haiku
$0.000057
$1/1M tokens
GPT-4-class
$0.000142
$2.5/1M tokens
Small/open model
$0.000011
$0.2/1M tokens

Prices are ballpark figures for intuition — check each provider's current pricing page before budgeting.

Why tokens matter

Large language models don't read characters or words — they read tokens, subword units produced by byte-pair encoding (BPE). Token count determines API cost, context-window usage, and why models struggle with character-level tasks. This playground uses a GPT-4-style cl100k tokenizer running fully in your browser: hover any chip to see its token ID, and notice how spaces attach to the following word, emoji explode into several tokens, and rare words split into pieces.

Your data stays private. All processing happens locally in your browser. No data is collected, stored, or sent to any server.