Token Peek
A tool that lets you paste in text and see, color-coded, how ChatGPT-family AI divides that text up as it reads.
ToolFreeNo sign-upWorks on PC and phone
The interface is in Japanese.
Released:
The interface is in Japanese.
What it can do
- It color-codes the text you paste in by the unit that AI reads, called a "token".
- You can check the token count and the character count of the whole text.
- You can compare the token counts of Japanese and English text that means the same thing.
- You can read some pieces of trivia about tokens.
How the text you enter is handled
The text you enter is processed inside your browser, and there is no mechanism that sends it to a server for this tool.
This introduction page uses access analytics to see how it is being viewed. For details, please see the privacy policy.
Honest notes
- The color-coding is accurate for the method used by OpenAI's ChatGPT family.
- Claude and Gemini are not covered by this tool, because their rules for dividing up text are not published.
- You can enter up to 5,000 characters.
Update history
- Released on urara.dev.
Related posts (in Japanese)
The person who made this
A personal-project tool made by 麗 (urara), built while talking it over with AI.