js-tiktoken
JavaScript implementation of OpenAI's tiktoken for counting and encoding tokens.
How js-tiktoken is used
js-tiktoken is a dependency in 5 of the 6,208 public GitHub projects set up for AI coding agents that we scanned, too few to describe it from them alone. Our scan also samples projects by dependency (Solana 2,661, Ethereum and EVM 1,496, 3D and games 2,818, agent frameworks and SDKs 10,983); counting those, it appears in 59 projects, and what it is used with below comes from all of them. The most used alternative in our scan is GPT Tokenizer (14 projects). Projects that use it most often also use Zod (76.3%), AI SDK (47.5%), Anthropic SDK (37.3%).
js-tiktoken popularity over time
Gid Score 51: downloads 80, use in agent projects 43, GitHub stars 13, downloads trend 63 (each 0-100). How it is calculated.
Often used with js-tiktoken
| Zod | 76.3% of js-tiktoken projects · 3.0× more than average |
|---|---|
| AI SDK | 47.5% of js-tiktoken projects · 10.8× more than average |
| Anthropic SDK | 37.3% of js-tiktoken projects · 10.0× more than average |
| react-markdown | 37.3% of js-tiktoken projects · 9.5× more than average |
| TypeScript | 89.8% of js-tiktoken projects · 1.6× more than average |
| OpenAI | 35.6% of js-tiktoken projects · 7.8× more than average |
| remark-gfm | 25.4% of js-tiktoken projects · 7.0× more than average |
| OpenTelemetry | 23.7% of js-tiktoken projects · 16.9× more than average |
From all 59 projects in our scan that use js-tiktoken, including the dependency samples.
Install
| Package | npm install js-tiktoken |
|---|---|
| Main stacks | TypeScript / JavaScript (43), Next.js (14), Python (2) |
| Example projects | EKKOLearnAI/ekko-studio, fdarkaou/open-deep-research, taracodlabs/aiden, ryoiki-tokuiten/Deepthink, supermemoryai/memorybench |
Alternatives to js-tiktoken
Tools that do the same job, so a project picks one or the other. Add-ons and libraries used alongside js-tiktoken are not listed here; they are in "Often used with".
| Tool | Gid Score | Used in agent projects | Downloads / wk | GitHub stars | License |
|---|---|---|---|---|---|
| GPT Tokenizer npm JavaScript BPE tokenizer with pricing and token counts for OpenAI models. |
44 | 14 (0.3%) | 1.2M | 855 | MIT |
| tiktoken (JavaScript) npm JavaScript/WASM bindings for OpenAI's tokenizer for token counting. |
43 | 10 (0.2%) | 806K | 1,073 | MIT |
| tokenx npm · new in the catalog Fast token count estimates for LLM prompts in a 2 kB bundle. |
37 | 2 (0.0%) | 1.5M | 185 | MIT |
More in Prompts, structured output and evals: Ultracite tiktoken (Python) tokenizers LangSmith Langfuse (Python) TokenLens @graphql-codegen/typescript-operations partial-json