# Tokensift

> Token-efficiency linter for LLM prompts and payloads - ritenv/tokensift

- **Website:** https://github.com/ritenv/tokensift
- **Pricing:** paid
- **Categories:** Developer Tools, Design
- **Tags:** developer-tools, design, writing-content
- **Platforms:** CLI
- **Last verified:** 2026-09-10
- **Canonical page:** https://linkrena.com/tools/tokensift

## About

Deterministic, local, tokenizer-level static analysis of prompt strings, Message[] arrays, and tool schemas.

Status : early, actively developed. Core engine, 20 rules, real dollar cost per finding, a CLI, and tokensift/matchers for vitest/jest all work today. OpenAI models are exact; Claude support is estimate-based, see Cost and pricing below. See DESIGN.md for tradeoffs made along the way.

LLM APIs charge per token, and token counts don't line up with characters or words as cleanly as you'd expect. A UUID, a base64-encoded file, an indented JSON blob: all of these cost more tokens than their length suggests, because the tokenizer can't find any reusable pattern in them. tokensift reads a prompt (or a whole message array, or a tool schema) and points out exactly where that's happening: this UUID cost 18 tokens and a short id would've cost 3, this block of instructions got pasted twice, this JSON would tokenize the same minified.

It does this by actually tokenizing the text with the encoder the provider uses, not by estimating from character count. For OpenAI models that means the real BPE vocabulary, so counts are exact.

## Related tools

- [Orion](https://linkrena.com/tools/orion): A backend and automation language compiled to bytecode in Rust.
- [Free QR Codes](https://linkrena.com/tools/free-qr-codes): Create a free static QR code with no signup, watermark or expiry.
- [ArcadeMaker](https://linkrena.com/tools/arcademaker): A cross-platform 2D game engine with its own programming language and IDE.
- [Makie](https://linkrena.com/tools/makie): Makie is a child-safe AI game builder where kids describe, build, remix, and share playable games.
- [ravynOS](https://linkrena.com/tools/ravynos): An open-source operating system based on Darwin and FreeBSD that aims to be compatible with macOS applications.
- [Litelink](https://linkrena.com/tools/litelink): Durable append-only capture into Iceberg tables; embedded and local-first.