llama.cpp Token Count

Token counting sounds like a developer utility, and it is - but it's also the difference between "why is my output truncated at 900 tokens" and actually knowing why. This node takes a string, tokenizes it with your running llama-server's model, and returns the token count plus the token details. It's the measuring tape for every max_tokens decision you make elsewhere in the pack.

LLMs charge by the token, not by the character, and different models tokenize the same text differently. A count from this node is the ground truth for your model, which is exactly the number you need when you're budgeting context or deciding whether to raise max_tokens.

How it works

The node calls llama-server's /tokenize endpoint through the pack's client, with model-aware routing - it uses the running direct model, or the exact router model ID you give it, because tokenization is model-specific. Text in, token IDs (and optionally token pieces) out.

Two toggles control how the tokenizer treats the input:

• add_special - whether to add the model's special boundary tokens (BOS/EOS and friends) around your text. Off by default. If you want to know how many tokens a real request will consume - which includes those boundary tokens - this should be on.
• parse_special - whether text like special-token names in the input is recognized as the tokens they name. On by default.

with_pieces asks the server to include the actual token pieces (the subword strings) in the response, which turns this from a counter into a small tokenizer inspector - useful when you're trying to figure out why a rare word eats six tokens.

Inputs and outputs that matter

Inputs: text (required, multiline), server_url / model (routing), add_special, parse_special, with_pieces, plus the usual connection / api_key_env / verify_tls / request_timeout plumbing.

Outputs: token_count (an INT) and tokens_json (the token IDs, or pieces when requested, as formatted JSON).

Wiring it

The practical workflow: feed the text you're about to send to a prompt node into Token Count, and use the token_count INT to sanity-check your max_tokens and context_size. Since it returns an INT, you can even gate or log it - or just keep it in the graph as a diagnostic that updates on every queue. The subtle gotcha is add_special: counting your prompt with it off undercounts what the actual generation request will consume, so for budgeting, flip it on.