FFTNet Generate

What it is and why you'd bother

FFTNet Generate is the second half of bemoregt/ComfyUI_FFTNet: feed it a loaded FFTNet model and a prompt, and it writes the continuation. It's a language model node living inside ComfyUI - a tool that's otherwise all images and video - and that's both its charm and its limit.

You'd reach for it to make a workflow generate its own text: prompts, captions, titles, filler description. Wire its output into a text node and you've got a pipeline that writes its own input. What it is not is a chat assistant. No conversation, no streaming, no system-prompt roleplaying beyond a single prefix - just "here's some text, finish it." If that's all you wanted, a real LLM setup will serve you better. This node is for the niche case of running text generation inside the graph, on the FFTNet architecture from the "The FFT Strikes Back" paper - where self-attention's O(n²) is replaced with O(n log n) FFT-based token mixing plus a local convolution branch.

How it works

The node tokenizes your input, then runs a hand-rolled autoregressive loop in fftnet_arch.py: for each of up to max_new_tokens steps, it feeds the last max_seq_len tokens through the model, applies repetition penalty, scales by temperature, truncates with top-k and top-p, samples one token via multinomial, and stops early if it hits the EOS token. Then it decodes only the newly generated tokens and hands you the string.

One implementation detail worth knowing: generation runs inside a fresh thread because ComfyUI wraps node execution in torch.inference_mode(), which breaks seeding and sampling. The node dodges it, but it means generation blocks your queue for as long as it runs - this is a "click and wait" node, not a streaming one.

Inputs and output

Most of the controls are the sampler knobs you've seen in every LLM UI:

• prompt - multiline; the text the model continues from.
• max_new_tokens (200) - how long the continuation gets. Bump it for longer output; the model's context window (max_seq_len, 512 by default) caps how much past text it can see, not how much it writes.
• temperature (0.8) - lower is more deterministic, higher is loopier.
• top_k (50) - sample from only the top 50 logits. 0 disables it.
• top_p (0.9) - nucleus sampling; keep the smallest set of tokens whose probability sums to 90%.
• repetition_penalty (1.1) - penalizes tokens already in the output. 1.0 turns it off.
• seed (-1) - -1 is random; any non-negative number seeds torch.manual_seed for reproducibility.
• system_prompt (optional) - prepended to your prompt before tokenizing, a crude version of a system message.

Output is one generated_text string, and since the node is marked as an output node it shows the text right on the canvas. From there wire it into any string input - Show Text, Save Text, or onward into prompt-scripting logic.

Installing and the traps

Same pack as the loader, so installation is shared: ComfyUI Manager → search ComfyUI_FFTNet, or git clone https://github.com/bemoregt/ComfyUI_FFTNet into custom_nodes/ and restart. Make sure tiktoken is installed - it's the sensible GPT-2 tokenizer default. Without it and without transformers, the pack falls back to a character-level tokenizer that produces near-gibberish and prints a warning to the console. That warning is your first stop when output looks broken.

The bigger reality check: output quality depends entirely on the model the loader gave you. A small or untrained FFTNet checkpoint happily generates plausible-looking nonsense, and a mismatch between the loader's hyperparameters and the actual weights gives you confident garbage. Also remember the context cap - long prompts get truncated at max_seq_len, and only the newly generated text comes out, not your prompt echoed back. Seed reproducibility is also "same workflow, same order of operations," not a hard guarantee. So: tiktoken installed, checkpoint trusted, sampler defaults are fine to keep. That's the whole recipe.