PreFlight Calibrate

Every calibration-flavoured tool I've used eventually turns out to be a dashboard nobody opens. PreFlight Calibrate avoids that by being almost nothing: no inputs, one string output, and two tables of text you read next to your workflow. Connect it to Show Text and it's on screen.

It reads the feedback store - the predictions PreFlight Report logged and the outcomes you recorded with PreFlight Outcome - and changes nothing. No machine learning, no auto-tuning, no weights. It's a statistics readout for a human who is about to edit a Python file.

Table one: per platform

For each of Instagram, TikTok and X, the store is split three ways:

• How often the real outcome fell inside the predicted best → worst range. Those are consistent predictions.
• How often it came in above the range - the platform was harsher than you predicted. This is the dangerous kind. Under-prediction means content you shipped that got removed or crushed because the engine was too optimistic.
• How often it came in below the range - over-prediction. You were scared of content that was actually fine, which in practice means reach you lost to your own caution.

UNKNOWN verdicts are skipped rather than counted as hits or misses, which is the honest treatment.

High over-prediction is not harmless. It's the feedback that stops the rules from becoming a machine for talking you out of perfectly publishable work.

Table two: per rule

This is the table that earns the node its place. For every rule ID that fired, you get how many posts it fired on and the clean / demoted / removed counts of their outcomes, per platform.

So the reading is direct: base.bikini fires 30 times, 28 of those come back clean on Instagram, and that rule is too harsh there. Another rule fires 12 times with 5 demotions and it's earning its keep. You're not tuning a threshold in the dark any more - you're looking at which specific hypothesis about platform behaviour is failing and on which platform.

What "runs it" means

The node re-computes only when the store has changed - a prediction or an outcome logged since its last run. Otherwise the queue returns the cached tables. That's the right call. You can leave it wired into a workflow permanently and it won't grind through the log file on every queue, but the moment you log an outcome it refreshes.

Install

cd ComfyUI/custom_nodes
git clone https://github.com/0xBeycan/ComfyUI-BCNodes
pip install -r ComfyUI-BCNodes/requirements.txt

Restart, search preflight. This node needs no model, no downloads and no third-party Python - it reads a JSONL file and produces text. Its one output is report (STRING), which exists to be shown: wire it to the pack's Show Text node, or to anything else that accepts a string.

Getting burned

It reads, it never writes. Rule changes are made by hand, in pipelines/preflight/rules.py, and then you bump ENGINE_VERSION in that file. Which is why every stored prediction carries the engine version that produced it: after an edit, the calibration tables can tell old predictions from new ones and you don't poison the statistics you're about to read. ENGINE_VERSION is currently 1.4.0 - if your numbers look mixed, check whether you bumped it.

You have to fill the store first. Empty store, empty tables. Calibrate is downstream of the whole loop: prediction logged, outcome entered, and then it has something to say. Ten posts is not enough to conclude anything; a hundred across a few platforms starts to show shape.

Parameters live on the wrong side of the fence, and it knows it. The distinction the pack makes is worth repeating: the observation and the rules are separate so a bad verdict is traceable to perception (the model misread the garment) or policy (the rule was too strict). Calibrate's per-rule table is the policy half. The perception half shows up as rules firing on observations that plainly describe the wrong thing - which is why raw_response on PreFlight Observe is worth keeping when something looks off.

And nothing here tunes itself. That's a limit, not a missing feature. Editing a rule with 30 data points behind it and re-reading the table next month is a better loop than an auto-tuner quietly drifting your thresholds while you sleep.