[English](README.md) | [日本語](README.ja.md) # ComfyUI-Info-Prompt-Toolkit This extension node collection is built around simplifying ComfyUI wiring and improving reusability, so trial results are easier to carry into the next production pass. Key features include Civitai-compatible image metadata saving, same-name `.txt` caption saving, XY Plot, Tiled Sampling (`SDXL (with ControlNet Tile)` and `Anima`), SAM3, Detailer, PixAI Tagger, OppaiOracle Tagger, wildcards, and Dynamic Prompts to strengthen your workflow. ## Installation ```bash cd ComfyUI/custom_nodes git clone https://github.com/kinorax/comfyui-info-prompt-toolkit.git cd comfyui-info-prompt-toolkit pip install -r requirements.txt ``` Installing via ComfyUI Manager is the easiest option. > Note: `SAM3 Prompt To Mask`, `PixAI Tagger`, and `OppaiOracle Tagger` require manual placement of model files under `ComfyUI/models` after installation. See the relevant sections in `Core Nodes` for details. ## Support Ongoing Development A paid note article is available for users who want polished results with less trial and error. It introduces production-oriented SDXL and Anima workflows built with these custom nodes. Purchasing the article helps support ongoing development. Article: [ComfyUI SDXL & Anima Workflow: Automatic Detailer, XY Plot, Caption File Generation, and More](https://note.com/vast_zinnia5253/n/n95a43362d467?hl=en) The workflow files use English labels throughout. ## Core Nodes ### SamplerCustom (Sampler Params, Tiled) / SamplerCustom (Sampler Params)

Accepts sampler_params and performs sampling using the official KSamplerSelect / BasicScheduler / SamplerCustom nodes (no custom sampling logic is implemented).
The Tiled version uses the same sampling settings as the standard version and runs inference by splitting into spatial tiles.
The Tiled version is useful not only under VRAM constraints, but also when you need output resolutions beyond a model's practical supported size.
Models with verified operation for the Tiled version are currently SDXL (with ControlNet Tile) and Anima. Other models may still work, but compatibility is not guaranteed.

### Sampler Params

Aggregates sampler/scheduler/steps/denoise/seed/cfg into a single sampler_params bundle.
Improves both wiring simplicity and configuration reuse.

### XY Plot Start / XY Plot Modifier / XY Plot End

XY Plot Modifier chains axis-specific change sets (label + changes) as an array and defines X/Y condition variations.
XY Plot Start applies X/Y modifiers to the base image_info, generates per-cell image_info, and outputs in list (all at once) or loop (iterative) mode.
XY Plot End receives xy_plot_info and generated images, then composes the axis-labeled grid image xy_plot_image.
In loop mode, Start and End cooperate via loop_control to process all cells sequentially.

### Image Info Context

Image Info Context reconstructs updated image_info by reflecting only the input fields that are actually connected.
When positive is connected, it extracts <lora:...:...> tags and applies the tag-removed positive text and extracted values to lora_stack.
extras is merged into existing image_info.extras, and duplicate keys are overwritten by input-side values.
In addition to updated fields, it outputs base_sampler_params (with fixed denoise=1.0) for easy connection to downstream sampling nodes.

### Image Info Fallback

Image Info Fallback fills missing fields in the primary image_info from image_info_fallback.
Fallback is limited to missing fields, and fields that already have values in the primary side are not overwritten.
extras is merged by adding only missing keys, while existing keys are preserved.
lora_stack is supplemented when it is missing or None; an explicit empty list means no LoRA and is preserved.

### Detailer Start / Detailer End

Detailer Start detects a bounding box (bbox) from regions where the input mask value is greater than 0.0, then creates an expanded processing region using crop_margin_scale.
The processing region is pre-upscaled by upscale_factor, then rounded up to a size divisible by mini_unit (8/16/32/64) and arranged with center padding.
For compositing back into the original image, the graded values of the input mask are preserved as a soft mask, and that information is stored in detailer_control.
Detailer End receives inpainted_image and composites it back into the original image using the preserved soft mask.
If inpainted_image resolution differs from the expected canvas size, it is automatically resized before restoration.
If no target region is detected, Detailer End does not evaluate inpainted_image and returns the original image unchanged.

### SAM3 Prompt To Mask

SAM3 Prompt To Mask is a single-purpose node that generates a soft mask from text prompts. Comma-separated prompt tokens are treated as OR conditions, and accepted results can be controlled with score_threshold and combine_mode (union / top1).
The output is a non-binarized soft mask; thresholding and region cleanup are intended to be handled by downstream nodes.
This node is designed with a strong focus on keeping additional pip-installed dependencies to a minimum, because some existing SAM3-related extensions can significantly alter Python environments.
Under that policy, SAM3-related functionality is intentionally limited to this single purpose, and no further feature additions are planned at this time.
Place sam3.pt in ComfyUI/models/sam3 (download: facebook/sam3; operation with SAM3.1 and later is unverified).

### PixAI Tagger

PixAI Tagger estimates tags from images and outputs general / character / ip.
Tag selection uses threshold_general / threshold_character and max_tags_general / max_tags_character.
sort_order controls the final display order for general tags only (score / tag_id); character and ip are always output in score order.
This node runs on a PyTorch implementation and does not depend on onnxruntime / onnxruntime-gpu, making the setup less susceptible to Python environment changes.
Place the PixAI Tagger v0.9 bundle (model_v0.9.pth / tags_v0.9_13k.json / char_ip_map.json) in ComfyUI/models/pixai_tagger or ComfyUI/models/pixai_tagger/v0.9 (download: pixai-labs/pixai-tagger-v0.9).
The previous released node remains available as PixAI Tagger (Deprecated) for existing workflows.

### OppaiOracle Tagger

OppaiOracle Tagger estimates tags from images and outputs tags / rating.
Tag selection uses threshold and max_tags; sort_order controls only the final display order (score / tag_id).
This node runs on a PyTorch implementation and does not depend on onnxruntime / onnxruntime-gpu, making the setup less susceptible to Python environment changes.
Place the OppaiOracle bundle (model.safetensors / config.json / preprocessing.json / vocabulary.json) in ComfyUI/models/oppai_oracle/V1.1 or ComfyUI/models/oppai_oracle/V1.0 (download: Grio43/OppaiOracle).

### Flatten Prompt for Caption / Merge Caption Tokens / Remove Caption Tokens

Flatten Prompt for Caption prepares prompt text for caption files by removing prompt emphasis syntax and weight values, then unescaping escaped parentheses and backslashes.
Merge Caption Tokens merges start and end caption token strings, removes duplicate tokens, and rebuilds the result as a single-line caption.
Caption tokens are split on , and . . Existing order is preserved, and the first occurrence of each token is kept.
Remove Caption Tokens removes exact token matches from string using the comma-separated remove input, while preserving the remaining token order.
These nodes are useful when combining tagger output, handwritten captions, and prompt text before writing caption files.

### Prompt Template

Prompt Template generates final prompts from template text. It handles // / /*...*/ comment removal, wildcard expansion, Dynamic Prompts expansion, and $key replacement in one node.
Wildcards are loaded from ComfyUI/user/info_prompt_toolkit/wildcards .txt files and support __name__ (random) and __name__#N (fixed selection), including weighted lines in weight::value format.
Dynamic Prompts supports {a|b} plus weighted options, multi-select, range selection, and the @ cycle sampler.
When seed is connected, random expansion becomes reproducible; without it, results are non-deterministic.
When extras is connected, $key placeholders are replaced while missing keys are left as-is. suffix is appended after template expansion.
Outputs include raw string and token-normalized normalized_string.

### Image Reader / Image Saver

Image Reader and Image Saver are core nodes that support reusable workflows for images that retain generation data (A1111 infotext / image_info).
Image Saver can save WebP or PNG images and embeds A1111 infotext into matching image metadata so prompt, model, and multiple LoRA references remain reusable.
Check Referenced Models... in the Image Reader right-click menu parses referenced entries from image infotext and lists Model / Refiner / Detailer / CLIP / VAE / LoRA items.
The same window also shows local availability for each reference (Present / Missing), and can jump to View Model Info... for deeper inspection.
Image Saver supports both automatic serial naming and explicit file_stem naming, so you can switch naming strategy by workflow.
When write_caption is enabled, a same-name .txt caption file is written alongside each image.

### Referenced Image Saver

Referenced Image Saver copies an existing file supplied through image_reference into the ComfyUI output directory and rewrites its A1111 infotext metadata from image_info without re-encoding the image pixels.
It supports PNG / WebP / JPEG metadata updates and preserves the source extension, making it useful for saving lossy WebP or JPEG images again without further quality loss.
When file_stem is unconnected, it uses the same automatic serial naming as Image Saver. When connected, it saves as <file_stem><source extension> and overwrites an existing file with the same name.
output_subdir can organize files by date, and enabling write_caption writes a same-name .txt caption alongside each image.

### Image Directory Reader

Image Directory Reader loads multiple images from subdirectories under ComfyUI input / output and returns corresponding data as aligned list outputs.
You can choose the source with path_source and path (relative directory), then control the selection range with start_index and image_load_limit (0 = unlimited).
Target images are selected after case-insensitive filename sorting, making batch selection results easier to reproduce.
For each image, it outputs image, image_info reconstructed and normalized from A1111 infotext, extensionless filename file_stem, and same-name .txt caption (empty string when missing).
It also outputs path_source and path, so it can be connected directly to Caption File Saver.

### Video Reader / Video Saver

Video Reader and Video Saver are a paired set of nodes that support video reuse workflows while retaining generation metadata (source_image_info / video_info).
Video Reader outputs image as a list. If downstream nodes expect batch-oriented image input or do not assume list expansion, route through Image List To Batch.
In Video Saver, the active crf field controls the quality/file-size tradeoff; lower values generally produce higher quality and larger files.
source_image_info keeps source-image information, while video_info keeps video-related information for easier review and comparison later.
Video Reader / Video Saver are available in environments where PyAV (av) is installed. Available codecs depend on the user's PyAV / FFmpeg build.

### Mask Overlay Comparer

Mask Overlay Comparer displays a slider-based comparison between the original image and a mask-overlayed view, making mask coverage easier to inspect visually.
The comparer switches between two views: an easier-to-read background image and the image with mask overlay applied.
The mask is visualized without binarization, using raw values in the 0.0..1.0 range.
Visualization behavior is: mask = 0.0 keeps the image unchanged, mask = 1.0 renders black, and intermediate values are shown as a blue-tinted overlay.
For preview generation, use image and mask at the same resolution.

### Aspect Ratio to Size

Aspect Ratio to Size recalculates width / height from node-UI widget edits before execution, so you can adjust values while checking actual_ratio.
It resolves size from width_ratio : height_ratio and a base size, using either width or height as the anchor and deriving the other side from the ratio.
The resolved size is aligned to min_unit (8 / 16 / 32 / 64), which helps keep resolutions on practical step sizes.
width x height is a combined size output that carries width and height on a single line, making it easy to pass into nodes that accept size input.

### Aspect Ratio to Size (Trim Margin) / Trim Image by Margin

Aspect Ratio to Size (Trim Margin) and Trim Image by Margin

Aspect Ratio to Size (Trim Margin) calculates a sampling size that extends beyond the desired image edges so that Trim Image by Margin can remove the margins afterward. The horizontal and vertical margin pairs stay linked, with each pair totaling either zero or min_unit. Its size outputs include all four margins and remain divisible by min_unit.
Edit actual_width / actual_height as the size remaining after trimming. The readonly width / height fields show the sampling size including margins, while actual_ratio describes the trimmed size.
The margin output bundles top, right, bottom, and left for Trim Image by Margin. When margin is unconnected, the trim node passes the image through unchanged.
Set Margin Extra / Get Margin Extra store margins in image-info extras and restore both the bundled margin and its four edge values.

### Load New Model / Use Loaded Model

Load New Model and Use Loaded Model are a paired workflow that separates model loading from model reuse.
Load New Model receives the selected model from selector nodes such as Checkpoint Selector or Diffusion Model Selector. Checkpoint selections can load model, clip, and vae from the checkpoint; diffusion model selections load the model and use the optional clip, vae, and vae_fallback inputs for the accompanying CLIP/VAE runtime. It uses a temporary strong cache only within the same prompt execution to avoid duplicate raw loads.
Use Loaded Model keeps the final runtime bundle in a process-local cache keyed by the selected model, runtime settings, lora_stack, and connected clip / vae conditions. On cache hit, the lazy loaded_model / loaded_clip / loaded_vae branch is skipped.
Use Loaded Model applies lora_stack internally by default. Set apply_lora_stack to false if loaded_model and loaded_clip already include the same LoRA application.
If you want to pass through Lora Stack Lorader or TorchCompile-related nodes before Use Loaded Model, place them after Load New Model, then connect their outputs to loaded_model, loaded_clip, and loaded_vae with apply_lora_stack=false.
Cache behavior is configurable in ComfyUI Settings > Info-Prompt-Toolkit > Use Loaded Model Cache. Auto always keeps the latest entry and prunes older entries by estimated memory size and Memory budget ratio; Fixed keeps up to Max cache entries regardless of the memory budget. Max cache entries defaults to 1 and can be set from 1 to 9.
Use cache logging prints cache hit / miss / store / evict / clear messages to the ComfyUI Python console, including estimated memory amounts for store and evict events.
lora_stack matching is order- and strength-sensitive, so changing order or values is treated as a different condition set.

### Model Merge

Model Merge records two IPT-Model selections and their merge ratios in a new IPT-Model value. It does not load or merge model weights immediately; the actual model runtimes are loaded and merged when the result is passed to Load New Model.
Connect the result to the corresponding model, refiner, or detailer input of Image Info Context or another image-info node to retain the complete merge configuration in image_info, including both source model references, their model types, and the model / CLIP ratios.
When that image_info is saved by Image Saver or another compatible saver, the configuration is serialized as JSON in the A1111 infotext metadata. Depending on its image_info field, it is written as Model Merge, Refiner Merge, or Detailer Merge instead of being flattened into an ordinary single-model entry.
When the saved image is loaded by Image Reader, the merge metadata is parsed back into the corresponding model, refiner, or detailer value in image_info. Image Info Context can then output the restored IPT-Model value for reuse with Load New Model.
The metadata stores the merge configuration, not the merged model weights. The referenced source model files must therefore be available locally when the configuration is loaded and executed again.
base_model and merge_model must both be checkpoints or both be diffusion models; mixing the two model types is not supported.
model_ratio controls the model merge from the base model at 0.0 to the second model at 1.0. For checkpoints, clip_ratio independently controls the CLIP merge over the same range.
Checkpoint merges retain the VAE from base_model. For diffusion models, clip_ratio is ignored and CLIP / VAE can be supplied to Load New Model as needed.

### Release Memory

Release Memory releases selected persistent runtime references and requests Python / CUDA cleanup after the step connected to after has completed.
The after input and then output are pass-through sockets, so the node can be placed near the end of a workflow without changing the data being passed onward.
Generation Runtime unloads ComfyUI generation models where possible and clears this extension's Use Loaded Model cache and recently loaded VAE runtime.
SAM3 Runtime clears the SAM3 Prompt To Mask processor cache, while Tagger Runtime clears PixAI Tagger and OppaiOracle Tagger torch model caches.
GC & CUDA Cleanup runs garbage collection and asks the available CUDA cleanup hooks to release allocator-held memory where possible.
The Release Now button performs the same cleanup manually when no prompt is currently running.

### Selector Nodes

Selector nodes centralize choices for reusable workflow inputs such as checkpoints, diffusion models, LoRAs, CLIP models, VAEs, samplers, and schedulers.
Model-related selectors include Checkpoint Selector, Diffusion Model Selector, Unet Model Selector, Lora Selector, Vae Selector, CLIP Selector, Dual CLIP Selector, Triple CLIP Selector, and Quadruple CLIP Selector.
Supported model-related selectors provide a right-click View Model Info... action that uses the selected asset's SHA256 as a key to fetch metadata from Civitai and display it in a model information window.
For Lora Selector, View Model Info... can also show a LoRA tag list with tag frequencies when metadata has been analyzed for the selected LoRA.
Common connection targets include Image Info Defaults, Image Info Context, Load New Model, Use Loaded Model, and XY Plot Modifier.

## Workflow Examples ### Save Images With Metadata Using `Image Saver`

Workflow sample: save images with metadata using Image Saver

Use `Image Saver` at the end of a generation workflow to save images with reusable metadata. The embedded `A1111 infotext` / `image_info` keeps prompt, model, LoRA, sampler, and related settings available for later reuse. ### Generate From Metadata Loaded by `Image Reader`

Workflow sample: generate from metadata loaded by Image Reader

Load a metadata-rich image with `Image Reader`, pass the restored `image_info` into the workflow, and generate from the saved settings as a starting point for reproduction or further iteration. ## License - The original source code of this project is offered under [GPL-3.0-or-later](LICENSE). - This repository also includes vendored `sam3`, which is distributed under the [SAM License](vendor/LICENSE.sam3). - See [THIRD_PARTY_LICENSES.md](THIRD_PARTY_LICENSES.md) for third-party license details, including notes on separately obtained optional models. - Model files and other external assets obtained separately by users remain subject to their respective upstream licenses and terms.

Flatten Prompt for Caption

Inputs (1)

Outputs (1)