FL PixelGen

Pixel-space diffusion text-to-image generation and LoRA training nodes for ComfyUI, powered by PixelGen. Generate high-quality images directly in pixel space using a JiT transformer denoiser with Qwen3 text conditioning and flow matching sampling — entirely within ComfyUI's node graph.

Features

• Pixel-Space Diffusion — Generates images directly in pixel space, no VAE decoding needed
• Live Sampling Preview — See the image form step-by-step during generation
• REPA LoRA Training — 4-component loss (flow matching + cosine alignment + LPIPS + DINO) with noise gating
• Real-Time Training UI — Live loss chart, progress bar, validation samples, and lightbox preview via WebSocket widget
• Qwen3 Text Encoder — Uses Qwen3-1.7B for text conditioning, auto-downloaded from HuggingFace
• Auto Model Download — Denoiser and text encoder models download automatically from HuggingFace on first use

Nodes

| Node | Category | Description | |------|----------|-------------| | FL PixelGen Model Loader | Loaders | Load the PixelGen denoiser model from checkpoint | | FL PixelGen Text Encoder Loader | Loaders | Load a Qwen3 text encoder for conditioning | | FL PixelGen Generate | Generation | Generate images from text using pixel-space diffusion | | FL PixelGen LoRA Loader | Loaders | Load and apply a trained LoRA adapter for inference | | FL PixelGen Training Config | Training | Configure LoRA training hyperparameters and REPA loss weights | | FL PixelGen Train | Training | Run REPA LoRA training with real-time progress widget |

Installation

Manual

cd ComfyUI/custom_nodes
git clone https://github.com/filliptm/ComfyUI-FL-PixelGen.git
cd ComfyUI-FL-PixelGen
pip install -r requirements.txt

The training widget JS is included in js/ and requires no build step.

Quick Start

Inference Pipeline

1. Load Model — Add FL PixelGen Model Loader and select PixelGen-XXL-T2I
2. Load Text Encoder — Add FL PixelGen Text Encoder Loader and select a Qwen3 variant
3. Generate — Connect both to FL PixelGen Generate, enter a prompt, and execute

Training Pipeline

1. Load Model + Text Encoder — Same loader nodes as inference
2. Configure — Set LoRA rank, learning rate, epochs, and REPA loss weights in FL PixelGen Training Config
3. Train — Connect model, text_encoder, and config to FL PixelGen Train with your image directory
4. Validate — Set a validation prompt to see sample outputs during training

Using Trained LoRAs

Connect a trained LoRA checkpoint to FL PixelGen LoRA Loader between the model loader and generate nodes. Adjust strength to control the effect.

Node Details

FL PixelGen Model Loader

Loads the PixelGen denoiser from a local checkpoint or HuggingFace.

| Input | Type | Default | Notes | |-------|------|---------|-------| | model_name | Dropdown | PixelGen-XXL-T2I | Auto-downloads from HF | | device | Dropdown | cuda | cuda / cpu | | dtype | Dropdown | bfloat16 | bfloat16 / float32 | | force_reload | BOOLEAN | False | Bypass model cache |

Output: PIXELGEN_MODEL

FL PixelGen Text Encoder Loader

Loads the Qwen3-1.7B language model as the text encoder for conditioning.

| Input | Type | Default | Notes | |-------|------|---------|-------| | model_name | Dropdown | Qwen/Qwen3-1.7B | Auto-downloads from HF | | device | Dropdown | cuda | cuda / cpu | | force_reload | BOOLEAN | False | Bypass encoder cache |

Output: PIXELGEN_TEXT_ENCODER

FL PixelGen Generate

Generates images from text prompts using flow matching with the AdamLM solver. Provides live step-by-step preview during sampling.

| Input | Type | Default | Notes | |-------|------|---------|-------| | model | PIXELGEN_MODEL | — | From Model Loader | | text_encoder | PIXELGEN_TEXT_ENCODER | — | From Text Encoder Loader | | prompt | STRING | — | Text prompt (multiline) | | negative_prompt | STRING | (empty) | Negative prompt (multiline) | | width | INT | 512 | 128–1024 (step 32) | | height | INT | 512 | 128–1024 (step 32) | | steps | INT | 25 | 1–100 | | cfg | FLOAT | 4.0 | 0.1–10.0 | | seed | INT | 0 | — | | batch_size | INT | 1 | 1–4 | | timeshift | FLOAT | 3.0 | 0.1–5.0 | | solver_order | INT | 2 | 1–4 |

Output: IMAGE — standard ComfyUI image tensor

FL PixelGen LoRA Loader

Loads and applies a trained LoRA adapter to the denoiser for inference. Automatically resolves checkpoint directory structures.

| Input | Type | Default | Notes | |-------|------|---------|-------| | model | PIXELGEN_MODEL | — | From Model Loader | | lora_path | STRING | — | Path to LoRA checkpoint (folder or .safetensors) | | strength | FLOAT | 1.0 | 0.0–2.0 |

Output: PIXELGEN_MODEL — model with LoRA weights applied

FL PixelGen Training Config

| Parameter | Default | Range | |-----------|---------|-------| | LoRA Rank | 8 | 1–128 | | LoRA Alpha | 16 | 1–256 | | LoRA Dropout | 0.1 | 0–0.5 | | Learning Rate | 1e-4 | 1e-6 – 1e-2 | | Max Epochs | 100 | 1–10000 | | Batch Size | 1 | 1–8 | | Gradient Accumulation | 4 | 1–32 | | Save Every N Epochs | 10 | 1–1000 | | Max Grad Norm | 1.0 | 0.1–10.0 | | Seed | 42 | — | | REPA Losses | | | | Feature Loss Weight | 0.5 | 0–10.0 | | LPIPS Weight | 1.0 | 0–10.0 | | DINO Weight | 1.0 | 0–10.0 | | Perceptual Ratio | 1.0 | 0–10.0 | | Noise Gating Threshold | 0.0 | 0–1.0 | | CFG Dropout | 0.1 | 0–1.0 |

Output: PIXELGEN_TRAINING_CONFIG

Target modules: qkv_x, kv_y, proj, w12, w3, qkv

FL PixelGen Train

Runs the REPA LoRA training loop with 4-component loss: MSE(v_pred, v_target) + feature_alignment + LPIPS + DINO.

| Input | Type | Default | Notes | |-------|------|---------|-------| | model | PIXELGEN_MODEL | — | From Model Loader | | text_encoder | PIXELGEN_TEXT_ENCODER | — | From Text Encoder Loader | | config | PIXELGEN_TRAINING_CONFIG | — | From Training Config | | image_dir | STRING | — | Path to images with .txt caption sidecars | | lora_name | STRING | my_pixelgen_lora | Subfolder name for outputs | | resolution | INT | 512 | 128–1024 (step 32) | | caption_ext | STRING | .txt | Caption file extension | | resume_from | STRING | (empty) | Path to checkpoint to resume | | validation_prompt | STRING | (empty) | Generate samples during training | | validation_steps | INT | 20 | 5–50 |

Outputs: LoRA path, status message

The training widget displays a live loss chart, epoch progress, and validation sample carousel with fullscreen lightbox preview via WebSocket (pixelgen.training.progress).

Dataset Format

Place images and matching text captions in a folder:

my_dataset/
├── image_001.png
├── image_001.txt    # "A photo of a sunset over the ocean"
├── image_002.jpg
├── image_002.txt    # "A portrait of a woman in a red dress"
└── ...

Supported image formats: .png, .jpg, .jpeg, .webp, .bmp

Requirements

• Python 3.10+
• NVIDIA GPU with 12GB+ VRAM (bf16 training)
• PyTorch 2.0+
• PEFT (Parameter-Efficient Fine-Tuning)
• Transformers (for Qwen3 text encoder)

License

Apache-2.0