Extensions/CS Hybrid 4K Image-to-Video
ComfyUI Extension

CS Hybrid 4K Image-to-Video

Compressed-sensing hybrid 4K image-to-video: RAFT flow transfer, dual-channel Laplacian recompose, CS tile scheduler, keyframe refresh helpers.

By lvqunx·Created 21 days ago·Updated 21 days ago· 0
lvqunx/comfyui-cs-hybrid
Nodes—
On cloudLocal install
Stars0
Updated21 days ago
Readme

CS Hybrid 4K Image-to-Video (ComfyUI nodes)

Generate 4K image-to-video results while only running the video diffusion model at low resolution (e.g. 480P). High-frequency detail is transferred from the 4K source image via optical flow, and only scheduled regions (keyframes / occlusions / random tile subsets, compressed-sensing style) are re-synthesized at full resolution — so VRAM stays an order of magnitude below native 4K video diffusion.

Pipeline

  1. Generate video at 480P with any i2v model (designed around LTX 2.x, works with others).
  2. Estimate optical flow + occlusion masks at 480P (RAFT).
  3. Rebuild 4K frames in two channels: low frequencies from the upscaled generated video, high-frequency Laplacian bands from the 4K still advected by the accumulated flow.
  4. Schedule full-resolution measurements: random rotating tiles per frame, occlusion union, full frames every K keyframes.
  5. Refresh keyframes with an img2img refiner (SDXL/Flux), merge back.

Nodes

| Node | Purpose | |---|---| | CS Flow Estimate (RAFT + occlusion) | Forward/backward RAFT flow between consecutive frames + forward-backward consistency occlusion mask. Weights auto-download on first run. | | CS Detail Recompose (dual-channel 4K) | Dual-channel 4K frame rebuild: upscaled low-res video + warped Laplacian high bands from the source still. Also propagates the occlusion mask to 4K. | | CS Tile Scheduler (measurement mask) | Compressed-sensing measurement schedule: random rotating tiles (ratio r), full measurement every K frames, union with occlusion mask. | | CS Batch Stride (keyframes) | Select frames t % stride == 0 from an image batch. | | CS Concat Images | Concatenate up to 4 image batches (chunked sampler outputs). | | CS Replace Keyframes | Replace keyframes of a draft batch with refined frames, in order. |

Requirements

  • ComfyUI (recent; torch + torchvision are bundled with ComfyUI — no extra dependencies).
  • RAFT weights are downloaded automatically by torchvision on first use of the flow node.

Install

Via ComfyUI-Manager: search CS Hybrid 4K Image-to-Video and install.

Manual: clone this repo into ComfyUI/custom_nodes/ and restart ComfyUI.

Notes

  • Flow accuracy degrades for stochastic textures (water, smoke, particles); raise the scheduler ratio for such content.
  • Keep keyframe_every * ratio >= 2 so every tile is measured at least twice per cycle.