The patches are applied automatically, some features require configuration, copy the jncomfy.yaml.example to ComfyUI/jncomfy.yaml and uncomment the configuration for the feature you need.

<details> <summary>Show patches</summary>

Preview device

Allows to change the device used for TAESD to render the preview.

The default behaviour is to use the same device used to render the image, so the preview consumes VRAM that could be used to render the image.

This feature is useful if you don't have enough VRAM to render the preview and image on the same device.

# ComfyUI/jncomfy.yaml

preview_device: cpu

Extension device

Allows to change the device used by custom nodes.

Custom nodes use the comfy.model_management.get_torch_device() to get the device they should use, it is the same device used to render the image, but some custom nodes performe actions that don't require the same device, so with this feature you can set another device based on which code is asking for the device.

It matches the package/class that calls the function comfy.model_management.get_torch_device() on the custom node.

Example, if it's called at foo.bar.MyCustomNode, any of these will work:

# ComfyUI/jncomfy.yaml

extension_device:
  foo: cpu
  foo.bar: cpu
  foo.bar.MyCustomNode: cpu

The first part is always the package/repository name, like in this real example:

# ComfyUI/jncomfy.yaml

extension_device:
  comfyui_controlnet_aux: cpu
  jn_comfyui.nodes.facerestore: cpu
  jn_comfyui.extra.facelib : cpu

It is easy to change the device for all custom nodes from the same repository, just use the directory name inside the custom_nodes directory.

If the custom nodes are inside custom_nodes/some_custom_nodes_package you can set:

# ComfyUI/jncomfy.yaml

extension_device:
  some_custom_nodes_package: cpu

But to specify specific nodes you need to know how the code of the custom node works and where it calls the comfy.model_management.get_torch_device().

Depending of how the custom node works it may not be possible to specify just a specific node.

Temperature

If your device don't have a good cooling system it can overheat after too many consecutive generations.

With this feature you can configure temperature limits to pause the generation and wait the device cool down.

You can set a limit to pause on the execution process between the nodes and another limit for the progress process between the steps of a node, for the nodes the show the progress bar.

Each limit has a safe and max temperature, once the temperature exceeds the max temperature it pauses the generation and waits for it to cool down to the safe temperature.

You can set how many seconds it waits before checking the temperature again, the seconds to wait is shown in the progress bar on the node that is waiting.

You can also set for how long it can wait, if you set it to zero it will wait for how long it needs to reach the safe temperature.

# ComfyUI/jncomfy.yaml

temperature:
  limit:
    execution: # Don't execute a node if the temperature is above the max, but wait cool down to the safe temperature.
      safe: 90
      max: 95
    progress: # Don't execute the next step of a node if the temperature is above the max, but wait cool down to the safe temperature.
      safe: 90
      max: 95
  cool_down:
    each: 5 # Seconds to wait the temperature cool down before checking the temperature again.
    max: 0 # Max seconds to wait the temperature cool down.

Memory estimation

The split attention, as the name suggests, splits the data used by the attention process in chunks and processes each chunk one after another.

The amount of chunks depends of how much VRAM is available, but the exact amount of memory required for the attention process depends of so many things that we can only estimate that value.

The size of the tensor is used as the base to calculate the memory required and that is multiplied by a value.

You can change that multiplier with the setting:

# ComfyUI/jncomfy.yaml

memory_estimation_multiplier: 1

Optimizations

Some features, like split attention and tiled VAE encode/decode, divide the process in steps.

This patch optimizes these features to find the best amount of steps for each process that fits your device and caches that value so the next generations will run faster.

It helps if you have low VRAM and enables you to generate bigger images.

If you have a good GPU it changes nothing and it won't slow down your generations.

Easy generic inputs

Some nodes require inputs of any type, there are some hacks out there to do it, but the LiteGraph, which is the javascript library used to create the graphs already has the generic type *.

This patch just finishes the integration of the generic type that already exists, so you don't have to do any fancy trick, just use the type *.

class PrintValue:
    CATEGORY = "_for_testing"
    RETURN_TYPES = ()
    FUNCTION = "run"
    OUTPUT_NODE = True

    @classmethod
    def INPUT_TYPES(s):
        return {
            "required": {
                "value": ("*",),
            },
        }

    def run(self, value):
        print(value)
        return {"ui": {}}

Easy multiple inputs

Some nodes require multiple inputs of the same type, a common approach to solve this problem is to add two inputs of the same type, output the result and use that output as the input of a copy of the same node, thus concatenating the results. Another common approach is to add an arbitrary number of inputs of the same type, usually 4 or 5, and hope it is enough.

That may do the job but it is not a good solution, a better solution is to add new inputs dynamically when you connect the input.

Some custom nodes already do it, but they do it as a hack on specific nodes and they cannot mix it with static inputs.

This patch allows any node to have multiple inputs of the same type that work alongside regular inputs.

It is easy to turn an input into multiple inputs, just add the "multiple": True option and the value will be an array of the type.

class ImageGrid:
    CATEGORY = "_for_testing"
    RETURN_TYPES = ("IMAGE",)
    FUNCTION = "run"

    @classmethod
    def INPUT_TYPES(s):
        return {
            "required": {
                "images": ("IMAGE", {"multiple": True}),
            },
        }

    def run(self, images):
        # receive the images as an array
        for image in images:
            # ... rest of the code ...
        return (image_grid,)

</details>

Nodes

<details> <summary>Show nodes</summary>

Audio

JN_AudioArrayToBatch - Audio Array To Batch
JN_AudioBatchToArray - Audio Batch To Array
JN_AudioCompare - Audio Compare
JN_AudioInfo - Audio Info
JN_AudioPlot - Audio Plot
JN_LoadAudioDirectory - Load Audio Directory
JN_PreviewAudio - Preview Audio
JN_SaveAudio - Save Audio

Audio > Channels

JN_AudioGetChannels - Audio Get Channels
JN_AudioSetChannels - Audio Set Channels
JN_AudioSplitChannels - Audio Split Channels
JN_AudioStackChannels - Audio Stack Channels

Audio > Samples

JN_AudioConcatenation - Audio Concatenation
JN_AudioSlice - Audio Slice
JN_AudioTrimSilence - Audio Trim Silence

Audio > Edit

JN_AudioAutoTune - Audio Auto Tune
JN_AudioNoiseReduction - Audio Noise Reduction
JN_AudioNormalize - Audio Normalize
JN_AudioPitch - Audio Pitch
JN_AudioReverberation - Audio Reverberation
JN_AudioSampleRate - Audio Sample Rate
JN_AudioSpeed - Audio Speed
JN_AudioTempo - Audio Tempo
JN_AudioVolume - Audio Volume

Audio > Meow

JN_MeowSentenceSplit - Meow Sentence Split
JN_MeowSaveVoice - Meow Save Voice
JN_MeowLoadVoice - Meow Load Voice

Audio > Meow > HRTF

JN_MeowHrtfAudio3d - Meow HRTF Audio 3D
JN_MeowHrtfModel - Meow HRTF Model
JN_MeowHrtfPosition - Meow HRTF Position

Audio > Meow > TTS

JN_MeowTts - Meow TTS
JN_MeowTtsAudioToContext - Meow TTS Audio To Context
JN_MeowTtsCoarse - Meow TTS Coarse
JN_MeowTtsDecode - Meow TTS Decode
JN_MeowTtsFine - Meow TTS Fine
JN_MeowTtsLoadContext - Meow TTS Load Context
JN_MeowTtsModel - Meow TTS Model
JN_MeowTtsModelCoarse - Meow TTS Model Coarse
JN_MeowTtsModelEncodec - Meow TTS Model Encodec
JN_MeowTtsModelFine - Meow TTS Model Fine
JN_MeowTtsModelHubert - Meow TTS Model Hubert
JN_MeowTtsModelSemantic - Meow TTS Model Semantic
JN_MeowTtsSaveContext - Meow TTS Save Context
JN_MeowTtsSemantic - Meow TTS Semantic
JN_MeowTtsTokenizerHubert - Meow TTS Tokenizer Hubert

Audio > Meow > VC

JN_MeowVc - Meow Voice Conversion
JN_MeowVcConvertVoice - Meow VC Convert Voice
JN_MeowVcEncodeSource - Meow VC Encode Source
JN_MeowVcEncodeTarget - Meow VC Encode Target
JN_MeowVcLoadSpeaker - Meow VC Load Speaker
JN_MeowVcModelFreeVC - Meow VC Model FreeVC
JN_MeowVcModelWavLM - Meow VC Model WavLM
JN_MeowVcSaveSpeaker - Meow VC Save Speaker

Image

JN_ImageAddBackground - Image Add Background
JN_ImageAddMask - Image Add Mask
JN_ImageBatch - Image Batch
JN_ImageCenterArea - Image Center Area
JN_ImageCrop - Image Crop
JN_ImageGrid - Image Grid
JN_ImageInfo: Image Info
JN_ImageRemoveBackground - Image Remove Background
JN_ImageSharpness - Image Sharpness
JN_ImageSquare - Image Square
JN_ImageToMask - Image To Mask
JN_ImageUncrop - Image Uncrop
JN_LoadImageDirectory - Load Image Directory
JN_MaskBatch - Mask Batch
JN_MaskInfo - Mask Info
JN_MaskToImage - Mask To Image
JN_PreviewImage - Preview Image
JN_PreviewMask - Preview Mask
JN_RemBGSession - RemBG Session
JN_SaveImage - Save Image

Image > Area

JN_AreaAround - Area Around
JN_AreaInfo - Area Info
JN_AreaNormalize - Area Normalize
JN_AreaToMask - Area To Mask
JN_AreaWidthHeight - Area Width Height
JN_AreaXY - Area X Y
JN_MaskToArea - Mask To Area

Image > Blip

JN_Blip - Blip
JN_BlipLoader - Blip Loader

Image > Face

JN_FaceCrop - Face Crop
JN_FaceRestoreModelLoader - Face Restore Model Loader
JN_FaceRestoreWithModel - Face Restore With Model

Sampling

JN_KSampler - KSampler
JN_KSamplerAdvancedParams - KSampler Advanced Params
JN_KSamplerFaceRestoreParams - KSampler Face Restore Params
JN_KSamplerResizeInputParams - KSampler Resize Input Params
JN_KSamplerResizeMaskAreaParams - KSampler Resize Mask Area Params
JN_KSamplerResizeOutputParams - KSampler Resize Output Params
JN_KSamplerSeamlessParams - KSampler Seamless Params
JN_KSamplerTileParams - KSampler Tile Params

Patch

JN_Seamless - Seamless
JN_SeamlessBorder - Seamless Border
JN_SeamlessBorderCrop - Seamless Border Crop

Primitive

JN_PrimitiveArrayInfo - ARRAY INFO
JN_PrimitiveBoolean - BOOLEAN
JN_PrimitiveFloat - FLOAT
JN_PrimitiveInt - INT
JN_PrimitivePrompt - PROMPT
JN_PrimitiveString - STRING
JN_PrimitiveStringMultiline - STRING MULTILINE

Primitive > Conversion

JN_PrimitiveBatchToArray - BATCH TO ARRAY
JN_PrimitiveStringToArray - STRING TO ARRAY
JN_PrimitiveToArray - TO ARRAY
JN_PrimitiveToBoolean - TO BOOLEAN
JN_PrimitiveToFloat - TO FLOAT
JN_PrimitiveToInt - TO INT
JN_PrimitiveToString - TO STRING

Primitive > Process

JN_BooleanOperation - Boolean Operation
JN_FirstActive - First Active
JN_KeyValue - Key Value
JN_LogicOperation - Logic Operation
JN_MathOperation - Math Operation
JN_MathOperationArray - Math Operation Array
JN_SelectItem - Select Item
JN_SliceOperation - Slice Operation
JN_TextConcatenation - Text Concatenation
JN_TextReplace - Text Replace

Workflow

JN_Condition - Condition
JN_Flow - Flow
JN_FlowOutput - Flow Output
JN_StopIf - Stop If
JN_StopIfOutput - Stop If Output

Other

JN_CoolDown - Cool Down
JN_CoolDownOutput - Cool Down Output
JN_DatetimeFormat - Datetime Format
JN_DatetimeInfo - Datetime Info
JN_DatetimeNow - Datetime Now
JN_Dump - Dump
JN_DumpOutput - Dump Output
JN_Exec - Exec
JN_ExecOutput - Exec Output
JN_Sleep - Sleep
JN_SleepOutput - Sleep Output
JN_TensorInfo - Tensor Info
JN_TimedeltaFormat - Timedelta Format
JN_TimedeltaInfo - Timedelta Info

</details>

References

mav-rik/facerestore_cf Face Restore
suno-ai/bark Text-To-Speech
OlaWod/FreeVC Voice Conversion
joe-habel/PyHRTFSweeper HRTF - Head-Related Transfer Functions
amini-allight/cipic-hrtf-database CIPIC HRTF Database

License

The MIT License (MIT). Please see License File for more information.