Nodes/Nilor Nodes by Nilor Corp/๐Ÿ‘บ Save Image To HF Dataset
ComfyUI Node

๐Ÿ‘บ Save Image To HF Dataset

Push generated images straight into a Hugging Face dataset repo as you make them

By nilor-corpยทCreated 2 years agoยทUpdated 6 months agoยท 6
๐Ÿ‘บ Save Image To HF Dataset
  • image
    โ—„repository_idnilor_datasetโ–บ
    โ—„hf_auth_tokenauth_tokenโ–บ
    โ—„filename_prefixnilor_imageโ–บ

    If you're building a training dataset the slow way - generate, save to disk, gather up the folder, upload it - this node kills the upload step. ๐Ÿ‘บ Save Image To HF Dataset takes the image batch at the end of your generation graph, converts each image to PNG, and uploads it to a Hugging Face dataset repository on the spot. Every queue run appends new images to the repo, so your dataset grows while you sleep, which is exactly the workflow this pack's author is clearly built around (the whole pack smells like internal tooling that got published).

    The mechanism is straightforward: for each tensor in the batch it converts to uint8, encodes PNG in memory, and pushes via the huggingface_hub library with repo_type="dataset". Filenames get a prefix, an index, and a timestamp, so successive runs don't collide. One detail worth flagging: the embedded PNG metadata is currently a hardcoded placeholder string rather than your actual workflow - so don't count on the "workflow included" trick carrying over to these uploads. If that matters for your pipeline, you'll be adding metadata yourself later.

    Inputs and output

    • image - the IMAGE batch to upload. Every frame in the batch becomes its own PNG.
    • repository_id - your Hugging Face dataset repo, e.g. yourname/mydataset (default nilor_dataset - you'll want to change this).
    • hf_auth_token - your HF write token. Default is literally auth_token, a placeholder; this is the field people forget and the #1 failure mode.
    • filename_prefix - base name for uploaded files (default nilor_image).
    • No outputs - it's an output node; the effect is uploads to your repo.

    Installing it

    Part of Nilor Nodes (nilor-corp/nilor-nodes):

    cd ComfyUI/custom_nodes
    git clone https://github.com/nilor-corp/nilor-nodes
    

    or ComfyUI Manager โ†’ search "Nilor Nodes" โ†’ install โ†’ restart. The relevant dependency here is huggingface_hub, which the pack installs as part of its heavier-than-expected stack (boto3, fastapi, openexr and friends come along regardless). The README lists Kijai's comfyui-kjnodes as a prerequisite. No model files.

    Troubleshooting

    Every failure mode here is an auth or plumbing problem, not a logic one. The default token is a placeholder, so first-run uploads fail until you paste a real write-capable HF token - and the token needs write permission on that specific repo, not just any token. The repo must exist before you run (or be created first). And if the repo is private, the node doesn't create it for you. The pack has no community support to consult, so the error from HfApi is your main diagnostic - it usually says exactly what's wrong with the credentials. It also can't skip duplicates: each run adds files with fresh timestamps, so re-running a queue creates copies, not idempotent dedupe. Cheap to work around, worth knowing before you upload the same batch nine times.

    CategoryNilor Nodes ๐Ÿ‘บ/IO

    Inputs (4)

    NameTypeDefaultDescription
    imageIMAGEโ€”
    repository_idSTRINGnilor_datasetโ€”
    hf_auth_tokenSTRINGauth_tokenโ€”
    filename_prefixSTRINGnilor_imageโ€”

    Outputs (0)

    No outputs