ComfyUI-huggingface_dataset
ComfyUI node to load a Hugging Face dataset
Nodes (17)
Filter a Hugging Face dataset by column β without writing a single lambda
Unwrap nested columns β turn that JSON-in-a-cell into real columns
Add a column to every row β constants, copies, or a row index, all in the graph
Drop the columns you don't need β before the rows ever hit your memory
Fix a column name in one box β no schema, no migration, no drama
Pick exact rows by index β comma lists, Python slices, all of it
Keep only the columns you need β the whitelist version of column trimming
Slice a dataset into N rough pieces and grab one β the parallel-work tool
Shuffle your dataset with a seed β reproducibility that actually reproduces
Skip the first N rows β the boring node that keeps showing up in real graphs
Sort by column, then flip with reverse β data wrangling's most underrated node
The honest train/test split β randomness you can pin with a seed
Keep the first N rows β your streaming dataset's best friend
Materialize a dataset as a Data List β the shape the loader's rows output promises
Turn a dataset into one flat Python LIST that every Basic node can eat
What values actually exist in this column? Ask once, get a Data List
Pull Hugging Face datasets into ComfyUI β no API key, no terminal Python
Hugging Face Dataset Loader
A custom ComfyUI node that loads a Hugging Face dataset and makes it available inside ComfyUI for further processing.
Every node ships in-app documentation and parameter tooltips β hover a node
and open its help (info) panel, or view the node's page in the Node Library
β and ready-made example workflows are available from the template browser
(Workflow β Browse Templates).
It loads a dataset from the Hugging Face Hub or from local/remote files using the
datasets library and exposes it
in two forms:
datasetβ the rawdatasets.Datasetobject of the selected split (a lazydatasets.IterableDatasetwhenstreamingis on), handed to ComfyUI as an opaqueHUGGINGFACE_DATASETvalue.rowsβ a ComfyUI Data List of row dicts (one dict per row), so every record can be processed further with generic data-handling nodes, for example from the Basic data handling node pack (convert to a ComfyUILISTorData List, access fields, filter, ...).
The pack also ships a set of processing nodes that consume the dataset
output and expose the data-wrangling methods of the datasets library
(shuffling, filtering, column selection, ...) as graph nodes, so a dataset can
be shaped before its rows are materialized β see
Dataset nodes.
Requirements
-
Python package
datasets(Hugging Face). It is installed automatically when you install this pack through ComfyUI-Manager, the Comfy Registry or apip installof the repo β there is nothing extra to do. The nodes import it lazily, so ComfyUI still starts even if it is missing; you only get a clear install message when you actually try to load a dataset. When you installed from a baregit clonewithout an installer, run once:pip install -r requirements.txt # or: pip install datasets
[!NOTE] JPEG XL images. If you want to load datasets whose images are stored as JPEG XL (
.jxl), make sure the Pillow JPEG XL plugin is installed as well.pip install pillow-jxl-pluginThe plugin is optional and you must install it yourself: the node starts and runs fine without it, only the JPEG XL images then fail to decode. The ComfyUI startup log tells you whether it is active β it prints
JPEG XL image support: enabledwhen the plugin is installed andJPEG XL image support: disabledotherwise.
Quickstart
Recommended Installation (ComfyUI-Manager or the Comfy Registry)
- Install ComfyUI-Manager (or
use the Registry tab in ComfyUI Desktop /
comfy node install). - Look up the "Hugging Face dataset" extension and install it.
- Restart ComfyUI.
The datasets dependency is installed for you automatically.
Alternative (Manual Installation)
- Install ComfyUI.
- Clone this repository under
ComfyUI/custom_nodes:git clone https://github.com/StableLlama/ComfyUI-huggingface_dataset.git - Install the runtime dependencies into ComfyUI's Python environment:
pip install -r requirements.txt - Restart ComfyUI.
Node
Load Hugging Face Dataset
Loads a dataset and outputs the loaded dataset plus a rows Data List.
| Input | Type | Default | Description |
| --- | --- | --- | --- |
| path | STRING | "" | Hub dataset id (e.g. stanfordnlp/imdb) or a local/remote file or directory (see Sources). |
| loader | STRING | auto | auto (infer from file extension), hub, or one of csv, json, parquet, arrow, text. |
| split | COMBO | train | Split to load. A dropdown listing the splits of the selected source (see Split selection). |
| config | STRING | "" | Config/subset name for Hub datasets that have several configs (e.g. nyu-mll/glue + config mrpc). |
| revision | STRING | "" | Optional Hub revision: tag, branch name, or commit hash. |
| streaming | BOOLEAN | False | Load the split as a lazy IterableDataset instead of downloading/caching it fully. |
| limit | INT | -1 | Maximum number of rows to materialize into rows. -1 = all rows. |
| Output | Type | Description |
| --- | --- | --- |
| dataset | HUGGINGFACE_DATASET | The raw datasets.Dataset of the selected split (or an IterableDataset with streaming on). |
| rows | * (Data List) | List of row dicts (one dict per row), capped by limit. |
Split selection
The split dropdown is filled with the splits the selected dataset actually
exposes:
- it refreshes automatically when you change
path,config,revisionorloader, and there is also a Refresh splits button to re-query manually; - a sensible default is picked (in the order
trainβvalidationβtest) when the dataset has notrainsplit - e.g. a dataset that only ships atestsplit selectstestautomatically; - listing the splits requires the
datasetspackage and (for Hub datasets) network access. When the splits cannot be determined (offline, or a multi-config dataset without aconfig), the dropdown falls back totrain/test/validationand the node validates the split when it runs. datasetsslicing syntax is still honoured for values that come from an older workflow, e.g. a storedtrain[:100]ortrain[:10%]continues to work.
Streaming vs. full load
With streaming off (the default) datasets downloads and caches the whole
split before the first limit rows are materialized into the rows Data
List - so limit caps the size of the returned rows, but not the download.
With streaming on, the node loads the split as a lazy
datasets.IterableDataset instead: nothing is downloaded until it is iterated,
which makes it a memory/bandwidth-friendly way to feed only the first limit
rows of a very large dataset into the graph. Note the dataset output is then
an IterableDataset (no len(), single-pass) rather than a datasets.Dataset.
Sources
- Hugging Face Hub β pass the repository id (
namespace/name) aspath, e.g.stanfordnlp/imdb,nyu-mll/glue(withconfig), orHuggingFaceFW/fineweb(with arevision). Leaveloaderatauto. - Local/remote files β
csv,tsv,json,jsonl,parquet,arrow,txt. Withloader = autothe format is inferred from the file extension, otherwise pick the matching file builder explicitly. Globs (e.g.data/*.parquet) and remotehttps:///hf://URLs work too. - Local dataset directory β a directory in Hugging Face dataset format.
Example workflows
Load the IMDb reviews stanfordnlp/imdb dataset and count its rows with the
Basic data handling pack (Basic β Data List β length):
flowchart LR
A[Load Hugging Face Dataset<br/>path=stanfordnlp/imdb<br/>split=train] -->|rows| B[Data List β length]
Take the review text of one specific row β rows is a Data List of row
dicts, so first pull a row, then read its text field:
flowchart LR
A[Load Hugging Face Dataset] -->|rows| B[Data List β get item<br/>index=0]
B --> C[DICT β get<br/>key=text]
[!TIP]
rowsis already a ComfyUI Data List β see Working with Basic data handling for many more ways to slice, map and consume it with the Basic data handling pack.
Dataset nodes
The loader's dataset output is an opaque HUGGINGFACE_DATASET value. The
processing nodes below take it as input, work on the underlying datasets
object and hand the result back as a new HUGGINGFACE_DATASET, so transforms
chain together (e.g. Shuffle β Filter β Map Column β Select Columns). Nodes
that only exist on a fully-loaded datasets.Dataset β marked loaded only
below β raise a clear error when given a streaming dataset instead of a cryptic
traceback: disable streaming on the loader for those.
Transform nodes
| Node | What it does | Streaming? |
| --- | --- | --- |
| π€ Dataset Shuffle | Randomly reorders the rows (shuffle(seed)). | β
|
| π€ Dataset Skip | Drops the first n rows (skip(n)). | β
|
| π€ Dataset Take | Keeps only the first n rows (take(n)). | β
|
| π€ Dataset Sort | Sorts the rows by a column (sort). | loaded only |
| π€ Dataset Shard | Keeps shard index of the dataset split into num_shards (shard). | β
|
| π€ Dataset Select Rows | Keeps rows by indices β comma-separated 0,2,4 and/or slices 0:100, 0:100:2 (select). | loaded only |
| π€ Dataset Select Columns | Keeps only the comma-separated columns (select_columns). | β
|
| π€ Dataset Remove Columns | Removes the comma-separated columns (remove_columns). | β
|
| π€ Dataset Rename Column | Renames one column (rename_column). | β
|
| π€ Dataset Flatten | Expands nested columns into top-level ones (flatten). | loaded only |
| π€ Dataset Filter | Keeps rows whose column satisfies an operator against value (filter). | β
|
| π€ Dataset Map Column | Adds/replaces column per row from a constant, a copy of another column, or the row index (map). | β
|
| π€ Dataset Train/Test Split | Randomly splits into train and test outputs (train_test_split). | loaded only |
| π€ Dataset Unique | Returns the unique column values as a Data List (unique). | loaded only |
Filter operators: ==, !=, <, <=, >, >= (the value text is
coerced to the column's type, so numeric columns compare with plain numbers),
contains / not contains / starts with / ends with, in / not in
(comma-separated value list), and is null / is not null.
Conversion nodes (LIST and Data List)
The "Basic data handling" pack distinguishes two list shapes, and either kind of dataset (fully-loaded or streaming) can be turned into both:
| Node | Output | Description |
| --- | --- | --- |
| π€ Dataset To LIST | LIST | One Python list value of all rows (a LIST as Basic data handling defines it, so it feeds LIST β length, LIST β get item, ...). With a column set, the list holds that column's values instead of row dicts. |
| π€ Dataset To Data List | * (Data List) | The same rows exposed as a ComfyUI Data List (like the loader's rows output): Basic Data List nodes receive the whole list in one call, other nodes run once per row. |
Both honour a limit widget (-1 = all rows) β handy for pulling only the
first rows of a large streaming dataset.
Example workflow
Load IMDb, keep only positive reviews (label == 1), add a row index and
feed the result to a Basic-data-handling Data List:
flowchart LR
A[Load Hugging Face Dataset<br/>path=stanfordnlp/imdb<br/>split=train] -->|dataset| B[π€ Dataset Filter<br/>column=label operator== value=1]
B -->|dataset| C[π€ Dataset Map Column<br/>column=row_id operation=row index]
C -->|dataset| D[π€ Dataset To Data List]
D -->|rows| E[Data List β length]
Working with Basic data handling
The Basic data handling pack provides everyday data nodes β lists, dicts, strings, maths, flow control and more. These Hugging Face nodes hand data to it in the two shapes it already understands, so rows can be counted, inspected, mapped and consumed with almost no manual conversion:
*Data List β one item per row: the loader'srowsoutput,π€ Dataset To Data List, and thevaluesoutput ofπ€ Dataset Unique.LISTβ one Python list value:π€ Dataset To LIST.
Rows are plain dicts keyed by the dataset's column names. Values are converted
to plain Python (numpy scalars/arrays are handled for you); objects such as
images or audio pass through unchanged. Install the Basic pack from
ComfyUI-Manager or the registry β its nodes appear under the Basic/β¦ menus
(Basic/Data List, Basic/LIST, Basic/DICT, Basic/STRING, Basic/cast,
...).
Whole-list nodes vs. per-row mapping
A Data List can be wired into Basic nodes in two ways:
- Whole-list nodes (
Basic/Data ListandBasic/LIST) receive the whole list in one call β e.g.length,count,first,get item,convert to LIST,convert to Data List. - Per-row mapping β connect the Data List into any other Basic node (a
Basic/DICT,Basic/STRINGorBasic/castnode). ComfyUI then runs that node once for every row, and its output is a new Data List with one result per row. This is the idiomatic way to transform every record in one go.
Recipes
Count the loaded rows, and read the review text of the first row:
flowchart LR
A[Load Hugging Face Dataset<br/>path=stanfordnlp/imdb] -->|rows| L[Data List β length]
A -->|rows| G[Data List β get item<br/>index=0]
G --> T[DICT β get<br/>key=text]
Map every row to one field β pull text out of each row with a per-row
DICT β get, then join the results into a single string:
flowchart LR
A[Load Hugging Face Dataset<br/>limit=100] -->|rows| G[DICT β get<br/>key=text]
G --> J[STRING β join (from data list)<br/>separator= ---]
- Turn whole rows into text β
rowsβBasic/cast β to STRING, then use anyBasic/STRINGnode. - One column of all rows as a
LISTβπ€ Dataset To LIST (column=text)βBasic/LIST β length/get item/first, orBasic/LIST β convert to Data Listto switch back to per-row processing. - Shape before materialising β build the rows you want with the transform
nodes first (
π€ Dataset Filter,Map Column,Select Columns, ...), then feedπ€ Dataset To Data List/To LISTinto Basic. - Unique values β
π€ Dataset Unique (column=label)βBasic/Data List β length, orBasic/Data List β convert to LIST.
These patterns work for every Data List the pack produces β the loader's rows
as well as any π€ Dataset To Data List fed from a transform chain β and the
limit widget (or a Take / Skip) bounds how many rows are materialised.
Example workflow templates
The repository ships ready-made workflows under
example_workflows/. Once the pack is installed they
appear in ComfyUI's template browser (Workflow β Browse Templates) under the
Hugging Face dataset entry:
imdb_filter_to_data_listβ load IMDb, keep only positive reviews and expose them as a Data List.imdb_map_select_to_listβ add a row index, keep thetextcolumn and export it as a LIST of strings.imdb_streaming_skip_takeβ load IMDb in streaming mode, skip/take rows and materialize a Data List without a full download.
Development
# Lint & format
python -m ruff check .
python -m ruff format .
# Type check (strict)
python -m mypy src tests
# Tests (no network / no datasets required)
python -m pytest tests/