Dataset → DataLoader

What it is

ComfyDL is Cynthia-lxx's d2l-derived pack for doing deep learning inside ComfyUI: textbook recipes as nodes, one universal DATASET type (features, optional labels, column names, passed by reference) gluing the data side together.

Dataset → DataLoader turns a DATASET into a cdlDataloader - the socket type the pack's batch-consuming nodes want. If Dataset → Tensors hands data out in one lump, this hands it out in mini-batches, which is what most real training code iterates over.

Whether you need it depends on the trainer. Linear Regression Train takes raw X/y tensors or a dataset and does its own batch sampling internally - no loader required. The loader is for nodes that expect to iterate a cdlDataloader: the pack's own DataLoader Info and batch inspection/statistics nodes, and any training-loop node built around a batch iterator. If you're not wiring into one of those, you probably want Dataset → Tensors instead.

How it works

It's not a new implementation - it forwards the dataset's features and labels into the pack's existing CdlLoadArray node, which calls the d2l helper:

dataset = torch.utils.data.TensorDataset(*data_arrays)
return torch.utils.data.DataLoader(dataset, batch_size, shuffle=is_train)

Two consequences fall straight out of that. First, it's a real torch.utils.data.DataLoader wrapping a TensorDataset, so iterating it yields tuples: (X_batch, y_batch) when there are labels, and 1-element tuples when there aren't. Second, the pack's shuffle flag is passed through as d2l's is_train parameter - that's why the widget is named "shuffle" and documented as "shuffle each epoch": in the d2l idiom, shuffling is the marker for a training-mode iterator.

Note also that a DataLoader is rebuilt from the tensors every time this node runs, so the shuffle order isn't preserved between runs; with the same source data you'll get a different batch order each execution unless the surrounding graph caches.

Inputs and outputs

Required: dataset (a forced input socket - it must be wired, not typed), batch_size (default 32, range 1–4096), and shuffle (default True).

Output: dataloader. Wire it to a batch consumer, not to anything expecting TENSOR.

Set two things: batch_size, and shuffle=False when order matters (a time series where sequential batches should stay in time order, or a reproducibility run). There's no drop_last, so the final batch of an epoch is whatever's left over - 1,000 rows at batch_size=32 gives you 31 full batches and one of 8.

Install

Manager → search ComfyDL, or:

cd ComfyUI/custom_nodes
git clone https://github.com/Cynthia-lxx/ComfyDL
pip install -r ./ComfyDL/requirements.txt

and restart. Requirements are light - matplotlib, IPython, matplotlib-inline - with pandas assumed present in your ComfyUI environment, since the dataset readers and writers need it. Remember the pack loads node modules in a try/except and only prints [ComfyDL] Warning: Failed to load <module>; if nodes are missing from the menu, that console line is your clue, not an error toast.

Where people get burned

Unlabelled datasets produce 1-tuples. With labels = None, TensorDataset wraps a single tensor, so iterating yields one-element tuples. A consumer written for (X, y) pairs will fail in a way that looks like a bug in the consumer. Check labels exist first (Dataset → Preview shows the label column; if it isn't there, it doesn't exist).

Nothing shuffles the source. shuffle=True reorders iteration; it doesn't randomise per-epoch with the graph re-running (there's no Generator seed exposed here - the whole graph just runs again).

Very large batch_size is not free. Up to 4,096 rows per batch means a batch held in memory per iteration, and the loader is built on the full tensors regardless - this node does not make a huge dataset fit in less RAM.

Wrong socket. A cdlDataloader isn't a tensor. If a node's input refuses the connection, that's the type system doing its job - put DataLoader Info in between to read num_batches, batch_size and dataset_size and confirm the thing is what you think it is.