Media Scraper

Media Scraper turns a URL into a batch of images inside your ComfyUI graph. You paste a gallery link (or several, one per line), hit Run, and out come the images as tensors plus a list of the file paths they were saved to. Under the hood it's just shelling out to gallery-dl, the mature Python scraper that supports essentially every gallery site that exists - Pixiv, Danbooru, Twitter/X, Reddit, e621, and the long tail documented in gallery-dl's supported-sites list. You get all of gallery-dl's site support for free, wrapped in a node.

Why you'd reach for it

The README frames it around dataset building, and that's the honest use case: the pack's example workflows pair the scraper with polymath_chat running the "DSD captioning" agent, so you can go from "a URL" to "a caption-labeled training folder" in one graph. For LoRA training that's a genuinely useful assembly line - grab 50 images of a character, caption them, save them into your dataset directory. It also works fine for the simpler thing: pull a bunch of references into your graph without manually downloading anything.

The inputs

• urls - one or more gallery URLs, one per line.
• output_file_path - where files land. Leave it empty and it uses ComfyUI/output/scraped_by_polymath.
• file_name - a prefix for renamed output. When you supply both output_file_path and file_name, the scraper downloads to a temp folder, then renames and moves everything to the final destination as yourprefix_1.jpg, yourprefix_2.jpg, …
• image_load_cap (default 0) - stop after N images. 0 means no cap. Handy when a gallery has 500 images and you only want 20.
• keep_temp_path - the author's tooltip says it best: "Images are saved temporarily, then renamed. Set true to keep the gallery_dl structure." In other words, leave it false to get clean numbered filenames in your chosen folder; set it true to preserve gallery-dl's per-site folder layout. For dataset work you almost always want it false.

The outputs

• images - an IMAGE list, ready to wire into a preview, a batch, or the polymath_chat node for captioning.
• filepath_texts - a STRING list of the absolute paths each image was saved to. This is the output that connects to the rest of your pipeline: hand these to a caption-saver node or feed them into a training-dataset tool.

Installation and gotchas

It's part of the pack, so: ComfyUI Manager → search "comfyui_LLM_Polymath", or clone into custom_nodes and pip install -r requirements.txt. The critical bit: the pack lists gallery-dl as a dependency, but the node calls the gallery-dl command as a subprocess. If it's not on your PATH, the node runs, downloads nothing, and silently returns an empty list. If you get no images out, verify gallery-dl is installed and callable from the same environment that launches ComfyUI (gallery-dl --version).

Realistic expectations, because this is where scraper nodes get people in trouble:

• Sites with login walls need cookies. Pixiv and Twitter/X, for example, require an authenticated cookies.txt configured in gallery-dl; the node doesn't handle that for you - you configure it the standard gallery-dl way and the node inherits it.
• Some sites will rate-limit or block you. gallery-dl is good about respecting robots and throttling, but scrape respectfully - it's a dataset tool, not a mirror-all-of-the-internet tool.
• You're trusting a 2025-era indie pack with a subprocess call. It runs whatever gallery-dl does on your machine, which is normal for the ecosystem but worth knowing. This is also squarely in "verify what the node's doing" territory - check where files landed before pointing a training run at them.

The polish is thin - errors are mostly swallowed, so "nothing happened" is your main failure mode and the console is where you'll find out why - but as a dataset pipeline starter, it's hard to beat for zero configuration on the sites gallery-dl already handles.