URL Fetcher

ComfyUI has no native way to reach the internet. You can load a checkpoint, sample a latent, pipe strings around all day - but the graph ends where your hard drive does. URL Fetcher is the bridge: paste in a list of URLs, and out comes the text content of every page, joined into one string you can wire straight into an LLM node, a prompt builder, or a text-to-tags pipeline. If you've ever wanted to hand your local model a news article and say "summarize this, then make me a prompt," this is the node you were missing.

The name is a lie in one respect: there's no API, no key, no model to download. It's a tiny pure-Python node that makes plain requests.get() calls - a couple of hundred lines, MIT-licensed, and it shows up in ComfyUI under the category "example," which is the author quietly telling you what this is: a utility, not a product.

How it works

One input, urls, a multiline text box. Feed it URLs separated by spaces or newlines - the node splits on any whitespace, so no commas, no bullets. Each URL gets fetched, and the node does its own crude scraping: it grabs the <title> tag, deletes <script> and <style> blocks, strips every remaining HTML tag, and flattens the whitespace. The result per page is formatted as title: page text, and all pages are joined into the single output string, one page per line.

The scraping is regex, not a proper parser - that matters, and we'll get to it. It's also a per-URL try/except, so one dead link doesn't kill the batch: a failure just becomes a line like Error fetching https://...: ... inside the output. That's friendlier than it sounds, and it's also the first trap.

Where people get burned

• No timeout on the request. requests.get() is called without a timeout=, so a server that accepts the connection and then stalls will hang your node - and your whole queue - indefinitely. If you batch a bunch of URLs, one slow site holds everything.
• The default User-Agent. Lots of sites 403 the python-requests default UA. You won't get an error, exactly - you'll get an error line inside the output and wonder why your LLM is summarizing Cloudflare's block page.
• It fetches every time. No cache, no "already loaded" state. Re-run the queue and every page is re-downloaded, so a 30-URL list is 30 network calls per run, plus the sites' robots policy on your machine.
• The text is raw. Nav bars, cookie banners, footers, and script leftovers all make it into the string. There's no readability extraction and no markdown conversion. With several pages concatenated, you can blow past your LLM's context window fast - keep the URL count small, or accept that the model will read a lot of junk.
• No JS, no login. Only server-rendered HTML works. A page built entirely by JavaScript returns an empty shell, and anything behind auth is out.

Installing it

The README's preferred path is the registry - in ComfyUI Manager, search "URL Fetcher" (pack title comfy-url-fetcher), or run:

comfy node registry-install url_fetcher

Or the manual route:

cd ComfyUI/custom_nodes
git clone https://github.com/molbal/comfy-url-fetcher

then restart ComfyUI. Dependencies are light: requests plus beautifulsoup4 in requirements.txt - though the shipped code never actually imports BeautifulSoup and does everything with regex, so that second one is dead weight the install pulls anyway. No models, no VRAM, nothing to download.

What to do with it

The genuinely useful pattern is the one llm-in-comfyui covers: an LLM sitting in the graph as a text tool. Wire output into a local LLM node, and you've got a "go read this page and rewrite my prompt around it" pipeline, or a caption-from-a-live-article loop, or a quick way to pull an API endpoint's text response into a string without leaving the canvas. The other thing worth knowing: this is a custom node whose whole job is reaching out over the network, which is exactly the shape of pack that has been weaponized before in this ecosystem. This one is tiny, open, and readable - but glance at the source before you trust it with anything sensitive, and know that the URLs you paste go out over the wire on every run.