Nodes/comfyui_LLM_schools/get HF dataset repo_id
ComfyUI Node

get HF dataset repo_id

Find the right HF dataset repo_id without opening a browser tab

By heshengtao·Created 2 years ago·Updated 2 years ago· 7
get HF dataset repo_id
    • repo_id_list
    keywordimdb
    is_enabletrue

    get_dataset_name is the pack's "what should I even download?" helper. It takes a search keyword, queries Hugging Face's dataset registry, and hands you back the list of matching repo_ids as JSON. In the LLM School pipeline it sits right before download_dataset: you search, read the returned names, and paste the one you want into download_dataset's repo_id field. It's the difference between guessing that a dataset exists and knowing its exact handle.

    Mechanically it's a one-liner: huggingface_hub.list_datasets(filter=keyword) returns the search results, and the node extracts each dataset's .id into a JSON array. That's it - no downloads, no caching, no token needed for public search. The output repo_id_list is a plain string of JSON, so in the UI it reads like a list of quoted names (["imdb", "stanfordnlp/imdb", ...]). Wire it to a Show Text node if you want it readable inline.

    The inputs that matter

    • keyword - the search term. Default is imdb, which is a fine example but a lousy default for the rest of this pack, since the trainer's preprocessing expects SQuAD-style QA columns (question, context, answers). If you're following the pack's example end to end, just skip the search and use rajpurkar/squad_v2 in download_dataset.
    • is_enable - every node in this pack has one; flip it off to mute the node without deleting it.

    One honest limitation

    Pagination isn't handled. list_datasets returns a page of results, and the node dumps whatever that first page was - you won't see all 10,000 repos matching "bert," just the first batch. Fine for finding a well-known dataset by exact-ish keyword; bad for broad discovery. For real browsing, the Hub's own search is still the better tool; this node is for the "I know roughly what it's called, give me the exact id" case.

    Install

    Same story as every node in the pack - ComfyUI Manager, search "comfyui_LLM_schools", or:

    cd ComfyUI/custom_nodes
    git clone https://github.com/heshengtao/comfyui_LLM_schools
    

    then restart. Dependencies are huggingface_hub, datasets, transformers, peft.

    Common issues

    • Node missing after install. The pack reads config.ini at import time (for the HF token), and if that file isn't there, dataset.py - which contains this node - fails to load while the pack's loader silently skips it. Copy config.ini.example to config.ini in the pack folder and restart.
    • Empty list for a term you know exists. Search matching on the Hub is fuzzy and paginated; try a shorter keyword, or a partial name. You can also just type the exact owner/name into download_dataset directly - the search is a convenience, not a requirement.

    It's a utility node, and it knows it. Don't expect cleverness - expect a faster way to get a repo_id into download_dataset than tab-switching to the Hub.

    Category大模型学校(llm_schools)/数据预处理(data preprocessing)

    Inputs (2)

    NameTypeDefaultDescription
    keywordSTRINGimdb
    is_enableBOOLEANtrue

    Outputs (1)

    NameTypeDescription
    repo_id_listSTRING