LLM Arguments

LLM_Arguments is the settings panel for the LLM School pack's trainer. It packages Hugging Face's TrainingArguments - the object that decides learning rate, batch size, epochs, and where checkpoints land - into a single node whose output plugs into CausalLM_trainer's args input. Nothing trains on its own; it's pure configuration that the trainer serializes into JSON and passes along.

Think of it as the "how long should this take and how hard should it try" node. The defaults are conservative on purpose: batch size 1, 3 epochs, learning rate 2e-5. That's a reasonable first-run starting point for a small model like GPT-2, and it's exactly the sort of recipe that will get you a very slow full fine-tune on anything bigger - which is your cue to pair it with a LoRA method rather than full fine-tuning.

The inputs that matter

• learning_rate - 2e-5 default. The single most consequential knob. For LoRA runs on LLMs the community typically stays in the 1e-4 to 5e-4 range; for full fine-tuning, 2e-5 is a sane floor. Raise it and watch loss - this is where most people go wrong on both sides.
• per_device_train_batch_size - default 1. Bump it only if your VRAM comfortably survives a full forward+backward pass. With quantization (dtype int8/int4 in the trainer) you can afford more.
• num_train_epochs - default 3. For QA-style fine-tuning on SQuAD-like data, a couple of epochs is plenty; more epochs on a small dataset just memorizes.
• output_dir - where checkpoints go. The gotcha: the default is results, a relative path. Because the trainer runs as a separate python subprocess, that resolves against whatever directory ComfyUI was launched from, not the pack folder. Give it an absolute path or you'll be hunting for your checkpoints.
• save_steps (1000) and save_total_limit (2) - how often to checkpoint and how many to keep.

Everything else - eval_strategy, per_device_eval_batch_size, weight_decay - has a sensible default and you can leave it alone for a first run.

The one input people expect but isn't here

There's no warmup_steps, no lr_scheduler_type, no gradient_accumulation_steps, no fp16 toggle. The node hardcodes the eight fields above and that's it. If you need gradient accumulation or a scheduler, you'll have to edit the source or handle it outside the node. That's the honest tradeoff of an argument-builder node: it's a curated subset, not the whole TrainingArguments surface.

Install and run

Same as every node in the pack - ComfyUI Manager, search "comfyui_LLM_schools":

cd ComfyUI/custom_nodes
git clone https://github.com/heshengtao/comfyui_LLM_schools

restart, and let Manager install huggingface_hub, datasets, transformers, peft. One pack-level gotcha that applies here too: if the node doesn't load at all, the pack's missing config.ini is usually the cause - copy config.ini.example to config.ini in the pack folder and restart.

Output is a single args connection into CausalLM_trainer. It's the least glamorous node in the pack and the one you'll touch on every single run - which is exactly why it exists: better to fiddle with a learning rate in the graph than to rewrite a training script.