Modal LLM

Modal LLM is a full multimodal language-model node that lives inside the graph: give it text, an IMAGE batch, a native VIDEO, or files, and it hands back a clean text response. The part nobody else does - and the reason it's in this pack at all - is where the model runs. Leave Run on Modal off and it runs locally on Apple Silicon through MLX. Flip the toggle on and it runs on the same Modal GPU worker as the rest of your remote island. One node, two execution targets, controlled by a switch you already use.

That duality is genuinely useful once you understand it. The community's default LLM-in-the-graph pattern is a small obedient model doing prompt enhancement or captioning - and the KB's take holds here: you don't want a big reasoning model for that job, you want something that follows a format and stops. The default model_profile (smolvlm2-2.2b-instruct) is exactly that kind of small-and-obedient VLM. The clever workflow is a MacBook: run this node locally on unified memory for captioning while the heavyweight samplers run on Modal - the planner even lets an unmarked LLM sit between two remote regions, keeping the model on the Mac and only moving tensors across the boundary.

Mechanically it's the most careful node in the pack. Every model goes through an immutable, revision-pinned profile: it inspects the Hugging Face repo and config.json before downloading anything, validates that the architecture and quantization are compatible with your chosen target, pins the exact commit, and reuses the completed snapshot on later runs. Local uses MLX-VLM (or mlx-dspark for text-only models); remote uses Transformers or vLLM on the Modal GPU. The per-request enable_reasoning toggle passes Qwen's hard thinking switch through all three backends - for a thinking model, max_new_tokens covers reasoning and the answer together, and MLX caps thinking at half that budget so a verbose model can't eat the whole allowance before responding. Loaded models stay resident in a process-global LRU (that's keep_model_loaded, default on), and before a cold load the manager asks ComfyUI to release idle managed models and enforces a free-VRAM reserve. Cancellation is wired straight into ComfyUI's interrupt.

The inputs that matter: model_profile (a curated name like the default, or any owner/model@revision), prompt, max_new_tokens, temperature (default 0.2 - set 0 for greedy), and enable_reasoning. reserve_free_vram_gb (24 default) and local_reserve_free_memory_gb (4) are the "don't starve ComfyUI" guards - the README's single-user B300 recipe also raises the local reserve to 24 via env vars. video_frames (default 12) controls how many uniformly-sampled frames it analyzes from a VIDEO input. It returns three strings: response (the clean answer), metadata_json (telemetry - backend, device, token rates, memory), and reasoning (the thinking trace, empty when you disable reasoning). The local_mlx_engine selector is only relevant on Apple hardware; auto picks correctly.

Install is the same as the pack: Manager, search Modal Sync, or git clone https://github.com/ttulttul/ComfyUI-Modal into custom_nodes/, then restart. The two real setup costs: Apple-local inference needs the pinned MLX runtime in the exact Python that launches ComfyUI -

uv pip install --python <comfyui-venv>/bin/python \
  "mlx-vlm==0.6.15" "mlx-dspark==0.13.1" "psutil>=7,<8" \
  "huggingface-hub==1.28.0" "hf-xet==1.6.0"

• and remote execution needs a Modal secret collection (default comfy), with HF_TOKEN added to it if you want gated models. First-run model staging takes minutes, but it's billed CPU/network time, not GPU; later runs reuse the volume snapshot.

Where people get tripped up: non-Mac users who forget the toggle get a clear error (the fix is enabling Run on Modal, not filing a bug). A profile generated for Modal can't be loaded through MLX, and vice versa - target is part of the content identity, so switching targets re-resolves the model. Unsupported quantizations (CUDA FP8, ModelOpt FP4 on the local side) are rejected before download, which saves you a 20 GB lesson. The whole pack is alpha, this node included, so keep expectations measured - but if you're already paying for Modal, this is the cleanest way to stop renting a second GPU just to caption your LoRA set.