VRAM Context Test (diagnostics)

A ComfyUI node in OTR/v2/Diagnostics with 5 inputs and 1 output.

By jbrick2070·Created 5 months ago·Updated about 5 hours ago· 7

◄model_idmistralai/Mistral-Nemo-Instruct-2407►

◄probe_lengths2048,4096,6144,8192,12288,16384,20480,24576►

◄max_new_tokens16►

◄optimization_profileStandard►

◄measurement_label►

CategoryOTR/v2/Diagnostics

Inputs (5)

Name	Type	Default	Description
model_id	COMBO	mistralai/Mistral-Nemo-Instruct-2407	Which LLM to probe. Mirrors the LLMScriptWriter dropdown so production-relevant models are first-class. Suffix tags ([ALPHA], (EXPERIMENTAL)) are stripped before the HF lookup -- same behaviour as the writer's request_slot path.
probe_lengths	STRING	2048,4096,6144,8192,12288,16384,20480,24576	Comma-separated token-length targets. Each probe builds a prompt of approximately this many tokens and runs a single 16-token generation, capturing VRAM nvml + VRAM torch + CPU RAM peaks. Default probes: 2048,4096,6144,8192,12288,16384,20480,24576
max_new_tokens	INT	164–256	Generation length per probe. 16 is the smallest useful sample for VRAM measurement (KV cache growth is the relevant scaling factor, not generation length itself). Raise to 64-128 if you want to also bench tokens/sec at this context size.
optimization_profileopt	COMBO	Standard	Same widget as LLMScriptWriter -- controls 4-bit NF4 vs full precision.
measurement_labelopt	STRING		Optional tag for this run, stamped into the ledger entry. Useful when running the same node across multiple workflows or after VRAM-affecting config changes.

Name	Type	Description
report	STRING	—