Nodes/ComfyUI-SwissArmyKnife/LLM Studio Structured Describe (Video)
ComfyUI Node

LLM Studio Structured Describe (Video)

Structured JSON captions for video, straight from a local model

By sammykumar·Created 12 months ago·Updated 8 months ago· 6
LLM Studio Structured Describe (Video)
    • json_output
    • field_1
    • field_2
    • field_3
    • field_4
    • field_5
    • frames_processed
    base_urlhttp://192.168.50.41:1234
    model_nameqwen3-vl-8b-thinking-mlx
    video_path
    sample_rate2.0
    max_duration5.0
    schema_presetvideo_description
    promptAnalyze this video sequence and provide a detailed description following the schema.
    temperature0.2
    verbosefalse

    The image version of this node gives you clean structured JSON from one picture. LLM Studio Structured Describe (Video) is the same promise for video: it samples frames from a clip, runs them through a local vision model with JSON-schema constrained output, and hands you guaranteed-valid JSON - plus the individual fields pulled out and a count of how many frames it actually processed.

    Inputs split into "where's the video" and "how should the model behave." video_path is a path to a video file, and sample_rate + max_duration control how much of it gets analyzed - the same framing as the other video nodes: frames per second to extract (default 2.0) and a maximum duration in seconds (default 5), so a long clip doesn't balloon into a slow, expensive sampling run. Then base_url and model_name (default qwen3-vl-8b-thinking-mlx) point at your LM Studio server, schema_preset picks the JSON schema (video_description, simple_description, or character_analysis), and prompt is the instruction text. temperature defaults low at 0.2, and verbose toggles logging.

    Outputs: json_output (the full valid JSON as a string), field_1 through field_5 (individual fields sliced out per the preset - wire one straight into a prompt builder), and frames_processed (an INT telling you how many frames were actually sent, which is your sanity check that sampling behaved).

    Why reach for this over the free-form Video Describe? If the destination is any kind of automation - a logging pipeline, a prompt-builder that expects named fields, a batch job - structured output saves you a parsing step and the JSON-schema enforcement means the shape never breaks between runs. The free-form node is friendlier to read; this one is friendlier to build on. The smaller default model (8B vs 30B) also makes it the budget option for iterating.

    Installing

    Ships with Swiss Army Knife:

    • ComfyUI Manager: search "Swiss Army Knife", install, restart.
    • Manual:
    cd ComfyUI/custom_nodes
    git clone https://github.com/sammykumar/ComfyUI-SwissArmyKnife.git
    cd ComfyUI-SwissArmyKnife
    pip install -e .
    

    Gotchas

    • LM Studio needs JSON-schema support for the loaded model, same as the image variant. Schema errors mean an outdated server or a model without structured-output support.
    • sample_rate × max_duration is your cost dial. 2 fps × 5 seconds = 10 frames per video. More frames means slower runs and more tokens; start low and raise it only if the captions are missing detail.
    • frames_processed is your friend. If it's 0 or suspiciously low, the video path is wrong or the sampler didn't find frames - check the path before blaming the model.
    • Fields are sliced per preset, so change the preset and field_1..5 mean different things.

    This is the node for when you want video captions that slot into code or other nodes without ceremony. Local, structured, and no cloud key anywhere in the chain.

    CategorySwiss Army Knife 🔪/Media Caption

    Inputs (9)

    NameTypeDefaultDescription
    base_urlSTRINGhttp://192.168.50.41:1234LM Studio server URL (e.g. http://192.168.50.41:1234)
    model_nameSTRINGqwen3-vl-8b-thinking-mlxModel name in LM Studio (e.g. qwen3-vl-8b-thinking-mlx)
    video_pathSTRINGPath to video file to analyze
    sample_rateFLOAT2.00.1–30Frames per second to extract
    max_durationFLOAT5.01–60Maximum duration in seconds to sample from
    schema_presetCOMBOvideo_descriptionJSON schema preset to use for structured output
    promptSTRINGAnalyze this video sequence and provide a detailed description following the schema.Prompt for video analysis
    temperatureFLOAT0.20–2Temperature for text generation
    verboseBOOLEANfalseShow detailed processing information

    Outputs (7)

    NameTypeDescription
    json_outputSTRING
    field_1STRING
    field_2STRING
    field_3STRING
    field_4STRING
    field_5STRING
    frames_processedINT