ꘇ mediamime

Somewhere in a box of ideas from 1981 there's Laurie Anderson wearing a drum suit, every hit on her body a sound. ꘇ mediamime is that, rebuilt in ComfyUI: draw zones on a schematic body, connect a live perception stream, press play, and the performer's limbs strike the zones - firing events, MIDI, OSC, WebSocket messages and note channels that play instruments with no MIDI hardware.

It makes no images and runs no model: it's an instrument panel wired to somebody else's landmarks.

Two boards, and what each one is for

The editor is Poliplan, and it gives you two boards. The stage board is the live image - connect the provider's clean image output - with the skeleton and hitter points over it, plus the body outline if you connect a segmentation mask. Marks drawn here live in normalized image coordinates and stay put relative to the frame.

The body board is a homunculus: a schematic figure facing you, performer's left on your right. Marks here anchor to the nearest bone frame - head, torso, upper arm, forearm, hand, thigh, shin, foot - and get re-placed on the live skeleton every frame. That's the whole trick. A shape on the homunculus becomes a zone that travels with the performer, dashed out on the stage board as the body moves: draw once, and it stays on the chest instead of on a rectangle of screen.

Closed shapes are zones, open strokes are tripwires, points are targets. Each mark becomes a region with its own hitters, triggers and action list, edited in a side panel.

Hitters, triggers, values - the three dials that matter

Hitters are the landmarks that can strike a region: presets for fingers, wrists, hands, feet, head, whole pose, or your own selectors. There's a thoughtful rule buried here - a body zone ignores hitters from its own limb, so the left forearm can only be struck by something else.

Triggers are per action: enter, exit, enter + exit (how you get note-on/off and latching CC), while inside (retriggered every ~120 ms), and cross for tripwires. Cooldown swallows a too-early re-entry; hold requires time inside. A lost stream releases every active zone - notes off, no screaming drone - while a paused source holds state.

Values are what an action sends: speed (how hard the strike was), normX/normY (position within the zone), distance from centre, coverage, count, x/y, or a constant.

The action is one of five kinds: a __ event for Livecode, a MIDI note, a MIDI CC, an OSC or WebSocket message. Every hit also emits a catch-all event named after publish (default mime) carrying region, x, y, speed and hitter.

Outputs, and the no-hardware trick

channels is a GENERETI_CHOP: per-region .active, .x, .y, .speed, .coverage, .count, .hits, plus held notes. events is a GENERETI_DAT of recent entries, exits and crossings. overlay is an IMAGE of zones and hits on transparency, so you can project it over the performer instead of watching your screen.

The part that saves you a shopping trip: notes are published as chC.noteN channels whether or not any MIDI device is connected. Wire channels into mod.drumkit or mod.synth and the body plays them directly; chop.select with ch10.* and ch1.* splits drums from synth. Web MIDI is opt-in via the MIDI button, OSC is armed explicitly.

Inputs

stream (default body) must match a running provider. publish names the catch-all event stream. min_confidence and max_age_ms gate stale and shaky detections. score holds your boards, regions and actions - ↓/↑ export and import it. The optional image is the clean frame under the stage board; the optional mask draws the body outline and enables the "Body (segmentation)" hitter, which fires when the masked body covers more than a threshold fraction of a zone.

Installing it

Manager → search Genereti → install → restart → hard-refresh the tab. Manual:

cd ComfyUI
git clone https://github.com/languel/genereti.git custom_nodes/genereti
./.venv/bin/python -m pip install -r custom_nodes/genereti/requirements.txt

That file is basically one line - comfyui-frontend-package>=1.53.6 - since torch, numpy, pillow and aiohttp come from ComfyUI. Perception models are a separate, explicit browser download into models/genereti/browser-perception.

Where people get burned

Nothing fires. There's a ▶ in the node; nothing evaluates until you press it, and a provider that never started is the same class of mistake.

No MIDI output. Web MIDI needs a browser that has it and a port to talk to; the node's error lists what it found ("No MIDI outputs. Enable an IAC bus or connect a device."). You need no MIDI at all if you route channels into the pack's instruments. OSC is loopback only - 127.0.0.1, so a phone on your LAN will never see it.

A body zone never triggers. Body marks need their bone's two landmarks every frame. If that limb isn't detected or sits below min_confidence, the zone has nowhere to be. Same when the performer leaves frame.

You queued it and got an error. mediamime raises on queue - it evaluates browser Live state, so there's no headless path. Press ▶ and keep the tab open. Related: the score is saved in the workflow, so an unsaved workflow has none. Export the JSON when a score is worth keeping.

The pack calls itself super alpha, early classroom testing, interfaces changing. Its demo ("ꘇ-Perception-08-Mediamime-Body-Drum") is the best starting point - and back up workflows before updating.