Lucy Edit - ComfyUI

<p align="center"> <img src="assets/logo.png" width="680" alt="Lucy Edit Dev Logo"/> </p> <p align="center"> 🧪 <a href="http://github.com/DecartAI/lucy-edit"><b>GitHub</b></a> &nbsp;|&nbsp; 🤗 <a href="https://huggingface.co/decart-ai/Lucy-Edit-Dev">Huggingface</a> &nbsp;|&nbsp; 📖 <a href="https://platform.decart.ai">Playground</a> &nbsp;|&nbsp; 📑 <a href="#">arXiv (Coming soon)</a> &nbsp;|&nbsp; 💬 <a href="https://discord.gg/decart">Discord</a> </p>

---

<img width="2559" height="812" alt="image" src="https://github.com/user-attachments/assets/291f41d2-f4a4-4d36-a0cf-f73a05fd0a0c" /> <div align="center"> <table> <tr> <td align="center"> <video src="https://github.com/user-attachments/assets/5084db41-be23-47a2-97a2-4f6bf7229809" width="100%" controls> Your browser does not support the video tag. </video> <br/> <em>Put the woman in gothic black jeans and leather jacket and crop top under it.</em> </td> <td align="center"> <video src="https://github.com/user-attachments/assets/f72e58e1-f00b-45a7-a2d4-28bea2aad11c" width="100%" controls> Your browser does not support the video tag. </video> <br/> <em>1.2) Put her in a clown outfit.</em> </td> <td align="center"> <video src="https://github.com/user-attachments/assets/51263d11-66e9-4bdc-a41d-b59ee628332d" width="100%" controls> Your browser does not support the video tag. </video> <br/> <em>1.3) Put the woman in a red bikini with an open thick coat above it.</em> </td> </tr> </table> </div>

Lucy Edit is a video editing model that performs instruction-guided edits on videos using free-text prompts — it supports a variety of edits, such as clothing & accessory changes, character changes, object insertions, and scene replacements while preserving the motion and composition perfectly.

• 🏃‍♂️ Motion Preservation - preserves the motion and composition of videos perfectly, allowing precise edits.
• 🎯 Edit reliability — edits are more robust when compared to common inference time methods.
• 🧢 Wardrobe & accessories — change outfits, add glasses/earrings/hats/etc.
• 🧌 Character Changes — replace characters with monsters, animals and known characters. (e.g., "Replace the person with a polar bear")
• 🗺️ Scenery swap — move the scene (e.g., "transform the scene into a 2D cartoon,")
• 📝 Pure text instructions — no finetuning, no masks required for common edits

---

🛠️ Quickstart

Installation

1. Clone this repo into custom_nodes folder.
2. Install dependencies: pip install -r requirements.txt

Usage

1. Load the workflow from examples/basic-api-lucy-edit.json.
2. Get an api key from: https://platform.decart.ai/ - You will get 5000 free credits.

Please refer to the "Prompting Guidelines & Supported Edits" section for the best experience.

🎬 Demos

<div align="center"> ### Sample 1 <table> <tr> <td align="center"> <video src="https://github.com/user-attachments/assets/0ac94178-ce03-4e9d-9326-676fe6146bc6" width="100%" controls> Your browser does not support the video tag. </video> <br/> <em>1.1) Replace the man with an alien wearing the same leather jacket.</em> </td> <td align="center"> <video src="https://github.com/user-attachments/assets/78275b81-04b4-4ee7-afa2-79fdcf54b688" width="100%" controls> Your browser does not support the video tag. </video> <br/> <em>1.2) Replace the man witha polar bear.</em> </td> <td align="center"> <video src="https://github.com/user-attachments/assets/3ad89caa-8b89-4322-a1ef-e92df45c907a" width="100%" controls> Your browser does not support the video tag. </video> <br/> <em>1.3) Make it snow.</em> </td> </tr> </table>

Sample 2

<table> <tr> <td align="center"> <video src="https://github.com/user-attachments/assets/443c36a8-dfc9-4a11-8873-4ed4985753ee" width="100%" controls> Your browser does not support the video tag. </video> <br/> <em>2.1) Replace the woman with Harley Quinn with full make up and a shirt with "Daddy's Lil Monster" written on it.</em> </td> <td align="center"> <video src="https://github.com/user-attachments/assets/e9654e91-e0f4-479e-8632-d567178ea72f" width="100%" controls> Your browser does not support the video tag. </video> <br/> <em>2.2) Replace the girl with a lego character.</em> </td> <td align="center"> <video src="https://github.com/user-attachments/assets/b58fcac9-2095-4686-9123-916250ccb9e3" width="100%" controls> Your browser does not support the video tag. </video> <br/> <em>2.3) Change the shit to a Manchester United jersey.</em> </td> </tr> </table> </div>

Note: The prompts above are not enriched, the model will react better to enriched prompts - as described in the prompt guideline section below.

---

🧭 Roadmap

• ✅ API based custom nodes.
• [ ] local inference custom nodes.

🔥 Latest News

• [2025-09-17]: Initial Lucy Edit Dev weights & reference code released.
• [2025-09-16]: Diffusers integration PR Merged. <a href="https://github.com/huggingface/diffusers/pull/12340">PR #12340</a>.
• [2025-09-16]: Diffusers integration PR opened. <a href="https://github.com/huggingface/diffusers/pull/12340">PR #12340</a>.

---

Prompting Guidelines & Supported Edits

When creating prompts for edits, always use the correct trigger word and provide 20–30 words of detail about the change.
The right phrasing and specificity significantly improve results.

Trigger Words

• Change → Clothing or color modifications
• Add → Adding animals or objects
• Replace → Human replacement (VACE) or object substitution
• Transform to → Global scene transformations

Edit Types

1. Clothing Changes

• Works well.
• Example: “Change the shirt to a kimono.”

2. Color Changes

• Mixed results (sometimes subtle, sometimes too bold).
• Example: “Change the shirt color to red.”

3. Human Replacement (VACE)

• Strong performance.
• Avoid vague phrasing like “Replace me with…”.
• Example: “Replace the man with a tiger, with fuzzy striped fur and bright green eyes.”

4. Add Animal

• Usually works well.
• Example: “Add a small parrot with green feathers sitting on the person’s shoulder.”
• Very out-of-distribution animals (e.g., swarms of bees) may fail unless extra detail is provided.

5. Add Object / General Add

• Often “glues” items to the person.
• Example: “Add a golden crown on the person’s head, with ornate jewels.”

6. Replace Object

• Good for structure-preserving swaps.
• Example: “Replace the apple with a crystal ball.”

7. Global Transformations

• Scene-wide changes work well, depending on complexity.
• Example: “Transform the desert landscape into a snowy, frozen tundra.”

8. VFX (fire, falling leaves, etc.)

• Not reliable.
• Current limitations prevent consistent results.

Additional Notes

• Prompts with ~20–30 descriptive words perform best.
• The model can generate NSFW content too easily — ensure moderation filters catch unwanted outputs.
• Generations at 81 frames look better compared to 29 frames.

---

📬 Contact

• GitHub Issues: <a href="http://github.com/DecartAI/Lucy-Edit-ComfyUI">DecartAI/lucy-edit</a>.
• Discord: Join our discord server, <a href="https://discord.gg/decart">here</a>.