Extensions/comfyui-tencent-hymt2
ComfyUI Extension

comfyui-tencent-hymt2

Run Tencent's state-of-the-art Hy-MT2 multilingual translation models directly inside ComfyUI with no HuggingFace cache pollution, models live cleanly inside your ComfyUI directory.

By mamorett·Created 3 months ago·Updated 3 months ago· 0
mamorett/comfyui-tencent-hymt2
Nodes—
On cloudLocal install
Stars0
Updated3 months ago
Readme

🌐 ComfyUI Hy-MT2 Translation Node

Run Tencent's state-of-the-art Hy-MT2 multilingual translation models directly inside ComfyUI — no HuggingFace cache pollution, models live cleanly inside your ComfyUI models/ directory.

Table of Contents

Overview

This custom node brings Tencent's Hy-MT2 family of large-language-model-based machine translation models into ComfyUI as a first-class node. Hy-MT2 models are instruction-tuned LLMs optimised specifically for high-quality multilingual translation, supporting dozens of language pairs out of the box.

Key features:

  • 🗂 Three model sizes — choose the right trade-off between speed and quality for your hardware.
  • 💾 Clean model storage — weights are downloaded once and stored under ComfyUI/models/hy-mt2/, never in ~/.cache/huggingface.
  • ⚡ In-process caching — once a model is loaded into VRAM it stays there for the lifetime of the ComfyUI session; switching workflows costs zero reload time.
  • 🔌 Fully composable — the node accepts and emits plain STRING values, so it chains naturally with prompt nodes, text display nodes, file loaders, and anything else in the ecosystem.
  • 🎛 Fine-grained generation control — temperature, top-p, top-k, repetition penalty, and max tokens are all exposed as node inputs.

Models

| Model ID | Parameters | Quantisation | VRAM (bf16) | Best for | |---|---|---|---|---| | tencent/Hy-MT2-1.8B | 1.8 B | none (bf16) | ~4 GB | Fast drafts, resource-constrained setups, CPU offload | | tencent/Hy-MT2-7B | 7 B | none (bf16) | ~15 GB | General-purpose high quality, recommended default | | tencent/Hy-MT2-7B-FP8 | 7 B | FP8 | ~8 GB | High quality at reduced VRAM — requires H100/H200 or RTX 40xx |

Note on FP8: The 7B-FP8 variant requires a GPU with native FP8 compute support (NVIDIA Hopper or Ada Lovelace architecture). On older GPUs it will either fail or silently fall back to a slower path. Use 7B (bf16) if unsure.

All three models are available on the Tencent HuggingFace organisation page.

Requirements

Python packages

| Package | Minimum version | Notes | |---|---|---| | torch | 2.1+ | CUDA build strongly recommended | | transformers | 4.40+ | Required for apply_chat_template | | accelerate | 0.27+ | Required for device_map="auto" | | sentencepiece | any | Tokeniser dependency |

Install all at once inside your ComfyUI Python environment:

pip install transformers accelerate sentencepiece

If you use the ComfyUI portable/standalone build on Windows, prefix pip with the path to the embedded Python interpreter, e.g.:
.\python_embeded\python.exe -m pip install transformers accelerate sentencepiece

Hardware

| Config | Recommended model | |---|---| | GPU ≥ 16 GB VRAM | Hy-MT2-7B | | GPU 8–15 GB VRAM | Hy-MT2-7B-FP8 (Hopper/Ada) or Hy-MT2-1.8B | | GPU < 8 GB VRAM | Hy-MT2-1.8B with CPU offload via device_map="auto" | | CPU only | Hy-MT2-1.8B — slow but functional |

Installation

Option A — Manual (recommended)

  1. Clone or download this repository into your ComfyUI custom nodes folder:
cd ComfyUI/custom_nodes/
git clone https://github.com/your-repo/comfyui-hy-mt2-translation hy_mt2_translation

Or, if you downloaded a ZIP, unzip it so the folder looks like:

ComfyUI/custom_nodes/hy_mt2_translation/
├── __init__.py
├── hy_mt2_translation_node.py
└── README.md

  1. Install the Python dependencies (see Requirements).

  2. Restart ComfyUI.

Option B — ComfyUI Manager

If you use ComfyUI Manager, search for "Hy-MT2 Translation" in the custom node browser and click Install.

File Structure

hy_mt2_translation/           ← the custom node package folder
├── __init__.py               ← ComfyUI entry point; exports NODE_CLASS_MAPPINGS
├── hy_mt2_translation_node.py ← all node logic lives here
└── README.md                 ← this file

After first use, ComfyUI's models directory will contain:

ComfyUI/
└── models/
    └── hy-mt2/
        ├── Hy-MT2-1.8B/      ← weights + tokeniser (downloaded on first use)
        │   ├── config.json
        │   ├── tokenizer.json
        │   ├── tokenizer_config.json
        │   └── model.safetensors (or shards)
        ├── Hy-MT2-7B/
        └── Hy-MT2-7B-FP8/

Model Storage

One of the most important design decisions in this node is where model weights live.

By default, HuggingFace's from_pretrained() stores everything in ~/.cache/huggingface/hub/ — a hidden folder that fills up silently and is separate from ComfyUI's own model management. This node deliberately avoids that:

How it works

  1. HF_HOME override — before any network call, the node sets HF_HOME to models/hy-mt2/.hf_home. This redirects all HuggingFace library internals away from your home directory.
  2. Direct save — after downloading, tokenizer.save_pretrained() and model.save_pretrained() write a clean, self-contained snapshot to models/hy-mt2/<ModelName>/.
  3. Presence detection — on subsequent ComfyUI starts, the node checks for .safetensors/.bin/.json files in the target directory. If found, it loads with local_files_only=True — no network call, no HuggingFace infrastructure involved.
  4. ComfyUI root detection — the node uses folder_paths.base_path (ComfyUI's own path registry) to find the installation root, with a filesystem-walk fallback in case the import fails outside a ComfyUI context.

Result

~/.cache/huggingface/    ← never touched by this node ✅
ComfyUI/models/hy-mt2/  ← all weights live here ✅

You can move, backup, or delete individual model folders just like any other ComfyUI model.

Node Reference

The node appears in ComfyUI under the category text / translation with the display name Hy-MT2 Translation (Tencent).

Inputs

| Input | Type | Required | Default | Description | |---|---|---|---|---| | text | STRING (multiline) | ✅ Yes | (empty) | The source text you want to translate. Accepts any length up to the model's context window. | | model | Dropdown | ✅ Yes | Hy-MT2-1.8B | Which model variant to use. See Model Comparison. | | instruction | STRING (multiline) | No | "Translate the following text into Chinese:" | The task instruction prepended to the source text. Changing this is how you control the target language — see Switching Target Languages. | | max_tokens | INT | No | 4096 | Hard cap on the number of new tokens the model may generate. Range: 64–8192. | | temperature | FLOAT | No | 0.7 | Sampling temperature. 0.0 = fully deterministic greedy decoding. Higher values increase variety. | | top_p | FLOAT | No | 0.6 | Nucleus sampling: restrict sampling to the smallest set of tokens whose cumulative probability exceeds this value. | | top_k | INT | No | 20 | Restrict sampling to the top-K most likely tokens at each step. 0 disables this filter. | | repetition_penalty | FLOAT | No | 1.05 | Penalise tokens that have already appeared in the output. 1.0 = no penalty. Values above 1.0 reduce repetition. |

Outputs

| Output | Type | Description | |---|---|---| | translated_text | STRING | The model's translation, with leading/trailing whitespace stripped. |

Generation Parameters

The default values are the ones recommended by Tencent in the model card:

{
  "temperature": 0.7,
  "top_p": 0.6,
  "top_k": 20,
  "repetition_penalty": 1.05,
  "max_tokens": 4096
}

They represent a good balance between output quality and diversity. See Parameter Tuning Guide for when and how to change them.

Usage Examples

Basic Translation

The simplest possible setup:

[Text Multiline Node] ──► text
                          model   = tencent/Hy-MT2-1.8B
                          (leave everything else at defaults)
                                          │
                                          ▼
                               [Show Text Node] ◄── translated_text
  1. Add Hy-MT2 Translation (Tencent) from the node menu (text → translation).
  2. Connect a Text Multiline node to the text input and type your source text.
  3. Connect the translated_text output to a Show Text or Display Any node.
  4. Queue the prompt. On first run the model downloads automatically; subsequent runs load instantly from disk.

Chaining with Other Nodes

Because both input and output are plain STRING values, the node fits naturally anywhere in a text pipeline:

[Load Text File] ──► text ──► [Hy-MT2 Translation] ──► translated_text ──► [Save Text File]

[CLIPTextEncode result (as string)] ──► text ──► [Hy-MT2 Translation] ──► translated_text ──► [CLIPTextEncode]

This last pattern is useful when you have a prompt in English and need a Chinese-language CLIP encoding for a Chinese diffusion model.

Switching Target Languages

The instruction input is free-form text. Simply describe what you want the model to do:

| Goal | instruction value | |---|---| | Translate to English | Translate the following text into English: | | Translate to Chinese | Translate the following text into Chinese: | | Translate to French | Translate the following text into French: | | Translate to Japanese | Translate the following text into Japanese: | | Translate to Spanish | Translate the following text into Spanish: | | Translate to German | Translate the following text into German: | | Translate to Arabic | Translate the following text into Arabic: | | Formal register | Translate the following text into formal English, preserving all technical terminology: | | Subtitle style | Translate the following text into English in a concise, subtitle-friendly style: |

The Hy-MT2 models are instruction-tuned, so they respond well to detailed natural-language instructions.

Tip: You can drive the instruction input from another node (e.g. a dropdown primitive or a text node) to make language selection a dynamic workflow parameter.

Parameter Tuning Guide

Temperature

Controls randomness. The model picks the next token by sampling from a probability distribution; temperature scales that distribution.

| Value | Effect | Use when… | |---|---|---| | 0.0 | Greedy / fully deterministic | You need reproducible, single-best output | | 0.3–0.5 | Low variance, conservative | Technical or legal text where precision > variety | | 0.7 (default) | Balanced | General-purpose translation | | 1.0+ | High variance, creative | Experimental or stylistic outputs |

When temperature is set to 0.0, the node automatically disables sampling (do_sample=False) and ignores top_p and top_k, which prevents a HuggingFace warning.

top_p (Nucleus Sampling)

Restricts sampling to a "nucleus" of tokens that together account for at least top_p of the probability mass. Tokens outside the nucleus are never chosen.

| Value | Effect | |---|---| | 1.0 | No restriction (all tokens eligible) | | 0.6 (default) | Moderate filtering — a good middle ground | | 0.3 | Aggressive filtering — only the most likely tokens |

top_k

Restricts sampling to the top-K most probable tokens at each step, regardless of their cumulative probability.

| Value | Effect | |---|---| | 0 | Disabled | | 20 (default) | Only the 20 most likely tokens are candidates | | 1 | Equivalent to greedy decoding |

top_p and top_k are applied together: a token must pass both filters to be eligible.

repetition_penalty

Reduces the probability of tokens that have already appeared. Particularly useful for long translations where the model might loop.

| Value | Effect | |---|---| | 1.0 | No penalty | | 1.05 (default) | Mild penalty — recommended for translation | | 1.2+ | Strong penalty — may distort output if set too high |

max_tokens

Hard ceiling on output length. Translation output is usually shorter than input, so 4096 is generous for most documents. Reduce to 512 or 1024 to limit cost on large batches.

Model Comparison

| | Hy-MT2-1.8B | Hy-MT2-7B | Hy-MT2-7B-FP8 | |---|---|---|---| | Quality | Good | Excellent | Excellent | | Speed (A100) | ~3× faster than 7B | Baseline | ~1.5× faster than 7B | | VRAM | ~4 GB | ~15 GB | ~8 GB | | FP8 GPU required | No | No | Yes (H100/H200/RTX 40xx) | | Best for | Drafts, edge devices, CPU | Production use | Production use, limited VRAM |

Recommended starting point: Hy-MT2-1.8B if you are unsure, then upgrade to Hy-MT2-7B if quality isn't sufficient.

First-Run Behaviour & Downloading

On the very first run after selecting a model, the node will:

  1. Print [HyMT2] Downloading 'tencent/Hy-MT2-<size>' → <path> to the ComfyUI console.
  2. Download all model shards from HuggingFace Hub. File sizes:
    • 1.8B — ~3.6 GB
    • 7B — ~14 GB
    • 7B-FP8 — ~7 GB
  3. Save a clean snapshot to ComfyUI/models/hy-mt2/<ModelName>/.
  4. Print [HyMT2] Ready: tencent/Hy-MT2-<size> when loading is complete.

On all subsequent runs (even after restarting ComfyUI), the node loads entirely from disk with local_files_only=True. The download happens exactly once per model per machine.

Slow download? HuggingFace Hub supports resuming interrupted downloads. Simply re-queue the prompt; the node will resume from where it left off (the partial files remain in the model directory).

Want to pre-download before your session? You can trigger the download outside ComfyUI by running:

python -c "
from transformers import AutoModelForCausalLM, AutoTokenizer
import torch, pathlib
out = pathlib.Path('ComfyUI/models/hy-mt2/Hy-MT2-1.8B')
out.mkdir(parents=True, exist_ok=True)
tok = AutoTokenizer.from_pretrained('tencent/Hy-MT2-1.8B', trust_remote_code=True, cache_dir=str(out))
mdl = AutoModelForCausalLM.from_pretrained('tencent/Hy-MT2-1.8B', torch_dtype=torch.bfloat16, trust_remote_code=True, cache_dir=str(out))
tok.save_pretrained(str(out)); mdl.save_pretrained(str(out))
print('Done.')
"

Performance Tips

GPU memory:

  • If you run a diffusion model and this translation node in the same workflow, the LLM may compete for VRAM. Consider running translation in a separate workflow pass, or use the 1.8B model which fits alongside most diffusion models on 24 GB GPUs.
  • device_map="auto" will automatically split layers across GPU and CPU RAM if VRAM is insufficient. This is slower but allows models to run on cards with less memory than the full bf16 footprint.

Batching: The current node processes one text per execution. For large-volume translation workflows, consider splitting text into chunks upstream (e.g. with a text-splitting node) and using ComfyUI's batch/loop primitives.

Model warm-up: The first inference after loading is slightly slower than subsequent ones due to PyTorch kernel compilation. This is normal.

CPU-only mode: If no CUDA GPU is available, device_map="auto" will place the model on CPU. The 1.8B model takes roughly 30–120 seconds per translation on a modern CPU. Not recommended for production use.

Troubleshooting

ModuleNotFoundError: No module named 'transformers'

The transformers package is not installed in your ComfyUI Python environment.

pip install transformers accelerate sentencepiece

On Windows portable builds:

.\python_embeded\python.exe -m pip install transformers accelerate sentencepiece

CUDA out of memory

The selected model requires more VRAM than is available.

  • Switch to Hy-MT2-1.8B.
  • If already on 1.8B, try closing other applications using the GPU.
  • device_map="auto" will offload layers to CPU RAM automatically, but this is slower.

The model downloads to the wrong location

Check that folder_paths is importable (i.e. the node is running inside a proper ComfyUI process, not a standalone Python script). The HF_HOME override only takes effect if set before the first HuggingFace import in the process — if another node or extension imported transformers first with a different HF_HOME, the override may not apply. In that case, set HF_HOME as a system environment variable pointing to ComfyUI/models/hy-mt2/.hf_home before starting ComfyUI.

Translation output is empty

  • Check the ComfyUI console for error messages.
  • Make sure the text input is not empty or whitespace-only (the node returns "" silently in that case by design).
  • Try reducing max_tokens to a smaller value if the model is running out of memory mid-generation.

trust_remote_code warning in console

This is expected. The Hy-MT2 models include custom tokeniser/model code that HuggingFace loads from the downloaded repository. The warning is informational; the node deliberately passes trust_remote_code=True as required by these models.

Slow loading every time (model redownloaded each start)

The node caches models in-process (within a single ComfyUI session). It does not persist across ComfyUI restarts in memory — but it will load from local disk rather than re-downloading. If you see the download step on every restart, check that the model files were correctly saved to ComfyUI/models/hy-mt2/<ModelName>/ and that the directory contains .safetensors or .bin files.

FAQ

Q: Can I use this node without a GPU?
A: Yes. device_map="auto" will place the model on CPU if no CUDA device is found. Inference will be significantly slower (minutes rather than seconds), but functional. The 1.8B model is the only practical choice for CPU-only setups.

Q: Which languages are supported?
A: Hy-MT2 is a general-purpose multilingual model trained on a wide range of language pairs. The instruction-tuned format means you simply specify the target language in the instruction field. Performance is best for high-resource language pairs (Chinese ↔ English, European languages) and may degrade for very low-resource languages.

Q: Can I use the 30B model (tencent/Hy-MT2-30B-A3B)?
A: The node currently supports the three recommended consumer-grade variants. The 30B model requires substantially more VRAM (60+ GB in bf16) and is intended for server deployments. You can add it to SUPPORTED_MODELS in hy_mt2_translation_node.py if your hardware supports it — the download and load logic will work identically.

Q: Is the instruction field a system prompt?
A: Functionally, yes — it is prepended to the source text and sent as the user turn in a chat template, giving the model context about what to do. It is not passed as a separate system message, which is consistent with the reference inference example from Tencent's model card.

Q: Why does the node use bfloat16 instead of float16?
A: bfloat16 has the same memory footprint as float16 but a much larger dynamic range (matching float32), which is important for LLM inference stability. It is the format used in Tencent's own reference code and is natively supported on all modern NVIDIA GPUs (Ampere and later). On older GPUs that do not support bf16, PyTorch will automatically fall back to fp32 for unsupported operations.

Q: Can I run two different models in the same workflow?
A: Yes. Each unique model ID gets its own entry in the in-process cache, so both models stay loaded simultaneously. Be mindful of combined VRAM usage.

Q: Is internet access required after the first download?
A: No. Once a model is saved to ComfyUI/models/hy-mt2/, the node loads it with local_files_only=True and makes no network calls.

Credits & License

Model: Hy-MT2 by Tencent. Please refer to Tencent's model card and licence terms before using the weights in production or commercial settings.

Node: Community contribution. The node code itself is released under the MIT License — see LICENSE for details.

Dependencies:

Made with ❤️ for the ComfyUI community.