ComfyUI Node
Tokenize
A ComfyUI node in ComfyDL/NLP Utils with 2 inputs and 1 output.
Tokenize
- tokens_str
◄textthe quick brown fox
jumps over the lazy dog►
◄token_modeword►
CategoryComfyDL/NLP Utils
Inputs (2)
| Name | Type | Default | Description |
|---|---|---|---|
| text | STRING | the quick brown fox jumps over the lazy dog | — |
| token_mode | COMBO | word | 2 options: word, char |
Outputs (1)
| Name | Type | Description |
|---|---|---|
| tokens_str | STRING | — |