Prompt Engineering

Prompt engineering plays an important role in generating quality images using Stable Diffusion via ComfyUI.

Positive Prompt Example

An example of a positive prompt used in image generation:

Weighted Terms in Prompts

• Placing words into parentheses and assigning weights alters their impact on the prompt.
• Examples: (word:1.2) increases the effect by 1.2, (word:0.9) slightly decreases the effect, and (word) is equivalent to (word:1.1).

Scene and Dialogue Examples

• Example: (cute:1.4) can be used to emphasize cuteness in an image.
• However, high weights like 1.4 may cause issues in the generated image.

Keep Prompts Simple

• Complexity in prompts can lead to conflicting terms and affect image quality.
• Stable Diffusion struggles with generating images that respect contradicting prompts.

Checkpoint Dependency

• Prompt effectiveness varies with different checkpoints.
• A prompt effective with one checkpoint may yield poor results with another.

Negative Prompts

Negative prompts helps in guiding Stable Diffusion to generate images according to user preferences.

Overview

Negative prompts are employed to convey to Stable Diffusion what elements should be avoided in the generated image. Unlike positive prompts that emphasize desired features, negative prompts help exclude specific aspects from the final result.

Negative Prompt Example

An example of a negative prompt is presented below:

Elements in Negative Prompts

The negative prompt typically includes elements the user wishes to avoid in the generated image. For instance, the inclusion of "(hands)" in the negative prompt aims to address the issue of anime models creating an excess of hands.

Vagueness in Negative Prompts

Stable Diffusion may struggle with vague concepts in negative prompts. Unlike positive prompts where specific features are emphasized, vague terms like "(bad image)" may not be effectively interpreted by the algorithm.

Algorithmic Treatment of Negative Prompts

During the sampling process, the negative prompt is treated similarly to the positive prompt. The algorithm calculates the noise predicted by the positive prompt and subtracts the noise predicted with the negative prompt. This subtraction mechanism works effectively for elements that can be subtracted from an image.

Importance of Negative Prompts

Negative prompts are as significant as positive prompts in influencing the image generation process. Experimenting with both positive and negative prompts allows users to fine-tune the generated images according to their preferences.