The guidance scale parameter controls how strongly the generation process is guided towards the prompt. A value of 0 is equivalent to no guidance, and the image is completely unrelated to the prompt (it can however, still be an interesting image). Increasing the guidance scale increases how closely the image resembles the prompt, but reduces consistency and tends to lead to an overstaturated and oversharpened image.
The noise predictor (unet) excepts a prompt, and it will generate images similar to that prompt (this is known as conditioning).
(stable-diffusion)
TLDR: The step count has very litle effect on image content, and increasing it has rapidly deminishing returns over 20 steps.
A lot of people run stable diffusion with very high step counts, like 100 or 200, but is this neccicary to produce good images? No. Here a a bunch of images generated with stable diffusion (K-LMS scheduler), with the same seed and prompt, with different step counts:
Prompt 5 steps 10 steps 20 steps 40 steps 80 steps 160 steps A fantasy landscape.
(AI, ML and if statements )
(stable-diffusion)
You have probably seen smooth animations made with stable diffusion like this: https://www.youtube.com/watch?v=Bo3VZCjDhGI But how was it done?
The answer is very simple, by running multiple prompts on the same image, the noise predictions can be combined to create an intermediate image. Changing the relative weights of the prompts, but keeping the initial latents (seed value) the same, a series of intermediate images can be created.
Starting prompt: “A fantasy landscape” end prompt: “A sprawling cityscape”, seed: 42
(AI, ML and if statements )
(stable-diffusion)
If you have not read the last post, I would highly recommend you read it first.
Full disclosure, I will be using pre-trained models from the internet for unet, CLIP TextEncoder, and vae.
Environment A cuda comparable (read NVIDIA) GPU will perform best, but a reasonably modern CPU with more than 8 GB of ram should work fine, as long as you are prepared to wait.
Here are all the python libraries I used, these can be installed with pip.
(AI, ML and if statements )
(stable-diffusion)
Stable Diffusion is a newish (Dec 20, 2021) image generation model. It can generate quickly generate highly detailed images with a concise text prompt:
Prompt: “A photograph of an astronaut riding a horse”:
It does this while being much faster and smaller than OpenAI’s DALL-E. Better yet, it it open source, meaning you can not only run it on your own hardware with out restrictions (It even performs decently on a CPU!
(AI, ML and if statements )
(stable-diffusion)