Please suggest a combination of model and settings for LCM or LCM Lora

[andrewjswan]

I'm trying LCM and LCM Lora, but I can't seem to get a good result. Request:

A very cute, happy and playful Mythological symbol of the zodiac sign Virgo, looking amazed at the starry sky. whimsical illustration style, sharp focus, high detail, crisp image

But no matter how I play with the parameters, it doesn't work out very well.

Can anyone suggest a combination of model and settings to get at least something close to this:

[monstruosoft]

Judging by your image resolution, it seems like you're using SD 1.5, which is ancient at this point. I suggest switching at least to an SDXL-based model.

[andrewjswan]

I suggest switching at least to an SDXL-based model.

I've tried all the models included with the FastSD CPU except OpenVINO, and the results are all very poor.
Do you have a specific setup that would give good results? I'll try it out.

[monstruosoft]

Well, in that case I'd suggest using LCM-LoRA mode with any SDXL-based model from Civitai, for example. For SDXL you can increase the image resolution to 1024x1024 if your PC can handle it.
Also, note that SD and SDXL aren't very good at following natural language prompts as more recent models are, so you might have to change your prompt to better describe what you want; instead of asking for a representation of the zodiac symbol Virgo, ask for a blonde girl wearing a white dress standing on a field.

[andrewjswan]

I've tried all the LCM and LCM Lora options, but the results are still bad. Actually, very bad.

{
    "lcm_model_id": "stabilityai/sdxl-turbo",
    "openvino_lcm_model_id": "rupeshs/sd-turbo-openvino",
    "use_offline_model": false,
    "use_lcm_lora": true,
    "lcm_lora": {
        "base_model_id": "segmind/SSD-1B",
        "lcm_lora_id": "latent-consistency/lcm-lora-sdxl"
    },
    "use_tiny_auto_encoder": false,
    "use_openvino": false,
    "prompt": "A very cute, happy and playful Mythological symbol of the zodiac sign Virgo, playing happily in the sunbeams in a magical landscape during a beautiful sunset with warm golden light. sharp focus, high detail, crisp image",
    "negative_prompt": "",
    "strength": 0.6,
    "image_height": 512,
    "image_width": 512,
    "inference_steps": 3,
    "guidance_scale": 1.5,
    "clip_skip": 1,
    "token_merging": 0.0,
    "number_of_images": 1,
    "seed": 123123,
    "use_seed": false,
    "use_safety_checker": false,
    "diffusion_task": "text_to_image",
    "lora": {
        "weight": 0.5,
        "fuse": true,
        "enabled": false
    },
    "controlnet": null,
    "rebuild_pipeline": false,
    "use_gguf_model": false,
    "gguf_model": {
        "gguf_models": "/app/models/gguf",
        "diffusion_path": null,
        "clip_path": null,
        "t5xxl_path": null,
        "vae_path": null
    },
    "image_seeds": [
        684328090
    ]
}

[monstruosoft]

I meant to reply to this, you can see my answer below.

[monstruosoft]

The default models included in the FastSD settings, particularly the LCM models, are meant to show how SD works on low-end PCs but they don't give particularly good results. SD Turbo results are terrible, though it works fine for img2img or upscaling modes. SSD-1B is a distilled version of SDXL, meaning you can expect it to be worse than the base SDXL model.
Ideally you should download a fine-tuned model from Civitai, add the model path to the FastSD config file stable-diffusion-models.txt and use that model in LCM-LoRA mode.
Also, as I mentioned, you should probably revise your prompt to be more specific on what you want. For example, this is the result when using your prompt on one of the SDXL models I currently have on my PC, it's just an average SDXL model from Civitai:

And this is the result after changing the prompt to more closely describe what you want:

The results may be not great but aren't terrible either.

[andrewjswan]

Ideally you should download a fine-tuned model from Civitai, add the model path to the FastSD config file stable-diffusion-models.txt and use that model in LCM-LoRA mode.

Do these models need to be downloaded? Or can they be specified in a file (is there an example?) and FastSD CPU will download them automatically?

What are their memory requirements?

Also, as I mentioned, you should probably revise your prompt to be more specific on what you want. For example, this is the result when using your prompt on one of the SDXL models I currently have on my PC, it's just an average SDXL model from Civitai:

An excellent result, although a bit anime-style. It's quite good, though. What models did you use?

[monstruosoft]

FastSD will automatically download models from huggingface, but Civitai models have to be downloaded manually. For example, try adding this line to the file configs/stable-diffusion-models.txt:

stablediffusionapi/nova-anime-illustrous8

SDXL requires at least 16 GB of RAM for image generation.