Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

LoRA loaded successfully but the effect wasn't applied #28

Open
EntroSanity opened this issue Oct 22, 2024 · 2 comments
Open

LoRA loaded successfully but the effect wasn't applied #28

EntroSanity opened this issue Oct 22, 2024 · 2 comments

Comments

@EntroSanity
Copy link

EntroSanity commented Oct 22, 2024

I was testing the pipeline's capability to apply a Flux LoRA effect, but I couldn't achieve the desired effect in my generated images. I experimented with two different methods. Both methods successfully displayed a "LoRA successfully loaded" message; however, the generated images did not reflect the applied LoRA effect. Could I get some assistance in case I'm missing any crucial steps?

  1. Using /lora endpoint:
import requests
import json

# Load the LoRA
lora_data = {
    "path": "path-to-my-lora",
    "action": "load",
    "scale": 1.0,  # Adjust this value as needed,
    "name": None  # Include this even if it's None
}

response = requests.post("http://localhost:8088/lora", json=lora_data)
print(response.json())
  1. Using flux pipeline:
from flux_pipeline import FluxPipeline

config_path = "path/to/config/file.json"
config_overrides = {
    #...
}
lora_path = "path/to/lora/file.safetensors"
pipeline = FluxPipeline.load_pipeline_from_config_path(config_path, **config_overrides)
pipeline.load_lora(lora_path, scale=1.0)
## 
run inference
@aredden
Copy link
Owner

aredden commented Oct 23, 2024

I have actually been thinking that I've implemented the qkv loras a little incorrectly, which I do plan to fix, though requires a bit of an overhaul since I have to keep track of exactly which section of the fused qkv weight to apply unfused lora weights to. Though also, are you using the current version? There was a pretty significant improvement in lora implementation that I did recently (aka like two weeks ago or so).

@EntroSanity
Copy link
Author

I initially believed that the LoRA scale was limited to a range of 0 to 1, but I noticed that within this range, it didn't seem to affect the generated image. However, when I tried using a scale larger than 1, it successfully applied the LoRA effect to the image.

lora_data = {
    "path": "path-to-my-lora",
    "action": "load",
    "scale": 1.5,  # Adjust this value as needed,
    "name": None  # Include this even if it's None
}

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants