Humble request for commented ipynb notebook with example face training

[AlongForTheR1de]

Hi folks,

I've been following all your awesome developments closely and have managed to get the basic lora training running with a few modifications of the kohya_ss repo to allow it to run in Linux. My outputs were not stellar after multiple attempts, but there are so many different ways I could be failing, and the methods available there are naturally lagging a bit behind the latest and greatest.

For a basically literate but not experienced person like me, it would be amazing if someone could but together a well commented Jupyter notebook that walks through implementing the best training methods you guys have arrived at thus far. If it would be possible to make the training dataset available so we could actually run through it in a tutorial style, that would be fantastic. No need for a video or anything, just a script that starts with training data, produces the lora network, then applies the network to a txt2img prompt.

Also -- it would be wonderful if this could include whatever the stand-alone command is to convert .pt to .safetensors format in order to make it possible use the final output in the Auto1111 extension. I've figured out that I can save things as a safetensor using --save_model_as=safetensors in the kohya train_network.py script, but for the life of me I cannot figure out what this ends up activating as their code comments are a bit sparse.

Thanks!!

[cian0]

I created one here:

https://github.com/cian0/lora/tree/master/training_scripts (see the notebook file, accompanying captioned training images and safetensors output)

@brian6091 and @cloneofsimo was actively assisting me to understand how to use the repo so I think we're on the same boat. Simu Liu was chosen as subject as I was having a hard time getting new concepts to train but was able to get it to work now. See output images:

[FunWithFaces]

Hey thanks so much for posting this! I'm still trying to get things working without OOM on 12GB, but once I do I will write up a step by step guide to get this running from nothing for people even more clueless than myself :)

[FunWithFaces]

OK, here is the short version of how I, a clueless person, managed to get this working with 12GB VRAM. This is the lazy version piggy backing off of the auto1111 venv, but it would probably be better to keep it in a separate venv tbh. If you know what you are doing at all, probably just install torch/xformers/accelerate in the venv/conda environment of your choosing. But if you know what you are doing this isn't for you anyhow...

For starters, I have to assume you have auto1111 installed as well as the auto1111 dreambooth extension functioning with xformers working. This is definitely not the place to ask for help if you can't get that to function (they may help you on the relevant discords). Also I'm on Ubuntu, so you'll need to translate this to your OS if you are not.

I didn't want to use the SD1.5 generic model for training, so I used the d8hazard DB extension to create a new trainable diffusor model beforehand. When you do this it will dump the path to the model in the terminal. Copy the path.

Navigate to the auto1111 stable-diffusion-webui folder in terminal and activate the virtual environment. For me:

source venv/bin/activate

now make a new folder somewhere else and git pull this repo (as branched by cian0 above, or the original cloneofsimo and copy the files from cian0 if you want to make sure you have the latest version).

If you have ~12GB VRAM, you'll probably need to modify 3 lines of code to use xformers:

#121 (reply in thread)

if you want to go the path of least resistance, just install jupyter labs in the venv and start it up:

pip install jupyterlab
jupyter-lab

now navigate in jupyter lab to cian0's training.ipynb. Everything is set up to run the demo for the simo liu example. Paste your model path (created earlier) into the MODEL_NAME line, eg

MODEL_NAME="/home/USER/stable-diffusion-webui/models/dreambooth/chosen_name/working"

run the cells in sequential order and you will have your result. Play around and try to learn what things do. Probably don't bother the creator of this repo if things don't work following the above, it's my fault (your fault?) not theirs. Good luck. If someone want to post a better way of doing things I will go ahead and delete this comment D=