is it possible to change the trigger word "sks" later down the line or while extracting to LoRA

[runner22k]

Hi, recently I came through some script and ways, through which we are able to extract custom training to a small LoRA file. In such process, can we also edit the trigger word used? I mean if a SD model is trained on a character with "sks" and now we want it to be named with something meaningful. can we do that? For example sks to john, while extracting?

This applies to the LoRA trainings too. If this can be done, it would be very useful to rename the trigger words to something meaningful down the line.

[ExponentialML]

In theory you should be able to. In the vocab.json file where your model is (model/tokenizer/vocab.json, you should be able to change any of the words to your own, and the word will reference the embedding vector.

So looking at the 1.5 checkpoint it would be like:

Original: "sks</w>": 48136
New: "my_word_here</w>": 48136

Then just merge the model with the LoRA as you normally would.

[runner22k]

🥇 Wow! No one answered this question, on any forum I posted this, till now. Thank you so much. But how to access vocab.json?

I have a model which was trained through colab cus_model.ckpt. What should I do to rename it and save it back?

[ExponentialML]

No problem. It's a bit difficult to say due to how Diffusers saves their models as blobs on the HDD, which I'm unfamiliar with.
I usually just download the checkpoints manually, then load them in from whatever directory I downloaded them to. You can find the link here: https://huggingface.co/runwayml/stable-diffusion-v1-5/tree/main

Open a terminal or command line in the directory where you want to download your model. Then do:

git lfs install
git clone https://huggingface.co/runwayml/stable-diffusion-v1-5

It will download your model files to your hard drive in a custom directory.
You can see the vocab.json file where I've mentioned. Then you just edit the words from there (best to back up the json file first).

[runner22k]

@ExponentialML Maybe we need to convert .ckpt file to diffusers to access these files and then edit and save. Do you know a good working script which can convert between any ckpt to diffusers and back?

[arceus-jia]

@ExponentialML Maybe we need to convert .ckpt file to diffusers to access these files and then edit and save. Do you know a good working script which can convert between any ckpt to diffusers and back?

https://github.com/huggingface/diffusers/blob/main/scripts/convert_original_stable_diffusion_to_diffusers.py
diffusers officially provides sample code