How to correctly load a model generated by SoVITS (G_xx.pth) ?

[xD0135]

Hi, is there a way to correctly load a model generated by SoVITS SVC (https://github.com/voicepaw/so-vits-svc-fork)?

I copied the generated model (G_xx.pth) to its appropriate folder under model_dir/6, created a params.json file, I can see the model in the UI but it basically produces no sound. I used this same model for inference in SoVITS and it works without issues.

Here's the params.json file for reference:

{
    "slot_index": 6,
    "voice_changer_type": "RVC",
    "name": "Sire",
    "description": "",
    "credit": "",
    "terms_of_use_url": "https://huggingface.co/wok000/vcclient_model/raw/main/rvc_v2_alpha/kikoto_kurage/terms_of_use.txt",
    "icon_file": "kikoto_kurage.png",
    "speakers": {},
    "model_file": "G_1401.pth",
    "index_file": null,
    "is_onnx": false,
    "inferencer_type": "pyTorchRVCv2",
    "sample_rate": 40000,
    "is_f0": true,
    "deprecated": false,
    "embedder": "hubert_base_l12",
    "pitch_estimator": "fcpe",
    "sample_id": null,
    "version": "v3.0",
    "chunk_sec": 0.05,
    "pitch_shift": 0,
    "index_ratio": 0.0,
    "protect_ratio": 0.5
}

And here's the error I see:

2024-09-29 00:43:19,480 - vcclient - validation_error_log - ERROR - VCClientError: {"code":701,"reason":"Model update failed.","action":"Please check model file and start converting.","detail":"Failed to update model:'config'"} - vcclient_dev\server\validation_error_logging_route.py - 26

Thanks in advance for the help.

[Kuuko-fokkusugaru]

On the model selection area, click on the edit button. It will open a pop-up window with a list of all the installed models. Look for a blank space and click on the upload button. Another pop-up window will open to search for the model and index files. There is a drop-down menu showing "RVC". Click there and select the type of model which in your case is SoVITS then load the proper pth file.

[xD0135]

Thank you for the reply! A couple of follow up questions:

1. SoVITS doesn't seem to generate an index file with the extension .index or .bin as required by the file uploader. Do you know how to get/generate them?
2. The drop-down for the Voice Changer Type doesn't have SoVITS as an option, only the following:
   

I'm using version vcclient_win_cuda_2.0.61-alpha

[Kuuko-fokkusugaru]

I am not so acknowledged about generating voice models but the index file is used to generate a proper "accent". Basically, while the pth model contains the generated "voice", the index contains details that would make you not just sound like the original voice but also speak in the same way, with the same accent, same way of pronunciation, etc. This is useful when you want to sound exactly like the source voice and in the same language. Using an index file to speak on a different language than the source one often results in a broken accent. So if you use a Japanese voice with index file to speak English it may actually sound closer to how a Japanese would try to speak a not so perfect English (in regards to pronunciation). That said, the index file is not a requirement and every voice works without one. How to generate one isn't something that I know neither if that's a possibility for SoVITS models.

About your screenshot, I do know that SoVITS it's there on v1 so consider switching to v1 and open a request to add SoVITS to v2. I am not aware of why v2 may not contain SoVITS, if it's a limitation or simply haven't been implemented yet.

[xD0135]

Thank you for the explanation. I downloaded MMVCServerSIO_win_onnxgpu-cuda_v.1.5.3.18a, added a new model using the steps that you had provided before, and it imports without issues. But when I try to use the model to generate sound, it sounds completely different than the expected voice; it has very high pitch and almost sound like a cartoon. Not something usable. I also tried with a couple of other SoVITS files available on HuggingFace and the results are all the same. I must say that all models I tried using are in English, and I'm also speaking in English.

Any other suggestions on how to address this issue?

[Kuuko-fokkusugaru]

There is a tune slider at the right of where the model picture is shown. The tune will change the "pitch" to fit that of the model. Usually, from guy to girl voice, you will increase it to around 12, for girl to guy voice you will do the opposite and set it to around -12. If is guy to guy or girl to girl, you may play between -4 and 4 depending on your voice depth. I recommend to use this setting to match as close as possible the source original voice and not to customize the voice. The reason is that the closer it gets to the original pitch, the less robotic will sound. I have seen some SoVITS models that aren't meant to work with RVC so if the Tune setting doesn't work as expected, it may be the case that isn't meant to be used with RVC and meant to be used with TTS instead.