CustomModelLoader Timing Diagrams

[prakashngit]

This is about the Custom Model Loader feature of the OpenVINO model server. I understand the Custom Model Loader is supposed to define the functions within the CustomLoaderInterface class template; however, I cannot find any documentation on how/when these functions are called by the OVMS model manager. Without such documentation, IMHO one cannot produce a meaningful implementation of the CustomLoaderInterface class.

From the logs of the SampleCustomLoader, I see rather "unintuitive behaviors" such as models getting loaded even before any inference request is placed. For example, if the CustomLoader is configured with a config.json file that contains multiple models, the custom loader attempts to pre-load all the models, even before any of these models are called for inference. Also, what is the difference between unload and retire models? When do these get called? After every inference?

It would be very helpful to obtain timing diagrams that describe usage of custom model loader functions by model manager. Thanks!

[prakashngit]

Moving this as an issue within model server repo. thanks!