Running over an rest api instead of socket based connection.

[ZTAP0011]

@xyzhang626 is there any possibility of running the embedding model behind a rest API server instead of socket-based connection?

[xyzhang626]

sorry I'm not the expert on severing stuff. Could you check the problem on llama.cpp, where it has already integrated embedding models in recent PR based on this one ?

[ZTAP0011]

Thank you for the reply. Can you please share repo link for llama.cpp. Wanted to be sure that I look the correct one.

[xyzhang626]

Check this out. ggerganov/llama.cpp#5423 (comment)