Ability to use async functions in the microservices

[geiseri]

Proposed change

Provide a way to supply a resolution callback instead of blocking on:

https://github.com/nats-io/nats.c/blob/29dcecc42afc454b7328ee7c3b1aba761689339a/src/micro_endpoint.c#L215C14-L215C14

Use case

Currently I use boost::asio for a grpc project. I would like to use the nats microservices. One struggle I have is that it is all blocking logic. Asio uses completion tokens that are VERY easy to use in an asynchronous manner, but they are VERY hard to use in a blocking manner because the result is populated by the final resolution of the token (usually a closure). One handy side-effect is you don't need to use threads just to wait for an async io request or in my case dispatch to a fixed size worker pool interfacing with external hardware. I know async code in C is painfully difficult, but would there be a way to have a response callback or something to allow the handler to notify completion?

Contribution

No, unfortunately I avoid C with anything involving memory safety. NATS does look very encouraging and I am very impressed with the approach to solve some really annoying (usually fatal) issues for horizontal scaling. Thanks!

[levb]

@geiseri Can you please elaborate, why use NATS microservices for this use case, rather than a "regular" NATS async subscription? What services framework features are valuable? Stats/monitoring? Error handling?

[geiseri]

TL;DR NATS microservices remove 2/3 of my current gRPC microservice's infrastructure with very few code changes on my part.

I have a project that started its life a long time ago. Originally it used AMQP as a backend and used message passing to communicate between different services. This was great for being able to distribute work and recover from failures. Over time the business logic ended up being multiple dependent steps, and it became very cumbersome to pass around correlation ids, timeouts, etc. In the end that code was moved to gRPC microservices. The framework used (Boost ASIO) helped keep the dispatch decoupled from the underlying thread so the only retained part was the socket connection. Dispatch could then be scheduled as seen fit. From the outside it looks more or less looks like A+ promises. So fast forward to now and it has become clear that the infrastructure overhead to handle service discovery, failover, and load balancing is a liability.

Currently old AMQP code has been replaced by NATS and it is looking like the microservices aspect could solve the pains with gRPC. So far the gRPC conversion has been a matter of just using the protobuf files over NATS messages. I have some python test code and that does the "subscribe"/"reply" approach but it smacks of the original issues had with AMQP. The key things microservices API solves here is service discovery (ie. $SRV.PING.EchoService) and health monitoring (ie $SRV.STATS.EchoService). Those two things alone would remove the current setup of HAProxy, keepalived, and SRV records I have to deal with right now.

The rub that I have is that each service call would happen in a separate thread. In prior projects I have been nailed by this when there is a burst of traffic. The nice thing about the way ASIO works is that the caller can return and when its complete the response is dispatched decoupled from the thread's lifetime.

I am not sure this clarifies things at all, but that is why the microservices look so attractive.

[codeandroid]

I'd like to add an additional use case which seems to be very much related: We have several services which are a few years old. Some of them based on a custom async Bond-over-gRPC implementation, some of them boost.asio-style HTTP services. If I read the description correctly, then they very much rely on the same pattern @geiseri described, i.e. on receiving a message, the message is then not necessarily processed immediately but posted to and processed in a thread pool and after processing at some later time, the response (or error) will be generated - all without blocking the RPC service handler. This is especially important for some of our services as sometimes the processing has to be serialized in a dedicated thread due to 3rd party API restrictions.
For some of these restrictions there are workarounds but this generates more friction as it changes the way we are developing concurrent services. (Also, the boost.asio way, while not perfect, is well-understood.)