Can I use JetStream KV as cache?

[tzq0301]

I tried to use JetStream KV as cache, but it seems to get worse than even no cache. And Redis goes well in the case.

I wonder if my test case is not good (not suitable for the cache-case).

The test code is nats-demo/cache

[ripienaar]

Try setting the KV to memory storage.

[ripienaar]

Redis stores its data in memory. So can nats and it would be faster than going to disk all the time when not needed.

For like for like comparison you should set KV to memory

[tzq0301]

Oh I find the type argument in jetstream.KeyValueConfig, and I set Storage: jetstream.MemoryStorage in func BenchmarkNatsCache. However, it stays slower than no-cache. Does my code uses the KV in a wrong way?

https://github.com/tzq0301/nats-demo/blob/2611e5eac7e7ca43a24619284dc5552df9b588c6/cache/post_test.go#L85

[ripienaar]

I don’t really think a cache is what our KV is going to be the best at, it’s not really been a goal we are more into distributed cluster scale stuff with replicated data etc. but it shouldn’t be bad

Regarding rhe test, be aware that MySQL when doing queries like this is incredibly fast since the data is almost all in RAM cached already and it’s optimised for that with its own caches so you aren’t really testing realistic use cases.

[tzq0301]

1. I don't understand that 'the data is almost all in RAM cached already'. MySQL is a disk-based storage and cache of MySQL has been deprecated since MySQL 8.
2. I don't understand that 'so you aren’t really testing realistic use cases'. In the benchmark, Redis do fasten the query and you can go test -bench= or just see the README.
3. I just try to use NATS as cache because someone in the slack (jetstream) say that it's fast enough.

If my test code is poor, looking forward to a guide.

[ripienaar]

While the query cache is indeed gone there are many other layers of caching happening. Block level caching in the database and lots in the OS.

From your OS and up with that little data you can see with your OS I/O monitoring tools that you would almost never be reading data from disk.

Better test is much more data - much more, more intensive queries reads in more random ways. With such a simple test the return on investment of the cache would be minimal.

Again Nats KV isn’t optimised as a very fast cache imo though

[derekcollison]

We can be on par with Redis when there is a real network.

You can also use KV watchers to have a subset of the KV space in your app, and kept up to date for even faster response times.

I disagree with R.I., no reason it can not be very fast IMO.

Should make sure direct get acces is on with your KV etc.

[jnmoyne]

To add a bit more concrete details to the answer above. How you can do a simple NATS and Redis latency comparison using the KV get() operation:

NATS setup is nats-server -js running locally on the machine and then using the nats bench command:
Populate 1M keys nats bench test --kv --pub 1 --msgs 1000000 --storage memory and then read them nats bench test --kv --sub 1 --msgs 1000000 --storage memory --no-progress

Redis setup is just start redis-server running locally and then using the redis-benchmark command:
Populate 1M keys redis-benchmark -d 128 -c 1 -r 1000000 -n 1000000 -t set then read them redis-benchmark -d 128 -c 1 -r 1000000 -n 1000000 -t get

On a Mac Studio M1-Ultra the NATS benchmark reports almost exactly 24,000 synchronous operations per second meaning an average latency of 41.666 micro-seconds per get. The redis benchmark reports an average of 18 micro-seconds.

These test are between processes on the same host over the loopback interface, in real-life deployments however client applications and NATS or Redis servers are going to talk to each other over the network, and that latency due to the network is going to be a number of times larger than the latency of NATS/Redis as measured above (e.g. in the hundreds of micro-seconds over a fast LAN) such that the 20 or so micro-seconds of difference in latency between NATS and Redis becomes relatively insignificant.

[tzq0301]

Thanks for your answer. Learn a lot!

[nickchomey]

If the nats nodes are on the same machine as the application that is using it as a kv cache (ie no network latency), is it safe to assume that the read latency should be close to the 41 and 18 microseconds that you got in the benchmark? This is how I intend to use NATS with my application

[jnmoyne]

You can look at the nats bench code, it's just using the KV's get() call from the standard nats.go client library, no magic involved. This numbers I got on my M1-Ultra machine: your mileage may vary. The whole point of nats bench is that you can easily install it on your target infrastructure and run it there to get an actual performance number.