zetcd performance

[IgorPerikov]

Is zetcd+etcd supposed to be faster than zk nowadays? I saw a few graphics, which says that etcd itself faster than zk and this benchmark about zetcd 0.0.1, is it still relevant?

[heyitsanthony]

There have been performance improvements since then on both etcd and zetcd, but another round of benchmarks would be needed to quantify the impact. If performance has gotten worse, there's a regression.

[IgorPerikov]

Thanks, Anthony. Any plans about this benchmarks? If you do should I convert this issue to tracking issue about them?

[jpiper]

I just ran zkboom on a deployment backed by a 5-node etcd v3.3.2 cluster, attempting to replicate the table shown here:https://coreos.com/blog/introducing-zetcd. I compiled the latest zetcd HEAD and vendored in etcd v3.3.2 and gRPC v1.7.5 as well.

For a single zetcd replica and small numbers of concurrent clients, things are much faster than the blog post. However, the write latencies are higher once you hit 50 concurrent clients. This may be expected - my 5-node cluster will have a higher write latency than a 3 or 1 node cluster (I don't know what was used in the blog post) Likewise, my etcd peer and client connections are secured with TLS which could incur a performance hit.

If I set an overall request limit to 500/s, I get an average write latency of 30ms, so it looks like running without a limit it just hammering zetcd, and it can't really handle more than 500 writes a second on my setup.

$ zkboom --key-size 128 --val-size 128 --total=10000 --rate=2500 --conns=$CONNS  --endpoints zetcd:2181 create

+-------+---------+--------------------------+
| conns |  HEAD   |  v0.0.1 (from blog post) |
+-------+---------+--------------------------+
|     1 | 3.2ms   | 12.5ms                   |
|     5 | 7.5ms   | 31.25ms                  |
|    10 | 13ms    | 37.5ms                   |
|    25 | 44ms    | 44ms                     |
|    50 | 100ms   | 44ms                     |
+-------+---------+--------------------------+

To put this in comparison with a real 5-node zookeeper deployment, I can run zkboom with 50 connections and get 2.8k op/s - 5x higher throughput than zetcd and with much lower latency. Unless I've done something terribly wrong, this only seems like zetcd is useful if you're expecting less than 500 writes a second.

Summary:
  Total:	3.9995 secs.
  Slowest:	0.0123 secs.
  Fastest:	0.0003 secs.
  Average:	0.0007 secs.
  Stddev:	0.0005 secs.
  Requests/sec:	2500.2825