OpenZipkin's 5 year anniversary
Today, April 16, 2020, we celebrate 5 years since Eirik Sletteberg clicked send on the pivotal Moving Zipkin out in the open email. Below will discuss why this email was special, and how its spirit impacts us positively today. We'll review what OpenZipkin is and the major accomplishments of the last five years. We'll close with some recent developments in 2020.
If you want to chat with us about the project, we're always available on Gitter. If you are a fan, make sure you star our main repository.
Without further adieu, let's talk about that email!
Eirik and Mick worked on Zipkin at Finn.no at the time. They had responsibility to run their Zipkin site, but were dependent on Twitter to improve Zipkin. Zipkin was still managed by Twitter at the time, and their needs were different than the external community. For example, when Twitter switched internally from Cassandra onto a non-open source alternative, it resulted in neglect in the Cassandra component. This was a dilemma for those who had no access to the alternative, and desired to run Cassandra!
There were other differences in priority client side as well. For example, Twitter was originally a ruby on rails application and so its first tracing library was Ruby. When they moved to Finagle (Scala), the Ruby tracer became neglected despite external usage. Even sites that ran Finagle had other libraries to trace: two years before Eirik's email, Kristof invented a Java tracing library Brave to fill the gaps.
Differences continued into how projects run. For example, Twitter was operating largely as a mono repo and it was typical to build distributions from source (to deploy to Mesos/Aurora at the time). Outside Twitter, deployments were very different. Few used Mesos or Aurora, even if they used similar container technology like Docker. This meant end users had to individually build this Scala project from source, usually with some modifications, to ultimately turn it into something they could deploy.
These forces led to Eirik's email, and the culture of OpenZipkin that remains today. Sites owners want not just problems solved, but ownership to solve them. After some months of deliberation, Twitter decided to get behind the community. This resulted in effort on their part for both the external and internal migrations. For example, it required a major change to switch from a push to a pull model for Zipkin code deployments internally. Twitter deserve a lot of credit, not just for the initial code drop of Zipkin, but also several months of full time effort by Adrian to create Zipkin's new home and readjust internal processes towards that.
The "Moving Zipkin out in the open" email transformed into a new GitHub org OpenZipkin. OpenZipkin committers are from different companies, usually volunteers who were at one point running a Zipkin site. No committer has more rights than another in OpenZipkin, and no company has more than two committers either. Committers routinely move in and out of focus as they switch roles or jobs, but the community itself behaves the same. While there are hundreds of contributors, we'll introduce the core team through some historical anecdotes.
For the last 5 years, OpenZipkin transformed the server experience. Zoltán reinvented our DevOps experience, leading to a Dockerized build, fully automated release process, Prometheus integration and our website. After the community decided collectively to use Java, Adrian (then at Pivotal) led a Spring Boot rewrite of the server. Later Adrian and Rag rewrote the I/O subsystems using the high performance Armeria library. Meanwhile, we wanted cross language support to be excellent. The core team painstakingly reverse engineered and documented the B3 header format as well the historical Thrift data format. Then, OpenZipkin created the most widely supported tracing format Zipkin v2 and a simplified 'b3' single header format. The UI was refreshed two times, first by Eirik, who converted it into pure Javascript, then by Raja who renovated it further. Most recently, Igarashi from LINE rewrote the UI with support from Huy, while Seo-yeon (also from LINE) led a website and logo refresh.
Instrumentation side was a similar combination of overhaul and new. With Jordi on point, folks at Medidata re-owned the abandoned zipkin-ruby. PH at Criteo donated zipkin4net, something later also re-owned also by Medidata. Eirik created zipkin-js currently led by José Carlos. José Carlos invented zipkin-php and co-leads zipkin-go invented by Bas. Kristof retired from Brave, but it's in good hands you'll see Adrian, Jorge, Jorg and Rag ever present on Java tracing. Incidentally, Brave is the most starred distributed-tracing client library on GitHub today!
We also have deep integrations run by the community. Kafka is not just a traced technology, but also a transport to move traces. There are too many to thank, but notably Jorge leads all things Kafka for us. Tommy weaved in our RabbitMQ integration. Cassandra remains an solid storage option, particularly high throughput sites. Mick and Lance are behind most of the thought work there. Elasticsearch support has been a continual effort, created and endured even today by Rag. Rag also co-leads zipkin-gcp with the Spring Cloud GCP team at Google, as well zipkin-aws originally created by Lance and Brian.
Being a community is more than code. We get together with end users and conferences, and we also hold workshops and pow-wows. One fun fact is that Tokyo is the site of the first Zipkin user group in the world, led by Maki-san. Japan is a center for Zipkin, including LINE, who we often meet with.
We've had meetups in other places, too Apache SkyWalking team held the first Zipkin meetup in China. Plus, not only did we get to see service graphs that look like noodles, we got to eat them, too!
Pow-wows are a special meeting, typically at an end user site. We spend long days on technical topics everyone is interested in, and enjoy the time between topics.
Here's a pow-wow in Fukuoka along with Jason from Haystack. A major contributor Koji from LINE organized this so we have a place to hash out some technical problems face to face.
Expedia also welcomed the core team, and not yet core team like Daniele to a Haystack+Zipkin pow-wow in Delhi:
Other times, they are informal gatherings with near zero planning, such as this time in Sitges (Catalonia):
We also run or join other people's working groups, such as Naver Pinpoint (well deserved dinner on the left!). This lets us not not only learn from the hosts, but learn from other smart friends they invite such as Jaana (holding the camera) and William (furiously studying here). As you'll notice Karaoke is a trend.
You can read more about our pow-wows and workshops here.
Beyond face-to-face, these last 5 years transformed our support model from google groups and 7 accounts on IRC in 2015 to an online community of over 2000 on Gitter. Jorg is on constant watch for issues, recently getting us under 200 on our main repo (which has had over 3000 issues to date!). Jorg and Tommy also keep a keen eye on our docs, to help ensure they are readable by those who read docs :D
To sum it up, Zipkin is constantly refreshed by friends who support and mentor each other: the site-centric spirit is as real today as when it started 5 years ago.
There's always a lot going on, and most of the highlights you can find out via Twitter or watching projects of interest. Here's a few neat things you might have missed!
The European Commission is a Zipkin site whose tracing team operates from Brussels. While this is a relatively new site, its lead Jorg Heymans is no stranger to Zipkin. Jorg began contributing nearly 3 years ago, starting in the Brave (java tracer) project, but eventually working on the server as well. He's contributed to boring and exciting features, from documentation and spring XML to gRPC and Kafka Streams. He helped us slim down the server build by replumbing the entire logging infrastructure. Recently, you'll notice our issues list is a lot smaller. Despite having 3000 issues contributed since the beginning of Zipkin, less than 200 remain with huge thanks to Jorg and his thoughtful responses.
You can see the formal invitation of Jorg to the core team here
As of version 2.20 Lens became the new default UI for Zipkin, replacing the "classic" UI. We noticed many haven't updated yet, so take a look!
Below is a real trace from Yelp on the most recent version of Zipkin's new (Lens) UI:
While Igarashi spent over a year developing the [Lens] UI (based on an internal UI at LINE), it is now a team project. You'll see many contributions by co-lead Rag, as well sporadic contributions by volunteers with something to add.
- brave (Java) has new apis for Baggage (request scoped properties)
- zipkin has new settings for Trace Archival and Dynamic Elasticsearch credentials
- zipkin4net added messaging support
- zipkin-aws remodeled sdkv2 support into a single span per AWS call
- zipkin-js dropped support for nodejs 8 and added aws-sqs transport
- zipkin-ruby added messaging support and Faraday 1.x