Skip to content

2019 01 30 Zipkin PPMC Pow wow

Adrian Cole edited this page Jun 24, 2019 · 2 revisions

Goal

The group formerly known as OpenZipkin are over 3 years old, and are now the PPMC of Apache Incubating Zipkin. A lot has happened since we formed, including changes to governance and tooling. Meanwhile, some of us have never met together. The goal of our pow-wow is to firstly socialize and reflect on the years we've been together. Secondarily, we can dig into long term topics such as "should we write a golang proxy?" or "how will we successfully releases in ASF?"

Date

30-31 Jan during working hours in CEST (UTC +1)

This is intentionally just prior to Fosdem

Location

A small town: Amersfoort, Netherlands. Details are sent offline

Output

We will add notes at the bottom of this document including links to things discussed and any takeaways.

Attendees

This pow-wow is for PPMC and routine contributors (iotw future committers). It will not be remote friendly or strictly scheduled, though we may have some ad-hoc video calls.

Attending on-site

  1. Bas van Beek (conceiver and who chose the lovely place)
  2. Adrian Cole
  3. Lance Linder
  4. Zoltán Nagy 
  5. Max people onsite are 5!

Homework

None yet

Agenda

If any segment is in bold, it will be firmly coordinated for remote folks. Other segments may be lax or completely open ended.

Wednesday, Jan 30

morning get to Amersfoort
afternoon work on ASF releases
evening Explore the town

Thursday, Jan 31

morning work on ASF release
afternoon explore VoltDB as an backend
evening enjoy the town

Friday, Feb 1

morning spike on VoltDB
afternoon spike on VoltDB
evening some go to fosdem

Outcomes

We came together with an idea to perform an ASF release of brave-karaf and a stretch goal to explore VoltDB as a potential MySQL replacement and possibly a late sampling solution. Wednesday to Thursday was mostly dedicated to the release, learning that driving via Jenkins was a path of less resistance. We ended up with snapshots deployed and github integration integrated with Jenkins thanks to Zoltan. We are waiting on ASF INFRA for validation of some permissions as well authorizing githooks to remove some manual steps. Because we don't have permissions to upload the the maven repos we couldn't perform an official release.

On Thursday night, we dove into zipkin-voltdb, using SQL to implement basic queries needed for the Zipkin http api. Working on an experimental repo, we could see the ingest rates were substantial and that this could be a good fit for late sampling as well. On Friday, Zoltan and Lance experimented with export tables to prove that we can push data directly from VoltDB to another place. This is a key part in late sampling, as once we decide to retain a trace, we'll need to move it somewhere else, such as an http call, elasticsearch or kafka message. The logic we need to implement late sampling is more complex than what we could easily do in SQL, so we also developed code to push java procedures into VoltDB. With these two pieces, we have the parts to implement late sampling, but we still need to window the data. Various options for implementing this were evaluated including using TTL logic, but none fit perfectly.

Clone this wiki locally