Tracing without performance (#2084)

This PR is 1/2 to enable **Tracing without Performance**, i.e. make sure all our events are connected even if they are not Transactions.

This enables use cases such as Errors / Transactions / Replays etc all being connected across services and not just Transactions.

### Summary of changes

* new `PropagationContext` class that generates trace/span ids and baggage irrespective of whether there are transactions/spans active or not
* this lives on the `Scope`
* three new top level methods that first check the span and fallback on the scope's propagation context
  * `Sentry.get_traceparent`
  * `Sentry.get_baggage`
  * `Sentry.get_trace_propagation_headers`
* move `dynamic_sampling_context` to `Event` from `TransactionEvent` since all events will now have this info
* use the new top level helpers in `net/http` patch to set propagation headers

closes #2056 
also see #2089 

---

This PR is 2/2 to enable Tracing without Performance, i.e. make sure all our events are connected even if they are not Transactions.

### Summary of changes
* Implement new top level `Sentry.continue_trace(env, **options)` API that standardizes continuing an incoming trace from a rack env like hash.
* Use this new API in rack/rails/sidekiq

part of #2056 
linked to #2084

CHANGELOG.md

@@ -11,6 +11,22 @@
   config.trace_propagation_targets = [/.*/]  # default is to all targets
   config.trace_propagation_targets = [/example.com/, 'foobar.org/api/v2']
   ```
+- Tracing without Performance
+  - Implement `PropagationContext` on `Scope` and add `Sentry.get_trace_propagation_headers` API [#2084](https://github.com/getsentry/sentry-ruby/pull/2084)
+  - Implement `Sentry.continue_trace` API [#2089](https://github.com/getsentry/sentry-ruby/pull/2089)
+
+  The SDK now supports connecting arbitrary events (Errors / Transactions / Replays) across distributed services and not just Transactions.  
+  To continue an incoming trace starting with this version of the SDK, use `Sentry.continue_trace` as follows.
+
+  ```rb
+  # rack application
+  def call(env)
+    transaction = Sentry.continue_trace(env, name: 'transaction', op: 'op')
+    Sentry.start_transaction(transaction: transaction)
+  end
+  ```
+
+  To inject headers into outgoing requests, use `Sentry.get_trace_propagation_headers` to get a hash of headers to add to your request.
 
 ### Bug Fixes
 

sentry-rails/lib/sentry/rails/action_cable.rb

@@ -33,11 +33,8 @@ def capture(connection, transaction_name:, extra_context: nil, &block)
           end
 
           def start_transaction(env, scope)
-            sentry_trace = env["HTTP_SENTRY_TRACE"]
-            baggage = env["HTTP_BAGGAGE"]
-
             options = { name: scope.transaction_name, source: scope.transaction_source, op: OP_NAME }
-            transaction = Sentry::Transaction.from_sentry_trace(sentry_trace, baggage: baggage, **options) if sentry_trace
+            transaction = Sentry.continue_trace(env, **options)
             Sentry.start_transaction(transaction: transaction, **options)
           end
 

sentry-rails/lib/sentry/rails/capture_exceptions.rb

@@ -32,16 +32,13 @@ def capture_exception(exception, env)
       end
 
       def start_transaction(env, scope)
-        sentry_trace = env["HTTP_SENTRY_TRACE"]
-        baggage = env["HTTP_BAGGAGE"]
-
         options = { name: scope.transaction_name, source: scope.transaction_source, op: transaction_op }
 
         if @assets_regexp && scope.transaction_name.match?(@assets_regexp)
           options.merge!(sampled: false)
         end
 
-        transaction = Sentry::Transaction.from_sentry_trace(sentry_trace, baggage: baggage, **options) if sentry_trace
+        transaction = Sentry.continue_trace(env, **options)
         Sentry.start_transaction(transaction: transaction, custom_sampling_context: { env: env }, **options)
       end
 

sentry-ruby/lib/sentry-ruby.rb

@@ -489,6 +489,42 @@ def add_global_event_processor(&block)
       Scope.add_global_event_processor(&block)
     end
 
+    # Returns the traceparent (sentry-trace) header for distributed tracing.
+    # Can be either from the currently active span or the propagation context.
+    #
+    # @return [String, nil]
+    def get_traceparent
+      return nil unless initialized?
+      get_current_hub.get_traceparent
+    end
+
+    # Returns the baggage header for distributed tracing.
+    # Can be either from the currently active span or the propagation context.
+    #
+    # @return [String, nil]
+    def get_baggage
+      return nil unless initialized?
+      get_current_hub.get_baggage
+    end
+
+    # Returns the a Hash containing sentry-trace and baggage.
+    # Can be either from the currently active span or the propagation context.
+    #
+    # @return [Hash, nil]
+    def get_trace_propagation_headers
+      return nil unless initialized?
+      get_current_hub.get_trace_propagation_headers
+    end
+
+    # Continue an incoming trace from a rack env like hash.
+    #
+    # @param env [Hash]
+    # @return [Transaction, nil]
+    def continue_trace(env, **options)
+      return nil unless initialized?
+      get_current_hub.continue_trace(env, **options)
+    end
+
     ##### Helpers #####
 
     # @!visibility private

sentry-ruby/lib/sentry/client.rb

@@ -148,6 +148,8 @@ def send_event(event, hint = nil)
       raise
     end
 
+    # @deprecated use Sentry.get_traceparent instead.
+    #
     # Generates a Sentry trace for distribted tracing from the given Span.
     # Returns `nil` if `config.propagate_traces` is `false`.
     # @param span [Span] the span to generate trace from.
@@ -160,7 +162,9 @@ def generate_sentry_trace(span)
       trace
     end
 
-    # Generates a W3C Baggage header for distribted tracing from the given Span.
+    # @deprecated Use Sentry.get_baggage instead.
+    #
+    # Generates a W3C Baggage header for distributed tracing from the given Span.
     # Returns `nil` if `config.propagate_traces` is `false`.
     # @param span [Span] the span to generate trace from.
     # @return [String, nil]

sentry-ruby/lib/sentry/event.rb

@@ -37,6 +37,11 @@ class Event
     # @return [RequestInterface]
     attr_reader :request
 
+    # Dynamic Sampling Context (DSC) that gets attached
+    # as the trace envelope header in the transport.
+    # @return [Hash, nil]
+    attr_accessor :dynamic_sampling_context
+
     # @param configuration [Configuration]
     # @param integration_meta [Hash, nil]
     # @param message [String, nil]
@@ -54,6 +59,7 @@ def initialize(configuration:, integration_meta: nil, message: nil)
       @tags          = {}
 
       @fingerprint = []
+      @dynamic_sampling_context = nil
 
       # configuration data that's directly used by events
       @server_name = configuration.server_name

sentry-ruby/lib/sentry/hub.rb

@@ -229,6 +229,50 @@ def with_session_tracking(&block)
       end_session
     end
 
+    def get_traceparent
+      return nil unless current_scope
+
+      current_scope.get_span&.to_sentry_trace ||
+        current_scope.propagation_context.get_traceparent
+    end
+
+    def get_baggage
+      return nil unless current_scope
+
+      current_scope.get_span&.to_baggage ||
+        current_scope.propagation_context.get_baggage&.serialize
+    end
+
+    def get_trace_propagation_headers
+      headers = {}
+
+      traceparent = get_traceparent
+      headers[SENTRY_TRACE_HEADER_NAME] = traceparent if traceparent
+
+      baggage = get_baggage
+      headers[BAGGAGE_HEADER_NAME] = baggage if baggage && !baggage.empty?
+
+      headers
+    end
+
+    def continue_trace(env, **options)
+      configure_scope { |s| s.generate_propagation_context(env) }
+
+      return nil unless configuration.tracing_enabled?
+
+      propagation_context = current_scope.propagation_context
+      return nil unless propagation_context.incoming_trace
+
+      Transaction.new(
+        hub: self,
+        trace_id: propagation_context.trace_id,
+        parent_span_id: propagation_context.parent_span_id,
+        parent_sampled: propagation_context.parent_sampled,
+        baggage: propagation_context.baggage,
+        **options
+      )
+    end
+
     private
 
     def current_layer

sentry-ruby/lib/sentry/net/http.rb

@@ -31,7 +31,10 @@ def request(req, body = nil, &block)
 
         Sentry.with_child_span(op: OP_NAME, start_timestamp: Sentry.utc_now.to_f) do |sentry_span|
           request_info = extract_request_info(req)
-          set_sentry_trace_header(req, sentry_span, request_info)
+
+          if propagate_trace?(request_info[:url], Sentry.configuration)
+            set_propagation_headers(req)
+          end
 
           super.tap do |res|
             record_sentry_breadcrumb(request_info, res)
@@ -49,17 +52,8 @@ def request(req, body = nil, &block)
 
       private
 
-      def set_sentry_trace_header(req, sentry_span, request_info)
-        return unless sentry_span
-
-        client = Sentry.get_current_client
-        return unless propagate_trace?(request_info[:url], client.configuration.trace_propagation_targets)
-
-        trace = client.generate_sentry_trace(sentry_span)
-        req[SENTRY_TRACE_HEADER_NAME] = trace if trace
-
-        baggage = client.generate_baggage(sentry_span)
-        req[BAGGAGE_HEADER_NAME] = baggage if baggage && !baggage.empty?
+      def set_propagation_headers(req)
+        Sentry.get_trace_propagation_headers&.each { |k, v| req[k] = v }
       end
 
       def record_sentry_breadcrumb(request_info, res)
@@ -96,8 +90,10 @@ def extract_request_info(req)
         result
       end
 
-      def propagate_trace?(url, trace_propagation_targets)
-        url && trace_propagation_targets.any? { |target| url.match?(target) }
+      def propagate_trace?(url, configuration)
+        url &&
+          configuration.propagate_traces &&
+          configuration.trace_propagation_targets.any? { |target| url.match?(target) }
       end
     end
   end

sentry-ruby/lib/sentry/propagation_context.rb

@@ -0,0 +1,134 @@
+# frozen_string_literal: true
+
+require "securerandom"
+require "sentry/baggage"
+
+module Sentry
+  class PropagationContext
+    SENTRY_TRACE_REGEXP = Regexp.new(
+      "^[ \t]*" +  # whitespace
+      "([0-9a-f]{32})?" +  # trace_id
+      "-?([0-9a-f]{16})?" +  # span_id
+      "-?([01])?" +  # sampled
+      "[ \t]*$"  # whitespace
+    )
+
+    # An uuid that can be used to identify a trace.
+    # @return [String]
+    attr_reader :trace_id
+    # An uuid that can be used to identify the span.
+    # @return [String]
+    attr_reader :span_id
+    # Span parent's span_id.
+    # @return [String, nil]
+    attr_reader :parent_span_id
+    # The sampling decision of the parent transaction.
+    # @return [Boolean, nil]
+    attr_reader :parent_sampled
+    # Is there an incoming trace or not?
+    # @return [Boolean]
+    attr_reader :incoming_trace
+    # This is only for accessing the current baggage variable.
+    # Please use the #get_baggage method for interfacing outside this class.
+    # @return [Baggage, nil]
+    attr_reader :baggage
+
+    def initialize(scope, env = nil)
+      @scope = scope
+      @parent_span_id = nil
+      @parent_sampled = nil
+      @baggage = nil
+      @incoming_trace = false
+
+      if env
+        sentry_trace_header = env["HTTP_SENTRY_TRACE"] || env[SENTRY_TRACE_HEADER_NAME]
+        baggage_header = env["HTTP_BAGGAGE"] || env[BAGGAGE_HEADER_NAME]
+
+        if sentry_trace_header
+          sentry_trace_data = self.class.extract_sentry_trace(sentry_trace_header)
+
+          if sentry_trace_data
+            @trace_id, @parent_span_id, @parent_sampled = sentry_trace_data
+
+            @baggage = if baggage_header && !baggage_header.empty?
+                        Baggage.from_incoming_header(baggage_header)
+                      else
+                        # If there's an incoming sentry-trace but no incoming baggage header,
+                        # for instance in traces coming from older SDKs,
+                        # baggage will be empty and frozen and won't be populated as head SDK.
+                        Baggage.new({})
+                      end
+
+            @baggage.freeze!
+            @incoming_trace = true
+          end
+        end
+      end
+
+      @trace_id ||= SecureRandom.uuid.delete("-")
+      @span_id = SecureRandom.uuid.delete("-").slice(0, 16)
+    end
+
+    # Extract the trace_id, parent_span_id and parent_sampled values from a sentry-trace header.
+    #
+    # @param sentry_trace [String] the sentry-trace header value from the previous transaction.
+    # @return [Array, nil]
+    def self.extract_sentry_trace(sentry_trace)
+      match = SENTRY_TRACE_REGEXP.match(sentry_trace)
+      return nil if match.nil?
+
+      trace_id, parent_span_id, sampled_flag = match[1..3]
+      parent_sampled = sampled_flag.nil? ? nil : sampled_flag != "0"
+
+      [trace_id, parent_span_id, parent_sampled]
+    end
+
+    # Returns the trace context that can be used to embed in an Event.
+    # @return [Hash]
+    def get_trace_context
+      {
+        trace_id: trace_id,
+        span_id: span_id,
+        parent_span_id: parent_span_id
+      }
+    end
+
+    # Returns the sentry-trace header from the propagation context.
+    # @return [String]
+    def get_traceparent
+      "#{trace_id}-#{span_id}"
+    end
+
+    # Returns the Baggage from the propagation context or populates as head SDK if empty.
+    # @return [Baggage, nil]
+    def get_baggage
+      populate_head_baggage if @baggage.nil? || @baggage.mutable
+      @baggage
+    end
+
+    # Returns the Dynamic Sampling Context from the baggage.
+    # @return [String, nil]
+    def get_dynamic_sampling_context
+      get_baggage&.dynamic_sampling_context
+    end
+
+    private
+
+    def populate_head_baggage
+      return unless Sentry.initialized?
+
+      configuration = Sentry.configuration
+
+      items = {
+        "trace_id" => trace_id,
+        "environment" => configuration.environment,
+        "release" => configuration.release,
+        "public_key" => configuration.dsn&.public_key,
+        "user_segment" => @scope.user && @scope.user["segment"]
+      }
+
+      items.compact!
+      @baggage = Baggage.new(items, mutable: false)
+    end
+  end
+end

sentry-ruby/lib/sentry/rack/capture_exceptions.rb

@@ -62,11 +62,8 @@ def capture_exception(exception, env)
       end
 
       def start_transaction(env, scope)
-        sentry_trace = env["HTTP_SENTRY_TRACE"]
-        baggage = env["HTTP_BAGGAGE"]
-
         options = { name: scope.transaction_name, source: scope.transaction_source, op: transaction_op }
-        transaction = Sentry::Transaction.from_sentry_trace(sentry_trace, baggage: baggage, **options) if sentry_trace
+        transaction = Sentry.continue_trace(env, **options)
         Sentry.start_transaction(transaction: transaction, custom_sampling_context: { env: env }, **options)
       end