1 - API Reference
Complete API documentation for the Tracer type and all methods
Complete API reference for the Tracer type and all tracing methods.
Tracer Type
type Tracer struct {
// contains filtered or unexported fields
}
The main entry point for distributed tracing. Holds OpenTelemetry tracing configuration and runtime state. All operations on Tracer are thread-safe.
Important Notes
- Immutable: Tracer is immutable after creation via
New(). All configuration must be done through functional options. - Thread-safe: All methods are safe for concurrent use.
- Global state: By default, does NOT set the global OpenTelemetry tracer provider. Use
WithGlobalTracerProvider() option if needed.
Constructor Functions
New
func New(opts ...Option) (*Tracer, error)
Creates a new Tracer with the given options. Returns an error if the tracing provider fails to initialize.
Default configuration:
- Service name:
"rivaas-service". - Service version:
"1.0.0". - Sample rate:
1.0 (100%). - Provider:
NoopProvider.
Example:
tracer, err := tracing.New(
tracing.WithServiceName("my-api"),
tracing.WithOTLP("localhost:4317"),
tracing.WithSampleRate(0.1),
)
if err != nil {
log.Fatal(err)
}
defer tracer.Shutdown(context.Background())
MustNew
func MustNew(opts ...Option) *Tracer
Creates a new Tracer with the given options. Panics if the tracing provider fails to initialize. Use this when you want to panic on initialization errors.
Example:
tracer := tracing.MustNew(
tracing.WithServiceName("my-api"),
tracing.WithStdout(),
)
defer tracer.Shutdown(context.Background())
Lifecycle Methods
Start
func (t *Tracer) Start(ctx context.Context) error
Initializes OTLP providers that require network connections. The context is used for the OTLP connection establishment. This method is idempotent; calling it multiple times is safe.
Required for: OTLP (gRPC and HTTP) providers
Optional for: Noop and Stdout providers (they initialize immediately in New())
Example:
tracer := tracing.MustNew(
tracing.WithOTLP("localhost:4317"),
)
if err := tracer.Start(context.Background()); err != nil {
log.Fatal(err)
}
Shutdown
func (t *Tracer) Shutdown(ctx context.Context) error
Gracefully shuts down the tracing system, flushing any pending spans. This should be called before the application exits to ensure all spans are exported. This method is idempotent - calling it multiple times is safe and will only perform shutdown once.
Example:
defer func() {
ctx, cancel := context.WithTimeout(context.Background(), 5*time.Second)
defer cancel()
if err := tracer.Shutdown(ctx); err != nil {
log.Printf("Error shutting down tracer: %v", err)
}
}()
Span Management Methods
StartSpan
func (t *Tracer) StartSpan(ctx context.Context, name string, opts ...trace.SpanStartOption) (context.Context, trace.Span)
Starts a new span with the given name and options. Returns a new context with the span attached and the span itself.
If tracing is disabled, returns the original context and a non-recording span. The returned span should always be ended, even if tracing is disabled.
Parameters:
ctx: Parent contextname: Span name (should be descriptive)opts: Optional OpenTelemetry span start options
Returns:
- New context with span attached
- The created span
Example:
ctx, span := tracer.StartSpan(ctx, "database-query")
defer tracer.FinishSpan(span, http.StatusOK)
tracer.SetSpanAttribute(span, "db.query", "SELECT * FROM users")
FinishSpan
func (t *Tracer) FinishSpan(span trace.Span, statusCode int)
Completes the span with the given status code. Sets the span status based on the HTTP status code:
- 2xx-3xx: Success (
codes.Ok) - 4xx-5xx: Error (
codes.Error)
This method is safe to call multiple times; subsequent calls are no-ops.
Parameters:
span: The span to finishstatusCode: HTTP status code (e.g., http.StatusOK)
Example:
defer tracer.FinishSpan(span, http.StatusOK)
SetSpanAttribute
func (t *Tracer) SetSpanAttribute(span trace.Span, key string, value any)
Adds an attribute to the span with type-safe handling.
Supported types:
string, int, int64, float64, bool: native OpenTelemetry handling- Other types: converted to string using
fmt.Sprintf
This is a no-op if tracing is disabled, span is nil, or span is not recording.
Parameters:
span: The span to add the attribute tokey: Attribute keyvalue: Attribute value
Example:
tracer.SetSpanAttribute(span, "user.id", 12345)
tracer.SetSpanAttribute(span, "user.premium", true)
tracer.SetSpanAttribute(span, "user.name", "Alice")
AddSpanEvent
func (t *Tracer) AddSpanEvent(span trace.Span, name string, attrs ...attribute.KeyValue)
Adds an event to the span with optional attributes. Events represent important moments in a span’s lifetime.
This is a no-op if tracing is disabled, span is nil, or span is not recording.
Parameters:
span: The span to add the event toname: Event nameattrs: Optional event attributes
Example:
import "go.opentelemetry.io/otel/attribute"
tracer.AddSpanEvent(span, "cache_hit",
attribute.String("key", "user:123"),
attribute.Int("ttl_seconds", 300),
)
Context Propagation Methods
func (t *Tracer) ExtractTraceContext(ctx context.Context, headers http.Header) context.Context
Extracts trace context from HTTP request headers. Returns a new context with the extracted trace information.
If no trace context is found in headers, returns the original context. Uses W3C Trace Context format by default.
Parameters:
ctx: Base contextheaders: HTTP headers to extract from
Returns:
- Context with extracted trace information
Example:
ctx := tracer.ExtractTraceContext(r.Context(), r.Header)
ctx, span := tracer.StartSpan(ctx, "operation")
defer tracer.FinishSpan(span, http.StatusOK)
InjectTraceContext
func (t *Tracer) InjectTraceContext(ctx context.Context, headers http.Header)
Injects trace context into HTTP headers. This allows trace context to propagate across service boundaries.
Uses W3C Trace Context format by default. This is a no-op if tracing is disabled.
Parameters:
ctx: Context containing trace informationheaders: HTTP headers to inject into
Example:
req, _ := http.NewRequestWithContext(ctx, "GET", url, nil)
tracer.InjectTraceContext(ctx, req.Header)
resp, _ := http.DefaultClient.Do(req)
Request Span Methods
These methods are used internally by the middleware but can also be used for custom HTTP handling.
StartRequestSpan
func (t *Tracer) StartRequestSpan(ctx context.Context, req *http.Request, path string, isStatic bool) (context.Context, trace.Span)
Starts a span for an HTTP request. This is used by the middleware to create request spans with standard attributes.
Parameters:
ctx: Request contextreq: HTTP requestpath: Request pathisStatic: Whether this is a static route
Returns:
- Context with span
- The created span
FinishRequestSpan
func (t *Tracer) FinishRequestSpan(span trace.Span, statusCode int)
Completes the span for an HTTP request. Sets the HTTP status code attribute and invokes the span finish hook if configured.
Parameters:
span: The span to finishstatusCode: HTTP response status code
Accessor Methods
IsEnabled
func (t *Tracer) IsEnabled() bool
Returns true if tracing is enabled.
ServiceName
func (t *Tracer) ServiceName() string
Returns the service name.
ServiceVersion
func (t *Tracer) ServiceVersion() string
Returns the service version.
GetTracer
func (t *Tracer) GetTracer() trace.Tracer
Returns the OpenTelemetry tracer.
GetPropagator
func (t *Tracer) GetPropagator() propagation.TextMapPropagator
Returns the OpenTelemetry propagator.
GetProvider
func (t *Tracer) GetProvider() Provider
Returns the current tracing provider.
Context Helper Functions
These are package-level functions for working with spans through context.
TraceID
func TraceID(ctx context.Context) string
Returns the current trace ID from the active span in the context. Returns an empty string if no active span or span context is invalid.
Example:
traceID := tracing.TraceID(ctx)
log.Printf("Processing request [trace=%s]", traceID)
SpanID
func SpanID(ctx context.Context) string
Returns the current span ID from the active span in the context. Returns an empty string if no active span or span context is invalid.
Example:
spanID := tracing.SpanID(ctx)
log.Printf("Processing request [span=%s]", spanID)
SetSpanAttributeFromContext
func SetSpanAttributeFromContext(ctx context.Context, key string, value any)
Adds an attribute to the current span from context. This is a no-op if tracing is not active.
Example:
func handleRequest(ctx context.Context) {
tracing.SetSpanAttributeFromContext(ctx, "user.role", "admin")
tracing.SetSpanAttributeFromContext(ctx, "user.id", 12345)
}
AddSpanEventFromContext
func AddSpanEventFromContext(ctx context.Context, name string, attrs ...attribute.KeyValue)
Adds an event to the current span from context. This is a no-op if tracing is not active.
Example:
import "go.opentelemetry.io/otel/attribute"
tracing.AddSpanEventFromContext(ctx, "cache_miss",
attribute.String("key", "user:123"),
)
TraceContext
func TraceContext(ctx context.Context) context.Context
Returns the context as-is (it should already contain trace information). Provided for API consistency.
Event Types
EventType
type EventType int
const (
EventError EventType = iota // Error events
EventWarning // Warning events
EventInfo // Informational events
EventDebug // Debug events
)
Event severity levels for internal operational events.
Event
type Event struct {
Type EventType
Message string
Args []any // slog-style key-value pairs
}
Internal operational event from the tracing package. Events are used to report errors, warnings, and informational messages about the tracing system’s operation.
EventHandler
type EventHandler func(Event)
Processes internal operational events from the tracing package. Implementations can log events, send them to monitoring systems, or take custom actions based on event type.
Example:
tracing.WithEventHandler(func(e tracing.Event) {
if e.Type == tracing.EventError {
sentry.CaptureMessage(e.Message)
}
slog.Default().Info(e.Message, e.Args...)
})
DefaultEventHandler
func DefaultEventHandler(logger *slog.Logger) EventHandler
Returns an EventHandler that logs events to the provided slog.Logger. This is the default implementation used by WithLogger.
If logger is nil, returns a no-op handler that discards all events.
Hook Types
SpanStartHook
type SpanStartHook func(ctx context.Context, span trace.Span, req *http.Request)
Called when a request span is started. It receives the context, span, and HTTP request. This can be used for custom attribute injection, dynamic sampling, or integration with APM tools.
Example:
hook := func(ctx context.Context, span trace.Span, req *http.Request) {
if tenantID := req.Header.Get("X-Tenant-ID"); tenantID != "" {
span.SetAttributes(attribute.String("tenant.id", tenantID))
}
}
tracer := tracing.MustNew(
tracing.WithSpanStartHook(hook),
)
SpanFinishHook
type SpanFinishHook func(span trace.Span, statusCode int)
Called when a request span is finished. It receives the span and the HTTP status code. This can be used for custom metrics, logging, or post-processing.
Example:
hook := func(span trace.Span, statusCode int) {
if statusCode >= 500 {
metrics.IncrementServerErrors()
}
}
tracer := tracing.MustNew(
tracing.WithSpanFinishHook(hook),
)
ContextTracing Type
type ContextTracing struct {
// contains filtered or unexported fields
}
A helper type for router context integration that provides convenient access to tracing functionality within HTTP handlers.
NewContextTracing
func NewContextTracing(ctx context.Context, tracer *Tracer, span trace.Span) *ContextTracing
Creates a new context tracing helper. Panics if ctx is nil.
Parameters:
ctx: The request contexttracer: The Tracer instancespan: The current span
Example:
ct := tracing.NewContextTracing(ctx, tracer, span)
ContextTracing Methods
TraceID
func (ct *ContextTracing) TraceID() string
Returns the current trace ID. Returns an empty string if no valid span.
SpanID
func (ct *ContextTracing) SpanID() string
Returns the current span ID. Returns an empty string if no valid span.
SetSpanAttribute
func (ct *ContextTracing) SetSpanAttribute(key string, value any)
Adds an attribute to the current span. No-op if span is nil or not recording.
AddSpanEvent
func (ct *ContextTracing) AddSpanEvent(name string, attrs ...attribute.KeyValue)
Adds an event to the current span. No-op if span is nil or not recording.
TraceContext
func (ct *ContextTracing) TraceContext() context.Context
Returns the trace context.
GetSpan
func (ct *ContextTracing) GetSpan() trace.Span
Returns the current span.
GetTracer
func (ct *ContextTracing) GetTracer() *Tracer
Returns the underlying Tracer.
ContextTracing Example
func handleRequest(w http.ResponseWriter, r *http.Request, tracer *tracing.Tracer) {
ctx := r.Context()
span := trace.SpanFromContext(ctx)
// Create context tracing helper
ct := tracing.NewContextTracing(ctx, tracer, span)
// Use helper methods
ct.SetSpanAttribute("user.id", "123")
ct.AddSpanEvent("processing_started")
// Get trace info for logging
log.Printf("Processing [trace=%s, span=%s]", ct.TraceID(), ct.SpanID())
}
Constants
Default Values
const (
DefaultServiceName = "rivaas-service"
DefaultServiceVersion = "1.0.0"
DefaultSampleRate = 1.0
)
Default configuration values used when not explicitly set.
Provider Types
const (
NoopProvider Provider = "noop"
StdoutProvider Provider = "stdout"
OTLPProvider Provider = "otlp"
OTLPHTTPProvider Provider = "otlp-http"
)
Available tracing providers.
Next Steps
2 - Tracer Options
All configuration options for Tracer initialization
Complete reference for all Option functions used to configure the Tracer.
Option Type
type Option func(*Tracer)
Configuration option function type used with New() and MustNew(). Options are applied during Tracer creation.
Service Configuration Options
WithServiceName
func WithServiceName(name string) Option
Sets the service name for tracing. This name appears in span attributes as service.name.
Parameters:
name: Service identifier like "user-api" or "order-service".
Default: "rivaas-service"
Example:
tracer := tracing.MustNew(
tracing.WithServiceName("user-api"),
)
WithServiceVersion
func WithServiceVersion(version string) Option
Sets the service version for tracing. This version appears in span attributes as service.version.
Parameters:
version: Service version like "v1.2.3" or "dev".
Default: "1.0.0"
Example:
tracer := tracing.MustNew(
tracing.WithServiceName("user-api"),
tracing.WithServiceVersion("v1.2.3"),
)
Provider Options
Only one provider can be configured at a time. Configuring multiple providers results in a validation error.
WithNoop
Configures noop provider. This is the default. No traces are exported. Use for testing or when tracing is disabled.
Example:
tracer := tracing.MustNew(
tracing.WithServiceName("my-service"),
tracing.WithNoop(),
)
WithStdout
Configures stdout provider for development/debugging. Traces are printed to standard output in pretty-printed JSON format.
Example:
tracer := tracing.MustNew(
tracing.WithServiceName("my-service"),
tracing.WithStdout(),
)
WithOTLP
func WithOTLP(endpoint string, opts ...OTLPOption) Option
Configures OTLP gRPC provider with endpoint. Use this for production deployments with OpenTelemetry collectors.
Parameters:
endpoint: OTLP endpoint in format "host:port" (e.g., "localhost:4317")opts: Optional OTLP-specific options (e.g., OTLPInsecure())
Requires: Call tracer.Start(ctx) before tracing
Example:
// Secure (TLS enabled by default)
tracer := tracing.MustNew(
tracing.WithServiceName("my-service"),
tracing.WithOTLP("collector.example.com:4317"),
)
// Insecure (local development)
tracer := tracing.MustNew(
tracing.WithServiceName("my-service"),
tracing.WithOTLP("localhost:4317", tracing.OTLPInsecure()),
)
WithOTLPHTTP
func WithOTLPHTTP(endpoint string) Option
Configures OTLP HTTP provider with endpoint. Use this when gRPC is not available or HTTP is preferred.
Parameters:
endpoint: OTLP HTTP endpoint with protocol (e.g., "http://localhost:4318", "https://collector:4318")
Requires: Call tracer.Start(ctx) before tracing
Example:
// HTTP (insecure - development)
tracer := tracing.MustNew(
tracing.WithServiceName("my-service"),
tracing.WithOTLPHTTP("http://localhost:4318"),
)
// HTTPS (secure - production)
tracer := tracing.MustNew(
tracing.WithServiceName("my-service"),
tracing.WithOTLPHTTP("https://collector.example.com:4318"),
)
OTLP Options
OTLPInsecure
func OTLPInsecure() OTLPOption
Enables insecure gRPC for OTLP. Default is false (uses TLS). Set to true for local development.
Example:
tracer := tracing.MustNew(
tracing.WithServiceName("my-service"),
tracing.WithOTLP("localhost:4317", tracing.OTLPInsecure()),
)
Sampling Options
WithSampleRate
func WithSampleRate(rate float64) Option
Sets the sampling rate (0.0 to 1.0). Values outside this range are clamped to valid bounds.
A rate of 1.0 samples all requests, 0.5 samples 50%, and 0.0 samples none. Sampling decisions are made per-request based on the configured rate.
Parameters:
rate: Sampling rate between 0.0 and 1.0
Default: 1.0 (100% sampling)
Example:
// Sample 10% of requests
tracer := tracing.MustNew(
tracing.WithServiceName("my-service"),
tracing.WithSampleRate(0.1),
)
Hook Options
WithSpanStartHook
func WithSpanStartHook(hook SpanStartHook) Option
Sets a callback that is invoked when a request span is started. The hook receives the context, span, and HTTP request, allowing custom attribute injection, dynamic sampling decisions, or integration with APM tools.
Type:
type SpanStartHook func(ctx context.Context, span trace.Span, req *http.Request)
Example:
startHook := func(ctx context.Context, span trace.Span, req *http.Request) {
if tenantID := req.Header.Get("X-Tenant-ID"); tenantID != "" {
span.SetAttributes(attribute.String("tenant.id", tenantID))
}
}
tracer := tracing.MustNew(
tracing.WithServiceName("my-service"),
tracing.WithSpanStartHook(startHook),
)
WithSpanFinishHook
func WithSpanFinishHook(hook SpanFinishHook) Option
Sets a callback that is invoked when a request span is finished. The hook receives the span and HTTP status code, allowing custom metrics recording, logging, or post-processing.
Type:
type SpanFinishHook func(span trace.Span, statusCode int)
Example:
finishHook := func(span trace.Span, statusCode int) {
if statusCode >= 500 {
metrics.IncrementServerErrors()
}
}
tracer := tracing.MustNew(
tracing.WithServiceName("my-service"),
tracing.WithSpanFinishHook(finishHook),
)
Logging Options
WithLogger
func WithLogger(logger *slog.Logger) Option
Sets the logger for internal operational events using the default event handler. This is a convenience wrapper around WithEventHandler that logs events to the provided slog.Logger.
Parameters:
logger: *slog.Logger for logging internal events
Example:
import "log/slog"
logger := slog.New(slog.NewJSONHandler(os.Stdout, &slog.HandlerOptions{
Level: slog.LevelDebug,
}))
tracer := tracing.MustNew(
tracing.WithServiceName("my-service"),
tracing.WithLogger(logger),
)
WithEventHandler
func WithEventHandler(handler EventHandler) Option
Sets a custom event handler for internal operational events. Use this for advanced use cases like sending errors to Sentry, custom alerting, or integrating with non-slog logging systems.
Type:
type EventHandler func(Event)
Example:
eventHandler := func(e tracing.Event) {
switch e.Type {
case tracing.EventError:
sentry.CaptureMessage(e.Message)
myLogger.Error(e.Message, e.Args...)
case tracing.EventWarning:
myLogger.Warn(e.Message, e.Args...)
case tracing.EventInfo:
myLogger.Info(e.Message, e.Args...)
case tracing.EventDebug:
myLogger.Debug(e.Message, e.Args...)
}
}
tracer := tracing.MustNew(
tracing.WithServiceName("my-service"),
tracing.WithEventHandler(eventHandler),
)
Advanced Options
WithTracerProvider
func WithTracerProvider(provider trace.TracerProvider) Option
Allows you to provide a custom OpenTelemetry TracerProvider. When using this option, the package will NOT set the global otel.SetTracerProvider() by default. Use WithGlobalTracerProvider() if you want global registration.
Use cases:
- Manage tracer provider lifecycle yourself
- Need multiple independent tracing configurations
- Want to avoid global state in your application
Important: When using WithTracerProvider, provider options (WithOTLP, WithStdout, etc.) are ignored since you’re managing the provider yourself. You are also responsible for calling Shutdown() on your provider.
Example:
import sdktrace "go.opentelemetry.io/otel/sdk/trace"
tp := sdktrace.NewTracerProvider(
// Your custom configuration
sdktrace.WithSampler(sdktrace.AlwaysSample()),
)
tracer := tracing.MustNew(
tracing.WithServiceName("my-service"),
tracing.WithTracerProvider(tp),
)
// You manage tp.Shutdown() yourself
defer tp.Shutdown(context.Background())
WithCustomTracer
func WithCustomTracer(tracer trace.Tracer) Option
Allows using a custom OpenTelemetry tracer. This is useful when you need specific tracer configuration or want to use a tracer from an existing OpenTelemetry setup.
Example:
tp := trace.NewTracerProvider(...)
customTracer := tp.Tracer("my-tracer")
tracer := tracing.MustNew(
tracing.WithServiceName("my-service"),
tracing.WithCustomTracer(customTracer),
)
WithCustomPropagator
func WithCustomPropagator(propagator propagation.TextMapPropagator) Option
Allows using a custom OpenTelemetry propagator. This is useful for custom trace context propagation formats. By default, uses the global propagator from otel.GetTextMapPropagator() (W3C Trace Context).
Example:
import "go.opentelemetry.io/otel/propagation"
// Use W3C Trace Context explicitly
prop := propagation.TraceContext{}
tracer := tracing.MustNew(
tracing.WithServiceName("my-service"),
tracing.WithCustomPropagator(prop),
)
Using B3 propagation:
import "go.opentelemetry.io/contrib/propagators/b3"
tracer := tracing.MustNew(
tracing.WithServiceName("my-service"),
tracing.WithCustomPropagator(b3.New()),
)
WithGlobalTracerProvider
func WithGlobalTracerProvider() Option
Registers the tracer provider as the global OpenTelemetry tracer provider via otel.SetTracerProvider(). By default, tracer providers are not registered globally to allow multiple tracing configurations to coexist in the same process.
Use when:
- You want
otel.GetTracerProvider() to return your tracer - Integrating with libraries that use the global tracer
- Single tracer for entire application
Example:
tracer := tracing.MustNew(
tracing.WithServiceName("my-service"),
tracing.WithOTLP("localhost:4317"),
tracing.WithGlobalTracerProvider(), // Register globally
)
Option Combinations
Development Configuration
tracer := tracing.MustNew(
tracing.WithServiceName("my-api"),
tracing.WithServiceVersion("dev"),
tracing.WithStdout(),
tracing.WithSampleRate(1.0),
tracing.WithLogger(slog.Default()),
)
Production Configuration
tracer := tracing.MustNew(
tracing.WithServiceName("user-api"),
tracing.WithServiceVersion(os.Getenv("VERSION")),
tracing.WithOTLP(os.Getenv("OTLP_ENDPOINT")),
tracing.WithSampleRate(0.1),
tracing.WithSpanStartHook(enrichSpan),
tracing.WithSpanFinishHook(recordMetrics),
)
Testing Configuration
tracer := tracing.MustNew(
tracing.WithServiceName("test-service"),
tracing.WithServiceVersion("v1.0.0"),
tracing.WithNoop(),
tracing.WithSampleRate(1.0),
)
Validation Errors
Configuration is validated when calling New() or MustNew(). Common validation errors:
Multiple Providers
// ✗ Error: multiple providers configured
tracer, err := tracing.New(
tracing.WithServiceName("my-service"),
tracing.WithStdout(),
tracing.WithOTLP("localhost:4317"), // Error!
)
// Returns: "validation errors: provider: multiple providers configured"
Solution: Only configure one provider.
Empty Service Name
// ✗ Error: service name cannot be empty
tracer, err := tracing.New(
tracing.WithServiceName(""),
)
// Returns: "invalid configuration: serviceName: cannot be empty"
Solution: Always provide a service name.
Invalid Sample Rate
// Values are automatically clamped
tracer := tracing.MustNew(
tracing.WithServiceName("my-service"),
tracing.WithSampleRate(1.5), // Clamped to 1.0
)
Sample rates outside 0.0-1.0 are automatically clamped to valid bounds.
Complete Option Reference
| Option | Description | Default |
|---|
WithServiceName(name) | Set service name | "rivaas-service" |
WithServiceVersion(version) | Set service version | "1.0.0" |
WithNoop() | Noop provider | Yes (default) |
WithStdout() | Stdout provider | - |
WithOTLP(endpoint, opts...) | OTLP gRPC provider | - |
WithOTLPHTTP(endpoint) | OTLP HTTP provider | - |
WithSampleRate(rate) | Sampling rate (0.0-1.0) | 1.0 |
WithSpanStartHook(hook) | Span start callback | - |
WithSpanFinishHook(hook) | Span finish callback | - |
WithLogger(logger) | Set slog logger | - |
WithEventHandler(handler) | Custom event handler | - |
WithTracerProvider(provider) | Custom tracer provider | - |
WithCustomTracer(tracer) | Custom tracer | - |
WithCustomPropagator(prop) | Custom propagator | W3C Trace Context |
WithGlobalTracerProvider() | Register globally | No |
Next Steps
3 - Middleware Options
All configuration options for HTTP middleware
Complete reference for all MiddlewareOption functions used to configure the HTTP tracing middleware.
MiddlewareOption Type
type MiddlewareOption func(*middlewareConfig)
Configuration option function type used with Middleware() and MustMiddleware(). These options control HTTP request tracing behavior.
Path Exclusion Options
Exclude specific paths from tracing to reduce noise and overhead.
WithExcludePaths
func WithExcludePaths(paths ...string) MiddlewareOption
Excludes specific paths from tracing. Excluded paths will not create spans or record any tracing data. This is useful for health checks, metrics endpoints, etc.
Maximum of 1000 paths can be excluded to prevent unbounded growth.
Parameters:
paths: Exact paths to exclude (e.g., "/health", "/metrics")
Performance: O(1) hash map lookup
Example:
handler := tracing.Middleware(tracer,
tracing.WithExcludePaths("/health", "/metrics", "/ready", "/live"),
)(mux)
WithExcludePrefixes
func WithExcludePrefixes(prefixes ...string) MiddlewareOption
Excludes paths with the given prefixes from tracing. This is useful for excluding entire path hierarchies like /debug/, /internal/, etc.
Parameters:
prefixes: Path prefixes to exclude (e.g., "/debug/", "/internal/")
Performance: O(n) where n = number of prefixes
Example:
handler := tracing.Middleware(tracer,
tracing.WithExcludePrefixes("/debug/", "/internal/", "/.well-known/"),
)(mux)
Matches:
/debug/pprof/debug/vars/internal/health/.well-known/acme-challenge
WithExcludePatterns
func WithExcludePatterns(patterns ...string) MiddlewareOption
Excludes paths matching the given regex patterns from tracing. The patterns are compiled once during configuration. Returns a validation error if any pattern fails to compile.
Parameters:
patterns: Regular expression patterns (e.g., "^/v[0-9]+/internal/.*")
Performance: O(p) where p = number of patterns
Validation: Invalid regex patterns cause the middleware to panic during initialization.
Example:
handler := tracing.Middleware(tracer,
tracing.WithExcludePatterns(
`^/v[0-9]+/internal/.*`, // Version-prefixed internal routes
`^/api/health.*`, // Any health-related endpoint
`^/debug/.*`, // All debug routes
),
)(mux)
Matches:
/v1/internal/status/v2/internal/debug/api/health/api/health/db/debug/pprof/heap
func WithHeaders(headers ...string) MiddlewareOption
Records specific request headers as span attributes. Header names are case-insensitive. Recorded as http.request.header.{name}.
Security: Sensitive headers (Authorization, Cookie, etc.) are automatically filtered out to prevent accidental exposure of credentials in traces.
Parameters:
headers: Header names to record (case-insensitive)
Recorded as: Lowercase header names (http.request.header.x-request-id)
Example:
handler := tracing.Middleware(tracer,
tracing.WithHeaders("X-Request-ID", "X-Correlation-ID", "User-Agent"),
)(mux)
Span attributes:
http.request.header.x-request-id: "abc123"http.request.header.x-correlation-id: "xyz789"http.request.header.user-agent: "Mozilla/5.0..."
The following headers are automatically filtered and will never be recorded, even if explicitly included:
AuthorizationCookieSet-CookieX-API-KeyX-Auth-TokenProxy-AuthorizationWWW-Authenticate
Example:
// Authorization is automatically filtered
handler := tracing.Middleware(tracer,
tracing.WithHeaders(
"X-Request-ID",
"Authorization", // ← Filtered, won't be recorded
"X-Correlation-ID",
),
)(mux)
Query Parameter Recording Options
Default Behavior
By default, all query parameters are recorded as span attributes.
WithRecordParams
func WithRecordParams(params ...string) MiddlewareOption
Specifies which URL query parameters to record as span attributes. Only parameters in this list will be recorded. This provides fine-grained control over which parameters are traced.
If this option is not used, all query parameters are recorded by default (unless WithoutParams is used).
Parameters:
params: Parameter names to record
Recorded as: http.request.param.{name}
Example:
handler := tracing.Middleware(tracer,
tracing.WithRecordParams("user_id", "request_id", "page", "limit"),
)(mux)
Request: GET /api/users?page=2&limit=10&user_id=123&secret=xyz
Span attributes:
http.request.param.page: ["2"]http.request.param.limit: ["10"]http.request.param.user_id: ["123"]secret is not recorded (not in whitelist)
WithExcludeParams
func WithExcludeParams(params ...string) MiddlewareOption
Specifies which URL query parameters to exclude from tracing. This is useful for blacklisting sensitive parameters while recording all others.
Parameters in this list will never be recorded, even if WithRecordParams includes them (blacklist takes precedence).
Parameters:
params: Parameter names to exclude
Example:
handler := tracing.Middleware(tracer,
tracing.WithExcludeParams("password", "token", "api_key", "secret"),
)(mux)
Request: GET /api/users?page=2&password=secret123&user_id=123
Span attributes:
http.request.param.page: ["2"]http.request.param.user_id: ["123"]password is not recorded (blacklisted)
WithoutParams
func WithoutParams() MiddlewareOption
Disables recording URL query parameters as span attributes. By default, all query parameters are recorded. Use this option if parameters may contain sensitive data.
Example:
handler := tracing.Middleware(tracer,
tracing.WithoutParams(),
)(mux)
No query parameters will be recorded regardless of the request.
Parameter Recording Precedence
When multiple parameter options are used:
WithoutParams() - If set, no parameters are recordedWithExcludeParams() - Blacklist takes precedence over whitelistWithRecordParams() - Only whitelisted parameters are recorded- Default - All parameters are recorded
Example:
// Whitelist with blacklist
handler := tracing.Middleware(tracer,
tracing.WithRecordParams("page", "limit", "sort", "api_key"),
tracing.WithExcludeParams("api_key", "token"), // Blacklist overrides
)(mux)
Result: page, limit, and sort are recorded, but api_key is excluded (blacklist wins).
Middleware Functions
Middleware
func Middleware(tracer *Tracer, opts ...MiddlewareOption) func(http.Handler) http.Handler
Creates a middleware function for standalone HTTP integration. Panics if any middleware option is invalid (e.g., invalid regex pattern).
Parameters:
tracer: Tracer instanceopts: Middleware configuration options
Returns: HTTP middleware function
Panics: If middleware options are invalid
Example:
tracer := tracing.MustNew(
tracing.WithServiceName("my-api"),
tracing.WithOTLP("localhost:4317"),
)
handler := tracing.Middleware(tracer,
tracing.WithExcludePaths("/health", "/metrics"),
tracing.WithHeaders("X-Request-ID"),
)(mux)
MustMiddleware
func MustMiddleware(tracer *Tracer, opts ...MiddlewareOption) func(http.Handler) http.Handler
Creates a middleware function for standalone HTTP integration. It panics if any middleware option is invalid (e.g., invalid regex pattern). This is a convenience wrapper around Middleware for consistency with MustNew.
Behavior: Identical to Middleware() - both panic on invalid options.
Example:
handler := tracing.MustMiddleware(tracer,
tracing.WithExcludePaths("/health", "/metrics"),
tracing.WithHeaders("X-Request-ID"),
)(mux)
Complete Examples
Minimal Middleware
// Trace everything with no filtering
handler := tracing.Middleware(tracer)(mux)
Production Middleware
handler := tracing.Middleware(tracer,
// Exclude observability endpoints
tracing.WithExcludePaths("/health", "/metrics", "/ready", "/live"),
// Exclude debug endpoints
tracing.WithExcludePrefixes("/debug/", "/internal/"),
// Record correlation headers
tracing.WithHeaders("X-Request-ID", "X-Correlation-ID"),
// Whitelist safe parameters
tracing.WithRecordParams("page", "limit", "sort", "filter"),
// Blacklist sensitive parameters
tracing.WithExcludeParams("password", "token", "api_key"),
)(mux)
Development Middleware
handler := tracing.Middleware(tracer,
// Only exclude metrics
tracing.WithExcludePaths("/metrics"),
// Record all headers (except sensitive ones)
tracing.WithHeaders("X-Request-ID", "X-Correlation-ID", "User-Agent"),
)(mux)
High-Security Middleware
handler := tracing.Middleware(tracer,
// Exclude health checks
tracing.WithExcludePaths("/health"),
// No headers recorded
// No query parameters recorded
tracing.WithoutParams(),
)(mux)
| Method | Complexity | Performance |
|---|
WithExcludePaths() | O(1) | ~9ns per request (hash lookup) |
WithExcludePrefixes() | O(n) | ~9ns per request (n prefixes) |
WithExcludePatterns() | O(p) | ~20ns per request (p patterns) |
Recommendation: Use exact paths when possible for best performance.
Memory Usage
- Path exclusion: ~100 bytes per path
- Header recording: ~50 bytes per header
- Parameter recording: ~30 bytes per parameter name
Limits
- Maximum excluded paths: 1000 (enforced by
WithExcludePaths) - No limit on: Prefixes, patterns, headers, parameters
Validation Errors
Configuration is validated when calling Middleware() or MustMiddleware(). Invalid options cause a panic.
Invalid Regex Pattern
// ✗ Panics: invalid regex
handler := tracing.Middleware(tracer,
tracing.WithExcludePatterns(`[invalid regex`),
)(mux)
// Panics: "middleware validation errors: excludePatterns: invalid regex..."
Solution: Ensure regex patterns are valid.
Option Reference Table
| Option | Description | Default Behavior |
|---|
WithExcludePaths(paths...) | Exclude exact paths | All paths traced |
WithExcludePrefixes(prefixes...) | Exclude by prefix | All paths traced |
WithExcludePatterns(patterns...) | Exclude by regex | All paths traced |
WithHeaders(headers...) | Record headers | No headers recorded |
WithRecordParams(params...) | Whitelist params | All params recorded |
WithExcludeParams(params...) | Blacklist params | No params excluded |
WithoutParams() | Disable params | All params recorded |
Best Practices
Always Exclude Health Checks
tracing.WithExcludePaths("/health", "/metrics", "/ready", "/live")
Health checks are high-frequency and low-value for tracing.
Use Exact Paths for Common Exclusions
// ✓ Good - fastest
tracing.WithExcludePaths("/health", "/metrics")
// ✗ Less optimal - slower
tracing.WithExcludePatterns("^/(health|metrics)$")
Blacklist Sensitive Parameters
tracing.WithExcludeParams(
"password", "token", "api_key", "secret",
"credit_card", "ssn", "access_token",
)
tracing.WithHeaders("X-Request-ID", "X-Correlation-ID", "X-Trace-ID")
Helps correlate traces with logs and other observability data.
Combine Exclusion Methods
handler := tracing.Middleware(tracer,
tracing.WithExcludePaths("/health", "/metrics"), // Exact
tracing.WithExcludePrefixes("/debug/", "/internal/"), // Prefix
tracing.WithExcludePatterns(`^/v[0-9]+/internal/.*`), // Regex
)(mux)
Next Steps
4 - Troubleshooting
Common issues and solutions for the tracing package
Common issues and solutions when using the tracing package.
Traces Not Appearing
Symptom
No traces appear in your tracing backend (Jaeger, Zipkin, etc.) even though tracing is configured.
Possible Causes & Solutions
1. OTLP Provider Not Started
Problem: OTLP providers require calling Start(ctx) before tracing.
Solution:
tracer := tracing.MustNew(
tracing.WithServiceName("my-service"),
tracing.WithOTLP("localhost:4317"),
)
// ✓ Required for OTLP providers
if err := tracer.Start(context.Background()); err != nil {
log.Fatal(err)
}
2. Sampling Rate Too Low
Problem: Sample rate is set too low. For example, 1% sampling means 99% of requests aren’t traced.
Solution: Increase sample rate or remove sampling for testing.
// Development - trace everything
tracer := tracing.MustNew(
tracing.WithServiceName("my-service"),
tracing.WithSampleRate(1.0), // 100% sampling
)
Problem: Using Noop provider (no traces exported).
Solution: Verify provider configuration:
// ✗ Bad - no traces exported
tracer := tracing.MustNew(
tracing.WithServiceName("my-service"),
tracing.WithNoop(), // No traces!
)
// ✓ Good - traces exported
tracer := tracing.MustNew(
tracing.WithServiceName("my-service"),
tracing.WithOTLP("localhost:4317"),
)
4. Paths Excluded from Tracing
Problem: Paths are excluded via middleware options.
Solution: Check middleware exclusions.
// Check if your paths are excluded
handler := tracing.Middleware(tracer,
tracing.WithExcludePaths("/health", "/api/users"), // ← Is this excluding your endpoint?
)(mux)
5. Shutdown Called Too Early
Problem: Application exits before spans are exported.
Solution: Ensure proper shutdown with timeout:
defer func() {
ctx, cancel := context.WithTimeout(context.Background(), 5*time.Second)
defer cancel()
if err := tracer.Shutdown(ctx); err != nil {
log.Printf("Error shutting down tracer: %v", err)
}
}()
6. OTLP Endpoint Unreachable
Problem: OTLP collector is not running or unreachable.
Solution: Verify collector is running:
# Check if collector is listening
nc -zv localhost 4317 # OTLP gRPC
nc -zv localhost 4318 # OTLP HTTP
Check logs for connection errors:
logger := slog.New(slog.NewTextHandler(os.Stdout, &slog.HandlerOptions{
Level: slog.LevelDebug,
}))
tracer := tracing.MustNew(
tracing.WithServiceName("my-service"),
tracing.WithOTLP("localhost:4317"),
tracing.WithLogger(logger), // See connection errors
)
Context Propagation Issues
Symptom
Services create separate traces instead of one distributed trace.
Possible Causes & Solutions
1. Context Not Propagated
Problem: Context is not passed through the call chain.
Solution: Always pass context:
// ✓ Good - context propagates
func handler(ctx context.Context) {
result := doWork(ctx) // Pass context
}
// ✗ Bad - context lost
func handler(ctx context.Context) {
result := doWork(context.Background()) // Lost!
}
2. Trace Context Not Injected
Problem: Trace context not injected into outgoing requests.
Solution: Always inject before making requests:
req, _ := http.NewRequestWithContext(ctx, "GET", url, nil)
// ✓ Required - inject trace context
tracer.InjectTraceContext(ctx, req.Header)
resp, _ := http.DefaultClient.Do(req)
Problem: Incoming requests don’t extract trace context.
Solution: Middleware automatically extracts, or do it manually:
// Automatic (with middleware)
handler := tracing.Middleware(tracer)(mux)
// Manual (without middleware)
func myHandler(w http.ResponseWriter, r *http.Request) {
ctx := tracer.ExtractTraceContext(r.Context(), r.Header)
// Use extracted context...
}
4. Different Propagators
Problem: Services use different propagation formats.
Solution: Ensure all services use the same propagator (default is W3C Trace Context):
// All services should use default (W3C) or same custom propagator
tracer := tracing.MustNew(
tracing.WithServiceName("my-service"),
// Default propagator is W3C Trace Context
)
Symptom
High CPU usage, increased latency, or memory consumption.
Possible Causes & Solutions
1. Too Much Sampling
Problem: Sampling 100% of high-traffic endpoints.
Solution: Reduce sample rate:
// For high-traffic services (> 1000 req/s)
tracer := tracing.MustNew(
tracing.WithServiceName("my-service"),
tracing.WithSampleRate(0.1), // 10% sampling
)
2. Not Excluding High-Frequency Endpoints
Problem: Tracing health checks and metrics endpoints.
Solution: Exclude them:
handler := tracing.Middleware(tracer,
tracing.WithExcludePaths("/health", "/metrics", "/ready", "/live"),
)(mux)
3. Too Many Span Attributes
Problem: Adding excessive attributes to every span.
Solution: Only add essential attributes:
// ✓ Good - essential attributes
tracer.SetSpanAttribute(span, "user.id", userID)
tracer.SetSpanAttribute(span, "request.id", requestID)
// ✗ Bad - too many attributes
for k, v := range req.Header {
tracer.SetSpanAttribute(span, k, v) // Don't do this!
}
4. Using Regex for Path Exclusion
Problem: Regex patterns are slower than exact paths.
Solution: Prefer exact paths or prefixes:
// ✓ Faster - O(1) hash lookup
tracing.WithExcludePaths("/health", "/metrics")
// ✗ Slower - O(p) regex matching
tracing.WithExcludePatterns("^/(health|metrics)$")
Configuration Errors
Error: "validation errors: provider: multiple providers configured"
Problem: Attempting to configure multiple providers.
Solution: Only configure one provider:
// ✗ Error - multiple providers
tracer, err := tracing.New(
tracing.WithServiceName("my-service"),
tracing.WithStdout(),
tracing.WithOTLP("localhost:4317"), // Error!
)
// ✓ Good - one provider
tracer := tracing.MustNew(
tracing.WithServiceName("my-service"),
tracing.WithOTLP("localhost:4317"),
)
Empty Service Name
Error: "invalid configuration: serviceName: cannot be empty"
Problem: Service name not provided or empty string.
Solution: Always provide a service name:
tracer := tracing.MustNew(
tracing.WithServiceName("my-service"), // Required
)
Invalid Regex Pattern
Error: Middleware panics with "middleware validation errors: excludePatterns: invalid regex..."
Problem: Invalid regex pattern in WithExcludePatterns.
Solution: Validate regex patterns:
// ✗ Invalid regex
tracing.WithExcludePatterns(`[invalid`)
// ✓ Valid regex
tracing.WithExcludePatterns(`^/v[0-9]+/internal/.*`)
OTLP Connection Issues
TLS Certificate Errors
Problem: TLS certificate verification fails.
Solution: Use insecure connection for local development:
tracer := tracing.MustNew(
tracing.WithServiceName("my-service"),
tracing.WithOTLP("localhost:4317", tracing.OTLPInsecure()),
)
For production, ensure proper TLS certificates are configured.
Connection Refused
Problem: Cannot connect to OTLP endpoint.
Solution:
Verify collector is running:
docker ps | grep otel-collector
Check endpoint is correct:
// Correct format: "host:port"
tracing.WithOTLP("localhost:4317")
// Not: "http://localhost:4317" (no protocol for gRPC)
Check network connectivity:
Wrong Endpoint for HTTP
Problem: Using gRPC endpoint for HTTP or vice versa.
Solution: Use correct provider and endpoint:
// OTLP gRPC (port 4317)
tracing.WithOTLP("localhost:4317")
// OTLP HTTP (port 4318, include protocol)
tracing.WithOTLPHTTP("http://localhost:4318")
Middleware Issues
Spans Not Created
Problem: Middleware doesn’t create spans for requests.
Solution: Ensure middleware is applied:
mux := http.NewServeMux()
mux.HandleFunc("/api/users", handleUsers)
// ✓ Middleware applied
handler := tracing.Middleware(tracer)(mux)
http.ListenAndServe(":8080", handler)
// ✗ Middleware not applied
http.ListenAndServe(":8080", mux) // No tracing!
Context Lost in Handlers
Problem: Context doesn’t contain trace information.
Solution: Use context from request:
func handleUsers(w http.ResponseWriter, r *http.Request) {
// ✓ Good - use request context
ctx := r.Context()
traceID := tracing.TraceID(ctx)
// ✗ Bad - creates new context
ctx := context.Background() // Lost trace context!
}
Testing Issues
Tests Fail to Clean Up
Problem: Tests hang or don’t complete cleanup.
Solution: Use testing utilities:
func TestSomething(t *testing.T) {
// ✓ Good - automatic cleanup
tracer := tracing.TestingTracer(t)
// ✗ Bad - manual cleanup required
tracer, _ := tracing.New(tracing.WithNoop())
defer tracer.Shutdown(context.Background())
}
Race Conditions in Tests
Problem: Race detector reports issues in parallel tests.
Solution: Use t.Parallel() correctly:
func TestParallel(t *testing.T) {
t.Parallel() // Each test gets its own tracer
tracer := tracing.TestingTracer(t)
// Use tracer...
}
Debugging Tips
Enable Debug Logging
See internal events:
logger := slog.New(slog.NewTextHandler(os.Stdout, &slog.HandlerOptions{
Level: slog.LevelDebug,
}))
tracer := tracing.MustNew(
tracing.WithServiceName("my-service"),
tracing.WithLogger(logger),
)
Use Stdout Provider
See traces immediately:
tracer := tracing.MustNew(
tracing.WithServiceName("my-service"),
tracing.WithStdout(), // Print traces to console
)
Check Trace IDs
Verify trace context is propagated:
func handleRequest(ctx context.Context) {
traceID := tracing.TraceID(ctx)
log.Printf("Processing request [trace=%s]", traceID)
if traceID == "" {
log.Printf("WARNING: No trace context!")
}
}
Verify Sampling
Log sampling decisions:
startHook := func(ctx context.Context, span trace.Span, req *http.Request) {
if span.SpanContext().IsValid() {
log.Printf("Request sampled: %s", req.URL.Path)
} else {
log.Printf("Request not sampled: %s", req.URL.Path)
}
}
tracer := tracing.MustNew(
tracing.WithServiceName("my-service"),
tracing.WithSpanStartHook(startHook),
)
Getting Help
Check Documentation
Check Logs
Enable debug logging to see internal events:
tracing.WithLogger(slog.Default())
Verify Configuration
Print configuration at startup:
tracer := tracing.MustNew(
tracing.WithServiceName("my-service"),
tracing.WithOTLP("localhost:4317"),
tracing.WithSampleRate(0.1),
)
log.Printf("Tracer configured:")
log.Printf(" Service: %s", tracer.ServiceName())
log.Printf(" Version: %s", tracer.ServiceVersion())
log.Printf(" Provider: %s", tracer.GetProvider())
log.Printf(" Enabled: %v", tracer.IsEnabled())
Common Pitfalls Checklist
Version Compatibility
Go Version
Minimum required: Go 1.25+
Error: go: module requires Go 1.25 or later
Solution: Upgrade Go version:
go version # Check current version
# Upgrade to Go 1.25+
OpenTelemetry Version
The tracing package uses OpenTelemetry SDK v1.x. If you have conflicts with other dependencies:
go mod tidy
go get -u rivaas.dev/tracing
Next Steps