Observability
4 minute read
Overview
The app package provides unified configuration for the three pillars of observability:
- Metrics - Prometheus or OTLP metrics with automatic HTTP instrumentation.
- Tracing - OpenTelemetry distributed tracing with context propagation.
- Logging - Structured logging with slog that includes request-scoped fields.
All three pillars use the same functional options pattern. They automatically receive service metadata (name and version) from app-level configuration.
Unified Observability Configuration
Configure all three pillars in one place.
a, err := app.New(
app.WithServiceName("orders-api"),
app.WithServiceVersion("v1.2.3"),
app.WithObservability(
app.WithLogging(logging.WithJSONHandler()),
app.WithMetrics(), // Prometheus is default
app.WithTracing(tracing.WithOTLP("localhost:4317")),
),
)
Logging
Basic Logging
Enable structured logging with slog:
a, err := app.New(
app.WithObservability(
app.WithLogging(logging.WithJSONHandler()),
),
)
Log Handlers
Choose from different log handlers.
// JSON handler (production)
app.WithLogging(logging.WithJSONHandler())
// Console handler (development)
app.WithLogging(logging.WithConsoleHandler())
// Text handler
app.WithLogging(logging.WithTextHandler())
Log Levels
Configure log level:
app.WithLogging(
logging.WithJSONHandler(),
logging.WithLevel(slog.LevelDebug),
)
Request-Scoped Logging
Use the request-scoped logger in handlers.
a.GET("/orders/:id", func(c *app.Context) {
orderID := c.Param("id")
// Logger automatically includes:
// - HTTP metadata (method, route, target, client IP)
// - Request ID (if present)
// - Trace/span IDs (if tracing enabled)
c.Logger().Info("processing order",
slog.String("order.id", orderID),
)
// Subsequent logs maintain context
c.Logger().Debug("fetching from database")
c.JSON(http.StatusOK, map[string]string{
"order_id": orderID,
})
})
Log output includes automatic context:
{
"time": "2024-01-18T10:30:00Z",
"level": "INFO",
"msg": "processing order",
"http.method": "GET",
"http.route": "/orders/:id",
"http.target": "/orders/123",
"network.client.ip": "203.0.113.1",
"trace_id": "abc...",
"span_id": "def...",
"order.id": "123"
}
Metrics
Prometheus Metrics (Default)
Enable Prometheus metrics on a separate server:
a, err := app.New(
app.WithObservability(
app.WithMetrics(), // Default: Prometheus on :9090/metrics
),
)
Custom Prometheus Configuration
Configure Prometheus address and path:
a, err := app.New(
app.WithObservability(
app.WithMetrics(metrics.WithPrometheus(":9091", "/custom-metrics")),
),
)
Mount Metrics on Main Router
Mount metrics endpoint on the main HTTP server:
a, err := app.New(
app.WithObservability(
app.WithMetricsOnMainRouter("/metrics"),
),
)
// Metrics available at http://localhost:8080/metrics
OTLP Metrics
Send metrics via OTLP to collectors like Prometheus, Grafana, or Datadog:
a, err := app.New(
app.WithObservability(
app.WithMetrics(metrics.WithOTLP("localhost:4317")),
),
)
Custom Metrics in Handlers
Record custom metrics in your handlers:
a.GET("/orders/:id", func(c *app.Context) {
orderID := c.Param("id")
// Increment counter
c.IncrementCounter("order.lookups",
attribute.String("order.id", orderID),
)
// Record histogram
c.RecordHistogram("order.processing_time", 0.250,
attribute.String("order.id", orderID),
)
c.JSON(http.StatusOK, order)
})
Tracing
OpenTelemetry Tracing
Enable OpenTelemetry tracing with OTLP exporter:
a, err := app.New(
app.WithObservability(
app.WithTracing(tracing.WithOTLP("localhost:4317")),
),
)
Stdout Tracing (Development)
Use stdout tracing for development:
a, err := app.New(
app.WithObservability(
app.WithTracing(tracing.WithStdout()),
),
)
Sample Rate
Configure trace sampling:
a, err := app.New(
app.WithObservability(
app.WithTracing(
tracing.WithOTLP("localhost:4317"),
tracing.WithSampleRate(0.1), // Sample 10% of requests
),
),
)
Span Attributes in Handlers
Add span attributes and events in your handlers:
a.GET("/orders/:id", func(c *app.Context) {
orderID := c.Param("id")
// Add span attribute
c.SetSpanAttribute("order.id", orderID)
// Add span event
c.AddSpanEvent("order_lookup_started")
// Fetch order...
c.AddSpanEvent("order_lookup_completed")
c.JSON(http.StatusOK, order)
})
Accessing Trace IDs
Get the current trace ID for correlation:
a.GET("/orders/:id", func(c *app.Context) {
traceID := c.TraceID()
c.JSON(http.StatusOK, map[string]string{
"order_id": orderID,
"trace_id": traceID,
})
})
Service Metadata Injection
Service name and version are automatically injected into all observability components:
a, err := app.New(
app.WithServiceName("orders-api"),
app.WithServiceVersion("v1.2.3"),
app.WithObservability(
app.WithLogging(), // Automatically gets service metadata
app.WithMetrics(), // Automatically gets service metadata
app.WithTracing(), // Automatically gets service metadata
),
)
You don’t need to pass service name/version explicitly - the app injects them automatically.
Overriding Service Metadata
If needed, you can override service metadata for specific components:
a, err := app.New(
app.WithServiceName("orders-api"),
app.WithServiceVersion("v1.2.3"),
app.WithObservability(
app.WithLogging(
logging.WithServiceName("custom-logger"), // Overrides injected value
),
),
)
Path Filtering
Exclude specific paths from observability (metrics, tracing, logging):
Exclude Paths
Exclude exact paths:
a, err := app.New(
app.WithObservability(
app.WithLogging(),
app.WithMetrics(),
app.WithTracing(),
app.WithExcludePaths("/healthz", "/readyz", "/metrics"),
),
)
Exclude Prefixes
Exclude path prefixes:
a, err := app.New(
app.WithObservability(
app.WithExcludePrefixes("/internal/", "/admin/debug/"),
),
)
Exclude Patterns
Exclude paths matching regex patterns:
a, err := app.New(
app.WithObservability(
app.WithExcludePatterns(`^/api/v\d+/health$`, `^/debug/.*`),
),
)
Default Exclusions
By default, the following paths are excluded:
/health,/healthz,/live,/livez/ready,/readyz/metrics/debug/*
To disable default exclusions:
a, err := app.New(
app.WithObservability(
app.WithoutDefaultExclusions(),
app.WithExcludePaths("/custom-health"), // Add your own
),
)
Access Logging
Enable/Disable Access Logging
Control access logging:
// Enable access logging (default)
a, err := app.New(
app.WithObservability(
app.WithAccessLogging(true),
),
)
// Disable access logging
a, err := app.New(
app.WithObservability(
app.WithAccessLogging(false),
),
)
Log Only Errors
Log only errors and slow requests (automatically enabled in production):
a, err := app.New(
app.WithObservability(
app.WithLogOnlyErrors(),
),
)
Slow Request Threshold
Mark requests as slow and log them:
a, err := app.New(
app.WithObservability(
app.WithLogOnlyErrors(),
app.WithSlowThreshold(500 * time.Millisecond),
),
)
Complete Example
Production-ready observability configuration:
package main
import (
"log"
"time"
"rivaas.dev/app"
"rivaas.dev/logging"
"rivaas.dev/metrics"
"rivaas.dev/tracing"
)
func main() {
a, err := app.New(
// Service metadata (automatically injected into all components)
app.WithServiceName("orders-api"),
app.WithServiceVersion("v2.1.0"),
app.WithEnvironment("production"),
// Unified observability configuration
app.WithObservability(
// Logging: JSON handler for production
app.WithLogging(
logging.WithJSONHandler(),
logging.WithLevel(slog.LevelInfo),
),
// Metrics: Prometheus on separate server
app.WithMetrics(
metrics.WithPrometheus(":9090", "/metrics"),
),
// Tracing: OTLP to Jaeger/Tempo
app.WithTracing(
tracing.WithOTLP("jaeger:4317"),
tracing.WithSampleRate(0.1), // 10% sampling
),
// Path filtering
app.WithExcludePaths("/healthz", "/readyz"),
app.WithExcludePrefixes("/internal/"),
// Access logging: errors and slow requests only
app.WithLogOnlyErrors(),
app.WithSlowThreshold(1 * time.Second),
),
)
if err != nil {
log.Fatal(err)
}
// Register routes...
a.GET("/orders/:id", handleGetOrder)
// Start server...
}
Next Steps
- Context - Use request-scoped logging in handlers
- Health Endpoints - Configure health checks
- Server - Start the server and view observability data
Feedback
Was this page helpful?
Glad to hear it! Please tell us how we can improve.
Sorry to hear that. Please tell us how we can improve.