This is the multi-page printable view of this section. Click here to print.

Return to the regular view of this page.

Metrics Package

API reference for rivaas.dev/metrics - Metrics collection for Go applications

1: API Reference
2: Configuration Options
3: Middleware Options
4: Troubleshooting

This is the API reference for the rivaas.dev/metrics package. For learning-focused documentation, see the Metrics Guide.

Package Information

Import Path: rivaas.dev/metrics
Go Version: 1.25+
Documentation: pkg.go.dev/rivaas.dev/metrics
Source Code: GitHub

Package Overview

The metrics package provides OpenTelemetry-based metrics collection for Go applications with support for multiple exporters including Prometheus, OTLP, and stdout.

Core Features

Multiple metrics providers (Prometheus, OTLP, stdout)
Built-in HTTP metrics via middleware
Custom metrics (counters, histograms, gauges)
Thread-safe operations
Context-aware methods
Automatic header filtering for security
Testing utilities

Architecture

The package is built on OpenTelemetry and provides a simplified interface for common metrics use cases.

graph TD
    App[Application Code]
    Recorder[Recorder]
    Provider[Provider Layer]
    Prom[Prometheus]
    OTLP[OTLP]
    Stdout[Stdout]
    Middleware[HTTP Middleware]
    
    App -->|Record Metrics| Recorder
    Middleware -->|Auto-Collect| Recorder
    Recorder --> Provider
    Provider --> Prom
    Provider --> OTLP
    Provider --> Stdout

Components

Main Package (`rivaas.dev/metrics`)

Core metrics collection including:

Recorder - Main metrics recorder
New() / MustNew() - Recorder initialization
Custom metrics methods - Counters, histograms, gauges
Middleware() - HTTP metrics collection
Testing utilities

Quick API Index

Recorder Creation

recorder, err := metrics.New(options...)     // With error handling
recorder := metrics.MustNew(options...)      // Panics on error

Lifecycle Management

err := recorder.Start(ctx context.Context)   // Start metrics server/exporter
err := recorder.Shutdown(ctx context.Context) // Graceful shutdown
err := recorder.ForceFlush(ctx context.Context) // Force immediate export

Recording Metrics

// Counters
err := recorder.IncrementCounter(ctx, name, attributes...)
err := recorder.AddCounter(ctx, name, value, attributes...)

// Histograms
err := recorder.RecordHistogram(ctx, name, value, attributes...)

// Gauges
err := recorder.SetGauge(ctx, name, value, attributes...)

HTTP Middleware

handler := metrics.Middleware(recorder, options...)(httpHandler)

Provider-Specific Methods

address := recorder.ServerAddress()          // Prometheus: actual address
handler, err := recorder.Handler()           // Prometheus: metrics handler
count := recorder.CustomMetricCount()        // Number of custom metrics

Testing Utilities

recorder := metrics.TestingRecorder(t, serviceName)
recorder := metrics.TestingRecorderWithPrometheus(t, serviceName)
err := metrics.WaitForMetricsServer(t, address, timeout)

Reference Pages

API Reference

Recorder type, lifecycle methods, and custom metrics API.

View →

Options

Configuration options for providers and service metadata.

View →

Middleware Options

HTTP middleware configuration and path exclusion.

View →

Troubleshooting

Common metrics issues and solutions.

View →

User Guide

Step-by-step tutorials and examples.

View →

Type Reference

Recorder

type Recorder struct {
    // contains filtered or unexported fields
}

Main metrics recorder. Thread-safe for concurrent access.

Methods: See API Reference for complete method documentation.

Option

type Option func(*Recorder)

Configuration option function type used with New() and MustNew().

Available Options: See Options for all options.

EventType

type EventType int

const (
    EventError   EventType = iota // Error events
    EventWarning                   // Warning events
    EventInfo                      // Informational events
    EventDebug                     // Debug events
)

Event severity levels for internal operational events.

Event

type Event struct {
    Type    EventType
    Message string
    Args    []any // slog-style key-value pairs
}

Internal operational event from the metrics package.

EventHandler

type EventHandler func(Event)

Processes internal operational events. Used with WithEventHandler option.

Common Patterns

Basic Usage

recorder := metrics.MustNew(
    metrics.WithPrometheus(":9090", "/metrics"),
    metrics.WithServiceName("my-api"),
)
recorder.Start(context.Background())
defer recorder.Shutdown(context.Background())

_ = recorder.IncrementCounter(ctx, "requests_total")

With HTTP Middleware

recorder := metrics.MustNew(
    metrics.WithPrometheus(":9090", "/metrics"),
    metrics.WithServiceName("my-api"),
)
recorder.Start(context.Background())

handler := metrics.Middleware(recorder,
    metrics.WithExcludePaths("/health"),
)(httpHandler)

http.ListenAndServe(":8080", handler)

With OTLP

recorder := metrics.MustNew(
    metrics.WithOTLP("http://localhost:4318"),
    metrics.WithServiceName("my-service"),
)
recorder.Start(ctx) // Required before recording metrics
defer recorder.Shutdown(context.Background())

Thread Safety

The Recorder type is thread-safe for:

All metric recording methods
Concurrent Start() and Shutdown() operations
Mixed recording and lifecycle operations

Not thread-safe for:

Concurrent modification during initialization

Performance Notes

Metric recording: ~1-2 microseconds per operation
HTTP middleware: ~1-2 microseconds overhead per request
Memory usage: Scales with number of unique metric names and label combinations
Histogram overhead: Proportional to bucket count

Best Practices:

Use fire-and-forget pattern for most metrics (ignore errors)
Limit metric cardinality (avoid high-cardinality labels)
Customize histogram buckets for your use case
Exclude high-traffic paths from middleware when appropriate

Built-in Metrics

When using HTTP middleware, these metrics are automatically collected:

Metric	Type	Description
`http_request_duration_seconds`	Histogram	Request duration distribution
`http_requests_total`	Counter	Total requests by method, path, status
`http_requests_active`	Gauge	Currently active requests
`http_request_size_bytes`	Histogram	Request body size distribution
`http_response_size_bytes`	Histogram	Response body size distribution
`http_errors_total`	Counter	HTTP errors by status code
`custom_metric_failures_total`	Counter	Failed custom metric creations

Version Compatibility

The metrics package follows semantic versioning. The API is stable for the v1 series.

Minimum Go version: 1.25

OpenTelemetry compatibility: Uses OpenTelemetry SDK v1.x

Next Steps

Read the API Reference for detailed method documentation
Explore Options for all available configuration options
Check Middleware Options for HTTP integration
Review Troubleshooting for common issues

For learning-focused guides, see the Metrics Guide.

1 - API Reference

Complete API documentation for the Recorder type and methods

Complete API reference for the metrics package core types and methods.

Recorder Type

The Recorder is the main type for collecting metrics. It is thread-safe. You can use it concurrently.

type Recorder struct {
    // contains filtered or unexported fields
}

Creation Functions

New

func New(opts ...Option) (*Recorder, error)

Creates a new Recorder with the given options. Returns an error if configuration is invalid.

Parameters:

opts ...Option - Configuration options.

Returns:

*Recorder - Configured recorder.
error - Configuration error, if any.

Example:

recorder, err := metrics.New(
    metrics.WithPrometheus(":9090", "/metrics"),
    metrics.WithServiceName("my-api"),
)
if err != nil {
    log.Fatal(err)
}

Errors:

Multiple provider options specified.
Invalid service name.
Invalid port or endpoint configuration.

MustNew

func MustNew(opts ...Option) *Recorder

Creates a new Recorder with the given options. Panics if configuration is invalid.

Parameters:

opts ...Option - Configuration options.

Returns:

*Recorder - Configured recorder.

Panics: If configuration is invalid.

Example:

recorder := metrics.MustNew(
    metrics.WithPrometheus(":9090", "/metrics"),
    metrics.WithServiceName("my-api"),
)

Use Case: Applications that should fail fast on invalid metrics configuration.

Lifecycle Methods

Start

func (r *Recorder) Start(ctx context.Context) error

Starts the metrics recorder. For Prometheus, starts the HTTP server. For OTLP, establishes connection. For stdout, this is a no-op but safe to call.

Parameters:

ctx context.Context - Lifecycle context for the recorder

Returns:

error - Startup error, if any

Example:

ctx, cancel := signal.NotifyContext(context.Background(), os.Interrupt)
defer cancel()

if err := recorder.Start(ctx); err != nil {
    log.Fatal(err)
}

Errors:

Port already in use (Prometheus with WithStrictPort)
Cannot connect to OTLP endpoint
Context already canceled

Provider Behavior:

Prometheus: Starts HTTP server on configured port
OTLP: Establishes connection to collector
Stdout: No-op, safe to call

Shutdown

func (r *Recorder) Shutdown(ctx context.Context) error

Gracefully shuts down the metrics recorder, flushing any pending metrics.

Parameters:

ctx context.Context - Shutdown context with timeout

Returns:

error - Shutdown error, if any

Example:

shutdownCtx, cancel := context.WithTimeout(context.Background(), 5*time.Second)
defer cancel()

if err := recorder.Shutdown(shutdownCtx); err != nil {
    log.Printf("Shutdown error: %v", err)
}

Behavior:

Stops accepting new metrics
Flushes pending metrics
Closes network connections
Stops HTTP server (Prometheus)
Idempotent (safe to call multiple times)

Best Practice: Always defer Shutdown with a timeout context.

ForceFlush

func (r *Recorder) ForceFlush(ctx context.Context) error

Forces immediate export of all pending metrics. Primarily useful for push-based providers (OTLP, stdout).

Parameters:

ctx context.Context - Flush context with timeout

Returns:

error - Flush error, if any

Example:

// Before critical operation
if err := recorder.ForceFlush(ctx); err != nil {
    log.Printf("Failed to flush metrics: %v", err)
}

Provider Behavior:

OTLP: Immediately exports all pending metrics
Stdout: Immediately prints all pending metrics
Prometheus: Typically a no-op (pull-based)

Use Cases:

Before deployment or shutdown
Checkpointing during long operations
Ensuring metrics visibility

Custom Metrics Methods

IncrementCounter

func (r *Recorder) IncrementCounter(ctx context.Context, name string, attrs ...attribute.KeyValue) error

Increments a counter metric by 1.

Parameters:

ctx context.Context - Context for the operation
name string - Metric name (must be valid)
attrs ...attribute.KeyValue - Optional metric attributes

Returns:

error - Error if metric name is invalid or limit reached

Example:

err := recorder.IncrementCounter(ctx, "requests_total",
    attribute.String("method", "GET"),
    attribute.String("status", "success"),
)

Naming Rules:

Must start with letter
Can contain letters, numbers, underscores, dots, hyphens
Cannot use reserved prefixes: __, http_, router_
Maximum 255 characters

AddCounter

func (r *Recorder) AddCounter(ctx context.Context, name string, value int64, attrs ...attribute.KeyValue) error

Adds a specific value to a counter metric.

Parameters:

ctx context.Context - Context for the operation
name string - Metric name (must be valid)
value int64 - Amount to add (must be non-negative)
attrs ...attribute.KeyValue - Optional metric attributes

Returns:

error - Error if metric name is invalid, value is negative, or limit reached

Example:

bytesProcessed := int64(1024)
err := recorder.AddCounter(ctx, "bytes_processed_total", bytesProcessed,
    attribute.String("direction", "inbound"),
)

RecordHistogram

func (r *Recorder) RecordHistogram(ctx context.Context, name string, value float64, attrs ...attribute.KeyValue) error

Records a value in a histogram metric.

Parameters:

ctx context.Context - Context for the operation
name string - Metric name (must be valid)
value float64 - Value to record
attrs ...attribute.KeyValue - Optional metric attributes

Returns:

error - Error if metric name is invalid or limit reached

Example:

start := time.Now()
// ... operation ...
duration := time.Since(start).Seconds()

err := recorder.RecordHistogram(ctx, "operation_duration_seconds", duration,
    attribute.String("operation", "create_user"),
)

Bucket Configuration: Use WithDurationBuckets or WithSizeBuckets to customize histogram boundaries.

SetGauge

func (r *Recorder) SetGauge(ctx context.Context, name string, value float64, attrs ...attribute.KeyValue) error

Sets a gauge metric to a specific value.

Parameters:

ctx context.Context - Context for the operation
name string - Metric name (must be valid)
value float64 - Value to set
attrs ...attribute.KeyValue - Optional metric attributes

Returns:

error - Error if metric name is invalid or limit reached

Example:

activeConnections := float64(pool.Active())
err := recorder.SetGauge(ctx, "active_connections", activeConnections,
    attribute.String("pool", "database"),
)

Provider-Specific Methods

ServerAddress

func (r *Recorder) ServerAddress() string

Returns the server address (port) for Prometheus provider. Returns empty string for other providers or if server is disabled.

Returns:

string - Server address in port format (e.g., :9090)

Example:

recorder := metrics.MustNew(
    metrics.WithPrometheus(":9090", "/metrics"),
    metrics.WithServiceName("my-api"),
)
recorder.Start(ctx)

address := recorder.ServerAddress()
log.Printf("Metrics at: http://localhost%s/metrics", address)

Use Cases:

Logging actual port (when not using strict mode)
Testing with dynamic port allocation
Health check registration

Note: Returns the port string (e.g., :9090), not a full hostname. Prepend localhost for local access.

Handler

func (r *Recorder) Handler() (http.Handler, error)

Returns the HTTP handler for metrics endpoint. Only works with Prometheus provider.

Returns:

http.Handler - Metrics endpoint handler
error - Error if not using Prometheus provider or server disabled

Example:

recorder := metrics.MustNew(
    metrics.WithPrometheus(":9090", "/metrics"),
    metrics.WithServerDisabled(),
    metrics.WithServiceName("my-api"),
)

handler, err := recorder.Handler()
if err != nil {
    log.Fatal(err)
}

http.Handle("/metrics", handler)
http.ListenAndServe(":8080", nil)

Errors:

Not using Prometheus provider
Server not disabled (use WithServerDisabled)

CustomMetricCount

func (r *Recorder) CustomMetricCount() int

Returns the number of custom metrics created.

Returns:

int - Number of custom metrics

Example:

count := recorder.CustomMetricCount()
log.Printf("Custom metrics: %d/%d", count, maxLimit)

// Expose as a metric
_ = recorder.SetGauge(ctx, "custom_metrics_count", float64(count))

Use Cases:

Monitoring metric cardinality
Debugging metric limit issues
Capacity planning

Note: Built-in HTTP metrics do not count toward this total.

Middleware Function

Middleware

func Middleware(recorder *Recorder, opts ...MiddlewareOption) func(http.Handler) http.Handler

Returns HTTP middleware that automatically collects metrics for requests.

Parameters:

recorder *Recorder - Metrics recorder
opts ...MiddlewareOption - Middleware configuration options

Returns:

func(http.Handler) http.Handler - Middleware function

Example:

handler := metrics.Middleware(recorder,
    metrics.WithExcludePaths("/health", "/metrics"),
    metrics.WithHeaders("X-Request-ID"),
)(httpHandler)

Collected Metrics:

http_request_duration_seconds - Request duration histogram
http_requests_total - Request counter
http_requests_active - Active requests gauge
http_request_size_bytes - Request size histogram
http_response_size_bytes - Response size histogram
http_errors_total - Error counter

Middleware Options: See Middleware Options for details.

Testing Functions

TestingRecorder

func TestingRecorder(tb testing.TB, serviceName string, opts ...Option) *Recorder

Creates a test recorder with stdout provider. Automatically registers cleanup via t.Cleanup().

Parameters:

tb testing.TB - Test or benchmark instance
serviceName string - Service name for metrics
opts ...Option - Optional additional configuration options

Returns:

*Recorder - Test recorder

Example:

func TestHandler(t *testing.T) {
    t.Parallel()
    
    recorder := metrics.TestingRecorder(t, "test-service")
    
    // Use recorder in tests...
    // Cleanup is automatic
}

// With additional options
func TestWithOptions(t *testing.T) {
    recorder := metrics.TestingRecorder(t, "test-service",
        metrics.WithMaxCustomMetrics(100),
    )
}

Features:

No port conflicts (uses stdout)
Automatic cleanup
Parallel test safe
Works with both *testing.T and *testing.B

TestingRecorderWithPrometheus

func TestingRecorderWithPrometheus(tb testing.TB, serviceName string, opts ...Option) *Recorder

Creates a test recorder with Prometheus provider and dynamic port allocation. Automatically registers cleanup via t.Cleanup().

Parameters:

tb testing.TB - Test or benchmark instance
serviceName string - Service name for metrics
opts ...Option - Optional additional configuration options

Returns:

*Recorder - Test recorder with Prometheus

Example:

func TestMetricsEndpoint(t *testing.T) {
    t.Parallel()
    
    recorder := metrics.TestingRecorderWithPrometheus(t, "test-service")
    
    // Wait for server
    err := metrics.WaitForMetricsServer(t, recorder.ServerAddress(), 5*time.Second)
    if err != nil {
        t.Fatal(err)
    }
    
    // Test metrics endpoint...
}

Features:

Dynamic port allocation
Real Prometheus endpoint
Automatic cleanup
Works with both *testing.T and *testing.B

WaitForMetricsServer

func WaitForMetricsServer(tb testing.TB, address string, timeout time.Duration) error

Waits for Prometheus metrics server to be ready.

Parameters:

tb testing.TB - Test or benchmark instance for logging
address string - Server address (e.g., :9090)
timeout time.Duration - Maximum wait time

Returns:

error - Error if server not ready within timeout

Example:

recorder := metrics.TestingRecorderWithPrometheus(t, "test-service")

err := metrics.WaitForMetricsServer(t, recorder.ServerAddress(), 5*time.Second)
if err != nil {
    t.Fatalf("Server not ready: %v", err)
}

// Server is ready, make requests

Event Types

EventType

type EventType int

const (
    EventError   EventType = iota // Error events
    EventWarning                   // Warning events
    EventInfo                      // Informational events
    EventDebug                     // Debug events
)

Severity levels for internal operational events.

Event

type Event struct {
    Type    EventType
    Message string
    Args    []any // slog-style key-value pairs
}

Internal operational event from the metrics package.

Example:

metrics.WithEventHandler(func(e metrics.Event) {
    switch e.Type {
    case metrics.EventError:
        sentry.CaptureMessage(e.Message)
    case metrics.EventWarning:
        log.Printf("WARN: %s", e.Message)
    case metrics.EventInfo:
        log.Printf("INFO: %s", e.Message)
    }
})

EventHandler

type EventHandler func(Event)

Function type for handling internal operational events.

Example:

handler := func(e metrics.Event) {
    slog.Default().Info(e.Message, e.Args...)
}

recorder := metrics.MustNew(
    metrics.WithPrometheus(":9090", "/metrics"),
    metrics.WithEventHandler(handler),
)

Error Handling

All metric recording methods return error. Common error types:

Invalid Metric Name

err := recorder.IncrementCounter(ctx, "__reserved")
// Error: metric name uses reserved prefix "__"

Metric Limit Reached

err := recorder.IncrementCounter(ctx, "new_metric_1001")
// Error: custom metric limit reached (1000/1000)

Provider Not Started

recorder := metrics.MustNew(metrics.WithOTLP("http://localhost:4318"))
err := recorder.IncrementCounter(ctx, "metric")
// Error: OTLP provider not started (call Start first)

Thread Safety

All methods are thread-safe and can be called concurrently:

// Safe to call from multiple goroutines
go func() {
    _ = recorder.IncrementCounter(ctx, "worker_1")
}()

go func() {
    _ = recorder.IncrementCounter(ctx, "worker_2")
}()

Next Steps

See Options for all configuration options
Check Middleware Options for HTTP middleware
Review Troubleshooting for common issues
Read Guides for usage patterns

2 - Configuration Options

Complete reference of all configuration options

Complete reference for all Option functions used to configure the Recorder.

Provider Options

Only one provider option can be used per Recorder. Using multiple provider options results in a validation error.

WithPrometheus

func WithPrometheus(port, path string) Option

Configures Prometheus provider with HTTP endpoint.

Parameters:

port string - Listen address like :9090 or localhost:9090.
path string - Metrics path like /metrics.

Example:

recorder := metrics.MustNew(
    metrics.WithPrometheus(":9090", "/metrics"),
    metrics.WithServiceName("my-api"),
)

Behavior:

Initializes immediately in New().
Starts HTTP server when Start() is called.
Metrics available at http://localhost:9090/metrics.

Related Options:

WithStrictPort() - Fail if port unavailable.
WithServerDisabled() - Manage HTTP server manually.

WithOTLP

func WithOTLP(endpoint string) Option

Configures OTLP (OpenTelemetry Protocol) provider for sending metrics to a collector.

Parameters:

endpoint string - OTLP collector HTTP endpoint like http://localhost:4318.

Example:

recorder := metrics.MustNew(
    metrics.WithOTLP("http://localhost:4318"),
    metrics.WithServiceName("my-service"),
)

Behavior:

Defers initialization until Start() is called.
Uses lifecycle context for network connections.
Important: Must call Start() before recording metrics.

Related Options:

WithExportInterval() - Configure export frequency.

WithStdout

func WithStdout() Option

Configures stdout provider for printing metrics to console.

Example:

recorder := metrics.MustNew(
    metrics.WithStdout(),
    metrics.WithServiceName("dev-service"),
)

Behavior:

Initializes immediately in New()
Works without calling Start() (but safe to call)
Prints metrics to stdout periodically

Use Cases:

Development and debugging
CI/CD pipelines
Unit tests

Related Options:

WithExportInterval() - Configure print frequency

Service Configuration Options

WithServiceName

func WithServiceName(name string) Option

Sets the service name for metrics identification.

Parameters:

name string - Service name

Example:

recorder := metrics.MustNew(
    metrics.WithPrometheus(":9090", "/metrics"),
    metrics.WithServiceName("payment-api"),
)

Best Practices:

Use lowercase with hyphens: user-service, payment-api
Be consistent across services
Avoid changing names in production

WithServiceVersion

func WithServiceVersion(version string) Option

Sets the service version for metrics.

Parameters:

version string - Service version

Example:

recorder := metrics.MustNew(
    metrics.WithPrometheus(":9090", "/metrics"),
    metrics.WithServiceName("my-api"),
    metrics.WithServiceVersion("v1.2.3"),
)

Best Practices:

Use semantic versioning: v1.2.3
Automate from CI/CD build information

Prometheus-Specific Options

WithStrictPort

func WithStrictPort() Option

Requires the metrics server to use the exact port specified. Fails if port is unavailable.

Example:

recorder := metrics.MustNew(
    metrics.WithPrometheus(":9090", "/metrics"),
    metrics.WithStrictPort(),  // Fail if 9090 unavailable
    metrics.WithServiceName("my-api"),
)

Default Behavior: Automatically searches up to 100 ports if requested port is unavailable.

With Strict Mode: Returns error if exact port is not available.

Production Recommendation: Always use WithStrictPort() for predictable behavior.

WithServerDisabled

func WithServerDisabled() Option

Disables automatic metrics server startup. Use Handler() to get metrics handler for manual serving.

Example:

recorder := metrics.MustNew(
    metrics.WithPrometheus(":9090", "/metrics"),
    metrics.WithServerDisabled(),
    metrics.WithServiceName("my-api"),
)

handler, err := recorder.Handler()
if err != nil {
    log.Fatal(err)
}

// Serve on your own server
http.Handle("/metrics", handler)
http.ListenAndServe(":8080", nil)

Use Cases:

Serve metrics on same port as application
Custom server configuration
Integration with existing HTTP servers

Histogram Bucket Options

WithDurationBuckets

func WithDurationBuckets(buckets ...float64) Option

Sets custom histogram bucket boundaries for duration metrics (in seconds).

Parameters:

buckets ...float64 - Bucket boundaries in seconds

Default: 0.005, 0.01, 0.025, 0.05, 0.1, 0.25, 0.5, 1, 2.5, 5, 10

Example:

// Fast API (most requests < 100ms)
recorder := metrics.MustNew(
    metrics.WithPrometheus(":9090", "/metrics"),
    metrics.WithDurationBuckets(0.001, 0.005, 0.01, 0.025, 0.05, 0.1, 0.5, 1),
    metrics.WithServiceName("fast-api"),
)

// Slow operations (seconds to minutes)
recorder := metrics.MustNew(
    metrics.WithPrometheus(":9090", "/metrics"),
    metrics.WithDurationBuckets(1, 5, 10, 30, 60, 120, 300, 600),
    metrics.WithServiceName("batch-processor"),
)

Trade-offs:

More buckets = better resolution, higher memory/storage
Fewer buckets = lower overhead, coarser resolution

WithSizeBuckets

func WithSizeBuckets(buckets ...float64) Option

Sets custom histogram bucket boundaries for size metrics (in bytes).

Parameters:

buckets ...float64 - Bucket boundaries in bytes

Default: 100, 1000, 10000, 100000, 1000000, 10000000

Example:

// Small JSON API (< 10KB)
recorder := metrics.MustNew(
    metrics.WithPrometheus(":9090", "/metrics"),
    metrics.WithSizeBuckets(100, 500, 1000, 5000, 10000, 50000),
    metrics.WithServiceName("json-api"),
)

// File uploads (KB to MB)
recorder := metrics.MustNew(
    metrics.WithPrometheus(":9090", "/metrics"),
    metrics.WithSizeBuckets(1024, 10240, 102400, 1048576, 10485760, 104857600),
    metrics.WithServiceName("file-service"),
)

Advanced Options

WithExportInterval

func WithExportInterval(interval time.Duration) Option

Sets export interval for push-based providers (OTLP and stdout).

Parameters:

interval time.Duration - Export interval

Default: 30 seconds

Example:

recorder := metrics.MustNew(
    metrics.WithOTLP("http://localhost:4318"),
    metrics.WithExportInterval(10 * time.Second),
    metrics.WithServiceName("my-service"),
)

Applies To:

OTLP (push-based)
Stdout (push-based)

Does NOT Apply To:

Prometheus (pull-based, scraped on-demand)

Trade-offs:

Shorter interval: More timely data, higher overhead
Longer interval: Lower overhead, delayed visibility

WithMaxCustomMetrics

func WithMaxCustomMetrics(maxLimit int) Option

Sets the maximum number of custom metrics allowed.

Parameters:

maxLimit int - Maximum custom metrics

Default: 1000

Example:

recorder := metrics.MustNew(
    metrics.WithPrometheus(":9090", "/metrics"),
    metrics.WithMaxCustomMetrics(5000),
    metrics.WithServiceName("my-api"),
)

Purpose:

Prevent unbounded metric cardinality
Protect against memory exhaustion
Enforce metric discipline

Note: Built-in HTTP metrics do not count toward this limit.

Monitor Usage:

count := recorder.CustomMetricCount()
log.Printf("Custom metrics: %d/%d", count, maxLimit)

WithLogger

func WithLogger(logger *slog.Logger) Option

Sets the logger for internal operational events.

Parameters:

logger *slog.Logger - Logger instance

Example:

recorder := metrics.MustNew(
    metrics.WithPrometheus(":9090", "/metrics"),
    metrics.WithLogger(slog.Default()),
    metrics.WithServiceName("my-api"),
)

Events Logged:

Initialization events
Error messages (metric creation failures)
Warning messages (port conflicts, limits reached)

Alternative: Use WithEventHandler() for custom event handling.

WithEventHandler

func WithEventHandler(handler EventHandler) Option

Sets a custom event handler for internal operational events.

Parameters:

handler EventHandler - Event handler function

Example:

recorder := metrics.MustNew(
    metrics.WithPrometheus(":9090", "/metrics"),
    metrics.WithEventHandler(func(e metrics.Event) {
        switch e.Type {
        case metrics.EventError:
            sentry.CaptureMessage(e.Message)
        case metrics.EventWarning:
            log.Printf("WARN: %s", e.Message)
        case metrics.EventInfo:
            log.Printf("INFO: %s", e.Message)
        }
    }),
    metrics.WithServiceName("my-api"),
)

Use Cases:

Send errors to external monitoring (Sentry, etc.)
Custom logging formats
Metric collection about metric collection

Event Types:

EventError - Error events
EventWarning - Warning events
EventInfo - Informational events
EventDebug - Debug events

Advanced Provider Options

WithMeterProvider

func WithMeterProvider(provider metric.MeterProvider) Option

Provides a custom OpenTelemetry meter provider for complete control.

Parameters:

provider metric.MeterProvider - Custom meter provider

Example:

mp := sdkmetric.NewMeterProvider(...)
recorder := metrics.MustNew(
    metrics.WithMeterProvider(mp),
    metrics.WithServiceName("my-service"),
)
defer mp.Shutdown(context.Background())

Use Cases:

Manage meter provider lifecycle yourself
Multiple independent metrics configurations
Avoid global state

Note: When using WithMeterProvider, provider options (WithPrometheus, WithOTLP, WithStdout) are ignored.

WithGlobalMeterProvider

func WithGlobalMeterProvider() Option

Registers the meter provider as the global OpenTelemetry meter provider.

Example:

recorder := metrics.MustNew(
    metrics.WithPrometheus(":9090", "/metrics"),
    metrics.WithGlobalMeterProvider(),  // Register globally
    metrics.WithServiceName("my-service"),
)

Default Behavior: Meter providers are NOT registered globally.

When to Use:

OpenTelemetry instrumentation libraries need global provider
Third-party libraries expect global meter provider
otel.GetMeterProvider() should return your provider

When NOT to Use:

Multiple services in same process
Avoid global state
Custom meter provider management

Configuration Examples

Production API

recorder := metrics.MustNew(
    metrics.WithPrometheus(":9090", "/metrics"),
    metrics.WithStrictPort(),
    metrics.WithServiceName("payment-api"),
    metrics.WithServiceVersion(version),
    metrics.WithLogger(slog.Default()),
    metrics.WithDurationBuckets(0.01, 0.1, 0.5, 1, 5, 10),
    metrics.WithMaxCustomMetrics(2000),
)

Development

recorder := metrics.MustNew(
    metrics.WithStdout(),
    metrics.WithServiceName("dev-api"),
    metrics.WithExportInterval(5 * time.Second),
)

OpenTelemetry Native

recorder := metrics.MustNew(
    metrics.WithOTLP(os.Getenv("OTEL_EXPORTER_OTLP_ENDPOINT")),
    metrics.WithServiceName(os.Getenv("SERVICE_NAME")),
    metrics.WithServiceVersion(os.Getenv("SERVICE_VERSION")),
    metrics.WithExportInterval(15 * time.Second),
    metrics.WithLogger(slog.Default()),
)

Embedded Metrics Server

recorder := metrics.MustNew(
    metrics.WithPrometheus(":9090", "/metrics"),
    metrics.WithServerDisabled(),
    metrics.WithServiceName("api"),
)

handler, _ := recorder.Handler()

// Serve on application port
mux := http.NewServeMux()
mux.Handle("/metrics", handler)
mux.HandleFunc("/", appHandler)
http.ListenAndServe(":8080", mux)

Option Validation

The following validation occurs during New() or MustNew():

Provider Conflicts: Only one provider option (WithPrometheus, WithOTLP, WithStdout) can be used
Service Name: Cannot be empty (default: "rivaas-service")
Service Version: Cannot be empty (default: "1.0.0")
Port Format: Must be valid address format for Prometheus
Custom Metrics Limit: Must be at least 1

Defaults: If no provider is specified, defaults to Prometheus on :9090/metrics.

Validation Errors:

// Multiple providers - ERROR
recorder, err := metrics.New(
    metrics.WithPrometheus(":9090", "/metrics"),
    metrics.WithOTLP("http://localhost:4318"),  // Error: conflicting providers
)

// Empty service name - ERROR
recorder, err := metrics.New(
    metrics.WithPrometheus(":9090", "/metrics"),
    metrics.WithServiceName(""),  // Error: service name cannot be empty
)

// No options - OK (uses defaults)
recorder, err := metrics.New()  // Uses default Prometheus on :9090/metrics

Next Steps

See API Reference for method documentation
Check Middleware Options for HTTP middleware configuration
Review Troubleshooting for common issues
Read Configuration Guide for detailed examples

3 - Middleware Options

HTTP middleware configuration options reference

Complete reference for MiddlewareOption functions used to configure the HTTP metrics middleware.

Overview

Middleware options configure which paths to exclude from metrics collection and which headers to record.

handler := metrics.Middleware(recorder,
    metrics.WithExcludePaths("/health", "/metrics"),
    metrics.WithExcludePrefixes("/debug/"),
    metrics.WithExcludePatterns(`^/admin/.*`),
    metrics.WithHeaders("X-Request-ID"),
)(httpHandler)

Path Exclusion Options

WithExcludePaths

func WithExcludePaths(paths ...string) MiddlewareOption

Excludes exact paths from metrics collection.

Parameters:

paths ...string - Exact paths to exclude

Example:

handler := metrics.Middleware(recorder,
    metrics.WithExcludePaths("/health", "/metrics", "/ready"),
)(mux)

Use Cases:

Health check endpoints.
Metrics endpoints.
Readiness and liveness probes.

Behavior:

Matches exact path only.
Case-sensitive.
Does not match path prefixes.

Examples:

// Excluded paths
/health          ✓ excluded
/metrics         ✓ excluded
/ready           ✓ excluded

// Not excluded (not exact matches)
/health/status   ✗ not excluded
/healthz         ✗ not excluded
/api/metrics     ✗ not excluded

WithExcludePrefixes

func WithExcludePrefixes(prefixes ...string) MiddlewareOption

Excludes all paths with specific prefixes from metrics collection.

Parameters:

prefixes ...string - Path prefixes to exclude

Example:

handler := metrics.Middleware(recorder,
    metrics.WithExcludePrefixes("/debug/", "/internal/", "/_/"),
)(mux)

Use Cases:

Debug endpoints (/debug/pprof/, /debug/vars/)
Internal APIs (/internal/)
Administrative paths (/_/)

Behavior:

Matches any path starting with prefix
Case-sensitive
Include trailing slash for directory prefixes

Examples:

// With prefix "/debug/"
/debug/pprof/heap      ✓ excluded
/debug/vars            ✓ excluded
/debug/                ✓ excluded

// Not excluded
/debuginfo             ✗ not excluded (no slash)
/api/debug             ✗ not excluded (doesn't start with prefix)

WithExcludePatterns

func WithExcludePatterns(patterns ...string) MiddlewareOption

Excludes paths matching regex patterns from metrics collection.

Parameters:

patterns ...string - Regular expression patterns

Example:

handler := metrics.Middleware(recorder,
    metrics.WithExcludePatterns(
        `^/v[0-9]+/internal/.*`,  // /v1/internal/*, /v2/internal/*
        `^/api/[0-9]+$`,           // /api/123, /api/456
        `^/admin/.*`,              // /admin/*
    ),
)(mux)

Use Cases:

Version-specific internal paths
High-cardinality routes (IDs in path)
Pattern-based exclusions

Behavior:

Uses Go’s regexp package
Matches full path
Case-sensitive (use (?i) for case-insensitive)

Examples:

// Pattern: `^/v[0-9]+/internal/.*`
/v1/internal/metrics   ✓ excluded
/v2/internal/debug     ✓ excluded

// Not excluded
/internal/api          ✗ not excluded (no version)
/api/v1/internal       ✗ not excluded (doesn't start with /v)

// Pattern: `^/api/[0-9]+$`
/api/123               ✓ excluded
/api/456               ✓ excluded

// Not excluded
/api/users             ✗ not excluded (not numeric)
/api/123/details       ✗ not excluded (has suffix)

Pattern Tips:

// Anchors
^      // Start of path
$      // End of path

// Character classes
[0-9]  // Any digit
[a-z]  // Any lowercase letter
.      // Any character
\d     // Any digit

// Quantifiers
*      // Zero or more
+      // One or more
?      // Zero or one
{n}    // Exactly n

// Grouping
(...)  // Group

// Case-insensitive
(?i)pattern  // Case-insensitive match

Combining Exclusions

Use multiple exclusion options together:

handler := metrics.Middleware(recorder,
    // Exact paths
    metrics.WithExcludePaths("/health", "/metrics", "/ready"),
    
    // Prefixes
    metrics.WithExcludePrefixes("/debug/", "/internal/", "/_/"),
    
    // Patterns
    metrics.WithExcludePatterns(
        `^/v[0-9]+/internal/.*`,
        `^/api/users/[0-9]+$`,  // User IDs in path
    ),
)(mux)

Evaluation Order:

Exact paths (WithExcludePaths)
Prefixes (WithExcludePrefixes)
Patterns (WithExcludePatterns)

If any exclusion matches, the path is excluded.

Header Recording Options

WithHeaders

func WithHeaders(headers ...string) MiddlewareOption

Records specific HTTP headers as metric attributes.

Parameters:

headers ...string - Header names to record

Example:

handler := metrics.Middleware(recorder,
    metrics.WithHeaders("X-Request-ID", "X-Correlation-ID", "X-Client-Version"),
)(mux)

Behavior:

Headers recorded as metric attributes
Header names normalized (lowercase, hyphens to underscores)
Sensitive headers automatically filtered

Header Normalization:

// Original header → Metric attribute
X-Request-ID       → x_request_id
X-Correlation-ID   → x_correlation_id
Content-Type       → content_type
User-Agent         → user_agent

Example Metric:

http_requests_total{
    method="GET",
    path="/api/users",
    status="200",
    x_request_id="abc123",
    x_correlation_id="def456"
} 1

Sensitive Header Filtering

The middleware automatically filters sensitive headers, even if explicitly requested.

Always Filtered Headers:

Authorization
Cookie
Set-Cookie
X-API-Key
X-Auth-Token
Proxy-Authorization
WWW-Authenticate

Example:

// Only X-Request-ID will be recorded
// Authorization and Cookie are automatically filtered
handler := metrics.Middleware(recorder,
    metrics.WithHeaders(
        "Authorization",      // ✗ Filtered (sensitive)
        "X-Request-ID",       // ✓ Recorded
        "Cookie",             // ✗ Filtered (sensitive)
        "X-Correlation-ID",   // ✓ Recorded
    ),
)(mux)

Why Filter?

Prevent credential leaks in metrics
Avoid exposing API keys
Comply with security policies
Prevent compliance violations

Configuration Examples

Basic Health Check Exclusion

handler := metrics.Middleware(recorder,
    metrics.WithExcludePaths("/health", "/ready"),
)(mux)

Development/Debug Exclusion

handler := metrics.Middleware(recorder,
    metrics.WithExcludePaths("/health", "/metrics"),
    metrics.WithExcludePrefixes("/debug/", "/_/"),
)(mux)

High-Cardinality Path Exclusion

handler := metrics.Middleware(recorder,
    // Exclude paths with IDs to avoid high cardinality
    metrics.WithExcludePatterns(
        `^/api/users/[0-9]+$`,         // /api/users/123
        `^/api/orders/[a-z0-9-]+$`,    // /api/orders/abc-123
        `^/files/[^/]+$`,              // /files/{id}
    ),
)(mux)

Request Tracing

handler := metrics.Middleware(recorder,
    metrics.WithExcludePaths("/health"),
    metrics.WithHeaders("X-Request-ID", "X-Correlation-ID", "X-Trace-ID"),
)(mux)

Production Configuration

handler := metrics.Middleware(recorder,
    // Exclude operational endpoints
    metrics.WithExcludePaths(
        "/health",
        "/ready",
        "/metrics",
        "/favicon.ico",
    ),
    
    // Exclude administrative paths
    metrics.WithExcludePrefixes(
        "/debug/",
        "/internal/",
        "/_/",
    ),
    
    // Exclude high-cardinality routes
    metrics.WithExcludePatterns(
        `^/api/v[0-9]+/internal/.*`,
        `^/api/users/[0-9]+$`,
        `^/api/orders/[a-z0-9-]+$`,
    ),
    
    // Record tracing headers
    metrics.WithHeaders(
        "X-Request-ID",
        "X-Correlation-ID",
        "X-Client-Version",
    ),
)(mux)

Best Practices

Path Exclusions

DO:

Exclude health and readiness checks
Exclude metrics endpoints
Exclude high-cardinality paths (IDs)
Exclude debug and administrative paths

DON’T:

Over-exclude (you need some metrics!)
Exclude business-critical endpoints
Use overly broad patterns

Header Recording

DO:

Record low-cardinality headers only
Use headers for request tracing
Consider privacy implications

DON’T:

Record sensitive headers (automatically filtered)
Record high-cardinality headers (user IDs, timestamps)
Record excessive headers (increases metric cardinality)

Cardinality Management

High cardinality leads to:

Excessive memory usage
Slow query performance
Storage bloat

Low Cardinality (Good):

// Headers with limited values
X-Client-Version: v1.0, v1.1, v2.0  (3 values)
X-Region: us-east-1, eu-west-1      (2 values)

High Cardinality (Bad):

// Headers with unbounded values
X-Request-ID: abc123, def456, ...   (millions of values)
X-Timestamp: 2025-01-18T10:30:00Z   (always unique)
X-User-ID: user123, user456, ...    (millions of values)

Performance Considerations

Path Evaluation Overhead

Exact paths: O(1) hash lookup
Prefixes: O(n) prefix checks (n = number of prefixes)
Patterns: O(n) regex matches (n = number of patterns)

Recommendation: Use exact paths when possible for best performance.

Header Recording Impact

Each header adds:

Additional metric attribute
Increased metric cardinality
Higher memory usage

Recommendation: Only record necessary headers.

Troubleshooting

Path Not Excluded

Check:

Path is exact match (use WithExcludePaths)
Prefix includes trailing slash
Pattern uses correct regex syntax
Pattern is anchored (^ and $)

Header Not Recorded

Check:

Header name is correct (case-insensitive)
Header is not in sensitive list
Header is present in request

High Memory Usage

Check:

Too many unique paths (exclude high-cardinality routes)
Too many header combinations
Recording high-cardinality headers

Next Steps

See API Reference for middleware function
Read Middleware Guide for detailed examples
Check Configuration for histogram tuning
Review Troubleshooting for common issues

4 - Troubleshooting

Common issues and solutions for the metrics package

Solutions to common issues when using the metrics package.

Metrics Not Appearing

OTLP Provider

Symptoms:

Metrics not visible in collector
No data in monitoring system
Silent failures

Solutions:

1. Call Start() Before Recording

The OTLP provider requires Start() to be called before recording metrics:

recorder := metrics.MustNew(
    metrics.WithOTLP("http://localhost:4318"),
    metrics.WithServiceName("my-service"),
)

// IMPORTANT: Call Start() before recording
if err := recorder.Start(ctx); err != nil {
    log.Fatal(err)
}

// Now recording works
_ = recorder.IncrementCounter(ctx, "requests_total")

2. Check OTLP Collector Reachability

Verify the collector is accessible:

# Test connectivity
curl http://localhost:4318/v1/metrics

# Check collector logs
docker logs otel-collector

3. Wait for Export Interval

OTLP exports metrics periodically (default: 30s):

// Reduce interval for testing
recorder := metrics.MustNew(
    metrics.WithOTLP("http://localhost:4318"),
    metrics.WithExportInterval(5 * time.Second),
    metrics.WithServiceName("my-service"),
)

Or force immediate export:

if err := recorder.ForceFlush(ctx); err != nil {
    log.Printf("Failed to flush: %v", err)
}

4. Enable Logging

Add logging to see what’s happening:

recorder := metrics.MustNew(
    metrics.WithOTLP("http://localhost:4318"),
    metrics.WithLogger(slog.Default()),
    metrics.WithServiceName("my-service"),
)

Prometheus Provider

Symptoms:

Metrics endpoint returns 404
Empty metrics output
Server not accessible

Solutions:

1. Call Start() to Start Server

recorder := metrics.MustNew(
    metrics.WithPrometheus(":9090", "/metrics"),
    metrics.WithServiceName("my-service"),
)

// Start the HTTP server
if err := recorder.Start(ctx); err != nil {
    log.Fatal(err)
}

2. Check Actual Address

If not using strict mode, server may use different port:

address := recorder.ServerAddress()
log.Printf("Metrics at: http://%s/metrics", address)

3. Verify Firewall/Network

Check if port is accessible:

# Test locally
curl http://localhost:9090/metrics

# Check from another machine
curl http://<server-ip>:9090/metrics

Stdout Provider

Symptoms:

No output to console
Metrics not visible

Solutions:

1. Wait for Export Interval

Stdout exports periodically (default: 30s):

recorder := metrics.MustNew(
    metrics.WithStdout(),
    metrics.WithExportInterval(5 * time.Second),  // Shorter interval
    metrics.WithServiceName("my-service"),
)

2. Force Flush

if err := recorder.ForceFlush(ctx); err != nil {
    log.Printf("Failed to flush: %v", err)
}

Port Conflicts

Symptoms

Error: address already in use
Metrics server fails to start
Different port than expected

Solutions

1. Use Strict Port Mode (Production)

Fail explicitly if port unavailable:

recorder := metrics.MustNew(
    metrics.WithPrometheus(":9090", "/metrics"),
    metrics.WithStrictPort(),  // Fail if 9090 unavailable
    metrics.WithServiceName("my-service"),
)

2. Check Port Usage

Find what’s using the port:

# Linux/macOS
lsof -i :9090
netstat -tuln | grep 9090

# Windows
netstat -ano | findstr :9090

3. Use Dynamic Port (Testing)

Let the system choose an available port:

recorder := metrics.MustNew(
    metrics.WithPrometheus(":0", "/metrics"),  // :0 = any available port
    metrics.WithServiceName("test-service"),
)
recorder.Start(ctx)

// Get actual port
address := recorder.ServerAddress()
log.Printf("Using port: %s", address)

4. Use Testing Utilities

For tests, use the testing utilities with automatic port allocation:

func TestMetrics(t *testing.T) {
    t.Parallel()
    recorder := metrics.TestingRecorderWithPrometheus(t, "test-service")
    // Automatically finds available port
}

Custom Metric Limit Reached

Symptoms

Error: custom metric limit reached
New metrics not created
Warning in logs

Solutions

1. Increase Limit

recorder := metrics.MustNew(
    metrics.WithPrometheus(":9090", "/metrics"),
    metrics.WithMaxCustomMetrics(5000),  // Increase from default 1000
    metrics.WithServiceName("my-service"),
)

2. Monitor Usage

Track how many custom metrics are created:

count := recorder.CustomMetricCount()
log.Printf("Custom metrics: %d/%d", count, maxLimit)

// Expose as a metric
_ = recorder.SetGauge(ctx, "custom_metrics_count", float64(count))

3. Review Metric Cardinality

Check if you’re creating too many unique metrics:

// BAD: High cardinality (unique per user)
_ = recorder.IncrementCounter(ctx, "user_"+userID+"_requests")

// GOOD: Low cardinality (use labels)
_ = recorder.IncrementCounter(ctx, "user_requests_total",
    attribute.String("user_type", userType),
)

4. Consolidate Metrics

Combine similar metrics:

// BAD: Many separate metrics
_ = recorder.IncrementCounter(ctx, "get_requests_total")
_ = recorder.IncrementCounter(ctx, "post_requests_total")
_ = recorder.IncrementCounter(ctx, "put_requests_total")

// GOOD: One metric with label
_ = recorder.IncrementCounter(ctx, "requests_total",
    attribute.String("method", "GET"),
)

What Counts as Custom Metric?

Counts:

Each unique metric name created with IncrementCounter, AddCounter, RecordHistogram, SetGauge

Does NOT count:

Built-in HTTP metrics
Different label combinations of same metric
Re-recording same metric name

Metrics Server Not Starting

Symptoms

Start() returns error
Server not accessible
No metrics endpoint

Solutions

1. Check Context

Ensure context is not canceled:

ctx, cancel := signal.NotifyContext(context.Background(), os.Interrupt)
defer cancel()

// Use context with Start
if err := recorder.Start(ctx); err != nil {
    log.Fatal(err)
}

2. Check Port Availability

See Port Conflicts section.

3. Enable Logging

recorder := metrics.MustNew(
    metrics.WithPrometheus(":9090", "/metrics"),
    metrics.WithLogger(slog.Default()),
    metrics.WithServiceName("my-service"),
)

4. Check Permissions

Ensure your process has permission to bind to the port (< 1024 requires root on Linux).

Invalid Metric Names

Symptoms

Error: invalid metric name
Metrics not recorded
Reserved prefix error

Solutions

1. Check Naming Rules

Metric names must:

Start with letter (a-z, A-Z)
Contain only: letters, numbers, underscores, dots, hyphens
Not use reserved prefixes: __, http_, router_
Maximum 255 characters

Valid:

_ = recorder.IncrementCounter(ctx, "orders_total")
_ = recorder.IncrementCounter(ctx, "api.v1.requests")
_ = recorder.IncrementCounter(ctx, "payment-success")

Invalid:

_ = recorder.IncrementCounter(ctx, "__internal")      // Reserved prefix
_ = recorder.IncrementCounter(ctx, "http_custom")     // Reserved prefix
_ = recorder.IncrementCounter(ctx, "router_gauge")    // Reserved prefix
_ = recorder.IncrementCounter(ctx, "1st_metric")      // Starts with number
_ = recorder.IncrementCounter(ctx, "my metric!")      // Invalid characters

2. Handle Errors

Check for naming errors:

if err := recorder.IncrementCounter(ctx, metricName); err != nil {
    log.Printf("Invalid metric name %q: %v", metricName, err)
}

High Memory Usage

Symptoms

Excessive memory consumption
Out of memory errors
Slow performance

Solutions

1. Reduce Metric Cardinality

Limit unique label combinations:

// BAD: High cardinality
_ = recorder.IncrementCounter(ctx, "requests_total",
    attribute.String("user_id", userID),        // Millions of values
    attribute.String("request_id", requestID),  // Always unique
)

// GOOD: Low cardinality
_ = recorder.IncrementCounter(ctx, "requests_total",
    attribute.String("user_type", userType),    // Few values
    attribute.String("region", region),          // Few values
)

2. Exclude High-Cardinality Paths

handler := metrics.Middleware(recorder,
    metrics.WithExcludePatterns(
        `^/api/users/[0-9]+$`,      // User IDs
        `^/api/orders/[a-z0-9-]+$`, // Order IDs
    ),
)(mux)

3. Reduce Histogram Buckets

// BAD: Too many buckets (15)
metrics.WithDurationBuckets(
    0.001, 0.005, 0.01, 0.025, 0.05,
    0.1, 0.25, 0.5, 1, 2.5,
    5, 10, 30, 60, 120,
)

// GOOD: Fewer buckets (7)
metrics.WithDurationBuckets(0.01, 0.1, 0.5, 1, 5, 10)

4. Monitor Custom Metrics

count := recorder.CustomMetricCount()
if count > 500 {
    log.Printf("WARNING: High custom metric count: %d", count)
}

Performance Issues

HTTP Middleware Overhead

Symptom: Slow request handling

Solution: Exclude high-traffic paths:

handler := metrics.Middleware(recorder,
    metrics.WithExcludePaths("/health"),  // Called frequently
    metrics.WithExcludePrefixes("/static/"),  // Static assets
)(mux)

Histogram Recording Slow

Symptom: High CPU usage

Solution: Reduce bucket count (see High Memory Usage).

Global State Issues

Symptoms

Multiple recorder instances conflict
Unexpected behavior with multiple services
Global meter provider issues

Solutions

1. Use Default Behavior (Recommended)

By default, recorders do NOT set global meter provider:

// These work independently
recorder1 := metrics.MustNew(
    metrics.WithPrometheus(":9090", "/metrics"),
    metrics.WithServiceName("service-1"),
)

recorder2 := metrics.MustNew(
    metrics.WithStdout(),
    metrics.WithServiceName("service-2"),
)

2. Avoid WithGlobalMeterProvider

Only use WithGlobalMeterProvider() if you need:

OpenTelemetry instrumentation libraries to use your provider
otel.GetMeterProvider() to return your provider

// Only if needed
recorder := metrics.MustNew(
    metrics.WithPrometheus(":9090", "/metrics"),
    metrics.WithGlobalMeterProvider(),  // Explicit opt-in
    metrics.WithServiceName("my-service"),
)

Thread Safety

All Recorder methods are thread-safe. No special handling needed for concurrent access:

// Safe to call from multiple goroutines
go func() {
    _ = recorder.IncrementCounter(ctx, "worker_1")
}()

go func() {
    _ = recorder.IncrementCounter(ctx, "worker_2")
}()

Shutdown Issues

Graceful Shutdown Not Working

Solution: Use proper timeout context:

shutdownCtx, cancel := context.WithTimeout(context.Background(), 10*time.Second)
defer cancel()

if err := recorder.Shutdown(shutdownCtx); err != nil {
    log.Printf("Shutdown error: %v", err)
}

Metrics Not Flushed on Exit

Solution: Always defer Shutdown():

recorder := metrics.MustNew(
    metrics.WithOTLP("http://localhost:4318"),
    metrics.WithServiceName("my-service"),
)
recorder.Start(ctx)

defer func() {
    shutdownCtx, cancel := context.WithTimeout(context.Background(), 5*time.Second)
    defer cancel()
    recorder.Shutdown(shutdownCtx)
}()

Testing Issues

Port Conflicts in Parallel Tests

Solution: Use testing utilities with dynamic ports:

func TestHandler(t *testing.T) {
    t.Parallel()  // Safe with TestingRecorder
    
    // Uses stdout, no port needed
    recorder := metrics.TestingRecorder(t, "test-service")
    
    // Or with Prometheus (dynamic port)
    recorder := metrics.TestingRecorderWithPrometheus(t, "test-service")
}

Server Not Ready

Solution: Wait for server:

recorder := metrics.TestingRecorderWithPrometheus(t, "test-service")

err := metrics.WaitForMetricsServer(t, recorder.ServerAddress(), 5*time.Second)
if err != nil {
    t.Fatal(err)
}

Getting Help

If you’re still experiencing issues:

Check logs: Enable logging with WithLogger(slog.Default())
Review configuration: Verify all options are correct
Test connectivity: Ensure network access to endpoints
Check version: Update to latest version
File an issue: GitHub Issues

Quick Reference

Common Patterns

Production Setup:

recorder := metrics.MustNew(
    metrics.WithPrometheus(":9090", "/metrics"),
    metrics.WithStrictPort(),
    metrics.WithServiceName("my-api"),
    metrics.WithServiceVersion(version),
    metrics.WithLogger(slog.Default()),
)

OTLP Setup:

recorder := metrics.MustNew(
    metrics.WithOTLP("http://localhost:4318"),
    metrics.WithServiceName("my-service"),
)
// IMPORTANT: Call Start() before recording
recorder.Start(ctx)

Testing Setup:

func TestMetrics(t *testing.T) {
    t.Parallel()
    recorder := metrics.TestingRecorder(t, "test-service")
    // Test code...
}

Next Steps

Review Configuration Guide for setup examples
Check API Reference for method details
See Examples for complete applications
Read Basic Usage for fundamentals

Metrics Package

Package Information

Package Overview

Core Features

Architecture

Components

Main Package (rivaas.dev/metrics)

Quick API Index

Recorder Creation

Lifecycle Management

Recording Metrics

HTTP Middleware

Provider-Specific Methods

Testing Utilities

Reference Pages

Type Reference

Recorder

Option

EventType

Event

EventHandler

Common Patterns

Basic Usage

With HTTP Middleware

With OTLP

Thread Safety

Performance Notes

Built-in Metrics

Version Compatibility

Next Steps

1 - API Reference

Recorder Type

Creation Functions

New

MustNew

Lifecycle Methods

Start

Shutdown

ForceFlush

Custom Metrics Methods

IncrementCounter

AddCounter

RecordHistogram

SetGauge

Provider-Specific Methods

ServerAddress

Handler

CustomMetricCount

Middleware Function

Middleware

Testing Functions

TestingRecorder

TestingRecorderWithPrometheus

WaitForMetricsServer

Event Types

EventType

Event

EventHandler

Error Handling

Invalid Metric Name

Metric Limit Reached

Provider Not Started

Thread Safety

Next Steps

2 - Configuration Options

Provider Options

WithPrometheus

WithOTLP

WithStdout

Service Configuration Options

WithServiceName

WithServiceVersion

Prometheus-Specific Options

WithStrictPort

WithServerDisabled

Histogram Bucket Options

WithDurationBuckets

WithSizeBuckets

Advanced Options

WithExportInterval

Main Package (`rivaas.dev/metrics`)