Configuration Management
Learn how to manage application configuration with the Rivaas config package
The Rivaas Config package provides configuration management for Go applications. It simplifies handling settings across different environments and formats. Follows the Twelve-Factor App methodology.
Features
- Easy Integration: Simple and intuitive API
- Flexible Sources: Load from files, environment variables (with custom prefixes), Consul, and easily extend with custom sources
- Dynamic Paths: Use
${VAR} in file and Consul paths for environment-based configuration - Format Agnostic: Supports JSON, YAML, TOML, and other formats via extensible codecs
- Type Casting: Built-in caster codecs for automatic type conversion (bool, int, float, time, duration, etc.)
- Hierarchical Merging: Configurations from multiple sources are merged, with later sources overriding earlier ones
- Struct Binding: Automatically map configuration data to Go structs
- Built-in Validation: Validate configuration using struct methods, JSON Schemas, or custom functions
- Dot Notation Access: Navigate nested configuration easily (e.g.,
cfg.String("database.host")) - Type-Safe Retrieval: Get values as specific types (
string, int, bool, etc.), with error-returning options for robust handling - Configuration Dumping: Save the effective configuration to files or other custom destinations
- Thread-Safe: Safe for concurrent access and configuration loading in multi-goroutine applications
- Nil-Safe Operations: All getter methods handle nil Config instances gracefully
Quick Start
Here’s a 30-second example to get you started:
package main
import (
"rivaas.dev/config"
"context"
"log"
)
func main() {
// Create config with multiple sources
cfg := config.MustNew(
config.WithFile("config.yaml"), // Auto-detects YAML format
config.WithFile("config.json"), // Auto-detects JSON format
config.WithEnv("APP_"), // Load environment variables with APP_ prefix
)
if err := cfg.Load(context.Background()); err != nil {
log.Fatalf("failed to load configuration: %v", err)
}
// Access configuration values - simple and clean!
port := cfg.Int("server.port")
host := cfg.StringOr("server.host", "localhost") // With default value
debug := cfg.Bool("debug")
log.Printf("Server running on %s:%d (debug: %v)", host, port, debug)
}
How It Works
- Sources are loaded in order; later sources override earlier ones
- Dot notation allows deep access:
cfg.Get("database.host") - Type-safe accessors:
String, Int, Bool, etc., plus generic Get[T], GetOr[T], GetE[T] for custom types - Context validation: Both
Load() and Dump() methods validate that context is not nil - Error handling: All methods return descriptive errors for easier debugging
Learning Path
Follow these guides to master configuration management with Rivaas:
- Installation - Get started with the config package
- Basic Usage - Learn the fundamentals of loading and accessing configuration
- Environment Variables - Master environment variable integration
- Struct Binding - Map configuration to Go structs automatically
- Validation - Ensure configuration correctness with validation
- Multiple Sources - Combine configuration from different sources
- Custom Codecs - Extend support to custom formats
- Examples - See real-world usage patterns
Next Steps
1 - Installation
Install and set up the Rivaas config package for your Go application
Get started with the Rivaas config package by installing it in your Go project.
Prerequisites
- Go 1.25 or higher - The config package requires Go 1.25+
- Basic familiarity with Go modules
Installation
Install the config package using go get:
This will add the package to your go.mod file and download the dependencies.
Verify Installation
Create a simple test to verify the installation is working:
package main
import (
"context"
"fmt"
"rivaas.dev/config"
)
func main() {
cfg := config.MustNew()
if err := cfg.Load(context.Background()); err != nil {
panic(err)
}
fmt.Println("Config package installed successfully!")
}
Save this as main.go and run:
If you see “Config package installed successfully!”, the installation is complete!
Import Path
Always import the config package using:
import "rivaas.dev/config"
Additional Packages
Depending on your use case, you may also want to import sub-packages:
import (
"rivaas.dev/config"
"rivaas.dev/config/codec" // For custom codecs
"rivaas.dev/config/dumper" // For custom dumpers
"rivaas.dev/config/source" // For custom sources
)
Common Issues
Go Version Too Old
If you get an error about Go version:
go: rivaas.dev/config requires go >= 1.25
Update your Go installation to version 1.25 or higher:
go version # Check current version
Visit golang.org/dl/ to download the latest version.
Module Not Found
If you get a “module not found” error:
go: rivaas.dev/config: module rivaas.dev/config: Get "https://rivaas.dev/config": dial tcp: lookup rivaas.dev
Make sure you have network connectivity and try:
go clean -modcache
go get rivaas.dev/config
Dependency Conflicts
If you experience dependency conflicts, ensure your go.mod is up to date:
Next Steps
Now that you have the config package installed:
For complete API documentation, visit the API Reference.
2 - Basic Usage
Learn the fundamentals of loading and accessing configuration with Rivaas
This guide covers the essential operations for working with the config package. Learn how to load configuration files, access values, and handle errors.
Loading Configuration Files
The config package automatically detects file formats based on the file extension. Supported formats include JSON, YAML, and TOML.
Simple File Loading
package main
import (
"context"
"log"
"rivaas.dev/config"
)
func main() {
cfg := config.MustNew(
config.WithFile("config.yaml"),
)
if err := cfg.Load(context.Background()); err != nil {
log.Fatalf("failed to load config: %v", err)
}
}
You can load multiple configuration files of different formats:
cfg := config.MustNew(
config.WithFile("config.yaml"), // YAML format
config.WithFile("config.json"), // JSON format
config.WithFile("config.toml"), // TOML format
)
Files are processed in order. Later files override values from earlier ones, enabling environment-specific overrides.
Environment Variables in Paths
You can use environment variables in file paths. This is useful when different environments use different directories:
// Use ${VAR} or $VAR in paths
cfg := config.MustNew(
config.WithFile("${CONFIG_DIR}/app.yaml"), // Expands to actual directory
config.WithFile("${APP_ENV}/overrides.yaml"), // e.g., "production/overrides.yaml"
)
This works with all path-based options: WithFile, WithFileAs, WithConsul, WithConsulAs, WithFileDumper, and WithFileDumperAs.
The config package includes built-in codecs for common formats:
| Format | Extension | Codec Type |
|---|
| JSON | .json | codec.TypeJSON |
| YAML | .yaml, .yml | codec.TypeYAML |
| TOML | .toml | codec.TypeTOML |
| Environment Variables | - | codec.TypeEnvVar |
Accessing Configuration Values
Once loaded, access configuration using dot notation and type-safe getters.
Dot Notation
Navigate nested configuration structures using dots:
// Given config: { "database": { "host": "localhost", "port": 5432 } }
host := cfg.String("database.host") // "localhost"
port := cfg.Int("database.port") // 5432
Type-Safe Getters
The config package provides type-safe getters for common data types:
// Basic types
stringVal := cfg.String("key")
intVal := cfg.Int("key")
boolVal := cfg.Bool("key")
floatVal := cfg.Float64("key")
// Time and duration
duration := cfg.Duration("timeout")
timestamp := cfg.Time("created_at")
// Collections
slice := cfg.StringSlice("tags")
mapping := cfg.StringMap("metadata")
Getters with Default Values
Use Or variants to provide fallback values:
host := cfg.StringOr("server.host", "localhost")
port := cfg.IntOr("server.port", 8080)
debug := cfg.BoolOr("debug", false)
timeout := cfg.DurationOr("timeout", 30*time.Second)
Generic Getters for Custom Types
For custom types or explicit error handling, use the generic GetE function:
// With error handling
port, err := config.GetE[int](cfg, "server.port")
if err != nil {
log.Printf("invalid port: %v", err)
port = 8080 // fallback
}
// For custom types
type DatabaseConfig struct {
Host string
Port int
}
dbConfig, err := config.GetE[DatabaseConfig](cfg, "database")
if err != nil {
log.Fatalf("invalid database config: %v", err)
}
Error Handling
The config package provides comprehensive error handling through different getter variants.
Short methods return zero values for missing keys:
cfg.String("nonexistent") // Returns ""
cfg.Int("nonexistent") // Returns 0
cfg.Bool("nonexistent") // Returns false
cfg.StringSlice("nonexistent") // Returns []string{}
cfg.StringMap("nonexistent") // Returns map[string]any{}
This approach is ideal when you want simple access with sensible defaults.
Or methods provide explicit fallback values:
host := cfg.StringOr("host", "localhost") // Returns "localhost" if missing
port := cfg.IntOr("port", 8080) // Returns 8080 if missing
debug := cfg.BoolOr("debug", false) // Returns false if missing
timeout := cfg.DurationOr("timeout", 30*time.Second) // Returns 30s if missing
Use GetE for explicit error handling:
port, err := config.GetE[int](cfg, "server.port")
if err != nil {
return fmt.Errorf("invalid port configuration: %w", err)
}
// Errors provide context
// Example: "config error: key 'server.port' not found"
ConfigError Structure
When errors occur during loading, they’re wrapped in ConfigError:
type ConfigError struct {
Source string // Where the error occurred (e.g., "source[0]")
Field string // Specific field with the error
Operation string // Operation being performed (e.g., "load")
Err error // Underlying error
}
Example error handling during load:
if err := cfg.Load(context.Background()); err != nil {
// Error message includes context:
// "config error in source[0] during load: file not found: config.yaml"
log.Fatalf("configuration error: %v", err)
}
Nil-Safe Operations
All getter methods handle nil Config instances gracefully:
var cfg *config.Config // nil
// Short methods return zero values (no panic)
cfg.String("key") // Returns ""
cfg.Int("key") // Returns 0
cfg.Bool("key") // Returns false
// Error methods return errors
port, err := config.GetE[int](cfg, "key")
// err: "config instance is nil"
Complete Example
Putting it all together:
package main
import (
"context"
"log"
"time"
"rivaas.dev/config"
)
func main() {
// Create config with file source
cfg := config.MustNew(
config.WithFile("config.yaml"),
)
// Load configuration
if err := cfg.Load(context.Background()); err != nil {
log.Fatalf("failed to load config: %v", err)
}
// Access values with different approaches
// Simple access (with zero values for missing keys)
host := cfg.String("server.host")
// With defaults
port := cfg.IntOr("server.port", 8080)
debug := cfg.BoolOr("debug", false)
// With error handling
timeout, err := config.GetE[time.Duration](cfg, "server.timeout")
if err != nil {
log.Printf("using default timeout: %v", err)
timeout = 30 * time.Second
}
log.Printf("Server: %s:%d (debug: %v, timeout: %v)",
host, port, debug, timeout)
}
Sample config.yaml:
server:
host: localhost
port: 8080
timeout: 30s
debug: true
Next Steps
For complete API details, see the API Reference.
3 - Environment Variables
Master environment variable integration with hierarchical naming conventions
The config package provides powerful environment variable support. It automatically maps environment variables to nested configuration structures. This follows the Twelve-Factor App methodology for configuration management.
Basic Usage
Enable environment variable support with a custom prefix:
cfg := config.MustNew(
config.WithFile("config.yaml"),
config.WithEnv("MYAPP_"), // Only env vars with MYAPP_ prefix
)
The prefix helps avoid conflicts with system or other application variables.
Naming Convention
The config package uses a hierarchical naming convention where underscores (_) in environment variable names create nested configuration structures.
- Strip prefix: Remove the configured prefix like
MYAPP_. - Convert to lowercase:
DATABASE_HOST → database_host. - Split by underscores:
database_host → ["database", "host"]. - Filter empty parts: Consecutive underscores create no extra levels.
- Create dot notation:
["database", "host"] → database.host.
Visualization
graph TB
EnvVar[MYAPP_DATABASE_HOST=localhost] --> StripPrefix[Strip Prefix MYAPP_]
StripPrefix --> Lowercase[database_host]
Lowercase --> SplitUnderscore[Split by underscore]
SplitUnderscore --> FilterEmpty[Filter empty parts]
FilterEmpty --> DotNotation[database.host = localhost]Examples
| Environment Variable | Configuration Path | Value |
|---|
MYAPP_SERVER_PORT | server.port | 8080 |
MYAPP_DATABASE_HOST | database.host | localhost |
MYAPP_DATABASE_USER_NAME | database.user.name | admin |
MYAPP_FOO__BAR | foo.bar | value |
MYAPP_A_B_C_D | a.b.c.d | nested |
Basic Example
export MYAPP_SERVER_HOST=localhost
export MYAPP_SERVER_PORT=8080
export MYAPP_DEBUG=true
cfg := config.MustNew(
config.WithEnv("MYAPP_"),
)
cfg.Load(context.Background())
host := cfg.String("server.host") // "localhost"
port := cfg.Int("server.port") // 8080
debug := cfg.Bool("debug") // true
Nested Configuration
Environment variables naturally create nested structures:
export MYAPP_DATABASE_PRIMARY_HOST=db1.example.com
export MYAPP_DATABASE_PRIMARY_PORT=5432
export MYAPP_DATABASE_REPLICA_HOST=db2.example.com
export MYAPP_DATABASE_REPLICA_PORT=5432
Access nested values:
primaryHost := cfg.String("database.primary.host") // "db1.example.com"
replicaHost := cfg.String("database.replica.host") // "db2.example.com"
Struct Field Mapping
When using struct binding, environment variables map directly to struct fields using the config tag:
type Config struct {
Port int `config:"port"`
Host string `config:"host"`
Database struct {
Host string `config:"host"`
Port int `config:"port"`
Username string `config:"username"`
Password string `config:"password"`
} `config:"database"`
}
Required environment variables:
export MYAPP_PORT=8080
export MYAPP_HOST=localhost
export MYAPP_DATABASE_HOST=db.example.com
export MYAPP_DATABASE_PORT=5432
export MYAPP_DATABASE_USERNAME=admin
export MYAPP_DATABASE_PASSWORD=secret123
Usage:
var appConfig Config
cfg := config.MustNew(
config.WithEnv("MYAPP_"),
config.WithBinding(&appConfig),
)
if err := cfg.Load(context.Background()); err != nil {
log.Fatalf("config error: %v", err)
}
// appConfig is now populated from environment variables
Advanced Nested Structures
For complex applications with deeply nested configuration:
type AppConfig struct {
Server struct {
Host string `config:"host"`
Port int `config:"port"`
TLS struct {
Enabled bool `config:"enabled"`
CertFile string `config:"cert_file"`
KeyFile string `config:"key_file"`
} `config:"tls"`
} `config:"server"`
Database struct {
Primary struct {
Host string `config:"host"`
Port int `config:"port"`
Database string `config:"database"`
} `config:"primary"`
Replica struct {
Host string `config:"host"`
Port int `config:"port"`
Database string `config:"database"`
} `config:"replica"`
} `config:"database"`
}
Environment variables:
export MYAPP_SERVER_HOST=0.0.0.0
export MYAPP_SERVER_PORT=8080
export MYAPP_SERVER_TLS_ENABLED=true
export MYAPP_SERVER_TLS_CERT_FILE=/etc/ssl/certs/server.crt
export MYAPP_SERVER_TLS_KEY_FILE=/etc/ssl/private/server.key
export MYAPP_DATABASE_PRIMARY_HOST=primary.db.example.com
export MYAPP_DATABASE_PRIMARY_PORT=5432
export MYAPP_DATABASE_PRIMARY_DATABASE=myapp
export MYAPP_DATABASE_REPLICA_HOST=replica.db.example.com
export MYAPP_DATABASE_REPLICA_PORT=5432
export MYAPP_DATABASE_REPLICA_DATABASE=myapp
Edge Cases
Consecutive Underscores
Multiple consecutive underscores are filtered to prevent empty parts:
export MYAPP_FOO__BAR=value # Becomes: foo.bar = "value"
export MYAPP_A___B=value # Becomes: a.b = "value"
Type Conflicts
If environment variables create conflicts between scalar and nested values, the nested structure takes precedence:
export MYAPP_FOO=scalar_value
export MYAPP_FOO_BAR=nested_value
# Result: foo.bar = "nested_value" (scalar "foo" is overwritten)
Whitespace Handling
Keys and values are automatically trimmed:
export MYAPP_KEY=" value " # Becomes: key = "value"
Best Practices
1. Use Descriptive Prefixes
Always use application-specific prefixes to avoid conflicts:
# Good - Application-specific
export MYAPP_DATABASE_HOST=localhost
export WEBAPP_DATABASE_HOST=localhost
# Avoid - Too generic
export DATABASE_HOST=localhost
2. Consistent Naming
Use consistent patterns across your application:
# Consistent pattern
export MYAPP_SERVER_HOST=localhost
export MYAPP_SERVER_PORT=8080
export MYAPP_SERVER_TIMEOUT=30s
3. Document Your Variables
Document required and optional environment variables:
# Required environment variables:
# MYAPP_SERVER_HOST - Server hostname (default: localhost)
# MYAPP_SERVER_PORT - Server port (default: 8080)
# MYAPP_DATABASE_HOST - Database hostname (required)
# MYAPP_DATABASE_PORT - Database port (default: 5432)
4. Validate Configuration
Use struct validation to ensure required variables are set:
func (c *Config) Validate() error {
if c.Server.Host == "" {
return errors.New("MYAPP_SERVER_HOST is required")
}
if c.Server.Port <= 0 {
return errors.New("MYAPP_SERVER_PORT must be positive")
}
return nil
}
Merging with Other Sources
Environment variables can override file-based configuration:
cfg := config.MustNew(
config.WithFile("config.yaml"), // Base configuration
config.WithFile("config.prod.yaml"), // Production overrides
config.WithEnv("MYAPP_"), // Environment overrides all files
)
Source precedence: Later sources override earlier ones. Environment variables, being last, have the highest priority.
Complete Example
package main
import (
"context"
"log"
"rivaas.dev/config"
)
type AppConfig struct {
Server struct {
Host string `config:"host"`
Port int `config:"port"`
} `config:"server"`
Database struct {
Host string `config:"host"`
Port int `config:"port"`
Username string `config:"username"`
Password string `config:"password"`
} `config:"database"`
}
func (c *AppConfig) Validate() error {
if c.Database.Host == "" {
return errors.New("database host is required")
}
if c.Database.Username == "" {
return errors.New("database username is required")
}
return nil
}
func main() {
var appConfig AppConfig
cfg := config.MustNew(
config.WithFile("config.yaml"), // Base config
config.WithEnv("MYAPP_"), // Override with env vars
config.WithBinding(&appConfig),
)
if err := cfg.Load(context.Background()); err != nil {
log.Fatalf("failed to load config: %v", err)
}
log.Printf("Server: %s:%d", appConfig.Server.Host, appConfig.Server.Port)
log.Printf("Database: %s:%d", appConfig.Database.Host, appConfig.Database.Port)
}
Next Steps
For technical details on the environment variable codec, see Codecs Reference.
4 - Struct Binding
Automatically map configuration data to Go structs with type safety
Struct binding allows you to automatically map configuration data to your own Go structs. This provides type safety and a clean, idiomatic way to work with configuration.
Basic Struct Binding
Define a struct and bind it during configuration initialization:
type Config struct {
Port int `config:"port"`
Host string `config:"host"`
}
var c Config
cfg := config.MustNew(
config.WithFile("config.yaml"),
config.WithBinding(&c),
)
if err := cfg.Load(context.Background()); err != nil {
log.Fatalf("failed to load config: %v", err)
}
// c.Port and c.Host are now populated
log.Printf("Server: %s:%d", c.Host, c.Port)
Important
Always pass a **pointer** to your struct with `WithBinding(&c)`, not the struct value itself.
Use the config tag to specify the configuration key for each field:
type Config struct {
Port int `config:"port"`
Host string `config:"host"`
Timeout int `config:"timeout"`
}
The tag value should match the key name at that struct’s level in the configuration hierarchy.
Tag Naming
- Tags are case-sensitive.
- Use snake_case or lowercase for consistency.
- Match the structure of your configuration files.
# config.yaml
port: 8080
host: localhost
timeout: 30
Default Values
Specify default values using the default tag:
type Config struct {
Port int `config:"port" default:"8080"`
Host string `config:"host" default:"localhost"`
Debug bool `config:"debug" default:"false"`
Timeout time.Duration `config:"timeout" default:"30s"`
}
Default values are used when:
- The configuration key is not found.
- The configuration file doesn’t exist.
- Environment variables don’t provide the value.
var c Config
cfg := config.MustNew(
config.WithFile("config.yaml"), // May not exist or be incomplete
config.WithBinding(&c),
)
cfg.Load(context.Background())
// Fields use defaults if not present in config.yaml
Nested Structs
Create hierarchical configuration by nesting structs:
type Config struct {
Server struct {
Host string `config:"host"`
Port int `config:"port"`
} `config:"server"`
Database struct {
Host string `config:"host"`
Port int `config:"port"`
Username string `config:"username"`
Password string `config:"password"`
} `config:"database"`
}
Corresponding YAML:
server:
host: localhost
port: 8080
database:
host: db.example.com
port: 5432
username: admin
password: secret
Usage:
var c Config
cfg := config.MustNew(
config.WithFile("config.yaml"),
config.WithBinding(&c),
)
cfg.Load(context.Background())
log.Printf("Server: %s:%d", c.Server.Host, c.Server.Port)
log.Printf("Database: %s:%d", c.Database.Host, c.Database.Port)
Pointer Fields for Optional Values
Use pointer fields when values are truly optional:
type Config struct {
Port int `config:"port"`
Host string `config:"host"`
CacheURL *string `config:"cache_url"` // Optional
Debug *bool `config:"debug"` // Optional
}
If the configuration key is missing, pointer fields remain nil:
var c Config
cfg := config.MustNew(
config.WithFile("config.yaml"),
config.WithBinding(&c),
)
cfg.Load(context.Background())
if c.CacheURL != nil {
log.Printf("Using cache: %s", *c.CacheURL)
} else {
log.Printf("Cache disabled")
}
Deeply Nested Structures
For complex applications, create deeply nested configuration:
type AppConfig struct {
Server struct {
HTTP struct {
Host string `config:"host"`
Port int `config:"port"`
} `config:"http"`
TLS struct {
Enabled bool `config:"enabled"`
CertFile string `config:"cert_file"`
KeyFile string `config:"key_file"`
} `config:"tls"`
} `config:"server"`
Database struct {
Primary struct {
Host string `config:"host"`
Port int `config:"port"`
Database string `config:"database"`
} `config:"primary"`
Replica struct {
Host string `config:"host"`
Port int `config:"port"`
Database string `config:"database"`
} `config:"replica"`
} `config:"database"`
}
Corresponding YAML:
server:
http:
host: 0.0.0.0
port: 8080
tls:
enabled: true
cert_file: /etc/ssl/certs/server.crt
key_file: /etc/ssl/private/server.key
database:
primary:
host: primary.db.example.com
port: 5432
database: myapp
replica:
host: replica.db.example.com
port: 5432
database: myapp
Slices and Maps
Bind slices and maps for collection data:
type Config struct {
Hosts []string `config:"hosts"`
Ports []int `config:"ports"`
Metadata map[string]string `config:"metadata"`
Features map[string]bool `config:"features"`
}
YAML:
hosts:
- localhost
- example.com
- api.example.com
ports:
- 8080
- 8081
- 8082
metadata:
version: "1.0.0"
environment: production
features:
auth: true
cache: true
debug: false
Type Conversion
The config package automatically converts between compatible types:
type Config struct {
Port int `config:"port"` // Converts from string "8080"
Debug bool `config:"debug"` // Converts from string "true"
Timeout time.Duration `config:"timeout"` // Converts from string "30s"
}
YAML (as strings):
port: "8080" # String converted to int
debug: "true" # String converted to bool
timeout: "30s" # String converted to time.Duration
Common Issues and Solutions
Issue: Struct Not Populating
Problem: Fields remain at zero values after loading.
Solutions:
- Pass a pointer: Use
WithBinding(&c), not WithBinding(c)
// Wrong
cfg := config.MustNew(config.WithBinding(c))
// Correct
cfg := config.MustNew(config.WithBinding(&c))
- Check tag names: Ensure
config tags match your configuration structure
// If your YAML has "server_port", use:
Port int `config:"server_port"`
// Not:
Port int `config:"port"`
- Verify nested tags: All nested structs need the
config tag
// Wrong - missing tag on Server struct
type Config struct {
Server struct {
Port int `config:"port"`
} // Missing `config:"server"`
}
// Correct
type Config struct {
Server struct {
Port int `config:"port"`
} `config:"server"`
}
Issue: Type Mismatch Errors
Problem: Error during binding due to type incompatibility.
Solution: Ensure your struct types match the configuration data types or are compatible:
// If YAML has: port: 8080 (number)
Port int `config:"port"` // Correct
// If YAML has: port: "8080" (string)
Port int `config:"port"` // Still works - automatic conversion
Issue: Optional Fields Always Present
Problem: Want to distinguish between “not set” and “set to zero value”.
Solution: Use pointer types:
type Config struct {
// Can't distinguish "not set" vs "set to 0"
MaxConnections int `config:"max_connections"`
// Can distinguish: nil = not set, &0 = set to 0
MaxConnections *int `config:"max_connections"`
}
Complete Example
package main
import (
"context"
"log"
"time"
"rivaas.dev/config"
)
type AppConfig struct {
Server struct {
Host string `config:"host" default:"localhost"`
Port int `config:"port" default:"8080"`
Timeout time.Duration `config:"timeout" default:"30s"`
} `config:"server"`
Database struct {
Host string `config:"host"`
Port int `config:"port" default:"5432"`
Username string `config:"username"`
Password string `config:"password"`
MaxConns *int `config:"max_connections"` // Optional
} `config:"database"`
Features struct {
EnableCache bool `config:"enable_cache" default:"true"`
EnableAuth bool `config:"enable_auth" default:"true"`
} `config:"features"`
}
func (c *AppConfig) Validate() error {
if c.Database.Host == "" {
return errors.New("database host is required")
}
if c.Database.Username == "" {
return errors.New("database username is required")
}
return nil
}
func main() {
var appConfig AppConfig
cfg := config.MustNew(
config.WithFile("config.yaml"),
config.WithEnv("MYAPP_"),
config.WithBinding(&appConfig),
)
if err := cfg.Load(context.Background()); err != nil {
log.Fatalf("failed to load config: %v", err)
}
log.Printf("Server: %s:%d (timeout: %v)",
appConfig.Server.Host,
appConfig.Server.Port,
appConfig.Server.Timeout)
log.Printf("Database: %s:%d",
appConfig.Database.Host,
appConfig.Database.Port)
if appConfig.Database.MaxConns != nil {
log.Printf("Max DB connections: %d", *appConfig.Database.MaxConns)
}
}
config.yaml:
server:
host: 0.0.0.0
port: 8080
timeout: 60s
database:
host: postgres.example.com
port: 5432
username: myapp
password: secret123
max_connections: 100
features:
enable_cache: true
enable_auth: true
Next Steps
For technical details, see the API Reference.
5 - Validation
Validate configuration to catch errors early and ensure application correctness
The config package supports multiple validation strategies. These help catch configuration errors early. They ensure your application runs with correct settings.
Validation Strategies
The config package provides three validation approaches:
- Struct-based validation - Implement
Validate() error on your struct. - JSON Schema validation - Validate against a JSON Schema.
- Custom validation functions - Use custom validation logic.
Struct-Based Validation
The most idiomatic approach for Go applications. Implement the Validate() method on your configuration struct:
type Validator interface {
Validate() error
}
Basic Example
type Config struct {
Port int `config:"port"`
Host string `config:"host"`
}
func (c *Config) Validate() error {
if c.Port <= 0 || c.Port > 65535 {
return errors.New("port must be between 1 and 65535")
}
if c.Host == "" {
return errors.New("host is required")
}
return nil
}
var cfg Config
config := config.MustNew(
config.WithFile("config.yaml"),
config.WithBinding(&cfg),
)
// Validation runs automatically during Load()
if err := config.Load(context.Background()); err != nil {
log.Fatalf("invalid configuration: %v", err)
}
Complex Validation
Validate nested structures and relationships:
type AppConfig struct {
Server struct {
Host string `config:"host"`
Port int `config:"port"`
TLS struct {
Enabled bool `config:"enabled"`
CertFile string `config:"cert_file"`
KeyFile string `config:"key_file"`
} `config:"tls"`
} `config:"server"`
Database struct {
Host string `config:"host"`
Port int `config:"port"`
MaxConns int `config:"max_connections"`
IdleConns int `config:"idle_connections"`
} `config:"database"`
}
func (c *AppConfig) Validate() error {
// Server validation
if c.Server.Port < 1 || c.Server.Port > 65535 {
return fmt.Errorf("server.port must be between 1-65535, got %d", c.Server.Port)
}
// TLS validation
if c.Server.TLS.Enabled {
if c.Server.TLS.CertFile == "" {
return errors.New("server.tls.cert_file required when TLS enabled")
}
if c.Server.TLS.KeyFile == "" {
return errors.New("server.tls.key_file required when TLS enabled")
}
}
// Database validation
if c.Database.Host == "" {
return errors.New("database.host is required")
}
if c.Database.MaxConns < c.Database.IdleConns {
return fmt.Errorf("database.max_connections (%d) must be >= idle_connections (%d)",
c.Database.MaxConns, c.Database.IdleConns)
}
return nil
}
Field-Level Validation
Create reusable validation helpers:
func validatePort(port int) error {
if port < 1 || port > 65535 {
return fmt.Errorf("invalid port %d: must be between 1-65535", port)
}
return nil
}
func validateHostname(host string) error {
if host == "" {
return errors.New("hostname cannot be empty")
}
if len(host) > 253 {
return errors.New("hostname too long (max 253 characters)")
}
return nil
}
func (c *Config) Validate() error {
if err := validatePort(c.Server.Port); err != nil {
return fmt.Errorf("server.port: %w", err)
}
if err := validateHostname(c.Server.Host); err != nil {
return fmt.Errorf("server.host: %w", err)
}
return nil
}
JSON Schema Validation
What is JSON Schema?
JSON Schema is a standard for describing the structure and validation rules of JSON data. It allows you to define required fields, data types, value constraints, and more.
Validate the merged configuration map against a JSON Schema:
schemaBytes, err := os.ReadFile("schema.json")
if err != nil {
log.Fatalf("failed to read schema: %v", err)
}
cfg := config.MustNew(
config.WithFile("config.yaml"),
config.WithJSONSchema(schemaBytes),
)
// Schema validation runs during Load()
if err := cfg.Load(context.Background()); err != nil {
log.Fatalf("configuration validation failed: %v", err)
}
JSON Schema validation is applied to the merged configuration map (`map[string]any`), not directly to Go structs. It happens before struct binding.
Example Schema
schema.json:
{
"$schema": "http://json-schema.org/draft-07/schema#",
"type": "object",
"required": ["server", "database"],
"properties": {
"server": {
"type": "object",
"required": ["host", "port"],
"properties": {
"host": {
"type": "string",
"minLength": 1
},
"port": {
"type": "integer",
"minimum": 1,
"maximum": 65535
}
}
},
"database": {
"type": "object",
"required": ["host", "port"],
"properties": {
"host": {
"type": "string",
"minLength": 1
},
"port": {
"type": "integer",
"minimum": 1,
"maximum": 65535
},
"username": {
"type": "string"
},
"password": {
"type": "string"
}
}
}
}
}
Custom Validation Functions
Register custom validation functions for flexible validation logic:
cfg := config.MustNew(
config.WithFile("config.yaml"),
config.WithValidator(func(data map[string]any) error {
// Validate the configuration map
port, ok := data["port"].(int)
if !ok {
return errors.New("port must be an integer")
}
if port <= 0 {
return errors.New("port must be positive")
}
return nil
}),
)
Multiple Validators
You can register multiple validators - all will be executed:
cfg := config.MustNew(
config.WithFile("config.yaml"),
config.WithValidator(validatePorts),
config.WithValidator(validateHosts),
config.WithValidator(validateFeatures),
)
func validatePorts(data map[string]any) error {
// Port validation logic
}
func validateHosts(data map[string]any) error {
// Host validation logic
}
func validateFeatures(data map[string]any) error {
// Feature flag validation logic
}
Validation Workflow
The validation process follows this order:
graph TB
Load[cfg.Load] --> Sources[Load Sources]
Sources --> Merge[Merge Data]
Merge --> JSONSchema{JSON Schema?}
JSONSchema -->|Yes| ValidateSchema[Validate Schema]
JSONSchema -->|No| CustomVal
ValidateSchema --> CustomVal{Custom Validator?}
CustomVal -->|Yes| RunCustom[Run Function]
CustomVal -->|No| Binding
RunCustom --> Binding{Binding?}
Binding -->|Yes| BindStruct[Bind Struct]
Binding -->|No| Success
BindStruct --> StructVal{Has Validate?}
StructVal -->|Yes| RunValidate[Run Validate]
StructVal -->|No| Success[Success]
RunValidate --> SuccessValidation order:
- Load and merge all sources
- Run JSON Schema validation (if configured)
- Run custom validation functions (if configured)
- Bind to struct (if configured)
- Run struct
Validate() method (if implemented)
Comparison Table
| Validation Type | For Structs | For Maps | When to Use |
|---|
Struct-based (Validate() error) | ✅ Yes | ❌ No | Type-safe validation with Go code |
| JSON Schema | ❌ No | ✅ Yes | Standard schema validation, language-agnostic |
| Custom Function | ✅ Yes | ✅ Yes | Complex logic, cross-field validation |
Combining Validation Strategies
You can combine multiple validation approaches:
type AppConfig struct {
Server struct {
Port int `config:"port"`
Host string `config:"host"`
} `config:"server"`
}
func (c *AppConfig) Validate() error {
if c.Server.Port <= 0 {
return errors.New("server.port must be positive")
}
return nil
}
var appConfig AppConfig
schemaBytes, _ := os.ReadFile("schema.json")
cfg := config.MustNew(
config.WithFile("config.yaml"),
config.WithJSONSchema(schemaBytes), // 1. Schema validation
config.WithValidator(customValidation), // 2. Custom validation
config.WithBinding(&appConfig), // 3. Struct binding + validation
)
func customValidation(data map[string]any) error {
// Custom validation logic
return nil
}
All three validations will run in sequence.
Error Handling
Validation errors are wrapped in ConfigError with context:
if err := cfg.Load(context.Background()); err != nil {
// Error format examples:
// "config error in json-schema during validate: server.port: value must be >= 1"
// "config error in binding during validate: port must be positive"
log.Printf("Validation failed: %v", err)
}
Best Practices
1. Prefer Struct Validation
For Go applications, struct-based validation is most idiomatic:
func (c *Config) Validate() error {
// Clear, type-safe validation logic
}
2. Provide Helpful Error Messages
Include field names and expected values:
// Bad
return errors.New("invalid value")
// Good
return fmt.Errorf("server.port must be between 1-65535, got %d", c.Server.Port)
3. Validate Relationships
Check dependencies between fields:
func (c *Config) Validate() error {
if c.TLS.Enabled && c.TLS.CertFile == "" {
return errors.New("tls.cert_file required when tls.enabled is true")
}
return nil
}
4. Use JSON Schema for APIs
When exposing configuration via APIs or accepting external config:
// Validate external configuration against schema
cfg := config.MustNew(
config.WithContent(externalConfigBytes, codec.TypeJSON),
config.WithJSONSchema(schemaBytes),
)
5. Fail Fast
Validate during initialization, not at runtime:
func main() {
cfg := loadConfig() // Validates during Load()
// If we reach here, config is valid
startServer(cfg)
}
Complete Example
package main
import (
"context"
"errors"
"fmt"
"log"
"os"
"rivaas.dev/config"
)
type AppConfig struct {
Server struct {
Host string `config:"host"`
Port int `config:"port"`
TLS struct {
Enabled bool `config:"enabled"`
CertFile string `config:"cert_file"`
KeyFile string `config:"key_file"`
} `config:"tls"`
} `config:"server"`
Database struct {
Host string `config:"host"`
Port int `config:"port"`
MaxConns int `config:"max_connections"`
} `config:"database"`
}
func (c *AppConfig) Validate() error {
// Server validation
if c.Server.Port < 1 || c.Server.Port > 65535 {
return fmt.Errorf("server.port must be 1-65535, got %d", c.Server.Port)
}
// TLS validation
if c.Server.TLS.Enabled {
if c.Server.TLS.CertFile == "" {
return errors.New("server.tls.cert_file required when TLS enabled")
}
if _, err := os.Stat(c.Server.TLS.CertFile); err != nil {
return fmt.Errorf("server.tls.cert_file not found: %w", err)
}
}
// Database validation
if c.Database.Host == "" {
return errors.New("database.host is required")
}
if c.Database.MaxConns <= 0 {
return errors.New("database.max_connections must be positive")
}
return nil
}
func main() {
var appConfig AppConfig
schemaBytes, err := os.ReadFile("schema.json")
if err != nil {
log.Fatalf("failed to read schema: %v", err)
}
cfg := config.MustNew(
config.WithFile("config.yaml"),
config.WithEnv("MYAPP_"),
config.WithJSONSchema(schemaBytes),
config.WithBinding(&appConfig),
)
if err := cfg.Load(context.Background()); err != nil {
log.Fatalf("configuration validation failed: %v", err)
}
log.Println("Configuration validated successfully!")
log.Printf("Server: %s:%d", appConfig.Server.Host, appConfig.Server.Port)
}
Next Steps
For technical details on error handling, see Troubleshooting.
6 - Multiple Sources
Combine configuration from files, environment variables, and remote sources
The config package supports loading configuration from multiple sources simultaneously. This enables powerful patterns like base configuration with environment-specific overrides.
Source Precedence
When multiple sources are configured, later sources override earlier ones:
cfg := config.MustNew(
config.WithFile("config.yaml"), // Base configuration
config.WithFile("config.prod.yaml"), // Production overrides
config.WithEnv("MYAPP_"), // Environment overrides all
)
Priority: Environment variables > config.prod.yaml > config.yaml
Visualization
graph LR
File1[config.yaml] --> Merge1[Merge]
File2[config.json] --> Merge2[Merge]
Merge1 --> Merge2
Env[Environment] --> Merge3[Merge]
Merge2 --> Merge3
Consul[Consul KV] --> Merge4[Merge]
Merge3 --> Merge4
Merge4 --> Final[Final Config]Hierarchical Merging
Sources are merged hierarchically - nested structures are combined intelligently:
config.yaml (base):
server:
host: localhost
port: 8080
timeout: 30s
database:
host: localhost
port: 5432
config.prod.yaml (overrides):
server:
host: 0.0.0.0
database:
host: db.example.com
Merged result:
server:
host: 0.0.0.0 # Overridden.
port: 8080 # From base.
timeout: 30s # From base.
database:
host: db.example.com # Overridden.
port: 5432 # From base.
File Sources
Load configuration from local files:
cfg := config.MustNew(
config.WithFile("config.yaml"),
config.WithFile("config.json"),
config.WithFile("config.toml"),
)
File formats are detected automatically from extensions:
config.WithFile("app.yaml") // YAML
config.WithFile("app.yml") // YAML
config.WithFile("app.json") // JSON
config.WithFile("app.toml") // TOML
Use explicit format when the file name doesn’t have an extension:
config.WithFileAs("config.txt", codec.TypeYAML)
config.WithFileAs("data.conf", codec.TypeJSON)
Environment Variable Sources
Load configuration from environment variables:
cfg := config.MustNew(
config.WithFile("config.yaml"),
config.WithEnv("MYAPP_"), // Prefix filter
)
Environment variables override file-based configuration. See Environment Variables for detailed naming conventions.
Content Sources
Load configuration from byte slices. This is useful for testing or when you have dynamic configuration:
configData := []byte(`{
"server": {
"port": 8080,
"host": "localhost"
}
}`)
cfg := config.MustNew(
config.WithContent(configData, codec.TypeJSON),
)
Use Cases for Content Sources
Testing:
func TestConfig(t *testing.T) {
testConfig := []byte(`port: 8080`)
cfg := config.MustNew(
config.WithContent(testConfig, codec.TypeYAML),
)
cfg.Load(context.Background())
assert.Equal(t, 8080, cfg.Int("port"))
}
Dynamic Configuration:
// Configuration from HTTP response
resp, _ := http.Get("https://config-server/config.json")
configBytes, _ := io.ReadAll(resp.Body)
cfg := config.MustNew(
config.WithContent(configBytes, codec.TypeJSON),
)
Remote Sources
Consul
Load configuration from HashiCorp Consul:
cfg := config.MustNew(
config.WithConsul("production/service.yaml"),
)
**Works without Consul:** If `CONSUL_HTTP_ADDR` isn't set, `WithConsul` does nothing. This means you can run your app locally without Consul. When you deploy to production, just set the environment variable and Consul will be used.
Environment variables:
export CONSUL_HTTP_ADDR=consul.example.com:8500
export CONSUL_HTTP_TOKEN=secret-token
Loading from Consul:
cfg := config.MustNew(
config.WithFile("config.yaml"), // Local defaults
config.WithConsul("staging/myapp.json"), // Staging overrides
config.WithEnv("MYAPP_"), // Environment overrides
)
The format is detected from the key path extension (.json, .yaml, .toml).
Custom Sources
Implement custom sources for any data source:
type Source interface {
Load(ctx context.Context) (map[string]any, error)
}
Example: Database Source
type DatabaseSource struct {
db *sql.DB
}
func (s *DatabaseSource) Load(ctx context.Context) (map[string]any, error) {
rows, err := s.db.QueryContext(ctx, "SELECT key, value FROM config")
if err != nil {
return nil, err
}
defer rows.Close()
config := make(map[string]any)
for rows.Next() {
var key, value string
if err := rows.Scan(&key, &value); err != nil {
return nil, err
}
config[key] = value
}
return config, nil
}
// Usage
cfg := config.MustNew(
config.WithSource(&DatabaseSource{db: db}),
)
Example: HTTP Source
type HTTPSource struct {
url string
codec codec.Codec
}
func (s *HTTPSource) Load(ctx context.Context) (map[string]any, error) {
req, _ := http.NewRequestWithContext(ctx, "GET", s.url, nil)
resp, err := http.DefaultClient.Do(req)
if err != nil {
return nil, err
}
defer resp.Body.Close()
data, err := io.ReadAll(resp.Body)
if err != nil {
return nil, err
}
var config map[string]any
if err := s.codec.Decode(data, &config); err != nil {
return nil, err
}
return config, nil
}
// Usage
cfg := config.MustNew(
config.WithSource(&HTTPSource{
url: "https://config-server/config.json",
codec: codec.JSON{},
}),
)
Multi-Environment Pattern
There are two ways to handle environment-specific configuration.
Using Path Expansion (Recommended)
The simplest approach is to use environment variables directly in paths:
cfg := config.MustNew(
config.WithFile("config.yaml"), // Base config
config.WithFile("${APP_ENV}/config.yaml"), // Environment-specific (e.g., "production/config.yaml")
config.WithEnv("MYAPP_"), // Environment variables
)
This is cleaner and works great when your config files are in environment-named folders.
Using String Concatenation
If you need more control or want to set a default, use Go code:
package main
import (
"context"
"log"
"os"
"rivaas.dev/config"
)
func loadConfig() *config.Config {
env := os.Getenv("APP_ENV")
if env == "" {
env = "development"
}
cfg := config.MustNew(
config.WithFile("config.yaml"), // Base config
config.WithFile("config."+env+".yaml"), // Environment-specific
config.WithEnv("MYAPP_"), // Environment variables
)
if err := cfg.Load(context.Background()); err != nil {
log.Fatalf("failed to load config: %v", err)
}
return cfg
}
func main() {
cfg := loadConfig()
// Use configuration
}
File structure:
config.yaml # Base configuration
config.development.yaml
config.staging.yaml
config.production.yaml
Dumping Configuration
Save the effective merged configuration to a file:
cfg := config.MustNew(
config.WithFile("config.yaml"),
config.WithFile("config.prod.yaml"),
config.WithEnv("MYAPP_"),
config.WithFileDumper("effective-config.yaml"),
)
cfg.Load(context.Background())
cfg.Dump(context.Background()) // Writes merged config to effective-config.yaml
Use Cases for Dumping
Debugging:
See the final merged configuration:
config.WithFileDumper("debug-config.json")
Configuration Snapshots:
Save configuration state for auditing:
timestamp := time.Now().Format("20060102-150405")
filename := fmt.Sprintf("config-snapshot-%s.yaml", timestamp)
config.WithFileDumper(filename)
Configuration Templates:
Generate configuration files:
cfg := config.MustNew(
config.WithEnv("MYAPP_"),
config.WithFileDumper("generated-config.yaml"),
)
Custom File Permissions
Control file permissions when dumping:
import "rivaas.dev/config/dumper"
// Default permissions (0644)
fileDumper := dumper.NewFile("config.yaml", encoder)
// Custom permissions (0600 - owner read/write only)
fileDumper := dumper.NewFileWithPermissions("config.yaml", encoder, 0600)
cfg := config.MustNew(
config.WithFile("config.yaml"),
config.WithDumper(fileDumper),
)
Custom Dumpers
Implement custom dumpers for any destination:
type Dumper interface {
Dump(ctx context.Context, data map[string]any) error
}
Example: S3 Dumper
type S3Dumper struct {
bucket string
key string
client *s3.Client
codec codec.Codec
}
func (d *S3Dumper) Dump(ctx context.Context, data map[string]any) error {
bytes, err := d.codec.Encode(data)
if err != nil {
return err
}
_, err = d.client.PutObject(ctx, &s3.PutObjectInput{
Bucket: aws.String(d.bucket),
Key: aws.String(d.key),
Body: bytes.NewReader(bytes),
})
return err
}
// Usage
cfg := config.MustNew(
config.WithFile("config.yaml"),
config.WithDumper(&S3Dumper{
bucket: "my-configs",
key: "app-config.json",
client: s3Client,
codec: codec.JSON{},
}),
)
Error Handling
Errors from sources include context about which source failed:
if err := cfg.Load(context.Background()); err != nil {
// Error format:
// "config error in source[0] during load: file not found: config.yaml"
// "config error in source[2] during load: consul key not found"
log.Printf("Configuration error: %v", err)
}
Complete Example
package main
import (
"context"
"log"
"os"
"rivaas.dev/config"
"rivaas.dev/config/codec"
)
type AppConfig struct {
Server struct {
Host string `config:"host"`
Port int `config:"port"`
} `config:"server"`
Database struct {
Host string `config:"host"`
Port int `config:"port"`
} `config:"database"`
}
func main() {
env := os.Getenv("APP_ENV")
if env == "" {
env = "development"
}
var appConfig AppConfig
cfg := config.MustNew(
// Base configuration
config.WithFile("config.yaml"),
// Environment-specific overrides
config.WithFile("config."+env+".yaml"),
// Remote configuration (production only)
func() config.Option {
if env == "production" {
return config.WithConsul("production/myapp.json")
}
return nil
}(),
// Environment variables (highest priority)
config.WithEnv("MYAPP_"),
// Struct binding
config.WithBinding(&appConfig),
// Dump effective config for debugging
config.WithFileDumper("effective-config.yaml"),
)
if err := cfg.Load(context.Background()); err != nil {
log.Fatalf("failed to load configuration: %v", err)
}
// Save effective configuration
if err := cfg.Dump(context.Background()); err != nil {
log.Printf("warning: failed to dump config: %v", err)
}
log.Printf("Server: %s:%d", appConfig.Server.Host, appConfig.Server.Port)
log.Printf("Database: %s:%d", appConfig.Database.Host, appConfig.Database.Port)
}
Next Steps
For complete API details, see Options Reference.
7 - Custom Codecs
Extend configuration support to custom formats with codec implementation
The config package allows you to extend configuration support to any format by implementing and registering custom codecs.
Codec Interface
A codec is responsible for encoding and decoding configuration data.
type Codec interface {
Encode(v any) ([]byte, error)
Decode(data []byte, v any) error
}
Methods:
Encode(v any) ([]byte, error) - Convert Go data structures to bytes.Decode(data []byte, v any) error - Convert bytes to Go data structures.
Built-in Codecs
The config package includes several built-in codecs.
| Codec | Type | Capabilities |
|---|
| JSON | codec.TypeJSON | Encode & Decode |
| YAML | codec.TypeYAML | Encode & Decode |
| TOML | codec.TypeTOML | Encode & Decode |
| EnvVar | codec.TypeEnvVar | Decode only |
Caster Codecs
Caster codecs handle type conversion.
| Codec | Type | Converts To |
|---|
| Bool | codec.TypeCasterBool | bool |
| Int | codec.TypeCasterInt | int |
| Int8/16/32/64 | codec.TypeCasterInt8, etc. | int8, int16, etc. |
| Uint | codec.TypeCasterUint | uint |
| Uint8/16/32/64 | codec.TypeCasterUint8, etc. | uint8, uint16, etc. |
| Float32/64 | codec.TypeCasterFloat32, codec.TypeCasterFloat64 | float32, float64 |
| String | codec.TypeCasterString | string |
| Time | codec.TypeCasterTime | time.Time |
| Duration | codec.TypeCasterDuration | time.Duration |
Implementing a Custom Codec
Let’s implement a simple INI file codec.
package inicodec
import (
"bufio"
"bytes"
"fmt"
"strings"
"rivaas.dev/config/codec"
)
type INICodec struct{}
func (c INICodec) Decode(data []byte, v any) error {
result := make(map[string]any)
scanner := bufio.NewScanner(bytes.NewReader(data))
var currentSection string
for scanner.Scan() {
line := strings.TrimSpace(scanner.Text())
// Skip empty lines and comments
if line == "" || strings.HasPrefix(line, ";") || strings.HasPrefix(line, "#") {
continue
}
// Section header
if strings.HasPrefix(line, "[") && strings.HasSuffix(line, "]") {
currentSection = strings.Trim(line, "[]")
if result[currentSection] == nil {
result[currentSection] = make(map[string]any)
}
continue
}
// Key-value pair
parts := strings.SplitN(line, "=", 2)
if len(parts) != 2 {
continue
}
key := strings.TrimSpace(parts[0])
value := strings.TrimSpace(parts[1])
if currentSection != "" {
section := result[currentSection].(map[string]any)
section[key] = value
} else {
result[key] = value
}
}
// Type assertion to set result
target := v.(*map[string]any)
*target = result
return scanner.Err()
}
func (c INICodec) Encode(v any) ([]byte, error) {
data, ok := v.(map[string]any)
if !ok {
return nil, fmt.Errorf("expected map[string]any, got %T", v)
}
var buf bytes.Buffer
for section, values := range data {
sectionMap, ok := values.(map[string]any)
if !ok {
// Top-level key-value
buf.WriteString(fmt.Sprintf("%s = %v\n", section, values))
continue
}
// Section header
buf.WriteString(fmt.Sprintf("[%s]\n", section))
// Section key-values
for key, value := range sectionMap {
buf.WriteString(fmt.Sprintf("%s = %v\n", key, value))
}
buf.WriteString("\n")
}
return buf.Bytes(), nil
}
func init() {
codec.RegisterEncoder("ini", INICodec{})
codec.RegisterDecoder("ini", INICodec{})
}
Using the Custom Codec
package main
import (
"context"
"log"
"rivaas.dev/config"
_ "yourmodule/inicodec" // Register codec via init()
)
func main() {
cfg := config.MustNew(
config.WithFileAs("config.ini", "ini"),
)
if err := cfg.Load(context.Background()); err != nil {
log.Fatalf("failed to load config: %v", err)
}
host := cfg.String("server.host")
port := cfg.Int("server.port")
log.Printf("Server: %s:%d", host, port)
}
config.ini:
[server]
host = localhost
port = 8080
[database]
host = db.example.com
port = 5432
Registering Codecs
Codecs must be registered before use:
import "rivaas.dev/config/codec"
func init() {
codec.RegisterEncoder("mytype", MyCodec{})
codec.RegisterDecoder("mytype", MyCodec{})
}
Registration functions:
RegisterEncoder(name string, encoder Codec) - Register for encodingRegisterDecoder(name string, decoder Codec) - Register for decoding
You can register the same codec for both or different codecs for each operation.
Decode-Only Codecs
Some codecs only support decoding (like the built-in EnvVar codec):
type EnvVarCodec struct{}
func (c EnvVarCodec) Decode(data []byte, v any) error {
// Decode environment variable format
// ...
}
func (c EnvVarCodec) Encode(v any) ([]byte, error) {
return nil, errors.New("encoding to environment variables not supported")
}
func init() {
codec.RegisterDecoder("envvar", EnvVarCodec{})
// Note: Not registering encoder
}
Advanced Example: XML Codec
A more complete example with error handling:
package xmlcodec
import (
"encoding/xml"
"fmt"
"rivaas.dev/config/codec"
)
type XMLCodec struct{}
func (c XMLCodec) Decode(data []byte, v any) error {
target, ok := v.(*map[string]any)
if !ok {
return fmt.Errorf("expected *map[string]any, got %T", v)
}
// XML unmarshaling to intermediate structure
var intermediate struct {
XMLName xml.Name
Content []byte `xml:",innerxml"`
}
if err := xml.Unmarshal(data, &intermediate); err != nil {
return fmt.Errorf("xml decode error: %w", err)
}
// Convert XML to map structure
result := make(map[string]any)
// ... conversion logic ...
*target = result
return nil
}
func (c XMLCodec) Encode(v any) ([]byte, error) {
data, ok := v.(map[string]any)
if !ok {
return nil, fmt.Errorf("expected map[string]any, got %T", v)
}
// Convert map to XML structure
xmlData, err := xml.MarshalIndent(data, "", " ")
if err != nil {
return nil, fmt.Errorf("xml encode error: %w", err)
}
return xmlData, nil
}
func init() {
codec.RegisterEncoder("xml", XMLCodec{})
codec.RegisterDecoder("xml", XMLCodec{})
}
Caster Codecs
Caster codecs provide type conversion. You typically don’t need to implement these - use the built-in casters:
import "rivaas.dev/config/codec"
// Get int value with automatic conversion
port := cfg.Int("server.port") // Uses codec.TypeCasterInt internally
// Get duration with automatic conversion
timeout := cfg.Duration("timeout") // Uses codec.TypeCasterDuration internally
Custom Caster Example
If you need custom type conversion:
type URLCaster struct{}
func (c URLCaster) Decode(data []byte, v any) error {
target, ok := v.(*url.URL)
if !ok {
return fmt.Errorf("expected *url.URL, got %T", v)
}
parsedURL, err := url.Parse(string(data))
if err != nil {
return fmt.Errorf("invalid URL: %w", err)
}
*target = *parsedURL
return nil
}
func (c URLCaster) Encode(v any) ([]byte, error) {
u, ok := v.(*url.URL)
if !ok {
return nil, fmt.Errorf("expected *url.URL, got %T", v)
}
return []byte(u.String()), nil
}
When to Create Custom Codecs
Use Custom Codecs For:
- Unsupported formats - XML, INI, HCL, proprietary formats
- Legacy formats - Converting old configuration formats
- Encrypted configurations - Decrypting config data
- Compressed data - Handling gzip/compressed configs
- Custom protocols - Special encoding schemes
Use Built-in Codecs For:
- Standard formats - JSON, YAML, TOML
- Type conversion - Use caster codecs (Int, Bool, Duration, etc.)
- Simple text formats - Can often use JSON/YAML
Best Practices
1. Error Handling
Provide clear error messages:
func (c MyCodec) Decode(data []byte, v any) error {
if len(data) == 0 {
return errors.New("empty data")
}
target, ok := v.(*map[string]any)
if !ok {
return fmt.Errorf("expected *map[string]any, got %T", v)
}
// ... decoding logic ...
if err != nil {
return fmt.Errorf("decode error at line %d: %w", line, err)
}
return nil
}
2. Type Validation
Validate expected types:
func (c MyCodec) Decode(data []byte, v any) error {
target, ok := v.(*map[string]any)
if !ok {
return fmt.Errorf("MyCodec requires *map[string]any, got %T", v)
}
// ...
}
3. Use init() for Registration
Register codecs in init() for automatic setup:
func init() {
codec.RegisterEncoder("myformat", MyCodec{})
codec.RegisterDecoder("myformat", MyCodec{})
}
4. Thread Safety
Ensure your codec is thread-safe:
type MyCodec struct {
// No mutable state
}
// OR use proper synchronization
type MyCodec struct {
mu sync.Mutex
cache map[string]any
}
5. Document Your Codec
Include usage examples:
// MyCodec implements encoding/decoding for the XYZ format.
//
// Example usage:
//
// import _ "yourmodule/mycodec"
//
// cfg := config.MustNew(
// config.WithFileAs("config.xyz", "xyz"),
// )
//
type MyCodec struct{}
Complete Example
package main
import (
"context"
"log"
"rivaas.dev/config"
_ "yourmodule/xmlcodec" // Custom XML codec
_ "yourmodule/inicodec" // Custom INI codec
)
func main() {
cfg := config.MustNew(
config.WithFile("config.yaml"), // Built-in YAML
config.WithFileAs("config.xml", "xml"), // Custom XML
config.WithFileAs("config.ini", "ini"), // Custom INI
config.WithEnv("MYAPP_"), // Built-in EnvVar
)
if err := cfg.Load(context.Background()); err != nil {
log.Fatalf("failed to load config: %v", err)
}
port := cfg.Int("server.port")
host := cfg.String("server.host")
log.Printf("Server: %s:%d", host, port)
}
Testing Custom Codecs
Write tests for your codecs:
func TestMyCodec_Decode(t *testing.T) {
codec := MyCodec{}
input := []byte(`
[server]
host = localhost
port = 8080
`)
var result map[string]any
err := codec.Decode(input, &result)
assert.NoError(t, err)
assert.Equal(t, "localhost", result["server"].(map[string]any)["host"])
assert.Equal(t, "8080", result["server"].(map[string]any)["port"])
}
func TestMyCodec_Encode(t *testing.T) {
codec := MyCodec{}
data := map[string]any{
"server": map[string]any{
"host": "localhost",
"port": 8080,
},
}
output, err := codec.Encode(data)
assert.NoError(t, err)
assert.Contains(t, string(output), "[server]")
assert.Contains(t, string(output), "host = localhost")
}
Next Steps
Tip: If you create a useful codec, consider contributing it to the community!
8 - Examples
Real-world examples and production-ready patterns for configuration management
Learn from practical examples that demonstrate different configuration patterns and use cases.
Example Repository
All examples are available in the GitHub repository with complete, runnable code.
Example Overview
1. Basic Configuration
Path: config/examples/basic/
A simple example showing the most basic usage. Load configuration from a YAML file into a Go struct.
Features:
- File source using YAML.
- Struct binding.
- Type conversion.
- Nested structures.
- Arrays and slices.
- Time and URL types.
Best for: Getting started, understanding basic concepts
Quick start:
cd config/examples/basic
go run main.go
2. Environment Variables
Path: config/examples/environment/
Demonstrates loading configuration from environment variables, following the Twelve-Factor App methodology.
Features:
- Environment variable source
- Struct binding
- Nested configuration
- Direct access methods
- Type conversion
Best for: Containerized applications, cloud deployments, 12-factor apps
Quick start:
cd config/examples/environment
export WEBAPP_SERVER_HOST=localhost
export WEBAPP_SERVER_PORT=8080
go run main.go
3. Mixed Configuration
Path: config/examples/mixed/
Shows how to combine YAML files and environment variables. Environment variables override YAML defaults.
Features:
- Mixed configuration sources.
- Configuration precedence.
- Environment variable mapping.
- Struct binding.
- Direct access.
Best for: Applications that need both default configuration files and environment-specific overrides
Quick start:
cd config/examples/mixed
export WEBAPP_SERVER_PORT=8080 # Override YAML default
go run main.go
4. Comprehensive Example
Path: config/examples/comprehensive/
A complete example demonstrating advanced features with a realistic web application configuration.
Features:
- Mixed configuration sources
- Complex nested structures
- Validation
- Comprehensive testing
- Production-ready patterns
Best for: Production applications, learning advanced features, understanding best practices
Quick start:
cd config/examples/comprehensive
go test -v
go run main.go
Dynamic Paths with Environment Variables
You can use environment variables in file and Consul paths. This makes it easy to use different configurations based on your environment.
Basic Path Expansion
// Set APP_ENV=production in your environment
cfg := config.MustNew(
config.WithFile("config.yaml"), // Base config
config.WithFile("${APP_ENV}/config.yaml"), // Becomes "production/config.yaml"
)
Multiple Variables
You can use several variables in one path:
// Set REGION=us-west and ENV=staging
cfg := config.MustNew(
config.WithFile("${REGION}/${ENV}/app.yaml"), // Becomes "us-west/staging/app.yaml"
)
Consul Paths
This also works with Consul:
// Set APP_ENV=production
cfg := config.MustNew(
config.WithFile("config.yaml"),
config.WithConsul("${APP_ENV}/service.yaml"), // Fetches from Consul: "production/service.yaml"
)
Output Paths
You can also use variables in dumper paths:
// Set LOG_DIR=/var/log/myapp
cfg := config.MustNew(
config.WithFile("config.yaml"),
config.WithFileDumper("${LOG_DIR}/effective-config.yaml"), // Writes to /var/log/myapp/
)
**Important:** Shell-style defaults like `${VAR:-default}` are NOT supported. If a variable is not set, it expands to an empty string. Set defaults in your code before calling the config options.
Production Configuration Example
Here’s a complete production-ready configuration pattern:
package main
import (
"context"
"errors"
"fmt"
"log"
"os"
"time"
"rivaas.dev/config"
)
type AppConfig struct {
Server struct {
Host string `config:"host" default:"localhost"`
Port int `config:"port" default:"8080"`
ReadTimeout time.Duration `config:"read_timeout" default:"30s"`
WriteTimeout time.Duration `config:"write_timeout" default:"30s"`
TLS struct {
Enabled bool `config:"enabled" default:"false"`
CertFile string `config:"cert_file"`
KeyFile string `config:"key_file"`
} `config:"tls"`
} `config:"server"`
Database struct {
Primary struct {
Host string `config:"host"`
Port int `config:"port" default:"5432"`
Database string `config:"database"`
Username string `config:"username"`
Password string `config:"password"`
SSLMode string `config:"ssl_mode" default:"require"`
} `config:"primary"`
Replica struct {
Host string `config:"host"`
Port int `config:"port" default:"5432"`
Database string `config:"database"`
} `config:"replica"`
Pool struct {
MaxOpenConns int `config:"max_open_conns" default:"25"`
MaxIdleConns int `config:"max_idle_conns" default:"5"`
ConnMaxLifetime time.Duration `config:"conn_max_lifetime" default:"5m"`
} `config:"pool"`
} `config:"database"`
Redis struct {
Host string `config:"host" default:"localhost"`
Port int `config:"port" default:"6379"`
Database int `config:"database" default:"0"`
Password string `config:"password"`
Timeout time.Duration `config:"timeout" default:"5s"`
} `config:"redis"`
Auth struct {
JWTSecret string `config:"jwt_secret"`
TokenDuration time.Duration `config:"token_duration" default:"24h"`
} `config:"auth"`
Logging struct {
Level string `config:"level" default:"info"`
Format string `config:"format" default:"json"`
Output string `config:"output" default:"/var/log/app.log"`
} `config:"logging"`
Monitoring struct {
Enabled bool `config:"enabled" default:"true"`
MetricsPort int `config:"metrics_port" default:"9090"`
HealthPath string `config:"health_path" default:"/health"`
} `config:"monitoring"`
Features struct {
RateLimit bool `config:"rate_limit" default:"true"`
Cache bool `config:"cache" default:"true"`
DebugMode bool `config:"debug_mode" default:"false"`
} `config:"features"`
}
func (c *AppConfig) Validate() error {
// Server validation
if c.Server.Port < 1 || c.Server.Port > 65535 {
return fmt.Errorf("server.port must be 1-65535, got %d", c.Server.Port)
}
// TLS validation
if c.Server.TLS.Enabled {
if c.Server.TLS.CertFile == "" {
return errors.New("server.tls.cert_file required when TLS enabled")
}
if c.Server.TLS.KeyFile == "" {
return errors.New("server.tls.key_file required when TLS enabled")
}
}
// Database validation
if c.Database.Primary.Host == "" {
return errors.New("database.primary.host is required")
}
if c.Database.Primary.Database == "" {
return errors.New("database.primary.database is required")
}
if c.Database.Primary.Username == "" {
return errors.New("database.primary.username is required")
}
if c.Database.Primary.Password == "" {
return errors.New("database.primary.password is required")
}
// Auth validation
if c.Auth.JWTSecret == "" {
return errors.New("auth.jwt_secret is required")
}
if len(c.Auth.JWTSecret) < 32 {
return errors.New("auth.jwt_secret must be at least 32 characters")
}
return nil
}
func loadConfig() (*AppConfig, error) {
var appConfig AppConfig
// Determine environment
env := os.Getenv("APP_ENV")
if env == "" {
env = "development"
}
cfg := config.MustNew(
// Base configuration
config.WithFile("config.yaml"),
// Environment-specific configuration
config.WithFile(fmt.Sprintf("config.%s.yaml", env)),
// Environment variables (highest priority)
config.WithEnv("MYAPP_"),
// Struct binding with validation
config.WithBinding(&appConfig),
)
if err := cfg.Load(context.Background()); err != nil {
return nil, fmt.Errorf("failed to load configuration: %w", err)
}
return &appConfig, nil
}
func main() {
appConfig, err := loadConfig()
if err != nil {
log.Fatalf("Configuration error: %v", err)
}
log.Printf("Server: %s:%d", appConfig.Server.Host, appConfig.Server.Port)
log.Printf("Database: %s:%d/%s",
appConfig.Database.Primary.Host,
appConfig.Database.Primary.Port,
appConfig.Database.Primary.Database)
log.Printf("Redis: %s:%d", appConfig.Redis.Host, appConfig.Redis.Port)
log.Printf("Features: RateLimit=%v, Cache=%v, Debug=%v",
appConfig.Features.RateLimit,
appConfig.Features.Cache,
appConfig.Features.DebugMode)
}
Multi-Environment Setup
Organize configuration for different environments:
File structure:
config/
├── config.yaml # Base configuration (shared defaults)
├── config.development.yaml # Development overrides
├── config.staging.yaml # Staging overrides
├── config.production.yaml # Production overrides
└── config.test.yaml # Test overrides
config.yaml (base):
server:
host: localhost
port: 8080
read_timeout: 30s
write_timeout: 30s
database:
pool:
max_open_conns: 25
max_idle_conns: 5
conn_max_lifetime: 5m
logging:
level: info
format: json
config.production.yaml:
server:
host: 0.0.0.0
port: 443
tls:
enabled: true
cert_file: /etc/ssl/certs/server.crt
key_file: /etc/ssl/private/server.key
database:
primary:
host: db.prod.example.com
ssl_mode: require
replica:
host: db-replica.prod.example.com
logging:
level: warn
output: /var/log/production/app.log
features:
debug_mode: false
config.development.yaml:
server:
host: localhost
port: 3000
database:
primary:
host: localhost
ssl_mode: disable
logging:
level: debug
format: text
output: stdout
features:
debug_mode: true
Integration with Rivaas App
Integrate configuration with the rivaas/app framework:
package main
import (
"context"
"log"
"os"
"os/signal"
"syscall"
"rivaas.dev/app"
"rivaas.dev/config"
)
type AppConfig struct {
Server struct {
Host string `config:"host" default:"localhost"`
Port int `config:"port" default:"8080"`
} `config:"server"`
}
func main() {
var appConfig AppConfig
cfg := config.MustNew(
config.WithFile("config.yaml"),
config.WithEnv("MYAPP_"),
config.WithBinding(&appConfig),
)
if err := cfg.Load(context.Background()); err != nil {
log.Fatalf("failed to load config: %v", err)
}
// Create rivaas/app with configuration from config
a := app.MustNew(
app.WithServiceName("myapp"),
app.WithServiceVersion("v1.0.0"),
app.WithHost(appConfig.Server.Host),
app.WithPort(appConfig.Server.Port),
)
// Define routes
a.GET("/", func(c *app.Context) {
c.JSON(200, map[string]string{"status": "ok"})
})
// Setup graceful shutdown
ctx, cancel := signal.NotifyContext(context.Background(), os.Interrupt, syscall.SIGTERM)
defer cancel()
if err := a.Start(ctx); err != nil {
log.Fatalf("server error: %v", err)
}
}
Testing Configuration
Example test patterns:
package main
import (
"context"
"testing"
"rivaas.dev/config"
"rivaas.dev/config/codec"
)
func TestConfigLoading(t *testing.T) {
testConfig := []byte(`
server:
host: localhost
port: 8080
database:
primary:
host: localhost
database: testdb
username: test
password: test123
`)
var appConfig AppConfig
cfg := config.MustNew(
config.WithContent(testConfig, codec.TypeYAML),
config.WithBinding(&appConfig),
)
if err := cfg.Load(context.Background()); err != nil {
t.Fatalf("failed to load config: %v", err)
}
// Assertions
if appConfig.Server.Host != "localhost" {
t.Errorf("expected localhost, got %s", appConfig.Server.Host)
}
if appConfig.Server.Port != 8080 {
t.Errorf("expected 8080, got %d", appConfig.Server.Port)
}
}
func TestConfigValidation(t *testing.T) {
invalidConfig := []byte(`
server:
host: localhost
port: 99999 # Invalid port
`)
var appConfig AppConfig
cfg := config.MustNew(
config.WithContent(invalidConfig, codec.TypeYAML),
config.WithBinding(&appConfig),
)
err := cfg.Load(context.Background())
if err == nil {
t.Error("expected validation error, got nil")
}
}
Common Patterns
Pattern 1: Secrets from Environment
Keep secrets out of config files:
# config.yaml - No secrets
database:
primary:
host: localhost
port: 5432
database: myapp
# username and password from environment
# Environment variables for secrets
export MYAPP_DATABASE_PRIMARY_USERNAME=admin
export MYAPP_DATABASE_PRIMARY_PASSWORD=secret123
Pattern 2: Feature Flags
Use configuration for feature flags:
type Config struct {
Features struct {
NewUI bool `config:"new_ui" default:"false"`
BetaFeatures bool `config:"beta_features" default:"false"`
Analytics bool `config:"analytics" default:"true"`
} `config:"features"`
}
// In application code
if appConfig.Features.NewUI {
// Use new UI
} else {
// Use old UI
}
Pattern 3: Dynamic Reloading
For applications that need dynamic configuration updates (advanced):
type ConfigManager struct {
cfg *config.Config
appCfg *AppConfig
mu sync.RWMutex
}
func (cm *ConfigManager) Reload(ctx context.Context) error {
cm.mu.Lock()
defer cm.mu.Unlock()
return cm.cfg.Load(ctx)
}
func (cm *ConfigManager) Get() *AppConfig {
cm.mu.RLock()
defer cm.mu.RUnlock()
return cm.appCfg
}
Next Steps
For questions or contributions, visit the GitHub repository.