Zig HTTP Sentinel

A robust, thread-safe HTTP client library for Zig 0.16.0, battle-tested in production environments.

Note: This library was extracted from production high-frequency trading systems where reliability and performance are critical.

Developed by QUANTUM ENCODING LTD
Contact: rich@quantumencoding.io

Features

Production-Grade API: Enterprise-level interface for all HTTP operations
Thread-Safe: Designed for concurrent use (each thread should use its own client instance)
Memory-Safe: Automatic memory management with proper cleanup
Full HTTP Support: GET, POST, PUT, PATCH, DELETE, HEAD methods
Configurable: Customizable request options including timeouts and body size limits
Production-Ready: Extensively tested under high-load conditions

Optional Advanced Modules

Retry Engine: Battle-tested resilience patterns including exponential backoff, circuit breakers, and adaptive retry strategies
Connection Pool: Enterprise-grade connection pooling with health monitoring, load balancing, and multi-host support

Installation

Add this library to your build.zig.zon:

.dependencies = .{
    .http_sentinel = .{
        .url = "https://github.com/YOUR_USERNAME/zig-http-sentinel/archive/refs/tags/v1.0.0.tar.gz",
        .hash = "YOUR_HASH_HERE",
    },
},

Then in your build.zig:

const http_sentinel = b.dependency("http_sentinel", .{
    .target = target,
    .optimize = optimize,
});
exe.root_module.addImport("http-sentinel", http_sentinel.module("http-sentinel"));

Production Deployment

const std = @import("std");
const HttpClient = @import("http-sentinel").HttpClient;

pub fn main() !void {
    var gpa = std.heap.GeneralPurposeAllocator(.{}){};
    defer _ = gpa.deinit();
    const allocator = gpa.allocator();

    // Create client
    var client = HttpClient.init(allocator);
    defer client.deinit();

    // Make a GET request
    const headers = [_]std.http.Header{
        .{ .name = "Accept", .value = "application/json" },
    };
    
    var response = try client.get("https://api.example.com/data", &headers);
    defer response.deinit();

    std.debug.print("Status: {}\n", .{response.status});
    std.debug.print("Body: {s}\n", .{response.body});
}

API Reference

Enterprise Client Initialization

var client = HttpClient.init(allocator);
defer client.deinit();

HTTP Methods

All methods return a Response struct that must be deinitialized:

pub const Response = struct {
    status: http.Status,
    body: []u8,
    allocator: std.mem.Allocator,
    
    pub fn deinit(self: *Response) void;
};

GET Request

var response = try client.get(url, headers);
defer response.deinit();

POST Request

var response = try client.post(url, headers, body);
defer response.deinit();

PUT Request

var response = try client.put(url, headers, body);
defer response.deinit();

PATCH Request

var response = try client.patch(url, headers, body);
defer response.deinit();

DELETE Request

var response = try client.delete(url, headers);
defer response.deinit();

HEAD Request

var response = try client.head(url, headers);
defer response.deinit();

Advanced Options

For more control, use the WithOptions variants:

const options = HttpClient.RequestOptions{
    .max_body_size = 50 * 1024 * 1024, // 50MB
    .timeout_ns = 30 * std.time.ns_per_s, // 30 seconds
};

var response = try client.getWithOptions(url, headers, options);
defer response.deinit();

Advanced Optional Modules

Retry Engine

The retry module provides enterprise resilience patterns extracted from production HFT systems:

const RetryEngine = @import("http-sentinel/retry");

// Configure retry strategy
const config = RetryEngine.RetryConfig{
    .max_attempts = 5,
    .initial_delay_ms = 100,
    .max_delay_ms = 10000,
    .backoff_multiplier = 2.0,
    .jitter_factor = 0.1,
};

var engine = RetryEngine.init(allocator, config);
defer engine.deinit();

// Use with HTTP requests
var attempt: u32 = 0;
while (attempt < config.max_attempts) : (attempt += 1) {
    var response = client.get(url, headers) catch |err| {
        const delay = engine.calculateDelay(attempt);
        std.time.sleep(delay * std.time.ns_per_ms);
        continue;
    };
    defer response.deinit();
    break; // Success
}

Patterns Included:

Exponential backoff with jitter
Circuit breaker pattern
Adaptive retry based on health scores
Configurable retry policies

Connection Pool

Enterprise-grade connection pooling for high-throughput scenarios:

const ConnectionPool = @import("http-sentinel/pool");

// Configure pool
const config = ConnectionPool.PoolConfig{
    .max_connections = 20,
    .max_idle_connections = 10,
    .connection_timeout_ms = 5000,
    .idle_timeout_ms = 30000,
    .max_connection_lifetime_ms = 300000,
};

var pool = try ConnectionPool.init(allocator, config);
defer pool.deinit();

// Acquire and use connections
const conn = try pool.acquire("api.example.com", 443);
defer pool.release(conn);

// Use connection for HTTP operations
var response = try conn.request(.GET, "/data", headers);
defer response.deinit();

Features:

Multi-host connection management
Connection health monitoring
Load balancing across backends
Automatic connection lifecycle management
Per-host statistics and metrics

Run the examples to see these patterns in action:

zig build retry-demo     # Demonstrates retry patterns
zig build pool-demo      # Shows connection pooling

Thread Safety & Concurrency

CRITICAL: HTTP Sentinel uses the client-per-worker pattern for concurrent operations. This is the only reliable way to do concurrent HTTP requests in Zig 0.16.0.

✅ Correct Pattern (Client-Per-Worker)

fn workerThread(allocator: std.mem.Allocator) void {
    // Each thread creates its own client
    var client = HttpClient.init(allocator);
    defer client.deinit();
    
    // Use the client safely for this worker's requests
}

❌ Wrong Pattern (Will Segfault)

// DON'T DO THIS - Shared client will segfault under load
var shared_client = HttpClient.init(allocator);
var mutex = std.Thread.Mutex{};

fn workerThread(client: *HttpClient, mutex: *std.Thread.Mutex) void {
    mutex.lock();
    defer mutex.unlock();
    // This WILL segfault even with mutex protection!
    const response = client.get(...);
}

Why: Zig 0.16.0's http.Client has internal state that is not thread-safe. The client-per-worker pattern avoids all concurrency issues.

See CONCURRENCY_PATTERN.md for detailed explanation and benchmarks.

Examples

See the examples/ directory for complete working examples including:

Basic GET/POST requests
JSON API interactions
Concurrent request handling
Error handling patterns

Run examples:

zig build examples

Example: High-Performance AI Client

Demonstrating zig-http-sentinel's enterprise capabilities with Anthropic's Claude API - proving universal applicability beyond financial systems.

Setup

Get your Anthropic API key from https://console.anthropic.com/

Set your environment variable:

export ANTHROPIC_API_KEY=your_api_key_here

Run the AI Client Demo

cd examples
zig run anthropic_client.zig --deps http-sentinel

Features Demonstrated

Production JSON Construction: Enterprise-grade payload building with proper escaping
Professional Header Management: Complete API authentication and versioning
Multi-Turn Conversations: Stateful conversation handling
Robust Error Handling: Comprehensive API error processing
Memory Safety: Proper allocation and cleanup patterns
Performance Metrics: Token usage tracking and optimization

Sample Output

=== Zig HTTP Sentinel: High-Performance AI Client ===

🚀 Initializing high-performance AI client...
📡 Using zig-http-sentinel for enterprise-grade HTTP operations

📝 Demo 1: Production Message Processing
🤖 Claude (claude-3-haiku-20240307):
Zig excels for HTTP clients through zero-cost abstractions, compile-time safety, 
manual memory management, and cross-platform compatibility. Its performance matches 
C while preventing common networking bugs through strong typing.
📊 Tokens: 23 in, 50 out

💬 Demo 2: Multi-Turn Conversation
🤖 Claude (claude-3-haiku-20240307):
zig-http-sentinel achieves these through Zig's allocator patterns for memory efficiency,
built-in thread safety, comprehensive error types, connection pooling architecture,
and clean generic interfaces that maintain zero-cost abstractions.
📊 Tokens: 67 in, 89 out

⚡ Demo 3: Technical Analysis
🤖 Claude (claude-3-haiku-20240307):
This pattern ensures deterministic cleanup, prevents memory leaks through RAII-style
resource management, enables zero-copy optimizations, and maintains explicit control
over allocation strategies—critical for high-frequency, low-latency systems.
📊 Tokens: 124 in, 156 out

✅ All demonstrations completed successfully!
💎 zig-http-sentinel: Enterprise-grade HTTP client for production AI systems

Integration in Your Projects

const AnthropicClient = @import("your_ai_module.zig").AnthropicClient;

var ai_client = AnthropicClient.init(allocator, api_key);
defer ai_client.deinit();

var response = try ai_client.sendMessage(
    "claude-3-sonnet-20240229",
    "Analyze this trading algorithm...",
    1000,
);
defer response.deinit();

// Use response.content for your application logic

This example proves zig-http-sentinel's versatility across industries—from algorithmic trading to AI applications, delivering consistent enterprise-grade performance.

Example: Enterprise NATS JetStream Bridge

Demonstrating production messaging infrastructure integration with NATS JetStream via HTTP gateway - proving enterprise messaging capabilities.

Features

V-Omega Protocol Compliance: Full integration with canonical V-Omega message patterns
JetStream Management: Stream and consumer lifecycle operations
High-Performance Publishing: Enterprise-grade message publishing with acknowledgments
Telemetry Integration: Compatible with existing NATS infrastructure (172.191.60.219:4222)
Multi-Domain Support: AI, HPC, and Quantum domain message routing

Run the NATS Demo

zig build nats-demo

V-Omega Message Pattern

// Canonical V-Omega message structure
vomega.{theater}.{domain}.{application}.{action}

// Examples:
vomega.azure.ai.hydra-chimera.telemetry.batch_complete
vomega.gcp.hpc.nuclear-fire-hose.telemetry.pps_report
vomega.aws.quantum.jetstream.telemetry.throughput

Integration Examples

const NatsJetStreamClient = @import("nats_bridge.zig").NatsJetStreamClient;

var client = NatsJetStreamClient.init(allocator, "172.191.60.219", 4222, "azure");
defer client.deinit();

// Publish AI telemetry
const ai_payload = std.json.Value{ .object = payload_map };
var response = try client.publishVOmegaMessage(
    "ai", "hydra-chimera", "telemetry.batch_complete", ai_payload
);
defer response.deinit();

// Create enterprise stream
const config = StreamConfig{ .max_msgs = 1000000, .storage = "file" };
var stream = try client.createVOmegaStream("quantum", "jetstream", config);
defer stream.deinit();

// Pull message batches
var batch = try client.pullMessages("VOMEGA_AZURE_AI", "processor", 100, 5000);
defer batch.deinit();

Supported Operations

Stream Management: Create, configure, and monitor JetStream streams
Consumer Operations: Durable consumers with acknowledgment policies
Message Publishing: V-Omega compliant message publishing with sequence tracking
Batch Processing: High-throughput message pulling and processing
Monitoring: Stream statistics and consumer health metrics

This integration demonstrates zig-http-sentinel's capability to bridge HTTP and enterprise messaging systems, enabling hybrid architectures with consistent performance patterns.

Testing

Run the test suite:

zig build test

Requirements

Zig 0.16.0 or later
No external dependencies

Performance

This library has been optimized for high-throughput scenarios and has been tested under production loads handling thousands of requests per second.

Contributing

Contributions are welcome! Please ensure:

All tests pass
Code follows Zig style conventions
New features include tests
Documentation is updated

License

MIT License - See LICENSE file for details

Acknowledgments

This library emerged from real-world production needs and represents lessons learned from building high-performance trading systems at QUANTUM ENCODING LTD.

quantum-encoding/zig-http-concurrent

Zig HTTP Sentinel

Features

Optional Advanced Modules

Installation

Production Deployment

API Reference

Enterprise Client Initialization

HTTP Methods

GET Request

POST Request

PUT Request

PATCH Request

DELETE Request

HEAD Request

Advanced Options

Advanced Optional Modules

Retry Engine

Connection Pool

Thread Safety & Concurrency

✅ Correct Pattern (Client-Per-Worker)

❌ Wrong Pattern (Will Segfault)

Examples

Example: High-Performance AI Client

Setup

Run the AI Client Demo

Features Demonstrated

Sample Output

Integration in Your Projects

Example: Enterprise NATS JetStream Bridge

Features

Run the NATS Demo

V-Omega Message Pattern

Integration Examples

Supported Operations

Testing

Requirements

Performance

Contributing

License

Acknowledgments