/go-metrics

A scalable metrics collection system for distributed environments, built in Go.

Primary LanguageGo

go-metrics

My pet project to learn Golang with Yandex Practicum Advanced Go Developer course.

Overview

go-metrics is a robust, scalable metrics collection system designed for distributed environments. It consists of two main components: an agent that collects and sends metrics in batches, and a server that receives, processes, and exposes these metrics through a RESTful API.

Server-Agent

Key Features

  • Efficient Metric Collection: Agents collect system and application metrics with minimal overhead.
  • Batch Processing: Metrics are sent in batches to optimize network usage and server processing.
  • Scalable Architecture: Designed to handle multiple agents sending data to a centralized server.
  • RESTful API: Server exposes collected metrics through a REST API.

Components

Agent

  • Lightweight process that runs on target systems
  • Collects various system metrics (CPU, memory, disk usage, network stats, etc.)
  • Aggregates metrics and sends them in configurable batches (with assymetric encryption support)
  • Implements retry logic

Server

  • Centralized metrics receiver and processor
  • Efficiently handles incoming batch metrics from multiple agents
  • Can store metrics in memory (with filesystem synchronization) or in PostgreSQL
  • Provides a RESTful API for querying and analyzing metrics

REST API Endpoints

  • GET / - list all metrics as an HTML page
  • POST /update { "id": "cpu", "type": "gauge", "value": 23.46 } - update one metric
  • POST /updates - update the batch of metrics
  • POST /value { "id": "cpu", "type": "gauge" } - get one metric
  • POST /update/gauge/cpu/23.46 - update one metric
  • GET /value/gauge/cpu - read metric value
  • GET /ping - check database status (if started in DB mode)

Tech Stack

TBD

Development

To get updates from template

Add git remote:

git remote add -m main template https://github.com/Yandex-Practicum/go-musthave-metrics-tpl.git

Update autotests:

git fetch template && git checkout template/main .github

Requirements to run autotests

  • Branch name should follow pattern iter<number>, where <number> — number of increment. E.g., branch iter4 will trigger autotests for increments 1-4.