/files-fu

A demo project about metrics collection and agregation with Akka Streams and InfluxDB

Primary LanguageScala

CPU metrcis collecition and analysis demo

Future of the Files-Fu - a revolutionary file sharing service - is in danger, as their client app seems to consume too much CPU, making the service unusable...

This is a proof-of-concept demo, addressing some aspects of the home assignment for the interview with Hive Streaming

Implementation overview

See Implementation Notes for techical discussion, motivations etc. This README focuses further on now to build and run the demo.

The demo

0. Prerequisites

  • JDK and SBT available on $PATHto build and run the service
  • Running InfluxDB instance to store the data and render the reports. This was tested with free tier of the Cloud service, but local install should work as well
    • an API token with full access rights (unsafe, but we assume you have separate organization in Influx for this demo, or at least don't have anything important in there)
  • InfluxDB CLI available on $PATHto initialize DB and run the query examples

1. Configuration

  • create src/main/resources/influx-auth.properties with 
    user="<your organization>"
password="<your API token>"
    this will be used by scala code in integration tests and simulation
  • run bin/init-config to initialize your ~/.influxdbv2/configs for the CLI

2. Building and running

  • Initialize the bucket in InfluxDB with bin/reset-data
  • Use the following SBT commands:
    • test : to build and unit test as usual
    • it:testOnly filesfu.Acceptance : small acceptance test to make sure data really ends up in influx
    • it:testOnly filesfu.Simulation : a simulation, feeding random data to illustrate particular scenario. After this is done, reports in the next section should show something sensible
      • the source can be modified to alter the scenario
    • run : to simply run the server to play with it using curl or similar tool
      • it currently only supports POST at the path /sessions and accepts JSON messages - one per line. The message structure is defined in Messages.scala
  • To empty the database and start over, use bin/reset-data any time

3. Report example

To illustrate how the report could look like we created some histograms for the InfluxDB web console. To recreate it, navigate to Dashboards->Create dashboard->Import dashboard. Import example-dashboard.json. With the data from the simulation above it would look like:

image