bootic/bootic_stats_aggregates

Subscribes to ZMQ event stream, tracks aggregates into Redis and exposes simple JSON API over HTTP

Bootic pageviews aggregator / API

• Subscribe to pageview and order events on Bootic's ZMQ events bus.
• Aggregate said events into incremental Redis hashes for different periods (hour, day, month, year)

Example

This script will take the current date and use it to increment redis hash counters keyed by the period name, so if your account is "acme" and today is September 5 2013, the following data structures are populated and incremented in Redis :

Current year

"acme/pageview/2013": {
  "08": 100,
  "09": 101 // <= this key gets incremented in September
}

Current month

"acme/pageview/2013/09": {
  "01": 10,
  "02": 5,
  "03": 3,
  "04": 23,
  "05": 101 // <= this key gets incremented today
}

... And so on with current day and current hour.

Stats API

This script also exposes a JSON HTTP API to query pageviews per period and account. Examples:

• /api/stats/track/all/pageview/2013 # yearly pageviews for all accounts
• /api/stats/track/acme/pageview/2013 # yearly pageviews for "acme" account
• /api/stats/track/acme/pageview/2013/02 # February 2013 pageviews for "acme" account
• /api/stats/track/acme/pageview/2013/02/02 # 2 February 2013 pageviews for "acme" account
• /api/stats/track/acme/pageview/2013/02/16 # 2 February 2013, 16hrs. pageviews for "acme" account

The JSON response for each period includes counters for the period's segments. So for example a month's segments are the month's days:

{
  "event": "pageview",
  "year": 2013,
  "month": 2,
  "day": null,
  "hour": null,
  "data": {
    1: 1489,
    2: 50148,
    3: 1250,
    4: 233,
    5: 108,
    6: 56,
    7: 39,
    8: 118,
    9: 51,
    10: 33,
    11: 25,
    12: 25,
    13: 22,
    14: 23,
    15: 10,
    16: 21,
    17: 12,
    18: 8,
    19: 10,
    20: 11,
    21: 6,
    22: 8,
    23: 10,
    24: 1,
    25: 1,
    26: 12,
    27: 2,
    28: 1,
    29: 1,
    30: 5,
    31: 7
  
}

Run

go run main.go --zmsocket=localhost:6666 --redishost=localhost:6379 --httphost=api.stats.com:80