JordiPolo/lorekeeper

Very fast logging library in JSON format

Lorekeeper

LoreKeeper contains a highly optimized JSON logger. It outputs messages as JSON and let the users to add their own customized fields. When used without extra fields it outputs 20% faster than the standard Logger for messages not longer than one line of text.

Installation

Add this line to your application's Gemfile:

gem 'lorekeeper', '~> 2.0'

And then execute:

bundle

Configuration

Configuration is set through environment variables.

• LOREKEEPER_DENYLIST
  • A comma separated list of keywords/phrases that will be excluded from the clean backtrace.
  • default: newrelic_rpm, active_support/callbacks.rb, zipkin-tracer, puma, phusion_passenger, opentelemetry

Usage

Normal logging methods

LoreKeeper::JSONLogger API is compatible with the stdlib's Logger's.

logger.error("This is a message")

Will output:

{
  "message": "This is a message",
  "timestamp": "1970-01-01T00:00:00.000+0100",
  "level": "debug"
}

Timestamps use ISO8601. Messages are JSON escaped so the total result is JSON parseable.

Log Levels

Method Name	JSON Property
debug	"level": "debug"
info	"level": "info"
warn	"level": "warning"
error	"level": "error"
fatal	"level": "fatal"

Adding keys to the output

Keys can be added to the output at any moment. These keys will be output in each message till they are removed again. If you want to output keys in only one message use the *_with_data method instead.

Keys can be added using the add_fields method which accepts a hash:

logger.add_fields("role" => "backend")
logger.error("This is a message")
logger.warn("This is another message")

Will output:

{
  "message": "This is a message",
  "timestamp": "1970-01-01T00:00:00.000+0100",
  "level": "error",
  "role": "backend"
}
{
  "message": "This is another message",
  "timestamp": "1970-01-01T00:00:00.000+0100",
  "level": "warning",
  "role": "backend"
}

Because of speed purposes the JSON dumping is done as simply as possible. If you provide a hash of keys like:

{ key: 'value' }

The output will include:

{ ":key": "value" }

Logging methods with data

All methods (info, debug, etc.) have a *_with_data equivalent: info_with_data, debug_with_data, etc. These methods accept and extra hash to add it to the JSON.

logger.error_with_data('message', { data1: 'Extra data', data2: 'Extra data2' })

Will output:

{
  "message": "This is a message",
  "timestamp": "1970-01-01T00:00:00.000+0100",
  "level": "debug",
  "data": {
    "data1": "Extra data",
    "data2": "Extra data2"
  }
}

Logging exceptions

There is a method to help you log exceptions in an standard way.

rescue => e
  logger.exception(e)
end

Will output:

{
  "message": "#{e.message}",
  "timestamp": "1970-01-01T00:00:00.000+0100",
  "level": "error",
  "exception": "<exception name>",
  "stack": [
    "<stacktraceline1>",
    "<stacktraceline2>"
  ]
}

This method also accepts a custom message, data and log level:

rescue => e
  logger.exception(e, "custom msg!", { some: { data: 123 } }, :warn)
end

Will output:

{
  "message": "custom msg!",
  "timestamp": "1970-01-01T00:00:00.000+0100",
  "level": "warning",
  "data": {
    ":some": {
      ":data": 123
    }
  },
  "exception": "<exception name>",
  "stack": [
    "<stacktraceline1>",
    "<stacktraceline2>"
  ]
}

Please note that due to the way Ruby 2.x automatically converts Hash objects to keyword arguments when they come last, you need to explicitly use the data keyword argument when you don't pass any other argument afterwards:

logger.exception(e, "custom msg!", { some: { data: 123 } })
# => ArgumentError: unknown keyword: some

logger.exception(e, "custom msg!", data: { some: { data: 123 } })
# => works

Ruby 3.x is unaffected by this issue since the conversion is not done automatically anymore.

The available keyword arguments are message, data and level. They can be used instead of the fixed arguments:

rescue => e
  logger.exception(e, message: "custom msg!", data: { some: { data: 123 } }, level: :warn)
end

This is especially useful when there is no custom message or data:

rescue => e
  logger.exception(e, level: :warn)
end

Backtrace Cleaner

The backtrace cleaner can be used independently:

Lorekeeper::BacktraceCleaner.instance.clean(backtrace)

License

The gem is available as open source under the terms of the MIT License.