Expanded Monitoring article: Logging, errors, and alerts

Question

Expanded Monitoring article: Logging, errors, and alerts

lorensr opened this issue 9 years ago · 11 comments

Currently the Guide shows the recent-history console.logs in Galaxy, and doesn't say much about errors.

What are the best practices for:

Logs

Recommended (or just an example) modern logging system, that allows for JSON logs, archiving, and levels, for example Winston (with for example Loggly). Related: #44 (comment)

Errors

logging errors: either with Winston or Kadira
environment: different behavior in dev vs production, eg to console in dev and in prod, client silenced and server to external system
sanitization: in production, while some errors are expected and handled by client code, the unexpected ones that a pentester would trigger you want to convert to some generic "server error" to not give away information
categorization: having different types of errors, some of which get sanitized, some of which get logged as normal or hidden (eg resume token expiration), some get logged as errors, for you to analyze later, and some to alert you (eg send email or pagerduty)
contextual info: eg catch all errors thrown from methods (rethrowing some, depending on type) and include in the JSON log what the method arguments were and the current userId

OKGrow and I use error name / message regex filtering with Kadira, like the below, but it's pretty basic:

It seems like some of this would fit well in a library. The only related packages I'm seeing on atmosphere are for client display of errors.

Answer 1 · 2016-04-13T23:11:15.000Z

I wonder if this topic is best covered in the more community-source cookbook like content we are looking to have (soon) on Stack Overflow

Answer 2 · 2016-04-13T23:11:38.000Z

Or maybe the dev tools article? #312

Answer 3 · 2016-04-13T23:14:09.000Z

It's not exactly dev tools though; more like something that would go under the "production" heading I think.

Answer 4 · 2016-04-13T23:17:46.000Z

Honestly we should probably split "deployment + monitoring" up and put this in the monitoring article.

Answer 5 · 2016-04-13T23:18:33.000Z

@lorensr: I went ahead and renamed the issue, I hope you don't mind.

Answer 6 · 2016-04-13T23:18:47.000Z

PS I suspect others know more about this than we do, please help!

Answer 7 · 2016-04-13T23:50:54.000Z

This would be a great place to move the Kadira content as well, pinging @arunoda

Answer 8 · 2016-04-13T23:52:39.000Z

@stubailo Thanks. I think that suites as well.

Answer 9 · 2016-04-14T08:27:16.000Z

We have an integration with loggly, universal logging. We had to monkey patch some stuff. If this interesting to ya'll, we should talk and get it into core/ or deliverable way

Answer 10 · 2016-04-17T19:12:09.000Z

Hi Abhi, I'd be interested in hearing what you had to monkey patch / seeing
code snippets. Could go into the recommended logging package.

On Thu, Apr 14, 2016 at 4:27 AM, Abhi Aiyer notifications@github.com
wrote:

We have an integration with loggly, universal logging. We had to monkey
patch some stuff. If this interesting to ya'll, we should talk and get it
into core/ or deliverable way

—
You are receiving this because you were mentioned.
Reply to this email directly or view it on GitHub
#376 (comment)

Answer 11 · 2021-02-03T13:48:56.000Z

I'm closing this issue because it's too old.

We are going to make many updates to the guide in the next weeks and it's better to focus on issues with recent activity. If you think this issue is still relevant please open a new one.