let4be/crusty

Investigate how we track errors and what is considered an error in grafana dashboard

Closed this issue · 2 comments

let4be commented

Right now it does not make any sense whatsoever...
blocked by

let4be/crusty-core#9
let4be/crusty-core#8

let4be commented

After blocking issues are resolved I wanna track the following:

  • FilteredOutByTerm = Redirect, task was considered a redirect and a new task with updated URL was emitted
  • FilteredOutByTerm = MaxRedirect, task was considered a redirect but we went over max number of allowed redirects
  • FilteredOutByTerm = ContentType, we do not want to deal with this content-type
  • FilteredOutByTerm = ..., every other TERM reason(right now - none)
  • FilteredOutByError - one of the filters returned an unhandled Error which we handled in crusty-core
  • FilteredOutByPanic - one of the filters paniced and we caught it in crusty-core
  • Error=LoadTimeout - timeout while waiting for http response
  • Error - there was some kind of actual error while trying to Follow/HeadFollow, but we don't know what's exactly went wrong(check logs?)
  • Ok - we reached Follow/HeadFollow state

all of this is happening on the receiving end of the job_state_update channel in Crusty but we need properly structured data to be provided by crusty-core

it's also gonna be stored completely denormalized in clickhouse, but we better avoid NULL values due to poor performance...
previous calculations on some metrics turned out to be incorrect because we did not store the data right