Write performance benchmark
MaxHalford opened this issue · 7 comments
This will allow to measure the impact of changes we make.
I would like to see from the /api/metrics endpoint a few more things. I am not, however, willing to get this information if it means any slowdown in the predict method.
Metrics I want:
Average Requests (count of all API calls)
Average Predictions (count of just predictions)
Average Predict Time (microsecond accuracy please)
I would be OK with having them simply be exponential moving averages so that you don't have to store the data. If that is the case please make them both short (10 periods) and long (at least 100k).
FYI these will be monitored by our health checks and also integrated into NetData dashboards.
This could also be it's own method as to not confuse with the "metric" class.
/api/stats
I agree, it would be nice to have some stats on the usage of the API :-)
It's a fair demand. I'll try to think of something convenient during the day.
Things are a bit tricky because, like with any other app, it's not supposed to be chantilly's job to monitor itself. The canonical way is to use a 3rd party tool such as DataDog or by monitoring logs (AFAIK NetData can monitor logs). I suppose though that, for convenience, we could provide some essential statistics through the API. I just need to think of the proper way to go about it implementing this.
Okay so I added some basic statistics that you can obtain from the /api/stats route. I've also added an example in the documentation.
As I said above, this route will provide some simple statistics, such as the average response time in nanoseconds, and an exponential weighted average of said response time, but that's it. I don't think it should be chantilly's job to monitor itself by reinventing the wheel, instead a purpose-built tool should be used. I'm open to providing a complete example of how to set this up at some point.
@kevinsteger good enough?
Hey @kevinsteger we now support using Redis as a storage backend.