golemfactory/ya-runtime-ai

Usage counters - GPU

Closed this issue · 0 comments

What:

  • Measure time of processing Requests on framework
  • Decide how we should measure simultaneous requests

Scenario:

  • after receiving request - start counter
  • proxy to framework(automatic)
  • on response stop counter

Counter can have binary values: 0/1, parallel execution doesn't increment the value.
Should be implemented in the library for requests
Should expose API to retrieve data
Keep in mind that there can be more counters in the future, e.g. for number of requests

Next steps:

  • migrate exe-unit\src\metrics and exe-unit\src\service\metrics.rs into exe-unit\components\metrics in a way it will no longer use ExeUnitContext, and instead take just an AgreementView
  • make new crate to export Metrics trait, some shared implementations of the trait, and MetricsService struct new ya-counters crate containing Counter related code (former Metrics from exe-unit codebase)
  • make http-gsb-proxy accept list of counters/metrics implementing Metrics and RequestMonitor trait as a parameter and move code updating response/request monitoring metrics there http-gsb-proxy produces implementations of Counter trait through requests_counter and requests_duration_counter functions.
  • make ya-counters create export exe-units proc module + proc using Metrics (under proc os feature)