Implement the metrics module
Opened this issue · 6 comments
tkornuta-semiotic commented
Implement the metrics module
tkornuta-semiotic commented
- Elementary metrics:
- served queries
- revenue (= served queries * price)
- Derivative metrics:
- query ratio (= served queries / total queries)
- regret (= customer budget - agent's revenue)
- regret ratio (= agent's revenue / customer budget)
tkornuta-semiotic commented
- time to convergence
- epsilon_1 = accepted difference between the customer budget b and agent's price p (mean)
- epsilon_2 = defines how wide the gaussian can be (stddev)
agent converged IF |b - p| <= epsilon_1 AND stddev <= epsilon_2
tkornuta-semiotic commented
note: think/discuss about the warm-up phase (dynamic budget and/or no queries)
tkornuta-semiotic commented
tkornuta-semiotic commented
Price efficiency = Indexer price / customer budget