How can we use ERUPT metrics to validate model ?

[CoteDave]

Hi,

How can we use the ERUPT metrics to validate/compare our models ? How can we interpret the ERUPT metrics ? I see the ERUPT curves in the examples but there is no interpretation made out of them. What is a bad/good model according to those metrics ?Specifically in the case where our treatment(s) is binary. (no examples)

Thanks !

[samcarlos]

Hi CoteDave,

I've left several citations to the ERUPT metric and have explained it in the examples as well. Did you take a look at this? https://github.com/Ibotta/mr_uplift/blob/master/examples/mr_uplift_one_response_example.ipynb

Thanks,
Sam

[CoteDave]

Hi,

"Below we can see that the model performs much better than the randomized treatments suggesting the model learned the heterogenity of the treatment effects well. If we deployed the model we expect to see profit to be ~ 0.16."

Thank you !

[samcarlos]

👍 np. let me know if you have another other questions.