gregdingle/genetify

What basis is confidence calculated on?

ddn opened this issue · 2 comments

ddn commented

In looking at the confidence calculations, I noticed it is not calculated in the same way as explained here:

http://visualwebsiteoptimizer.com/split-testing-blog/what-you-really-need-to-know-about-mathematics-of-ab-split-testing/

For example, for the same data set genetify gives me a .496 vs a .749 p-value.

Can you point me to any links on what the basis is? I found this http://faculty.vassar.edu/lowry/fcall.js and poked around, but I can't say I understood it.

Greg, I know this is a mostly dead project for you so feel free to ignore. Hopefully someone interested will notice the activity and jump on board.

Genetify uses a f-test to determine differences between n groups on a continuous variable. See http://en.wikipedia.org/wiki/F_statistic . The blog article describes a binomial test. The f-test is more general but less powerful -- should be lower confidence on the same data.

On Jan 5, 2013, at 9:58 AM, ddn notifications@github.com wrote:

In looking at the confidence calculations, I noticed it is not calculated in the same way as explained here:

http://visualwebsiteoptimizer.com/split-testing-blog/what-you-really-need-to-know-about-mathematics-of-ab-split-testing/

For example, for the same data set genetify gives me a .496 vs a .749 p-value.

Can you point me to any links on what the basis is? I found this http://faculty.vassar.edu/lowry/fcall.js and poked around, but I can't say I understood it.

Greg, I know this is a mostly dead project for you so feel free to ignore. Hopefully someone interested will notice the activity and jump on board.


Reply to this email directly or view it on GitHub.

ddn commented

Ah, excellent. Thank you.

On Jan 5, 2013, at 12:32 PM, Greg Dingle notifications@github.com wrote:

Genetify uses a f-test to determine differences between n groups on a continuous variable. See http://en.wikipedia.org/wiki/F_statistic . The blog article describes a binomial test. The f-test is more general but less powerful -- should be lower confidence on the same data.

On Jan 5, 2013, at 9:58 AM, ddn notifications@github.com wrote:

In looking at the confidence calculations, I noticed it is not calculated in the same way as explained here:

http://visualwebsiteoptimizer.com/split-testing-blog/what-you-really-need-to-know-about-mathematics-of-ab-split-testing/

For example, for the same data set genetify gives me a .496 vs a .749 p-value.

Can you point me to any links on what the basis is? I found this http://faculty.vassar.edu/lowry/fcall.js and poked around, but I can't say I understood it.

Greg, I know this is a mostly dead project for you so feel free to ignore. Hopefully someone interested will notice the activity and jump on board.


Reply to this email directly or view it on GitHub.


Reply to this email directly or view it on GitHub.