DeepPSP/DBCI

Computation of confidence intervals for binomial proportions and for difference of binomial proportions.

Confidence Intervals for Difference of Binomial Proportions

Computation of confidence intervals for binomial proportions and for difference of binomial proportions.

[GitHub Pages] [Read the Docs]

🚀 NEW 🚀 Streamlit support! See here for an app deployed on Streamlit Community Cloud.

Installation

Run

python -m pip install diff-binom-confint

or install the latest version in GitHub using

python -m pip install git+https://github.com/DeepPSP/DBCI.git

or git clone this repository and install locally via

cd DBCI
python -m pip install .

Numba accelerated version

Install using

python -m pip install diff-binom-confint[acc]

Usage examples

from diff_binom_confint import compute_difference_confidence_interval

n_positive, n_total = 84, 101
ref_positive, ref_total = 89, 105

confint = compute_difference_confidence_interval(
    n_positive,
    n_total,
    ref_positive,
    ref_total,
    conf_level=0.95,
    method="wilson",
)

Implemented methods

Confidence intervals for binomial proportions

Click to view!

Method (type)	Implemented
wilson	✔️
wilson-cc	✔️
wald	✔️
wald-cc	✔️
agresti-coull	✔️
jeffreys	✔️
clopper-pearson	✔️
arcsine	✔️
logit	✔️
pratt	✔️
witting	✔️
mid-p	✔️
lik	✔️
blaker	✔️
modified-wilson	✔️
modified-jeffreys	✔️

Confidence intervals for difference of binomial proportions

Click to view!

Method (type)	Implemented
wilson	✔️
wilson-cc	✔️
wald	✔️
wald-cc	✔️
haldane	✔️
jeffreys-perks	✔️
mee	✔️
miettinen-nurminen	✔️
true-profile	✔️
hauck-anderson	✔️
agresti-caffo	✔️
carlin-louis	✔️
brown-li	✔️
brown-li-jeffrey	✔️
miettinen-nurminen-brown-li	✔️
exact	❌
mid-p	❌
santner-snell	❌
chan-zhang	❌
agresti-min	❌
wang	❌
pradhan-banerjee	❌

Creating report

One can use the make_risk_report function to create a report of the confidence intervals for difference of binomial proportions.

from diff_binom_confint import make_risk_report

# df_train and df_test are pandas.DataFrame providing the data
table = make_risk_report((df_train, df_test), target = "binary_target")
# or if df_data is a pandas.DataFrame containing both training and testing data
table = make_risk_report(df_data, target = "binary_target")

For more details, see corresponding documenation. The produced table is similar to the following:

Click to view!

References

1. SAS
2. PASS
3. statsmodels.stats.proportion
4. scipy.stats._binomtest
5. corplingstats
6. DescTools.StatsAndCIs
7. Newcombee

NOTE

Reference 1 has errors in the description of the methods Wilson CC, Mee, Miettinen-Nurminen. The correct computation of Wilson CC is given in Reference 5. The correct computation of Mee, Miettinen-Nurminen are given in the code blocks in Reference 1

Test data

Test data are

1. taken (with slight modification, e.g. the upper_bound of miettinen-nurminen-brown-li method in the edge case file) from Reference 1 for automatic test of the correctness of the implementation of the algorithms.

2. generated using DescTools.StatsAndCIs via

   library("DescTools")
   library("data.table")
   
   results = data.table()
   for (m in c("wilson", "wald", "waldcc", "agresti-coull", "jeffreys",
                   "modified wilson", "wilsoncc", "modified jeffreys",
                   "clopper-pearson", "arcsine", "logit", "witting", "pratt",
                   "midp", "lik", "blaker")){
       ci = BinomCI(84,101,method = m)
       new_row = data.table("method" = m, "ratio"=ci[1], "lower_bound" = ci[2], "upper_bound" = ci[3])
       results = rbindlist(list(results, new_row))
   }
   fwrite(results, "./test/test-data/example-84-101.csv")  # with manual slight adjustment of method names

3. taken from Reference 7 (Table II).

The filenames has the following pattern:

# for computing confidence interval for difference of binomial proportions
"example-(?P<n_positive>[\\d]+)-(?P<n_total>[\\d]+)-vs-(?P<ref_positive>[\\d]+)-(?P<ref_total>[\\d]+)\\.csv"

# for computing confidence interval for binomial proportions
"example-(?P<n_positive>[\\d]+)-(?P<n_total>[\\d]+)\\.csv"

Note that the out-of-range values (e.g. > 1) are left as empty values in the .csv files.

Known Issues

1. Edge cases incorrect for the method true-profile.