pytorch/opacus

PRV Accountant fails for specific input values for its args

gauriprdhn opened this issue · 2 comments

gauriprdhn commented

🐛 Bug

Please reproduce using our template Colab and post here the link

Check Here

To Reproduce

Steps to reproduce the behavior:

  1. Just run each cell of the shared opacus bug report notebook
  2. You should have access to edit/ view the file.

Expected behavior

---------------------------------------------------------------------------
RuntimeError                              Traceback (most recent call last)
[<ipython-input-3-e9e4db84336a>](https://localhost:8080/#) in <cell line: 4>()
     11                                                                                         target_epochs,
     12                                                                                         target_delta))
---> 13         noise_sigma = get_noise_multiplier(
     14                     target_epsilon = eps,
     15                     epochs = target_epochs,

4 frames
[/content/opacus/opacus/accountants/analysis/prv/prvs.py](https://localhost:8080/#) in discretize(prv, domain)
    173 
    174     if np.abs(mean_shift) >= domain.dt / 2:
--> 175         raise RuntimeError("Discrete mean differs significantly from continuous mean.")
    176 
    177     domain_shifted = domain.shift_right(mean_shift)

RuntimeError: Discrete mean differs significantly from continuous mean.

I expected it to return the estimated sigma value of noise to be used for DP just like it did for other values of the arguments. However, for some configurations of the input to the noise_multiplier, the abovementioned error pops up. For example, for the setting epsilon = 5, batch size = 50, epochs = 10, delta = 1e-05 and epsilon = 4, batch size = 50, epochs = 5, delta = 1e-05. However, the error is NOT limited to just these 2 settings.

Environment

Please copy and paste the output from our
environment collection script
(or fill out the checklist below manually).

You can get the script and run it with:

wget https://raw.githubusercontent.com/pytorch/pytorch/master/torch/utils/collect_env.py
# For security purposes, please check the contents of collect_env.py before running it.
python collect_env.py
  • PyTorch Version (e.g., 1.0): 2.0.1+cu118
  • OS (e.g., Linux): Ubuntu 22.04.2 LTS (x86_64)
  • How you installed PyTorch (conda, pip, source): pip
  • Build command you used (if compiling from source):
  • Python version: 3.10.12
  • CUDA/cuDNN version: CUDA used to build PyTorch: 11.8
  • cuDNN version: Probably one of the following:
    /usr/lib/x86_64-linux-gnu/libcudnn.so.8.9.0
    /usr/lib/x86_64-linux-gnu/libcudnn_adv_infer.so.8.9.0
    /usr/lib/x86_64-linux-gnu/libcudnn_adv_train.so.8.9.0
    /usr/lib/x86_64-linux-gnu/libcudnn_cnn_infer.so.8.9.0
    /usr/lib/x86_64-linux-gnu/libcudnn_cnn_train.so.8.9.0
    /usr/lib/x86_64-linux-gnu/libcudnn_ops_infer.so.8.9.0
    /usr/lib/x86_64-linux-gnu/libcudnn_ops_train.so.8.9.0
  • GPU models and configuration: GPU 0: Tesla T4
  • Any other relevant information: NA

Additional context

Solosneros commented

See PR #606.