sdv-dev/SDGym

Passing None as synthesizers runs all of them

amontanez24 opened this issue · 2 comments

Environment Details

Please indicate the following details about the environment in which you found the bug:

  • SDGym version: master
  • Python version: Any
  • Operating System: Any

Error Description

If you pass an empty list or None for synthesizers into the benchmark_single_table function it runs all of the synthesizers. This is undesirable because sometimes you only want to run custom synthesizers. Instead of defaulting to all when synthesizers is None, we should just treat it as None.

Steps to reproduce

from sdgym import benchmark_single_table

benchmark_single_table(
    synthesizers=None,
    custom_synthesizers=[TestSynthesizer],
    sdv_datasets=['student_placements']
)

@npatki What should happen if the user passes None for both synthesizers and custom_synthesizers?

npatki commented

@amontanez24 in this case, I suppose we can just return an empty DataFrame, as there is nothing to run.

Also, I think the empty list should be treated the same way as None:

from sdgym import benchmark_single_table

benchmark_single_table(
    synthesizers=[],
    custom_synthesizers=[TestSynthesizer],
    sdv_datasets=['student_placements']
)