/synbols-benchmarks

Benchmarks for the Synbols project. Synbols is a ServiceNow Research project that was started at Element AI.

Primary LanguagePythonApache License 2.0Apache-2.0

ServiceNow completed its acquisition of Element AI on January 8, 2021. All references to Element AI in the materials that are part of this project should refer to ServiceNow.

#Synbols

Probing Learning Algorithms with Synthetic Datasets

License

Synbols

Description

This repository contains the code for reproducing experiments in [1].

To use the Synbols tool for generating new datasets, please visit https://github.com/ElementAI/synbols

Pytorch Dataset Quick Setup

from pytorch_examples.datasets import Synbols
from torchvision import transforms as tt

dataset_path = "./"
dataset_name = "default_n=100000_2020-Oct-19.h5py"

synbols = Synbols(args.data_path,
                  dataset_name=args.dataset)
train_dataset = synbols.get_split('train', tt.ToTensor())
val_dataset = synbols.get_split('val', tt.ToTensor())

For a complete example run ./pytorch_examples/minimal_classification.py from the root folder of this project:

python -m pytorch_examples.minimal_classification

It should reach >70% accuracy.

Bibliography

[1] Lacoste, A., Rodríguez, P., Branchaud-Charron, F., Atighehchian, P., Caccia, M., Laradji, I., Drouin, A., Craddock, M., Charlin, L. and Vázquez, D., 2020. Synbols: Probing Learning Algorithms with Synthetic Datasets.