facebookresearch/dynalab
The Python library with command line tools to interact with Dynabench(https://dynabench.org/), such as uploading models.
PythonMIT
Issues
- 0
- 0
- 0
- 0
Add BART style NLI example handler
#119 opened by Ishita-0112 - 0
Add BERT style NLI example handler
#123 opened by fzchriha - 0
- 0
- 1
Could an example task.json be provided
#113 opened by walterwang - 1
The same model works successfully for small tracks but takendown for the full track
#109 opened by BaohaoLiao - 2
getting no model_files path specified issue
#110 opened by nishantgupta08 - 0
Test failing after step 25/25
#111 opened by bpopeters - 17
Large model not evaluating on the devtest set
#103 opened by jcblaisecruz02 - 0
- 1
Taken down for full track issue[hurry]
#106 opened by haoyanghua - 2
- 13
Taken Down in the Dynabench for Large Track
#90 opened by CSJianYang - 1
Submit models with own extensions to fairseq
#105 opened by dannigt - 4
- 3
- 4
- 5
The shown average score is different with the manual calculation of mean of individual scores
#100 opened by BaohaoLiao - 2
Failed to submit large model
#102 opened by yingwaner - 8
The status of the model is "takendown"
#87 opened by yingwaner - 9
Large Size Model > 2GB
#91 opened by CSJianYang - 2
Submission shows "Takendown" status
#96 opened by jcblaisecruz02 - 5
Help with submission
#93 opened by gwenzek - 3
504 Server Error: Gateway Time-out
#94 opened by mazhiyi - 5
The same model failed today, but worked before
#101 opened by BaohaoLiao - 2
Installing on py37 gives error AttributeError: type object 'Callable' has no attribute '_abc_registry'
#73 opened by chauhang - 1
- 5
- 0
Better management of submitted files
#89 opened by mazhiyi - 1
Improved Reporting of Model Upload Status
#85 opened by maxbartolo - 0
Non-Leaderboard Datasets Leaderboard
#86 opened by maxbartolo - 3
Add a warning explaining that '_' is not allowed in filename (or general model naming convention guidelines)
#81 opened by maxbartolo - 8
Question about the dockerfile
#77 opened by CSJianYang - 0
[open source] pull dockerfiles from github file and remove install by --editable requirement
#67 opened by mazhiyi - 4
Double check to make sure that labels are unified with dynabench and the template handlers are ok
#60 opened by TristanThrush - 0
In verify_response, prevent extra keys too
#53 opened by mazhiyi - 0
Add prob key to response
#54 opened by mazhiyi - 0
- 0
Integrated test taking too much memory
#58 opened by mazhiyi - 0
dynalab init --rename
#42 opened by mazhiyi - 0
[be] use schema validation for verify_response
#56 opened by mazhiyi - 0
- 0
- 0
docker image name cannot contain upper case letters
#43 opened by mazhiyi - 1
- 1
docker build timeout when libraries in requirements.txt or setup.py are too big
#32 opened by mazhiyi - 0
Add a check before dynalab-cli upload really uploads to prevent accidental submission
#34 opened by mazhiyi