mlcommons/policies

Issues

• Allow for larger scale submissions in Inference when moving from Preview to Available

  #176 opened 4 months ago by psyhtest
  9

• Open/Preview submissions from Inference v2.1

  #140 opened 2 years ago by psyhtest
  3

• In https://mlcommons.org/en/training-normal-30/, an link for result guidelines is broken

  #158 opened a year ago by kcoms555
  0

• Submitter and non-submitter definition for unverified result disclosures

  #157 opened a year ago by psyhtest
  0

• Allowing Software Preview Submissions

  #122 opened 2 years ago by arjunsuresh
  22

• adding visualization to the results table

  #65 opened 4 years ago by christ1ne
  1

• Starting weights filename field is URL or not?

  #149 opened a year ago by arjunsuresh
  0

• training submission directory structure in benchmarks/

  #132 opened 2 years ago by plvlnisse
  0

• Clarity regarding source code requirement in Available category

  #123 opened 2 years ago by arjunsuresh
  1

• Clarify dates at which software must be available for the available category

  #83 opened 3 years ago by johntran-nv
  1

• For preview submission - requesting exemption to not include processor model name and core count

  #97 opened 2 years ago by rnchukka
  2

• Update compatibility table

  #112 opened 2 years ago by TheKanter
  0

• Add Paper Citations to Repos

  #86 opened 3 years ago by codyaustun
  1

• Contributing instructions are wrong

  #81 opened 3 years ago by TheKanter
  2

• how to generate submission_checker_log.txt for training v1.0 submission

  #85 opened 3 years ago by tkunlin
  0

• Update inference submission directory structure

  #48 opened 3 years ago by georgelyuan
  1

• Update TOU?

  #42 opened 3 years ago by TheKanter
  2

• Proposal for improved submission process

  #61 opened 4 years ago by georgelyuan
  34

• Write down rules for patching training frameworks

  #62 opened 4 years ago by TheKanter
  3

• Update review committee

  #78 opened 4 years ago by TheKanter
  0

• Updating submission rules

  #77 opened 4 years ago by TheKanter
  0

• Update table's field names in measurements JSON file

  #76 opened 4 years ago by mthmsn
  0

• clarify how to handle objections after publication

  #64 opened 4 years ago by christ1ne
  0

• Allow placing NOTES.txt under measurements/

  #59 opened 4 years ago by psyhtest
  1

• allowing compliance testing results to be submitted one week after submission deadline

  #60 opened 4 years ago by christ1ne
  2

• submission rules did not specify the need of truncating the accuracy logs

  #58 opened 4 years ago by christ1ne
  1

• Allow placing user.conf under results/

  #57 opened 4 years ago by psyhtest
  0

• Schedule deadline for public-facing materials

  #21 opened 5 years ago by DilipSequeira
  2

• Require package verifier, no more human readable summaries.

  #29 opened 4 years ago by petermattson
  2

• Require disclosure of Accuracy when quoting Open division results

  #24 opened 4 years ago by DilipSequeira
  2

• The benchmark names: {"resnet", "ssd-small", "ssd-large"} or {"resnet50", "ssd-mobilenet", "ssd-resnet34"}

  #52 opened 4 years ago by nvpohanh
  0

• The benchmark names: {"resnet", "ssd-small", "ssd-large"} or {}

  #50 opened 4 years ago by nvpohanh
  0

• The benchmark names: {"resnet", "ssd-small", "ssd-large"} or {"resnet50", "ssd-mobilenet", "ssd-resnet"}

  #51 opened 4 years ago by nvpohanh
  0

• Update section 5.6.2

  #44 opened 4 years ago by georgelyuan
  2

• Software updates for submission

  #40 opened 4 years ago by TheKanter
  1

• Update section 5.7 (system description json)

  #39 opened 4 years ago by TheKanter
  0

• Wrong schedule in public-facing documentation

  #37 opened 4 years ago by TheKanter
  0

• [URGENT] Is today (06/26) or August, first Friday to submit Training Result？

  #36 opened 4 years ago by GonChen
  0

• Link to TERMS OF USE is broken

  #19 opened 4 years ago by itayhubara
  1

• Submission time, change back from 4pm San Jose to 1pm San Jose

  #13 opened 4 years ago by petermattson
  1

• Reviewer assignment guarantee #250

  #25 opened 4 years ago by petermattson
  2

• Minigo real games played for each model should be logged and consider a tunnable hyperparameter

  #23 opened 4 years ago by delock
  1

• Section 5.11 in submission_rules is blank

  #26 opened 4 years ago by jonathan-cohen-nvidia
  0

• add few required files to accuracy run

  #6 opened 5 years ago by guschmue
  1

• Inference Submission: name of <system_desc_id>_<implementation_id>_<scenario>.json

  #12 opened 5 years ago by bitfort
  7

• Inference Submission: accuracy.txt/.json

  #10 opened 5 years ago by bitfort
  3

• Inference Submission: Top Level Dirs

  #9 opened 5 years ago by bitfort
  3

• Inference Submission: required files in accuracy

  #11 opened 5 years ago by bitfort
  3

• Add results field list to doc

  #8 opened 5 years ago by petermattson
  0

• Where should `implementation_id` go in results/?

  #7 opened 5 years ago by nvpohanh
  0