Pinned Repositories
ailuminate
The AILuminate v1.1 benchmark suite is an AI risk assessment benchmark developed with broad involvement from leading AI companies, academia, and civil society.
algorithmic-efficiency
MLCommons Algorithmic Efficiency is a benchmark and competition measuring neural network training speedups due to algorithmic improvements in both training algorithms and models.
ck
Collective Knowledge (CK), Collective Mind (CM/CMX) and MLPerf automations: community-driven projects to facilitate collaborative and reproducible research and to learn how to run AI, ML, and other emerging workloads more efficiently and cost-effectively across diverse models, datasets, software, and hardware using MLPerf methodology and benchmarks
croissant
Croissant is a high-level format for machine learning datasets that brings together four rich layers.
inference
Reference implementations of MLPerf™ inference benchmarks
inference_results_v5.0
This repository contains the results and code for the MLPerf™ Inference v5.0 benchmark.
modelbench
Run safety benchmarks against AI models and view detailed reports showing how well they performed.
tiny
MLPerf™ Tiny is an ML benchmark suite for extremely low-power systems such as microcontrollers
training
Reference implementations of MLPerf® training benchmarks
training_results_v4.1
This repository contains the results and code for the MLPerf™ Training v4.1 benchmark.
MLCommons's Repositories
mlcommons/training
Reference implementations of MLPerf® training benchmarks
mlcommons/inference
Reference implementations of MLPerf™ inference benchmarks
mlcommons/ck
Collective Knowledge (CK), Collective Mind (CM/CMX) and MLPerf automations: community-driven projects to facilitate collaborative and reproducible research and to learn how to run AI, ML, and other emerging workloads more efficiently and cost-effectively across diverse models, datasets, software, and hardware using MLPerf methodology and benchmarks
mlcommons/croissant
Croissant is a high-level format for machine learning datasets that brings together four rich layers.
mlcommons/tiny
MLPerf™ Tiny is an ML benchmark suite for extremely low-power systems such as microcontrollers
mlcommons/algorithmic-efficiency
MLCommons Algorithmic Efficiency is a benchmark and competition measuring neural network training speedups due to algorithmic improvements in both training algorithms and models.
mlcommons/GaNDLF
A generalizable application framework for segmentation, regression, and classification using PyTorch
mlcommons/medperf
An open benchmarking platform for medical artificial intelligence using Federated Evaluation.
mlcommons/storage
MLPerf® Storage Benchmark Suite
mlcommons/chakra
Repository for MLCommons Chakra schema and tools
mlcommons/training_policies
Issues related to MLPerf™ training policies, including rules and suggested changes
mlcommons/modelbench
Run safety benchmarks against AI models and view detailed reports showing how well they performed.
mlcommons/inference_policies
Issues related to MLPerf™ Inference policies, including rules and suggested changes
mlcommons/mobile_app_open
Mobile App Open
mlcommons/hpc
Reference implementations of MLPerf™ HPC training benchmarks
mlcommons/logging
MLPerf™ logging library
mlcommons/policies
General policies for MLPerf™ including submission rules, coding standards, etc.
mlcommons/dynabench
mlcommons/power-dev
Dev repo for power measurement for the MLPerf™ benchmarks
mlcommons/cm4mlops
Legacy CM repository with a collection of portable, reusable and cross-platform CM automations for MLOps and MLPerf to simplify the process of building, benchmarking and optimizing AI systems across diverse models, data sets, software and hardware
mlcommons/ailuminate
The AILuminate v1.1 benchmark suite is an AI risk assessment benchmark developed with broad involvement from leading AI companies, academia, and civil society.
mlcommons/cm4mlperf-results
CM interface and automation recipes to analyze MLPerf Inference, Tiny and Training results. The goal is to make it easier for the community to visualize, compare and reproduce MLPerf results and add derived metrics such as Performance/Watt or Performance/$
mlcommons/GaNDLF-Synth
Extension for GaNDLF [gandlf.org] to enable synthesis.
mlcommons/submissions_algorithms
mlcommons/mlcflow
MLCFlow: Simplifying MLPerf Automations
mlcommons/mlperf-automations
This repository contains automation scripts designed to run MLPerf Inference benchmarks. Originally developed for the Collective Mind (CM) automation framework, these scripts have been adapted to leverage the MLC automation framework, maintained by the MLCommons Benchmark Infrastructure Working Group.
mlcommons/datasets-knowledge-graphs
mlcommons/inference_results_visualization_template
mlcommons/mlperf_inference_submissions
mlcommons/mlperf_inference_test_submissions_v5.0