MLCommons

All

99 repositories

inference
Public
Reference implementations of MLPerf™ inference benchmarks
benchmark machine-learning
Python
•
Apache License 2.0
•539•1.3k•160•14•Updated Dec 19, 2024Dec 19, 2024
dynabench
Public
Python
•
MIT License
•16•21•7•5•Updated Dec 19, 2024Dec 19, 2024
mlperf-automations
Public
Python
•
Apache License 2.0
•3•0•45•0•Updated Dec 19, 2024Dec 19, 2024
modelbench
Public
Run safety benchmarks against AI models and view detailed reports showing how well they performed.
Python
•
Apache License 2.0
•12•64•174•6•Updated Dec 18, 2024Dec 18, 2024
mlperf_inference_test_submissions_v5.0
Public
Mermaid
•1•0•0•0•Updated Dec 18, 2024Dec 18, 2024
inference_policies
Public
Issues related to MLPerf™ Inference policies, including rules and suggested changes
Apache License 2.0
•52•57•18•2•Updated Dec 18, 2024Dec 18, 2024
policies
Public
General policies for MLPerf™ including submission rules, coding standards, etc.
Python
•
Apache License 2.0
•54•28•35•38•Updated Dec 18, 2024Dec 18, 2024
GaNDLF
Public
A generalizable application framework for segmentation, regression, and classification using PyTorch
machine-learning framework deep-learning regression pytorch medical-imaging classification segmentation data-augmentation biomedical-image-processing
Python
•
Apache License 2.0
•80•167•16•3•Updated Dec 18, 2024Dec 18, 2024
power-dev
Public
Dev repo for power measurement for the MLPerf™ benchmarks
Python
•
Apache License 2.0
•25•18•10•3•Updated Dec 17, 2024Dec 17, 2024
mobile_app_open
Public
Mobile App Open
C++
•
Apache License 2.0
•24•48•29•2•Updated Dec 17, 2024Dec 17, 2024
medperf
Public
An open benchmarking platform for medical artificial intelligence using Federated Evaluation.
Python
•
Apache License 2.0
•32•148•52•23•Updated Dec 16, 2024Dec 16, 2024
tiny
Public
MLPerf™ Tiny is an ML benchmark suite for extremely low-power systems such as microcontrollers
C
•
Apache License 2.0
•91•362•13•5•Updated Dec 13, 2024Dec 13, 2024
algorithmic-efficiency
Public
MLCommons Algorithmic Efficiency is a benchmark and competition measuring neural network training speedups due to algorithmic improvements in both training algorithms and models.
Python
•
Apache License 2.0
•71•339•30•7•Updated Dec 12, 2024Dec 12, 2024
training_results_v4.1
Public
This repository contains the results and code for the MLPerf™ Training v4.1 benchmark.
Python
•4•3•1•0•Updated Dec 11, 2024Dec 11, 2024
mlperf-automations_archived
Public archive
This repository contains the automations and scripts used to automate MLPerf benchmarks (mainly MLPerf inference for now)
automation mlperf
Python
•
Apache License 2.0
•24•0•46•0•Updated Dec 11, 2024Dec 11, 2024
mlperf_client
Public
MLPerf Client is a benchmark for Windows and macOS, focusing on client form factors in ML inference scenarios.
benchmarking machine-learning ml
C++
•
Apache License 2.0
•0•5•0•0•Updated Dec 10, 2024Dec 10, 2024
logging
Public
MLPerf™ logging library
Python
•
Apache License 2.0
•46•31•41•2•Updated Dec 10, 2024Dec 10, 2024
ck-mlops
Public
A collection of portable workflows, automation recipes and components for MLOps in a unified CK format. Note that this repository is outdated - please check the 2nd generation of the CK workflow automation meta-framework with portable MLOps and DevOps components here:
machine-learning automation optimization artificial-intelligence open-science systems reproducibility workflows portability co-design
Python
•
Apache License 2.0
•8•17•0•0•Updated Dec 10, 2024Dec 10, 2024
cm4mlops
Public
A collection of portable, reusable and cross-platform automation recipes (CM scripts) to make it easier to build and benchmark AI systems across diverse models, data sets, software and hardware
workflow devops machine-learning automation cross-platform reusable workflow-automation mlops mlperf
Python
•
Apache License 2.0
•24•17•8•0•Updated Dec 10, 2024Dec 10, 2024
ck
Public
Collective Minds (CM) and Collective Knowledge (CK): learning how to run AI, ML and other emerging workloads in a more efficient and cost-effective way across diverse models, datasets, software and hardware using MLPerf, CM and CK
education automation optimization best-practices collaboration cm modularity workflows portability reusability
Python
•
Apache License 2.0
•121•612•14•0•Updated Dec 9, 2024Dec 9, 2024
chakra
Public
Repository for MLCommons Chakra schema and tools
Python
•
Apache License 2.0
•40•75•25•9•Updated Dec 6, 2024Dec 6, 2024
croissant
Public
Croissant is a high-level format for machine learning datasets that brings together four rich layers.
machine-learning schema-org json-ld datasets
Python
•
Apache License 2.0
•45•476•120•19•Updated Dec 5, 2024Dec 5, 2024
datasets-knowledge-graphs
Public
Apache License 2.0
•0•0•0•0•Updated Dec 3, 2024Dec 3, 2024
cm4abtf
Public
CM interface and automation recipes for ABTF
workflow automation
Python
•
Apache License 2.0
•6•2•2•0•Updated Nov 29, 2024Nov 29, 2024
common-crawl-dmlr
Public
Apache License 2.0
•0•0•0•1•Updated Nov 26, 2024Nov 26, 2024
cm4mlperf-inference
Public
Apache License 2.0
•35•1•2•2•Updated Nov 21, 2024Nov 21, 2024
training_policies
Public
Issues related to MLPerf™ training policies, including rules and suggested changes
Python
•
Apache License 2.0
•66•93•109•3•Updated Nov 21, 2024Nov 21, 2024
submissions_algorithms
Public
Apache License 2.0
•2•1•1•1•Updated Nov 13, 2024Nov 13, 2024
cla-bot
Public
TypeScript
•
Apache License 2.0
•2•0•0•0•Updated Nov 7, 2024Nov 7, 2024
inference_results_v4.0
Public
This repository contains the results and code for the MLPerf™ Inference v4.0 benchmark.
Apache License 2.0
•17•10•5•5•Updated Nov 7, 2024Nov 7, 2024