MLperf
New to topics? Read the documentation here!
mlcommons.org/en/ Their homepage is not amazingly organized, but it does the job.
Benchmark focused on deep learning. It has two parts:Furthermore, a specific network model is specified for each benchmark in the closed category: so it goes beyond just specifying the dataset.
Results can be seen e.g. at:
- training: mlcommons.org/en/training-normal-21/
- inference: mlcommons.org/en/inference-datacenter-21/
And there are also separate repositories for each:
E.g. on mlcommons.org/en/training-normal-21/ we can see what the the benchmarks are:
| Dataset | Model |
|---|---|
| ImageNet | ResNet |
| KiTS19 | 3D U-Net |
| OpenImages | RetinaNet |
| COCO dataset | Mask R-CNN |
| LibriSpeech | RNN-T |
| Wikipedia | BERT |
| 1TB Clickthrough | DLRM |
| Go | MiniGo |