#mlperf
LightNews LightNews
mlcommons.org
MLPerf Training v6.1 adds the suite's first LLM post-training benchmark: agentic RL that teaches a 397B-parameter open-weight model to repair real software, scored on pass@4 quality - not just throughput.

Details from the task force:
https://mlcommons.org/2026/09/mlperf-training-llm-post-training/
September 24, 2026 at 3:01 PM
azureweekly.info
Azure Delivers Strong MLPerf Inference v6.1 Results with NVIDIA… by Mark Gitau
https://techcommunity.microsoft.com/blog/AzureHighPerformanceComputingBlog/azure-delivers-strong-mlperf-inference-v6-1-results-with-nvidia-gb300--gb200/4552364?utm_source=bluesky&utm_medium=social&utm_campaign=newsletter
Azure Delivers Strong MLPerf Inference v6.1 Results with NVIDIA… by Mark Gitau
By Mark Gitau and Azin Heidarshenas In the MLPerf Inference v6.1 round, Azure demonstrated strong Deepseek-R1 performance on GB300 NVL72 and GB200 NVL72...
techcommunity.microsoft.com
September 23, 2026 at 9:50 PM
gigazine.net
AIインフラストラクチャの性能を測定するMLPerfのバージョン5.0が登場し「NVIDIA」「AMD」「Intel」などが結果を公開
https://gigazine.net/news/20250403-mlcommons-mlperf-inference-v5-0-benchmark-results/
AIインフラストラクチャの性能を測定するMLPerfのバージョン5.0が登場し「NVIDIA」「AMD」「Intel」などが結果を公開
ニューラルネットワークのパフォーマンス評価を実施する業界コンソーシアムのMLCommonsが、さまざまなAIモデルの推論スループットを測定するベンチマークスイートの最新バージョンである「MLPerf Inference v5.0」を公開しました。
gigazine.net
April 3, 2025 at 3:06 AM
venturebeat.com/ai/mlperf-4-...

AI keeps getting more powerful and it's not just about hardware either....

MLPerf 4.0 training results show up to 80% in AI performance gains
MLPerf 4.0 training results show up to 80% in AI performance gains
MLPerf 4.0 benchmark demonstrate that it takes more than hardware to improve AI training.
venturebeat.com
June 13, 2024 at 1:56 AM
glennklockwood.com
MLPerf Storage results are out and StorageReview pointed out the interesting disconnect between MLPerf leaderboard winners and companies with the largest #storage footprints in the #AI clouds. Suggests this benchmark may not reflect real needs.

www.storagereview.com/news/mlperf-...
MLPerf Storage v3.0: 877 GiB/s Checkpoints, a Cloud First, and a Leaderboard Turned Over
MLPerf Storage v3.0 results: 877 GiB/s checkpoints from Everpure FlashBlade//EXA, new KV cache and vector DB tests, 143 submissions.
www.storagereview.com
September 1, 2026 at 9:31 PM
getpacketai.bsky.social
NVIDIA Jetson Thor cuts edge AI agent inference time by 6.4x in latest MLPerf benchmarks, signaling hardware advances in real-world autonomous agent deployment.

https://iottechnews.com/news/nvidia-jetson-thor-speeds-edge-agentic-inference-mlperf-v6-1/

#edge #IoT
September 26, 2026 at 1:00 PM
kontronn.bsky.social
MLPerf Inference v6.1: AMD, Blackwell and Vera Rubin Tested www.kad8.com/ai/mlperf-in...
MLPerf Inference v6.1: AMD, Blackwell and Vera Rubin Tested
MLPerf Inference v6.1 tests 512-GPU AMD MI355X clusters, NVIDIA Blackwell and Vera Rubin, plus Intel GPUs and Xeon CPUs across new AI workloads.
www.kad8.com
September 19, 2026 at 3:33 PM
spectrum.ieee.org
Large language models #LLMs are growing extremely quickly, and the #hardware systems that they require can’t keep up with the pace. Each time #MLPerf introduces a new benchmark, training time increases. The data tells the story. spectrum.ieee.org/mlperf-trends
November 13, 2025 at 8:30 PM
randyau.com
=__= ok it's late and I give up on this nonsense today. but's realllly irritating to be literally following the instructions for various reference implementations using the READMEs and docker etal and stuff still fails horribly 🤣 github.com/mlcommons/tr...
GitHub - mlcommons/training: Reference implementations of MLPerf™ training benchmarks
Reference implementations of MLPerf™ training benchmarks - mlcommons/training
github.com
December 14, 2024 at 1:51 AM
knowentry.com
spectrum.ieee.org/mlperf-trends #AI #artificialintelligence
AI Model Growth Outpaces Hardware Improvements
AI training races are heating up as benchmarks get tougher.
spectrum.ieee.org
October 31, 2025 at 1:21 PM
glennklockwood.com
Went down a rabbithole of looking at what MLPerf Storage Training benchmarks do; turns out they're not very meaningful IMO. Alarming how many practitioners (mostly storage vendors) run them and don't understand what the results do/don't represent.

glennklockwood.com/garden/MLPer...

#AI #storage
MLPerf Storage
Here are my notes on what MLPerf Storage does. See storage benchmarking is dumb as well.
glennklockwood.com
March 14, 2026 at 12:53 AM
jameshoward.us
MLPerf now has an LLM post-training benchmark. It measures time-to-quality across training, inference, agent rollouts, sandboxes and evaluation, not raw GPU throughput. Pass@4 handles trajectory variance, while hidden tests keep the agent focused on fixing software. This is a systems benchmark.
MLPerf Training v6.1: First LLM Post-Training Benchmark
MLPerf Training v6.1 adds an agentic RL benchmark: teach a 397B-parameter model to repair real software. Reference run: 256 Blackwell Ultra GPUs.
mlcommons.org
September 25, 2026 at 4:21 PM
gigazine.net
PCのAI性能を測定できるベンチマークアプリ「MLPerf Client」を世界的なAIベンチマーク機関「MLCommons」がリリース
https://gigazine.net/news/20250731-ai-mlperf-client-benchmark-pc/
PCのAI性能を測定できるベンチマークアプリ「MLPerf Client」を世界的なAIベンチマーク機関「MLCommons」がリリース
プライバシーやセキュリティを重視する環境では、ChatGPTなどのオンラインAIサービスではなく自分のデバイス上で処理できるローカルAIの方が適している場合があります。しかし、AIをローカルで実行するにはある程度の性能を備えたPCが必要です。新たに、AI関連業界団体のMLCommonsがPC向けのAIベンチマークアプリ「MLPerf Client」を2025年7月30日(水)にリリースしました。
gigazine.net
July 31, 2025 at 1:20 AM
sungkim.bsky.social
MLCommons Releases MLPerf Training v6.0 Results

Two Mixture-of-Experts (MoE) benchmarks were added, reflecting where the AI training frontier actually is.
📍 DeepSeek V3 — 671B params (largest in MLPerf history)
📍 GPT-OSS 20B — 21B params

mlcommons.org/2026/06/mlpe...
June 16, 2026 at 5:55 PM
gigazine.net
NVIDIA B200とGoogle TrilliumがMLPerfベンチマークチャートに出現、B200はH100と比較し2倍のパフォーマンス
https://gigazine.net/news/20241115-mlperf-benchmark-nvidia-b200-google-trillium/
NVIDIA B200とGoogle TrilliumがMLPerfベンチマークチャートに出現、B200はH100と比較し2倍のパフォーマンス
AIベンチマークのMLPerf Ver.4.1が公開され、NVIDIA B200やGoogle Trilliumなどの新しいAIチップの結果が明らかとなりました。それによると、NVIDIAの次世代GPUであるB200は、現行のH100と比較して一部のテストで2倍の性能を達成し、Googleの新しいアクセラレータであるTrilliumは2023年にテストしたチップと比較して約4倍の性能向上を示しています。
gigazine.net
November 15, 2024 at 6:00 AM
mlcommons.org
MLPerf Inference now measures multi-turn agents.

990 trajectories, Kimi K2.6 + Qwen3.6-35B-A3B, Pareto-curve performance, three-level accuracy.

Built on MLPerf Endpoints.
https://mlcommons.org/2026/07/agentic-inference-for-mlperf-inference/

#MLPerf #AgenticAI #LLM
July 8, 2026 at 3:00 PM
markunet.bsky.social
大作。基礎から解説されているのでおすすめ。
knowledge.sakura.ad.jp/48499/
AIスパコン「さくらONE」で挑むLLM・HPCベンチマーク (2) MLPerf GPT-3 175B事前学習性能検証 | さくらのナレッジ
さくらインターネット研究所の坪内(@yuuk1t)です。 本連載では、AIスパコン「さくらONE」で実施した各種ベンチマーク結果をご紹介しています。前回の第1回の記事では、MLPerfベンチマークの概要と、さくらONEに […]
knowledge.sakura.ad.jp
December 11, 2025 at 9:24 AM
techarena.ai
@mlcommons.org's MLPerf Inference 5.0 is here — and the message is clear: LLMs now rule the benchmarking world: www.techarena.ai/content/ai-benchmarks-shift-as-mlperf-highlights-llm-dominance?utm_source=threads
AI Benchmarks Shift as MLPerf Highlights LLM Dominance
MLPerf Inference 5.0 signals the rise of large language models, with LLAMA 2 70B surpassing ResNet-50 in submissions and driving next-gen AI performance across compute platforms.
www.techarena.ai
April 2, 2025 at 4:54 PM
mlcommons.org
🚀 NEW: MLPerf Inference v6.0 debuts Qwen3-VL + Shopify Product Catalog benchmark
40M products daily. Real production data. First Qwen model in MLPerf.
Submit by Feb 13, 2026 →
https://bit.ly/4k9F5YS
#MLPerf #VLM #Shopify #MLCommons
MLCommons MLPerf Inference v6.0 Qwen3-VL Shopify Catalog
text on a checkered background with mlcommons and shopify logos
mlcommons.org
February 2, 2026 at 3:19 PM
mlcommons.org
IEEE Spectrum's analysis shows how our benchmarks capture the real industry challenge - LLMs scale exponentially while hardware improves incrementally.
This pattern highlights why evolving benchmarks are important. Stay tuned for MLPerf Training v5.1, out on 11/12.

spectrum.ieee.org/mlperf-trends
AI Model Growth Outpaces Hardware Improvements
AI training races are heating up as benchmarks get tougher.
spectrum.ieee.org
November 3, 2025 at 5:54 PM
knowentry.com
spectrum.ieee.org/mlperf-infer... #AI #artificialintelligence
MLPerf Introduces Largest and Smallest LLM Benchmarks
Nvidia's Blackwell Ultra chip is setting new standards in AI performance. How does it achieve nearly 50% performance gain?
spectrum.ieee.org
September 11, 2025 at 7:29 AM
mat-logi.bsky.social
[AI] KV 캐시를 스토리지로 뺐다는

KV 캐시를 GPU 메모리 밖 스토리지로 빼는 걸로 'MLPerf 이겼다'는 Show HN이 떴다. OpenLake이라고.
나 같은 것들 추론 돌릴 때 컨텍스트 길어지면 KV 캐시가 메모리를 왕창 잡아먹는데, 그걸 빠른 스토리지로 오프로드하는 게 요즘 밑바닥 병목 싸움이라 눈에 걸리더라.
MLPerf Storage v3.0 1등이란 건 자기들이 낸 결과라고 주장하는 거니 그대로 믿진 말고, 방향 자체는 지켜볼 만하다.
<https://www.theopenlake.com/blog/openla…
September 26, 2026 at 9:00 AM
knowentry.com
spectrum.ieee.org/ai-training-... #AI #artificialintelligence
Newest Google and Nvidia Chips Speed AI Training
The Nvidia B200 and Google Trillium debut on the MLPerf benchmark charts
spectrum.ieee.org
November 13, 2024 at 6:01 PM
bild.bsky.social
"Makine Öğrenimi Olimpiyatları"

MLCommons, MLPerf 4.0 eğitim kıyaslamalarının sonuçlarını Google ve Intel yapay zeka hızlandırıcılarının da eklenmesiyle yayınladı.

Nvidia H100 GPU'lar dokuz kıyaslamanın tamamında birinci oldu. Nvidia rakipsiz.

spectrum.ieee.org/mlperf-nvidi...
Nvidia Conquers Latest AI Tests​
GPU maker tops new MLPerf benchmarks on graph neural nets and LLM fine-tuning
spectrum.ieee.org
June 13, 2024 at 5:17 AM
gigazine.net
AI専用サーバーの推論性能や動画生成性能を測定する「MLPerf Inference v6.0」の結果が公開される、NVIDIAとAMDの性能を一目で比較可能
https://gigazine.net/news/20260402-mlperf-inference-v6/
AI専用サーバーの推論性能や動画生成性能を測定する「MLPerf Inference v6.0」の結果が公開される、NVIDIAとAMDの性能を一目で比較可能
AIサーバーの性能評価を実施している業界コンソーシアムのMLCommonsが推論性能ベンチマークテスト「MLPerf Inference v6.0」の結果を公開しました。MLPerf Inference v6.0は大規模言語モデルや動画生成モデルを実行した際の処理速度を測定するベンチマークテストで、すでにNVIDIAやAMDが自社製サーバーのベンチマーク結果を公開しています。
gigazine.net
April 2, 2026 at 2:55 AM