Azure Delivers Strong MLPerf Inference v6.1 Results with NVIDIA… by Mark Gitau
https://techcommunity.microsoft.com/blog/AzureHighPerformanceComputingBlog/azure-delivers-strong-mlperf-inference-v6-1-results-with-nvidia-gb300--gb200/4552364?utm_source=bluesky&utm_medium=social&utm_campaign=newsletter
https://techcommunity.microsoft.com/blog/AzureHighPerformanceComputingBlog/azure-delivers-strong-mlperf-inference-v6-1-results-with-nvidia-gb300--gb200/4552364?utm_source=bluesky&utm_medium=social&utm_campaign=newsletter
Azure Delivers Strong MLPerf Inference v6.1 Results with NVIDIA… by Mark Gitau
By Mark Gitau and Azin Heidarshenas
In the MLPerf Inference v6.1 round, Azure demonstrated strong Deepseek-R1 performance on GB300 NVL72 and GB200 NVL72...
techcommunity.microsoft.com
September 23, 2026 at 9:50 PM
Azure Delivers Strong MLPerf Inference v6.1 Results with NVIDIA… by Mark Gitau
https://techcommunity.microsoft.com/blog/AzureHighPerformanceComputingBlog/azure-delivers-strong-mlperf-inference-v6-1-results-with-nvidia-gb300--gb200/4552364?utm_source=bluesky&utm_medium=social&utm_campaign=newsletter
https://techcommunity.microsoft.com/blog/AzureHighPerformanceComputingBlog/azure-delivers-strong-mlperf-inference-v6-1-results-with-nvidia-gb300--gb200/4552364?utm_source=bluesky&utm_medium=social&utm_campaign=newsletter
MLPerf Training v6.1 adds the suite's first LLM post-training benchmark: agentic RL that teaches a 397B-parameter open-weight model to repair real software, scored on pass@4 quality - not just throughput.
Details from the task force:
https://mlcommons.org/2026/09/mlperf-training-llm-post-training/
Details from the task force:
https://mlcommons.org/2026/09/mlperf-training-llm-post-training/
September 24, 2026 at 3:01 PM
MLPerf Training v6.1 adds the suite's first LLM post-training benchmark: agentic RL that teaches a 397B-parameter open-weight model to repair real software, scored on pass@4 quality - not just throughput.
Details from the task force:
https://mlcommons.org/2026/09/mlperf-training-llm-post-training/
Details from the task force:
https://mlcommons.org/2026/09/mlperf-training-llm-post-training/
MLPerf Inference v6.1: AMD, Blackwell and Vera Rubin Tested www.kad8.com/ai/mlperf-in...
MLPerf Inference v6.1: AMD, Blackwell and Vera Rubin Tested
MLPerf Inference v6.1 tests 512-GPU AMD MI355X clusters, NVIDIA Blackwell and Vera Rubin, plus Intel GPUs and Xeon CPUs across new AI workloads.
www.kad8.com
September 19, 2026 at 3:33 PM
MLPerf Inference v6.1: AMD, Blackwell and Vera Rubin Tested www.kad8.com/ai/mlperf-in...
AIインフラストラクチャの性能を測定するMLPerfのバージョン5.0が登場し「NVIDIA」「AMD」「Intel」などが結果を公開
https://gigazine.net/news/20250403-mlcommons-mlperf-inference-v5-0-benchmark-results/
https://gigazine.net/news/20250403-mlcommons-mlperf-inference-v5-0-benchmark-results/
AIインフラストラクチャの性能を測定するMLPerfのバージョン5.0が登場し「NVIDIA」「AMD」「Intel」などが結果を公開
ニューラルネットワークのパフォーマンス評価を実施する業界コンソーシアムのMLCommonsが、さまざまなAIモデルの推論スループットを測定するベンチマークスイートの最新バージョンである「MLPerf Inference v5.0」を公開しました。
gigazine.net
April 3, 2025 at 3:06 AM
AIインフラストラクチャの性能を測定するMLPerfのバージョン5.0が登場し「NVIDIA」「AMD」「Intel」などが結果を公開
https://gigazine.net/news/20250403-mlcommons-mlperf-inference-v5-0-benchmark-results/
https://gigazine.net/news/20250403-mlcommons-mlperf-inference-v5-0-benchmark-results/
venturebeat.com/ai/mlperf-4-...
AI keeps getting more powerful and it's not just about hardware either....
MLPerf 4.0 training results show up to 80% in AI performance gains
AI keeps getting more powerful and it's not just about hardware either....
MLPerf 4.0 training results show up to 80% in AI performance gains
MLPerf 4.0 training results show up to 80% in AI performance gains
MLPerf 4.0 benchmark demonstrate that it takes more than hardware to improve AI training.
venturebeat.com
June 13, 2024 at 1:56 AM
venturebeat.com/ai/mlperf-4-...
AI keeps getting more powerful and it's not just about hardware either....
MLPerf 4.0 training results show up to 80% in AI performance gains
AI keeps getting more powerful and it's not just about hardware either....
MLPerf 4.0 training results show up to 80% in AI performance gains
MLPerf Storage results are out and StorageReview pointed out the interesting disconnect between MLPerf leaderboard winners and companies with the largest #storage footprints in the #AI clouds. Suggests this benchmark may not reflect real needs.
www.storagereview.com/news/mlperf-...
www.storagereview.com/news/mlperf-...
MLPerf Storage v3.0: 877 GiB/s Checkpoints, a Cloud First, and a Leaderboard Turned Over
MLPerf Storage v3.0 results: 877 GiB/s checkpoints from Everpure FlashBlade//EXA, new KV cache and vector DB tests, 143 submissions.
www.storagereview.com
September 1, 2026 at 9:31 PM
MLPerf Storage results are out and StorageReview pointed out the interesting disconnect between MLPerf leaderboard winners and companies with the largest #storage footprints in the #AI clouds. Suggests this benchmark may not reflect real needs.
www.storagereview.com/news/mlperf-...
www.storagereview.com/news/mlperf-...
MLPerf now has an LLM post-training benchmark. It measures time-to-quality across training, inference, agent rollouts, sandboxes and evaluation, not raw GPU throughput. Pass@4 handles trajectory variance, while hidden tests keep the agent focused on fixing software. This is a systems benchmark.
MLPerf Training v6.1: First LLM Post-Training Benchmark
MLPerf Training v6.1 adds an agentic RL benchmark: teach a 397B-parameter model to repair real software. Reference run: 256 Blackwell Ultra GPUs.
mlcommons.org
September 25, 2026 at 4:21 PM
MLPerf now has an LLM post-training benchmark. It measures time-to-quality across training, inference, agent rollouts, sandboxes and evaluation, not raw GPU throughput. Pass@4 handles trajectory variance, while hidden tests keep the agent focused on fixing software. This is a systems benchmark.
Large language models #LLMs are growing extremely quickly, and the #hardware systems that they require can’t keep up with the pace. Each time #MLPerf introduces a new benchmark, training time increases. The data tells the story. spectrum.ieee.org/mlperf-trends
November 13, 2025 at 8:30 PM
Large language models #LLMs are growing extremely quickly, and the #hardware systems that they require can’t keep up with the pace. Each time #MLPerf introduces a new benchmark, training time increases. The data tells the story. spectrum.ieee.org/mlperf-trends
=__= ok it's late and I give up on this nonsense today. but's realllly irritating to be literally following the instructions for various reference implementations using the READMEs and docker etal and stuff still fails horribly 🤣 github.com/mlcommons/tr...
GitHub - mlcommons/training: Reference implementations of MLPerf™ training benchmarks
Reference implementations of MLPerf™ training benchmarks - mlcommons/training
github.com
December 14, 2024 at 1:51 AM
=__= ok it's late and I give up on this nonsense today. but's realllly irritating to be literally following the instructions for various reference implementations using the READMEs and docker etal and stuff still fails horribly 🤣 github.com/mlcommons/tr...
Went down a rabbithole of looking at what MLPerf Storage Training benchmarks do; turns out they're not very meaningful IMO. Alarming how many practitioners (mostly storage vendors) run them and don't understand what the results do/don't represent.
glennklockwood.com/garden/MLPer...
#AI #storage
glennklockwood.com/garden/MLPer...
#AI #storage
MLPerf Storage
Here are my notes on what MLPerf Storage does. See storage benchmarking is dumb as well.
glennklockwood.com
March 14, 2026 at 12:53 AM
Went down a rabbithole of looking at what MLPerf Storage Training benchmarks do; turns out they're not very meaningful IMO. Alarming how many practitioners (mostly storage vendors) run them and don't understand what the results do/don't represent.
glennklockwood.com/garden/MLPer...
#AI #storage
glennklockwood.com/garden/MLPer...
#AI #storage
MLPerf 6.1: more total output, less per device? Try this fictional comparison—not MLPerf results.
Source (16 Sep): mlcommons.org/2026/09/mlpe...
SignalSage by MerlaTech. Some features paid.
iPhone & Android: merlatech-stock-analysis.blogspot.com/p/download-s...
Source (16 Sep): mlcommons.org/2026/09/mlpe...
SignalSage by MerlaTech. Some features paid.
iPhone & Android: merlatech-stock-analysis.blogspot.com/p/download-s...
September 18, 2026 at 6:26 AM
MLPerf 6.1: more total output, less per device? Try this fictional comparison—not MLPerf results.
Source (16 Sep): mlcommons.org/2026/09/mlpe...
SignalSage by MerlaTech. Some features paid.
iPhone & Android: merlatech-stock-analysis.blogspot.com/p/download-s...
Source (16 Sep): mlcommons.org/2026/09/mlpe...
SignalSage by MerlaTech. Some features paid.
iPhone & Android: merlatech-stock-analysis.blogspot.com/p/download-s...
PCのAI性能を測定できるベンチマークアプリ「MLPerf Client」を世界的なAIベンチマーク機関「MLCommons」がリリース
https://gigazine.net/news/20250731-ai-mlperf-client-benchmark-pc/
https://gigazine.net/news/20250731-ai-mlperf-client-benchmark-pc/
PCのAI性能を測定できるベンチマークアプリ「MLPerf Client」を世界的なAIベンチマーク機関「MLCommons」がリリース
プライバシーやセキュリティを重視する環境では、ChatGPTなどのオンラインAIサービスではなく自分のデバイス上で処理できるローカルAIの方が適している場合があります。しかし、AIをローカルで実行するにはある程度の性能を備えたPCが必要です。新たに、AI関連業界団体のMLCommonsがPC向けのAIベンチマークアプリ「MLPerf Client」を2025年7月30日(水)にリリースしました。
gigazine.net
July 31, 2025 at 1:20 AM
PCのAI性能を測定できるベンチマークアプリ「MLPerf Client」を世界的なAIベンチマーク機関「MLCommons」がリリース
https://gigazine.net/news/20250731-ai-mlperf-client-benchmark-pc/
https://gigazine.net/news/20250731-ai-mlperf-client-benchmark-pc/
MLCommons Releases MLPerf Training v6.0 Results
Two Mixture-of-Experts (MoE) benchmarks were added, reflecting where the AI training frontier actually is.
📍 DeepSeek V3 — 671B params (largest in MLPerf history)
📍 GPT-OSS 20B — 21B params
mlcommons.org/2026/06/mlpe...
Two Mixture-of-Experts (MoE) benchmarks were added, reflecting where the AI training frontier actually is.
📍 DeepSeek V3 — 671B params (largest in MLPerf history)
📍 GPT-OSS 20B — 21B params
mlcommons.org/2026/06/mlpe...
June 16, 2026 at 5:55 PM
MLCommons Releases MLPerf Training v6.0 Results
Two Mixture-of-Experts (MoE) benchmarks were added, reflecting where the AI training frontier actually is.
📍 DeepSeek V3 — 671B params (largest in MLPerf history)
📍 GPT-OSS 20B — 21B params
mlcommons.org/2026/06/mlpe...
Two Mixture-of-Experts (MoE) benchmarks were added, reflecting where the AI training frontier actually is.
📍 DeepSeek V3 — 671B params (largest in MLPerf history)
📍 GPT-OSS 20B — 21B params
mlcommons.org/2026/06/mlpe...
NVIDIA B200とGoogle TrilliumがMLPerfベンチマークチャートに出現、B200はH100と比較し2倍のパフォーマンス
https://gigazine.net/news/20241115-mlperf-benchmark-nvidia-b200-google-trillium/
https://gigazine.net/news/20241115-mlperf-benchmark-nvidia-b200-google-trillium/
NVIDIA B200とGoogle TrilliumがMLPerfベンチマークチャートに出現、B200はH100と比較し2倍のパフォーマンス
AIベンチマークのMLPerf Ver.4.1が公開され、NVIDIA B200やGoogle Trilliumなどの新しいAIチップの結果が明らかとなりました。それによると、NVIDIAの次世代GPUであるB200は、現行のH100と比較して一部のテストで2倍の性能を達成し、Googleの新しいアクセラレータであるTrilliumは2023年にテストしたチップと比較して約4倍の性能向上を示しています。
gigazine.net
November 15, 2024 at 6:00 AM
NVIDIA B200とGoogle TrilliumがMLPerfベンチマークチャートに出現、B200はH100と比較し2倍のパフォーマンス
https://gigazine.net/news/20241115-mlperf-benchmark-nvidia-b200-google-trillium/
https://gigazine.net/news/20241115-mlperf-benchmark-nvidia-b200-google-trillium/
MLPerf Inference now measures multi-turn agents.
990 trajectories, Kimi K2.6 + Qwen3.6-35B-A3B, Pareto-curve performance, three-level accuracy.
Built on MLPerf Endpoints.
https://mlcommons.org/2026/07/agentic-inference-for-mlperf-inference/
#MLPerf #AgenticAI #LLM
990 trajectories, Kimi K2.6 + Qwen3.6-35B-A3B, Pareto-curve performance, three-level accuracy.
Built on MLPerf Endpoints.
https://mlcommons.org/2026/07/agentic-inference-for-mlperf-inference/
#MLPerf #AgenticAI #LLM
July 8, 2026 at 3:00 PM
MLPerf Inference now measures multi-turn agents.
990 trajectories, Kimi K2.6 + Qwen3.6-35B-A3B, Pareto-curve performance, three-level accuracy.
Built on MLPerf Endpoints.
https://mlcommons.org/2026/07/agentic-inference-for-mlperf-inference/
#MLPerf #AgenticAI #LLM
990 trajectories, Kimi K2.6 + Qwen3.6-35B-A3B, Pareto-curve performance, three-level accuracy.
Built on MLPerf Endpoints.
https://mlcommons.org/2026/07/agentic-inference-for-mlperf-inference/
#MLPerf #AgenticAI #LLM
大作。基礎から解説されているのでおすすめ。
knowledge.sakura.ad.jp/48499/
knowledge.sakura.ad.jp/48499/
AIスパコン「さくらONE」で挑むLLM・HPCベンチマーク (2) MLPerf GPT-3 175B事前学習性能検証 | さくらのナレッジ
さくらインターネット研究所の坪内(@yuuk1t)です。 本連載では、AIスパコン「さくらONE」で実施した各種ベンチマーク結果をご紹介しています。前回の第1回の記事では、MLPerfベンチマークの概要と、さくらONEに […]
knowledge.sakura.ad.jp
December 11, 2025 at 9:24 AM
大作。基礎から解説されているのでおすすめ。
knowledge.sakura.ad.jp/48499/
knowledge.sakura.ad.jp/48499/
@mlcommons.org's MLPerf Inference 5.0 is here — and the message is clear: LLMs now rule the benchmarking world: www.techarena.ai/content/ai-benchmarks-shift-as-mlperf-highlights-llm-dominance?utm_source=threads
AI Benchmarks Shift as MLPerf Highlights LLM Dominance
MLPerf Inference 5.0 signals the rise of large language models, with LLAMA 2 70B surpassing ResNet-50 in submissions and driving next-gen AI performance across compute platforms.
www.techarena.ai
April 2, 2025 at 4:54 PM
@mlcommons.org's MLPerf Inference 5.0 is here — and the message is clear: LLMs now rule the benchmarking world: www.techarena.ai/content/ai-benchmarks-shift-as-mlperf-highlights-llm-dominance?utm_source=threads
🚀 NEW: MLPerf Inference v6.0 debuts Qwen3-VL + Shopify Product Catalog benchmark
40M products daily. Real production data. First Qwen model in MLPerf.
Submit by Feb 13, 2026 →
https://bit.ly/4k9F5YS
#MLPerf #VLM #Shopify #MLCommons
40M products daily. Real production data. First Qwen model in MLPerf.
Submit by Feb 13, 2026 →
https://bit.ly/4k9F5YS
#MLPerf #VLM #Shopify #MLCommons
MLCommons MLPerf Inference v6.0 Qwen3-VL Shopify Catalog
text on a checkered background with mlcommons and shopify logos
mlcommons.org
February 2, 2026 at 3:19 PM
🚀 NEW: MLPerf Inference v6.0 debuts Qwen3-VL + Shopify Product Catalog benchmark
40M products daily. Real production data. First Qwen model in MLPerf.
Submit by Feb 13, 2026 →
https://bit.ly/4k9F5YS
#MLPerf #VLM #Shopify #MLCommons
40M products daily. Real production data. First Qwen model in MLPerf.
Submit by Feb 13, 2026 →
https://bit.ly/4k9F5YS
#MLPerf #VLM #Shopify #MLCommons
🤖 Beyond the Ground · AI & DeepTech
Anthropic's first flagship after 'pace the frontier': how routing architecture shifted the axis of frontier competition
🔗 Read the full investigation:
https://beyondtheground.com/article/445
#AI #DeepTech #BeyondTheGround
Anthropic's first flagship after 'pace the frontier': how routing architecture shifted the axis of frontier competition
🔗 Read the full investigation:
https://beyondtheground.com/article/445
#AI #DeepTech #BeyondTheGround
Anthropic's first flagship after 'pace the frontier': how routing architecture shifted…
Anthropic's new flagship doesn't win on benchmarks—it routes requests to smaller models, cutting costs. But its key metrics are self-reported and unverifiable by public benchmarks like MLPerf or HELM.
beyondtheground.com
September 23, 2026 at 5:26 PM
🤖 Beyond the Ground · AI & DeepTech
Anthropic's first flagship after 'pace the frontier': how routing architecture shifted the axis of frontier competition
🔗 Read the full investigation:
https://beyondtheground.com/article/445
#AI #DeepTech #BeyondTheGround
Anthropic's first flagship after 'pace the frontier': how routing architecture shifted the axis of frontier competition
🔗 Read the full investigation:
https://beyondtheground.com/article/445
#AI #DeepTech #BeyondTheGround
IEEE Spectrum's analysis shows how our benchmarks capture the real industry challenge - LLMs scale exponentially while hardware improves incrementally.
This pattern highlights why evolving benchmarks are important. Stay tuned for MLPerf Training v5.1, out on 11/12.
spectrum.ieee.org/mlperf-trends
This pattern highlights why evolving benchmarks are important. Stay tuned for MLPerf Training v5.1, out on 11/12.
spectrum.ieee.org/mlperf-trends
AI Model Growth Outpaces Hardware Improvements
AI training races are heating up as benchmarks get tougher.
spectrum.ieee.org
November 3, 2025 at 5:54 PM
IEEE Spectrum's analysis shows how our benchmarks capture the real industry challenge - LLMs scale exponentially while hardware improves incrementally.
This pattern highlights why evolving benchmarks are important. Stay tuned for MLPerf Training v5.1, out on 11/12.
spectrum.ieee.org/mlperf-trends
This pattern highlights why evolving benchmarks are important. Stay tuned for MLPerf Training v5.1, out on 11/12.
spectrum.ieee.org/mlperf-trends
"Makine Öğrenimi Olimpiyatları"
MLCommons, MLPerf 4.0 eğitim kıyaslamalarının sonuçlarını Google ve Intel yapay zeka hızlandırıcılarının da eklenmesiyle yayınladı.
Nvidia H100 GPU'lar dokuz kıyaslamanın tamamında birinci oldu. Nvidia rakipsiz.
spectrum.ieee.org/mlperf-nvidi...
MLCommons, MLPerf 4.0 eğitim kıyaslamalarının sonuçlarını Google ve Intel yapay zeka hızlandırıcılarının da eklenmesiyle yayınladı.
Nvidia H100 GPU'lar dokuz kıyaslamanın tamamında birinci oldu. Nvidia rakipsiz.
spectrum.ieee.org/mlperf-nvidi...
Nvidia Conquers Latest AI Tests
GPU maker tops new MLPerf benchmarks on graph neural nets and LLM fine-tuning
spectrum.ieee.org
June 13, 2024 at 5:17 AM
"Makine Öğrenimi Olimpiyatları"
MLCommons, MLPerf 4.0 eğitim kıyaslamalarının sonuçlarını Google ve Intel yapay zeka hızlandırıcılarının da eklenmesiyle yayınladı.
Nvidia H100 GPU'lar dokuz kıyaslamanın tamamında birinci oldu. Nvidia rakipsiz.
spectrum.ieee.org/mlperf-nvidi...
MLCommons, MLPerf 4.0 eğitim kıyaslamalarının sonuçlarını Google ve Intel yapay zeka hızlandırıcılarının da eklenmesiyle yayınladı.
Nvidia H100 GPU'lar dokuz kıyaslamanın tamamında birinci oldu. Nvidia rakipsiz.
spectrum.ieee.org/mlperf-nvidi...
AI専用サーバーの推論性能や動画生成性能を測定する「MLPerf Inference v6.0」の結果が公開される、NVIDIAとAMDの性能を一目で比較可能
https://gigazine.net/news/20260402-mlperf-inference-v6/
https://gigazine.net/news/20260402-mlperf-inference-v6/
AI専用サーバーの推論性能や動画生成性能を測定する「MLPerf Inference v6.0」の結果が公開される、NVIDIAとAMDの性能を一目で比較可能
AIサーバーの性能評価を実施している業界コンソーシアムのMLCommonsが推論性能ベンチマークテスト「MLPerf Inference v6.0」の結果を公開しました。MLPerf Inference v6.0は大規模言語モデルや動画生成モデルを実行した際の処理速度を測定するベンチマークテストで、すでにNVIDIAやAMDが自社製サーバーのベンチマーク結果を公開しています。
gigazine.net
April 2, 2026 at 2:55 AM
AI専用サーバーの推論性能や動画生成性能を測定する「MLPerf Inference v6.0」の結果が公開される、NVIDIAとAMDの性能を一目で比較可能
https://gigazine.net/news/20260402-mlperf-inference-v6/
https://gigazine.net/news/20260402-mlperf-inference-v6/