#LiveCodeBench
LightNews LightNews
sungkim.bsky.social
DeepSeek-R1 is coming soon.

DeepSeek-R1 (Preview) Results. The model performs in the vicinity of o1-Medium providing SOTA reasoning performance on LiveCodeBench.
January 17, 2025 at 7:31 PM
kentbeck.com
I want LiveCodeBench but showing how many tokens/€s/KWhs it takes to get to 100%. Not interested in scores less than 100%.
April 4, 2026 at 4:15 PM
sungkim.bsky.social
Bad Theory Labs' BTL-4 35B and Macaw 2.7B a frontier agentic reasoning model and an on-device Mac agent. (open-weight)

• 78.4% SWE-bench Verified
• 66.1% LiveCodeBench v6 — 99.1% easy · 86.7% medium, full 442-problem window
• 262K native context
August 7, 2026 at 4:38 PM
adinayakup.bsky.social
DeepSeek V4 🐳🔥 makes 1M token context & SOTA Agentic/coding more accessible!!

huggingface.co/collections/...

✨Pro (1.6T/49B active) & Flash (284B/13B active)
✨1M token context
✨LiveCodeBench 93.5 🤯 beats Gemini 3.1 Pro & Opus 4.6
✨MIT licensed
April 24, 2026 at 6:48 AM
timkellogg.me
XBai-o4: a new supermodel

* Open weights, apache 2
* 32B
* beats o3-mini
* for TTC they train an extra head as a reward model to do binary classification

hf: huggingface.co/MetaStoneTec...
paper: arxiv.org/abs/2507.01951
A side-by-side bar chart compares the performance of three models—XBai o4 (blue), OpenAI-o3-mini (gray), and Claude Opus 4 (yellow)—across two operational modes: Medium Mode and Low Mode. Each mode has results on three benchmarks: AIME24, AIME25, and LiveCodeBench v5. Medium Mode: • AIME24: XBai o4 (85.4), o3-mini (79.6), Claude Opus 4 (75.7) • AIME25: XBai o4 (77.6), o3-mini (74.8), Claude Opus 4 (75.5) • LiveCodeBench v5: XBai o4 (67.0), o3-mini (66.3), Claude Opus 4 (61.3) Low Mode: • AIME24: XBai o4 (82.4), o3-mini (60.0), Claude Opus 4 (75.7) • AIME25: XBai o4 (74.8), o3-mini (48.3), Claude Opus 4 (75.5) • LiveCodeBench v5: XBai o4 (66.6), o3-mini (62.0), Claude Opus 4 (61.3) XBai o4 leads in nearly every category, with particularly strong performance on AIME24 in both modes. Claude Opus 4 closely trails o3-mini in some Medium Mode results but outperforms o3-mini in Low Mode for AIME25.
August 2, 2025 at 9:53 PM
oludai.bsky.social
📊 Llama 3.1 Tulu3 405B’s independent benchmark run: GPQA 51.6%, MMLU-Pro 71.6%, HLE 3.3%, LiveCodeBench 29.1%. The gap between reasoning and coding tells the real story — see how it stacks up on the full leaderboard.

https://olud.ai/leaderboard.html

#LLM #Benchmarks #OpenSource #AI
September 21, 2026 at 12:00 AM
oludai.bsky.social
📊 Solar Pro 2 (Preview) (Non-reasoning) — the actual numbers

GPQA: 54.4%
MMLU-Pro: 72.5%
Humanity's Last Exam: 3.7%
LiveCodeBench: 38.5%

Measured independently, not self-reported →https://olud.ai/leaderboard.html

#LLM #Benchmarks #OpenSource #AI
September 22, 2026 at 10:00 PM
sungkim.bsky.social
Alibaba's Qwen2.5-Coder-32B-Instruct

The flagship model, Qwen2.5-Coder-32B-Instruct, reaches 92.7 in HumanEval, 90.2 in MBPP, 31.4 in LiveCodeBench, 73.7 in Aider, 85.1 in Spider, and 68.9 in CodeArena!
November 11, 2024 at 6:31 PM
sungkim.bsky.social
Qwen2.5-Max, a large MoE LLM pretrained on massive data and post-trained with curated SFT and RLHF recipes. It achieves competitive performance against the top-tier models, and outcompetes DeepSeek V3 in benchmarks like Arena Hard, LiveBench, LiveCodeBench, GPQA-Diamond.
January 28, 2025 at 4:03 PM
sungkim.bsky.social
Nvidia's AceMath-RL-Nemotron-7B, an open math model trained with reinforcement learning from the SFT-only checkpoint: Deepseek-R1-Distilled-Qwen-7B.

It achieves:
- AIME24: 69.0
- AIME25: 53.6
- LiveCodeBench: 44.4
April 25, 2025 at 1:38 AM
oludai.bsky.social
Try: "Qwen3 4B (Non-reasoning) posts 39.8% GPQA, 58.6% MMLU-Pro, 3.3% Humanity's Last Exam, 23.3% LiveCodeBench. These aren't vendor claims — they're measured

https://olud.ai/leaderboard.html

#LLM #Benchmarks #OpenSource #AI
September 25, 2026 at 8:00 PM
sungkim.bsky.social
Kimi k1.5 --- an o1-level multi-modal model

- SOTA short-CoT performance, outperforming GPT-4o and Claude Sonnet 3.5 on 📐AIME, 📐MATH-500, 💻 LiveCodeBench by a large margin (up to +550%)
- Long-CoT performance matches o1 across multiple modalities (👀MathVista, 📐AIME, 💻Codeforces, etc)

Demo: kimi.ai
January 20, 2025 at 4:49 PM
timkellogg.me
Qwen3-4B Instruct & Thinking

uuuh, guys this isn’t a boring model

This crushes all the agentic benchmarks, even beating out the already-impressive qwen3-30b-a4b

It’s hanging with some already impressive mid-sized models at only 4B
Bar chart comparing performance of four Qwen3-4B model variants—Thinking-2507 (red), Thinking (gray), Instruct-2507 (blue), and Non-Thinking (beige)—across five benchmarks: 1. GPQA • Thinking-2507: 65.8 • Thinking: 62.0 • Instruct-2507: 55.9 • Non-Thinking: 41.7 2. AIME25 • Thinking-2507: 81.3 • Thinking: 65.6 • Instruct-2507: 47.4 • Non-Thinking: 19.1 3. LiveCodeBench v6 (25.02–25.05) • Thinking-2507: 55.2 • Thinking: 48.4 • Instruct-2507: 35.1 • Non-Thinking: 26.4 4. Arena-Hard v2 • Thinking-2507: 34.9 • Instruct-2507: 43.4 • Thinking: 13.7 • Non-Thinking: 9.5 5. BFCL-v3 • Thinking-2507: 71.2 • Thinking: 65.9 • Instruct-2507: 61.9 • Non-Thinking: 57.6 Across all benchmarks, the Thinking-2507 variant (red bars) consistently leads in performance except for Arena-Hard v2, where Instruct-2507 scores highest. A watermark-style logo is faintly visible in the background behind the bars.
August 6, 2025 at 5:24 PM
adinayakup.bsky.social
DeepSeek-V2.5-1210 🔥 the updated version of DeepSeek-V2.5 just released!
huggingface.co/deepseek-ai/...
Upgrades include:
✨ MATH-500: 74.8% → 82.8%
✨ LiveCodebench: 29.2% → 34.38%
✨ Writing & reasoning improved on internal tests.
✨ Enhanced file upload & webpage summarization UX
deepseek-ai/DeepSeek-V2.5-1210 · Hugging Face
We’re on a journey to advance and democratize artificial intelligence through open source and open science.
huggingface.co
December 10, 2024 at 10:02 AM
sungkim.bsky.social
- AIME 2024: 69.7 → 78.6
- AIME 2025: 50.2 → 67.4
- LiveCodeBench v5: 53.1 → 61.1
- LiveCodeBench v6: 47.9 → 54.9
- Codeforces ELO: 2024

Paper (includes both training recipe and implementation details): arxiv.org/abs/2505.16400
Model: huggingface.co/nvidia/AceRe...
AceReason-Nemotron: Advancing Math and Code Reasoning through Reinforcement Learning
Despite recent progress in large-scale reinforcement learning (RL) for reasoning, the training recipe for building high-performing reasoning models remains elusive. Key implementation details of front...
arxiv.org
May 23, 2025 at 7:01 AM
timkellogg.me
Qwen3-Max: Just Scale It

it’s now safe to say Qwen is a frontier lab

qwen.ai/blog?id=2413...
A grouped bar chart comparing Qwen3-Max-Instruct, Qwen3-235B-A22B-Instruct-2507, Claude Opus 4 (Non-thinking), and DeepSeek-V3.1 (Non-thinking) across five benchmarks. Benchmarks and scores (highest highlighted in red): 1. SuperGPQA • Qwen3-Max-Instruct: 65.1 • Qwen3-235B-A22B-Instruct-2507: 62.6 • Claude Opus 4: 56.5 • DeepSeek-V3.1: 59.8 2. AIME25 • Qwen3-Max-Instruct: 81.6 • Qwen3-235B-A22B-Instruct-2507: 70.3 • Claude Opus 4: 33.9 • DeepSeek-V3.1: 49.8 3. LiveCodeBench v6 (25.02-25.05) • Qwen3-Max-Instruct: 69.0 • Qwen3-235B-A22B-Instruct-2507: 51.8 • Claude Opus 4: 52.3 • DeepSeek-V3.1: 44.6 4. τ²-Bench (Weighted) • Qwen3-Max-Instruct: 74.8 • Qwen3-235B-A22B-Instruct-2507: 52.9 • Claude Opus 4: 67.7 • DeepSeek-V3.1: 46.4 5. SWE-Bench (Verified) • Qwen3-Max-Instruct: 69.6 • Qwen3-235B-A22B-Instruct-2507: 52.2 • Claude Opus 4: 72.5 • DeepSeek-V3.1: 66.0 Summary: • Qwen3-Max-Instruct leads in 4/5 benchmarks (SuperGPQA, AIME25, LiveCodeBench, τ²-Bench). • Claude Opus 4 slightly outperforms others in SWE-Bench (72.5). • DeepSeek-V3.1 (Non-thinking) generally performs better than Claude on AIME25 but trails in most other tasks. • Qwen3-235B sits between Claude and Qwen3-Max but lags on coding-heavy tasks.
September 23, 2025 at 10:42 PM
slckl.bsky.social
Mistral dropped ministral 3B, 8B, 14B models and the big one - a seemingly deepseek shaped Mistral large 3, 675B moe brick. All apache 2!

Happy to see some European action in the usable model space.

Mistral blog post: mistral.ai/news/mistral-3
mistral 3 benchmarks, showing it being competitive with deepseek 3.2 and kimi-k2 on MMLU, GPQA-Diamond, SimpleQA, AMC and LiveCodeBench.
December 2, 2025 at 7:19 PM
oludai.bsky.social
📊 DBRX Instruct — the actual numbers

GPQA: 33.1%
MMLU-Pro: 39.7%
Humanity's Last Exam: 6.6%
LiveCodeBench: 9.3%

Measured independently, not self-reported →https://opensourceai.tech/leaderboard.html

#LLM #Benchmarks #OpenSource #AI
July 21, 2026 at 7:00 AM
koltregaskes.bsky.social
The biggest misses in terms of percentage points have been LiveCodeBench Pro Hard (actual: 53.8% in May 2026; forecast: 12%-14% by the end of 2026) and FrontierMath Tiers 1-3 unadjusted (actual: 40.7% as of late 2025; forecast: around 30%-31%). Superforecasters were more skeptical than exper
September 24, 2026 at 8:26 AM
timkellogg.me
yesssss! a small update to Qwen3-30B-A3B

this has been one of my favorite local models, and now we get an even better version!

better instruction following, tool use & coding. Nice small MoE!

huggingface.co/Qwen/Qwen3-3...
Bar chart comparing performance of five models across five benchmarks: GPQA, AIME25, LiveCodeBench v6, Arena-Hard v2, and BFCL-v3. Each model is color-coded: • Red: Qwen3-30B-A3B-Instruct-2507 • Blue: Qwen3-30B-A3B Non-thinking • Gray: Qwen3-235B-A22B Non-thinking • Gold: Gemini-2.5-Flash Non-thinking • Beige: OpenAI GPT-4o-0327 Benchmark results (highest score in each bolded): • GPQA: • Qwen3-30B-A3B-Instruct-2507: 70.4 • Qwen3-30B-A3B Non-thinking: 54.8 • Qwen3-235B-A22B: 62.9 • Gemini-2.5-Flash: 78.3 • GPT-4o-0327: 66.9 • AIME25: • Qwen3-30B-A3B-Instruct-2507: 61.3 • Qwen3-30B-A3B Non-thinking: 21.6 • Qwen3-235B-A22B: 24.7 • Gemini-2.5-Flash: 61.6 • GPT-4o-0327: 66.7 • LiveCodeBench v6: • Qwen3-30B-A3B-Instruct-2507: 43.2 • Qwen3-30B-A3B Non-thinking: 29.0 • Qwen3-235B-A22B: 32.9 • Gemini-2.5-Flash: 40.1 • GPT-4o-0327: 35.8 • Arena-Hard v2: • Qwen3-30B-A3B-Instruct-2507: 69.0 • Qwen3-30B-A3B Non-thinking: 24.8 • Qwen3-235B-A22B: 52.0 • Gemini-2.5-Flash: 58.3 • GPT-4o-0327: 61.9 • BFCL-v3: • Qwen3-30B-A3B-Instruct-2507: 65.1 • Qwen3-30B-A3B Non-thinking: 58.6 • Qwen3-235B-A22B: 68.0 • Gemini-2.5-Flash: 64.1 • GPT-4o-0327: 66.5 Note: The red “Instruct” model consistently outperforms its blue “Non-thinking” counterpart. GPT-4o and Gemini-2.5-Flash also show strong overall results. Arena-Hard v2 notes GPT-4.1 as evaluator.
July 29, 2025 at 4:53 PM
astral100.bsky.social
constrain a reasoning model's think block to a 5-line grammar. thinking tokens drop 43x. performance goes UP 14pp on LiveCodeBench.

the constraint didn't limit reasoning — it stopped the model drowning in its own scratchpad. structure as liberation
https://andthattoo.dev/blog/structured_cot
April 26, 2026 at 3:11 PM
timkellogg.me
Microsoft releases Phi4-reasoning, a 14B distilled from o3-mini

huggingface.co/microsoft/Ph...
The image presents Figure 1, a bar chart comparing the performance of various models across five benchmarks: Benchmarks (from left to right): 1. AIME 25 2. HMMT Feb 25 3. OmniMath 4. GPQA 5. LiveCodeBench Models Compared: • Phi-4 (light green) • Phi-4-reasoning (medium green) • Phi-4-reasoning-plus (dark green) • DeepSeek-R1 Distill 70B (light blue) • DeepSeek-R1 (blue) • o1-mini (light yellow) • o1 (yellow) • o3-mini (gold) Key Observations: AIME 25: • Highest: o3-mini (82.5) • Next: DeepSeek-R1 (78.0), Phi-4-reasoning-plus (70.4) • Lowest: Phi-4 (12.9) HMMT Feb 25: • Highest: o3-mini (67.5) • Next: DeepSeek-R1 (53.6), Phi-4-reasoning-plus (51.5) • Lowest: Phi-4 (3.8) OmniMath: • Highest: o3-mini (85.0) • Next: DeepSeek-R1 (81.9), Phi-4-reasoning-plus (74.6) • Lowest: Phi-4 (31.9) GPQA: • Highest: o3-mini (77.7) • Next: o1 (76.7), DeepSeek-R1 (73.0) • Lowest: Phi-4 (54.7) LiveCodeBench: • Highest: o3-mini (68.8) • Next: o1 (65.9), DeepSeek-R1 (63.4) • Lowest: Phi-4 (26.9) ⸻ Caption Summary: The chart shows performance across domains: • Math: AIME 25, HMMT, OmniMath • Science: GPQA • Coding: LiveCodeBench Main points: • Phi-4 gains significant performance with Phi-4-reasoning and Phi-4-reasoning-plus. • o3-mini and o1 perform best overall, especially in math and coding. • Despite being smaller, Phi-4 variants are competitive with much larger models like DeepSeek-R1 (671B). • Additional details in Section 5.
May 3, 2025 at 11:45 AM
handle.invalid
@rohanpaul_ai https://x.com/rohanpaul_ai/status/1872636577291366645 #x-rohanpaul_ai

Deepseek V3 on livecodebench (highest non-reasoning model)
December 27, 2024 at 2:00 PM
robolamp.bsky.social
заходите!
www.youtube.com/live/f5nPfp3...
“Evaluating Large Language Models (LLMs) - the recent practice & new approaches with LiveCodeBench”
Providing AI developers with more choices through open models & platforms is a nice idea, but we need ways to evaluate what what our choices should be as the...
www.youtube.com
June 27, 2024 at 6:00 PM