#MATH500
LightNews LightNews
shimmermathlabs.com
here is a PDF of 10 randomly selected problems from MATH-500 to test yourself. can you get 48.1% or higher and beat the little fixed Qwen3.5-0.8B that astra helped?

answers not included 😈

shimmermathlabs.com/math500-prac...
November 2, 2026 at 9:48 PM
aidailypost.com
webAI just dropped TwIL-8B, an 8.8B reasoning beast that hit 98.2 on MATH-500. Beats IBM Granite and tops GPQA Diamond. Check out the open-source details and see why the community’s buzzing! #TwIL8B #MATH500 #IBMGranite

🔗 aidailypost.com/news/webai-r...
November 4, 2026 at 11:43 PM
sparkllm.bsky.social
DSpark for Spark-X2.6-4B is here! ✨
Average BF16 speedup: 2.14x RTX 4090 / 2.50x M5 Pro.
❤️ Like & download!
ModelScope: https://modelscope.cn/models/XHToken/Spark-X2.6-4B-DSpark-GGUF
https://huggingface.co/XHToken/Spark-X2.6-4B-DSpark-GGUF
Spark-X2.6-4B DSpark: average BF16 speedup 2.14x on RTX 4090 and 2.50x on M5 Pro. Peak 2.94x on MATH500 BF16 M5 Pro; 532 tok/s on GSM8K Q4_K_M RTX 4090. Includes full benchmark and acceptance-length tables.
October 31, 2026 at 6:58 AM
victorgrosser.bsky.social
Good evening 🌙 Tonight's read: DTCA uses the denoising process itself to assign RL credit to tokens in diffusion LMs. On LLaDA-8B (no SFT), MATH500 goes from 36.4 to 43.2 at a 256-token budget, vs 41.2 for RLDF. Clever credit signal ✨ huggingface.co/papers/2610.35201
October 29, 2026 at 6:27 PM
paper.bsky.social
2506.06395
大規模言語モデル(LLM)は推論に優れているが、タスクのゴールとその動作を一致させるためには、事後学習が重要である。既存の強化学習（RL）手法は、多くの場合、コストのかかる人間のアノテーションや外部の報酬モデルに依存している。私たちは、ラベル、選好モデル、報酬工学の必要性を排除し、報酬シグナ...
大規模言語モデル(LLM)は推論に優れているが、タスクのゴールとその動作を一致させるためには、事後学習が重要である。 既存の強化学習（RL）手法は、多くの場合、コストのかかる人間のアノテーションや外部の報酬モデルに依存している。 私たちは、ラベル、選好モデル、報酬工学の必要性を排除し、報酬シグナルとしてモデル自身の自信を使用する、自己信頼による強化学習（RLSC）を提案する。 Qwen2.5-Math-7Bに、1問あたり16サンプル、10または20のトレーニングステップのみを適用した場合、RLSCはAIME2024で+13.4%、MATH500で+21.2%、Minerva Mathで+21.7%、Olympiadbenchで+20.8%、AMC23で+9.7%精度を向上させた。 RLSCは、推論モデルのためのシンプルでスケーラブルなポストトレーニングメソッドを提供する。
July 1, 2025 at 12:07 AM
paper.bsky.social
2505.05315
大規模推論モデル(LRM)は、拡張思考連鎖(CoT)を生成することにより、複雑なタスクにおいて目覚ましい進歩を遂げてきた。しかし、トークン、レイテンシ、コンピュートなどの推論時間バジェットが厳しく制限されている実世界での展開には、出力長を制御できないことが大きな課題となる。我々は、スケーラブルな...
大規模推論モデル(LRM)は、拡張思考連鎖(CoT)を生成することにより、複雑なタスクにおいて目覚ましい進歩を遂げてきた。 しかし、トークン、レイテンシ、コンピュートなどの推論時間バジェットが厳しく制限されている実世界での展開には、出力長を制御できないことが大きな課題となる。 我々は、スケーラブルな思考の連鎖のための新しいフレームワークであるElastic Reasoningを提案する。Elastic Reasoningは、推論を2つのフェーズ（思考と解決）に明確に分離し、独立に割り当てられた予算を持つ。 テスト時には、Elastic Reasoningはソリューションセグメントの完全性を優先し、厳しいリソース制約の下で信頼性を大幅に向上させる。 これは、思考プロセスが短縮されたときに適応的に推論するようにモデルを学習し、追加の訓練なしで未知の予算制約に効果的に汎化する。 数学（AIME、MATH500）およびプログラミング（LiveCodeBench、Codeforces）ベンチマークでの実証結果は、Elastic Reasoningが厳しい予算制約の下でも堅牢に動作し、ベースライン手法よりも学習コストが大幅に低いことを示している。 驚くべきことに、私たちのアプローチは、制約のない設定においても、より簡潔で効率的な推論を生み出す。 我々のコードはhttps://github.com/SalesforceAIResearch/Elastic-Reasoning。
May 31, 2025 at 2:50 AM
paper.bsky.social
[28/30] 120 Likes, 11 Comments, 5 Posts
2504.20571, cs․LG | cs․AI | cs․CL, 29 Apr 2025

🆕Reinforcement Learning for Reasoning in Large Language Models with One Training Example

Yiping Wang, Qing Yang, Zhiyuan Zeng, Liliang Ren, Lucas Liu, Baolin Peng, Hao Cheng, Xuehai He, Kuan Wang, Jianfen...
We show that reinforcement learning with verifiable reward using one training example (1-shot RLVR) is effective in incentivizing the math reasoning capabilities of large language models (LLMs). Applying RLVR to the base model Qwen2.5-Math-1.5B, we identify a single example that elevates model performance on MATH500 from 36.0% to 73.6%, and improves the average performance across six common mathematical reasoning benchmarks from 17.6% to 35.7%. This result matches the performance obtained using the 1.2k DeepScaleR subset (MATH500: 73.6%, average: 35.9%), which includes the aforementioned example. Similar substantial improvements are observed across various models (Qwen2.5-Math-7B, Llama3.2-3B-Instruct, DeepSeek-R1-Distill-Qwen-1.5B), RL algorithms (GRPO and PPO), and different math examples (many of which yield approximately 30% or greater improvement on MATH500 when employed as a single training example). In addition, we identify some interesting phenomena during 1-shot RLVR, including cross-domain generalization, increased frequency of self-reflection, and sustained test performance improvement even after the training accuracy has saturated, a phenomenon we term post-saturation generalization. Moreover, we verify that the effectiveness of 1-shot RLVR primarily arises from the policy gradient loss, distinguishing it from the "grokking" phenomenon. We also show the critical role of promoting exploration (e.g., by adding entropy loss with an appropriate coefficient) in 1-shot RLVR training. As a bonus, we observe that applying entropy loss alone, without any outcome reward, significantly enhances Qwen2.5-Math-1.5B's performance on MATH500 by 27.4%. These findings can inspire future work on RLVR data efficiency and encourage a re-examination of both recent progress and the underlying mechanisms in RLVR. Our code, model, and data are open source at https://github.com/ypwang61/One-Shot-RLVR
May 6, 2025 at 12:05 AM
saxon.me
In order to sample a more balanced distribution of questions across the difficulty spectrum, we introduce DUMB500, the Waluigi to MATH500 which consists of stupid easy Qs.

This way we can get a more comprehensive view of overthinking, from the hardest GPQA and ZebraLogic Qs to literally "2+2=?"
April 21, 2025 at 11:21 PM
adinayakup.bsky.social
New dataset 🔥

UltraData-Math 🔢 math pre-training dataset released by OpenBMB

huggingface.co/datasets/ope...

✨ Apache2.0
✨ 290B+ tokens with tiered quality (L1 → L3)
✨ Math-aware parsing that keeps formulas intact
✨ Proven to improve math reasoning (GSM8K & MATH500)
openbmb/UltraData-Math · Datasets at Hugging Face
We’re on a journey to advance and democratize artificial intelligence through open source and open science.
huggingface.co
February 10, 2026 at 9:01 AM
swanai.bsky.social
1/ In reasoning mode, the model achieved impressive benchmarks with 72.1% on AIME25, 97.8% on MATH500, 64.0% on GPQA, and 71.1% on LiveCodeBench.
August 20, 2025 at 5:00 AM
swanai.bsky.social
Nvidia launches Nemotron-Nano-9B-V2, a 9B parameter model that achieved 72.1% on AIME25 and 97.8% on MATH500 while fitting on a single A10 GPU

Here are the key takeaways:
Thread media
August 20, 2025 at 5:00 AM
alpharee.mastodon.social.ap.brid.gy
A team at UC Berkeley has trained Sky-T1-32B-Preview, a reasoning model that matches the performance of O1-Preview on popular reasoning and coding benchmarks.
What makes this even more remarkable is that the training was completed in 19 hours for less than […]

[Original post on mastodon.social]
Sky-T1-32B-Preview Qwen-2.5-32B-Instruct QwQ o1-preview Math500 82.4 76.2 85.4 81.4 AIME2024 43.3 16.7 50.0 40.0 LiveCodeBench-Easy 86.3 84.6 90.7 92.9 LiveCodeBench-Medium 56.8 40.8 56.3 54.9 LiveCodeBench-Hard 17.9 9.8 17.1 16.3 GPQA-Diamond 56.8 45.5 52.5 75.2
January 14, 2025 at 1:57 PM
jandotai.bsky.social
LN-Ultra introduces a system prompt ("detailed thinking on/off") to switch between standard chat and multi-step reasoning. No separate models. On GPQA-Diamond, accuracy jumps from 46% (chat) to 76% (reasoning). Same pattern holds for MATH500 (80% → 97%) and LiveCodeBench.
May 6, 2025 at 10:28 AM
cscv-bot.bsky.social
parameters, delivers competitive performance, achieving a score of 69.0 on the MMMU benchmark and 67.5 on MathVista. Meanwhile, it maintains robust textual reasoning performance, evidenced by impressive scores of 72.0 on AIME and 94.0 on MATH500. The [5/6 of https://arxiv.org/abs/2504.05599v1]
April 9, 2025 at 5:59 AM
fsconklin.bsky.social
UC Berkeley Researchers Released Sky-T1-32B-Preview: An Open-Source Reasoning LLM Trained for Under $450 Surpasses OpenAI-o1 on Benchmarks like Math500, AIME, and Livebench
January 14, 2025 at 6:51 AM
superposition.bsky.social
Group Reasoning Boosts AI Response Quality With Limited Tokens
Recent advances demonstrate that employing a modular thinking strategy, facilitated by the MOTIF reinforcement learning training method, enhances the reasoning capabilities of large language models like Qwen2.5, achieving up to 3.8% accuracy improvements on complex mathematical benchmarks such as MATH500 and AIME2024, while simultaneously increasing sample efficiency by utilising only 15% of training data.
quantumzeitgeist.com
July 6, 2025 at 9:13 AM
ancadweb.com
On n'ira pas nier qu'il y a de gros progrès vérifiables des LLMs en maths, qui confirment plus ou moins ceux de leurs résultats dans des benchmarks comme Math500.
( www.vals.ai/benchmarks/m... )

(je remet le lien dessous car buggé par le smiley dans son message)
gemini.google.com/share/364afc...
July 14, 2025 at 12:03 PM
getnews-me.bsky.social
SelfBudgeter predicts a token budget before reasoning, cutting response length by 61 % for a 1.5 B model and 48 % for a 7 B model on benchmarks like GSM8K, MATH500 and AIME2025. https://getnews.me/selfbudgeter-introduces-adaptive-token-budgets-for-llm-reasoning/ #selfbudgeter #llm #tokenbudget
SelfBudgeter Introduces Adaptive Token Budgets for LLM Reasoning
October 6, 2025 at 1:54 PM
getnews-me.bsky.social
Guided Speculative Inference (GSI) adds an auxiliary decoder and reward‑guided rescoring, delivering higher accuracy on MATH500 while cutting inference cost. Read more: https://getnews.me/guided-speculative-inference-boosts-test-time-alignment-of-llms/ #guidedspeculativeinference #llmalignment
Guided Speculative Inference Boosts Test‑Time Alignment of LLMs
October 3, 2025 at 8:14 AM
getnews-me.bsky.social
Researchers introduced Adaptive Monte Carlo Search (AMCS), which allocates more samples to uncertain steps, raising a Process Reward Model to 76.2% accuracy on the MATH500 benchmark with GLM‑4‑9B. https://getnews.me/adaptive-monte-carlo-search-enhances-ai-math-process-supervision/ #amcs #math500
Adaptive Monte Carlo Search Enhances AI Math Process Supervision
September 30, 2025 at 8:21 PM
getnews-me.bsky.social
FR-Ponder uses a sub-million-parameter controller to add steering vectors, letting LLMs allocate steps, cut FLOPs while maintaining accuracy on GSM8K and MATH500 benchmarks. https://getnews.me/fr-ponder-enables-adaptive-reasoning-depth-for-large-language-models/ #frponder #llm
FR-Ponder Enables Adaptive Reasoning Depth for Large Language Models
September 30, 2025 at 7:09 PM
getnews-me.bsky.social
A new study shows that LLMs trained with shared decision pivots improve accuracy on LogiQA, MedQA and MATH500 benchmarks, producing shorter, more reliable chain‑of‑thought explanations. Read more: https://getnews.me/shared-decision-pivots-boost-llm-reasoning-accuracy/ #llm #chainofthought
Shared Decision Pivots Boost LLM Reasoning Accuracy
September 29, 2025 at 6:20 AM
getnews-me.bsky.social
The d2 framework improves diffusion language models, achieving state‑of‑the‑art results on Countdown, Sudoku, GSM8K and MATH500 using only reinforcement learning. https://getnews.me/d2-framework-boosts-reasoning-capabilities-of-diffusion-language-models/ #dlm #reasoning #ml
d2 Framework Boosts Reasoning Capabilities of Diffusion Language Models
September 29, 2025 at 5:34 AM
getnews-me.bsky.social
IGPO adds brief verified reasoning fragments to diffusion language models. Tested on GSM8K, Math500 and AMC, it achieved new state-of-the-art accuracy on these math benchmarks. https://getnews.me/inpainting-guided-policy-optimization-boosts-diffusion-llm-performance/ #inpainting #diffusionllm
Inpainting-Guided Policy Optimization Boosts Diffusion LLM Performance
September 17, 2025 at 6:39 AM
saylortwift.hf.co
Results on AIME show only a small drop when using the 2025 version, suggesting the model isn't contaminated on AIME 2024 and results on MATH500 show strong math skills.

Instruction following? Extremely strong. 📈 Claude excels at understanding and executing complex prompts.

2/3
February 25, 2025 at 3:03 PM