Gemini 3 leak shows solid gains on math, vision, and SimpleQA
Sonnet still ahead on SWE-bench though, while Gemini takes TerminalBench
Nice to see models getting better at different things
Sonnet still ahead on SWE-bench though, while Gemini takes TerminalBench
Nice to see models getting better at different things
November 18, 2025 at 2:41 PM
Gemini 3 leak shows solid gains on math, vision, and SimpleQA
Sonnet still ahead on SWE-bench though, while Gemini takes TerminalBench
Nice to see models getting better at different things
Sonnet still ahead on SWE-bench though, while Gemini takes TerminalBench
Nice to see models getting better at different things
Meituan, China's largest food delivery company, has decided to enter the open-weight LLM game.
LongCat-Flash-Chat!
▫️ 560B Total Params | 18.6B-31.3B Dynamic Activation
▫️ Trained on 20T Tokens | 100+ tokens/sec Inference
▫️ High Performance: TerminalBench 39.5 | τ²-Bench 67.7
LongCat-Flash-Chat!
▫️ 560B Total Params | 18.6B-31.3B Dynamic Activation
▫️ Trained on 20T Tokens | 100+ tokens/sec Inference
▫️ High Performance: TerminalBench 39.5 | τ²-Bench 67.7
August 31, 2025 at 11:43 PM
Meituan, China's largest food delivery company, has decided to enter the open-weight LLM game.
LongCat-Flash-Chat!
▫️ 560B Total Params | 18.6B-31.3B Dynamic Activation
▫️ Trained on 20T Tokens | 100+ tokens/sec Inference
▫️ High Performance: TerminalBench 39.5 | τ²-Bench 67.7
LongCat-Flash-Chat!
▫️ 560B Total Params | 18.6B-31.3B Dynamic Activation
▫️ Trained on 20T Tokens | 100+ tokens/sec Inference
▫️ High Performance: TerminalBench 39.5 | τ²-Bench 67.7
the benchmark scores are inexplicably >100% of full-precision on a few of them, and more like 75% on TerminalBench 2.1 and SWE-bench verified, which the whitepaper acknowledges. overall much more noise than signal in benchmarks in general
September 19, 2026 at 7:15 AM
the benchmark scores are inexplicably >100% of full-precision on a few of them, and more like 75% on TerminalBench 2.1 and SWE-bench verified, which the whitepaper acknowledges. overall much more noise than signal in benchmarks in general
In high-sparsity scenarios, N-gram embeddings yield a better Pareto frontier than just adding more MoE experts.
LongCat-Flash-Lite - a model built on this insight.
⚙️ 68.5B Total Params(37.13B non-embedding) | 2.9B~4.5B Active
📊 High Performance: SWE-Bench 54.4 | τ²-Bench 72.8 | TerminalBench 33.75
LongCat-Flash-Lite - a model built on this insight.
⚙️ 68.5B Total Params(37.13B non-embedding) | 2.9B~4.5B Active
📊 High Performance: SWE-Bench 54.4 | τ²-Bench 72.8 | TerminalBench 33.75
January 29, 2026 at 2:58 AM
In high-sparsity scenarios, N-gram embeddings yield a better Pareto frontier than just adding more MoE experts.
LongCat-Flash-Lite - a model built on this insight.
⚙️ 68.5B Total Params(37.13B non-embedding) | 2.9B~4.5B Active
📊 High Performance: SWE-Bench 54.4 | τ²-Bench 72.8 | TerminalBench 33.75
LongCat-Flash-Lite - a model built on this insight.
⚙️ 68.5B Total Params(37.13B non-embedding) | 2.9B~4.5B Active
📊 High Performance: SWE-Bench 54.4 | τ²-Bench 72.8 | TerminalBench 33.75
Qwen3.8-Max: a 2.4T soon-to-be-open beast of a model that truly rivals Fable and Sol
qwen.ai/blog?id=qwen...
qwen.ai/blog?id=qwen...
August 3, 2026 at 4:10 AM
Qwen3.8-Max: a 2.4T soon-to-be-open beast of a model that truly rivals Fable and Sol
qwen.ai/blog?id=qwen...
qwen.ai/blog?id=qwen...
Terminal-Bench-Science 0.1 was announced in August 2026, built by @StevenDillmann and researchers at Stanford with the @terminalbench team and a global community of open source scientific contributors.
September 25, 2026 at 12:16 AM
Terminal-Bench-Science 0.1 was announced in August 2026, built by @StevenDillmann and researchers at Stanford with the @terminalbench team and a global community of open source scientific contributors.
wow that terminalbench result is *way* lower than what openai reported
April 23, 2026 at 10:42 PM
wow that terminalbench result is *way* lower than what openai reported
GPT-5.6: Sol, Terra & Luna
3 model sizes, like Anthropic does. Sol appears to be a Mythos class model at peak reasoning
Also comes with max reasoning effort and a new ultra mode that uses subagents server-side behind the API
openai.com/index/previe...
3 model sizes, like Anthropic does. Sol appears to be a Mythos class model at peak reasoning
Also comes with max reasoning effort and a new ultra mode that uses subagents server-side behind the API
openai.com/index/previe...
June 26, 2026 at 8:25 PM
GPT-5.6: Sol, Terra & Luna
3 model sizes, like Anthropic does. Sol appears to be a Mythos class model at peak reasoning
Also comes with max reasoning effort and a new ultra mode that uses subagents server-side behind the API
openai.com/index/previe...
3 model sizes, like Anthropic does. Sol appears to be a Mythos class model at peak reasoning
Also comes with max reasoning effort and a new ultra mode that uses subagents server-side behind the API
openai.com/index/previe...
Yeah I think basically bash is the strongest meta-interface to anything for a LLM. "everything is a file" includes virtual abstractions created inside that virtual fs. The file system *is* the interface. The LLMs are tuned so hard on TerminalBench they are shell ops masters.
June 7, 2026 at 6:40 PM
Yeah I think basically bash is the strongest meta-interface to anything for a LLM. "everything is a file" includes virtual abstractions created inside that virtual fs. The file system *is* the interface. The LLMs are tuned so hard on TerminalBench they are shell ops masters.
Key stats:
- 560B total parameters, 18.6B–31.3B dynamic activation
- Trained on 20T tokens
- 100+ tokens/sec inference
- TerminalBench: 39.5 | τ²-Bench: 67.7
- 560B total parameters, 18.6B–31.3B dynamic activation
- Trained on 20T tokens
- 100+ tokens/sec inference
- TerminalBench: 39.5 | τ²-Bench: 67.7
September 3, 2025 at 3:20 AM
Key stats:
- 560B total parameters, 18.6B–31.3B dynamic activation
- Trained on 20T tokens
- 100+ tokens/sec inference
- TerminalBench: 39.5 | τ²-Bench: 67.7
- 560B total parameters, 18.6B–31.3B dynamic activation
- Trained on 20T tokens
- 100+ tokens/sec inference
- TerminalBench: 39.5 | τ²-Bench: 67.7
Speed Meets Performance
With over 100 tokens/sec inference and stellar scores on TerminalBench (39.5) and τ²-Bench (67.7), LongCat-Flash-Chat isn’t just big—it’s blazing fast and benchmark-proven.
Huggingface: huggingface.co/meituan-long...
With over 100 tokens/sec inference and stellar scores on TerminalBench (39.5) and τ²-Bench (67.7), LongCat-Flash-Chat isn’t just big—it’s blazing fast and benchmark-proven.
Huggingface: huggingface.co/meituan-long...
meituan-longcat/LongCat-Flash-Chat · Hugging Face
We’re on a journey to advance and democratize artificial intelligence through open source and open science.
huggingface.co
September 3, 2025 at 3:20 AM
Speed Meets Performance
With over 100 tokens/sec inference and stellar scores on TerminalBench (39.5) and τ²-Bench (67.7), LongCat-Flash-Chat isn’t just big—it’s blazing fast and benchmark-proven.
Huggingface: huggingface.co/meituan-long...
With over 100 tokens/sec inference and stellar scores on TerminalBench (39.5) and τ²-Bench (67.7), LongCat-Flash-Chat isn’t just big—it’s blazing fast and benchmark-proven.
Huggingface: huggingface.co/meituan-long...
GitHub's HydraFusion routes Copilot across 3 model providers per request, matching Claude Opus 5 quality while cutting costs up to 67% on TerminalBench 2.1.
GitHub Copilot's Project HydraFusion Promises Frontier Level Performance Through Multi-Model Routing
Four Signals — The Wire
www.foursignals.dev
September 13, 2026 at 4:00 PM
GitHub's HydraFusion routes Copilot across 3 model providers per request, matching Claude Opus 5 quality while cutting costs up to 67% on TerminalBench 2.1.
Coding benchmarks are just a mess. Having to report on 3 versions of TerminalBench…
DeepSeek-V4.1-Flash
🔹 552B-parameter MoE.
🔹 New Causal Encoder–Decoder architecture: just 8B active parameters for input, 16B for output.
🔹 New pre-training methods + larger-scale RL post-training deliver benchmark results ahead of flagship models, including DeepSeek-V4-Pro.
🔹 552B-parameter MoE.
🔹 New Causal Encoder–Decoder architecture: just 8B active parameters for input, 16B for output.
🔹 New pre-training methods + larger-scale RL post-training deliver benchmark results ahead of flagship models, including DeepSeek-V4-Pro.
September 10, 2026 at 7:01 AM
Coding benchmarks are just a mess. Having to report on 3 versions of TerminalBench…
GitHub Copilotが複数AIモデルを自動で使い分ける「HydraFusion」を公開、TerminalBench 2.1でClaude Opus 5比コスト67％減・スコア4.9ポイント向上
https://gigazine.net/news/20260907-github-copilot-hydrafusion/
https://gigazine.net/news/20260907-github-copilot-hydrafusion/
GitHub Copilotが複数AIモデルを自動で使い分ける「HydraFusion」を公開、TerminalBench 2.1でClaude Opus 5比コスト67％減・スコア4.9ポイント向上
GitHubは2026年9月4日、GitHub Copilotでタスクに応じて複数のAIモデルを使い分ける「Project HydraFusion」を研究プレビューとして発表しました。GitHubによる評価では、TerminalBench 2.1でClaude Opus 5と比べて推定コストを67％削減しながら正しく完了できたタスクの割合が4.9ポイント高くなったとのことです。
gigazine.net
September 7, 2026 at 6:11 AM
GitHub Copilotが複数AIモデルを自動で使い分ける「HydraFusion」を公開、TerminalBench 2.1でClaude Opus 5比コスト67％減・スコア4.9ポイント向上
https://gigazine.net/news/20260907-github-copilot-hydrafusion/
https://gigazine.net/news/20260907-github-copilot-hydrafusion/
AutoAgent, an open-source library by Kevin Gu (thirdlayer.inc), autonomously iterated an agent harness via a meta-agent loop and, in a 24-hour run, reached #1 on SpreadsheetBench (96.5%) and #1 GPT-5 on TerminalBench (55.1%).
ThirdLayer
Applied research
thirdlayer.inc
April 6, 2026 at 2:17 PM
AutoAgent, an open-source library by Kevin Gu (thirdlayer.inc), autonomously iterated an agent harness via a meta-agent loop and, in a 24-hour run, reached #1 on SpreadsheetBench (96.5%) and #1 GPT-5 on TerminalBench (55.1%).
comparing to gpt-oss:120b, looks like shimmer is the better agentic model, but gpt-oss is better at traditional LLM tasks
August 10, 2026 at 2:15 PM
comparing to gpt-oss:120b, looks like shimmer is the better agentic model, but gpt-oss is better at traditional LLM tasks
🌲 Tree search for agentic RL: a meta-agent that squeezes richer signal from each training run.
It branches the run into many continuations and sees how they end. It then uses that signal to improve credit assignment in RL training.
2x GRPO's gains on TerminalBench-2.
It branches the run into many continuations and sees how they end. It then uses that signal to improve credit assignment in RL training.
2x GRPO's gains on TerminalBench-2.
July 1, 2026 at 5:04 PM
🌲 Tree search for agentic RL: a meta-agent that squeezes richer signal from each training run.
It branches the run into many continuations and sees how they end. It then uses that signal to improve credit assignment in RL training.
2x GRPO's gains on TerminalBench-2.
It branches the run into many continuations and sees how they end. It then uses that signal to improve credit assignment in RL training.
2x GRPO's gains on TerminalBench-2.
they'd be damn good at it. sonnet 4 plus letta nearly beats opus on terminalbench, and i think pattern's probably better still if i were to give them the right tools for coding. get to take advantage of that gemini context window while also having sonnet's reasoning skills, & still cheaper than opus
August 9, 2025 at 11:24 PM
they'd be damn good at it. sonnet 4 plus letta nearly beats opus on terminalbench, and i think pattern's probably better still if i were to give them the right tools for coding. get to take advantage of that gemini context window while also having sonnet's reasoning skills, & still cheaper than opus
Show HN: OSS Agent I built topped the TerminalBench on Gemini-3-flash-preview
https://github.com/dirac-run/dirac
https://news.ycombinator.com/item?id=47920787
https://github.com/dirac-run/dirac
https://news.ycombinator.com/item?id=47920787
April 27, 2026 at 7:20 PM
Show HN: OSS Agent I built topped the TerminalBench on Gemini-3-flash-preview
https://github.com/dirac-run/dirac
https://news.ycombinator.com/item?id=47920787
https://github.com/dirac-run/dirac
https://news.ycombinator.com/item?id=47920787
We're expanding the Epoch AI Benchmarking Hub with five new external benchmarks: TerminalBench, METR Time Horizons, DeepResearchBench, GSO, and WebDevArena! These benchmarks test AI's ability to perform complex tasks through coding or tool use. 🧵
August 15, 2025 at 3:55 PM
We're expanding the Epoch AI Benchmarking Hub with five new external benchmarks: TerminalBench, METR Time Horizons, DeepResearchBench, GSO, and WebDevArena! These benchmarks test AI's ability to perform complex tasks through coding or tool use. 🧵
Skill Learning is the proof of concept. 36.8% improvement on TerminalBench from learned skills, with 15.7% cost reduction. Errors in context normally degrade performance. Skills turn failure into transferable knowledge.
February 14, 2026 at 3:03 AM
Skill Learning is the proof of concept. 36.8% improvement on TerminalBench from learned skills, with 15.7% cost reduction. Errors in context normally degrade performance. Skills turn failure into transferable knowledge.
GitHub Copilotが複数AIモデルを自動で使い分ける「HydraFusion」を公開、TerminalBench 2.1でClaude Opus 5比コスト67％減・スコア4.9ポイント向上
gigazine.net/news/2026090...
gigazine.net/news/2026090...
GitHub Copilotが複数AIモデルを自動で使い分ける「HydraFusion」を公開、TerminalBench 2.1でClaude Opus 5比コスト67％減・スコア4.9ポイント向上
GitHubは2026年9月4日、GitHub Copilotでタスクに応じて複数のAIモデルを使い分ける「Project HydraFusion」を研究プレビューとして発表しました。GitHubによる評価では、TerminalBench 2.1でClaude Opus 5と比べて推定コストを67％削減しながら正しく完了できたタスクの割合が4.9ポイント高くなったとのことです。
gigazine.net
September 7, 2026 at 11:06 PM
GitHub Copilotが複数AIモデルを自動で使い分ける「HydraFusion」を公開、TerminalBench 2.1でClaude Opus 5比コスト67％減・スコア4.9ポイント向上
gigazine.net/news/2026090...
gigazine.net/news/2026090...
🏆 Chinese startup #FeelingAI achieves 72.9% on #TerminalBench 2.0 with CodeBrain-1, ranking #2 globally — only behind #OpenAI's Simple Codex (77.3%). First Chinese team in top 10 for Agentic AI coding benchmarks! 🧠
🧵👇#AI
🧵👇#AI
February 12, 2026 at 1:28 AM
🏆 Chinese startup #FeelingAI achieves 72.9% on #TerminalBench 2.0 with CodeBrain-1, ranking #2 globally — only behind #OpenAI's Simple Codex (77.3%). First Chinese team in top 10 for Agentic AI coding benchmarks! 🧠
🧵👇#AI
🧵👇#AI
GitHub Copilotが複数AIモデルを自動で使い分ける「HydraFusion」を公開、TerminalBench 2.1でClaude Opus 5比コスト67％減・スコア4.9ポイント向上
https://gigazine.net/news/20260907-github-copilot-hydrafusion/
https://gigazine.net/news/20260907-github-copilot-hydrafusion/
GitHub Copilotが複数AIモデルを自動で使い分ける「HydraFusion」を公開、TerminalBench 2.1でClaude Opus 5比コスト67％減・スコア4.9ポイント向上
GitHubは2026年9月4日、GitHub Copilotでタスクに応じて複数のAIモデルを使い分ける「Project HydraFusion」を研究プレビューとして発表しました。GitHubによる評価では、TerminalBench 2.1でClaude Opus 5と比べて推定コストを67％削減しながら正しく完了できたタスクの割合が4.9ポイント高くなったとのことです。
gigazine.net
September 7, 2026 at 11:34 AM
GitHub Copilotが複数AIモデルを自動で使い分ける「HydraFusion」を公開、TerminalBench 2.1でClaude Opus 5比コスト67％減・スコア4.9ポイント向上
https://gigazine.net/news/20260907-github-copilot-hydrafusion/
https://gigazine.net/news/20260907-github-copilot-hydrafusion/
Worth separating rate-card value from deployed value: a TerminalBench score is one signal, but cost per successful task also moves with retries, token shape, and the agent harness. A 2–4 point gap can vanish—or matter—there.
August 3, 2026 at 5:37 PM
Worth separating rate-card value from deployed value: a TerminalBench score is one signal, but cost per successful task also moves with retries, token shape, and the agent harness. A 2–4 point gap can vanish—or matter—there.