#FrontierCode
LightNews LightNews
ai-linkstream.bsky.social
FrontierCode benchmarks AI-generated code quality with a focus on mergeability. Crafted by 20 open-source developers, it addresses real-world coding standards and achieves 81% fewer misclassifications, challenging top AI models. https://cognition.ai/blog/frontier-code
Introducing FrontierCode | Cognition
FrontierCode benchmarks AI-generated code quality with a focus on mergeability. Crafted by 20 open-source developers, it addresses real-world coding standards and achieves 81% fewer misclassifications
cognition.ai
June 9, 2026 at 11:10 PM
davidcrespo.bsky.social
heh here's one. competitive! (these benchmarks are all shit though)
x.com/cognition/st...
Screenshot of an X post by Cognition (@cognition): GPT-6 Sol and Luna are now available in Devin. On FrontierCode 1.1, GPT-6 Sol matches GPT-5.6 Sol's score at 61% lower cost per task. GPT-6 Luna scores above GPT-5.6 Luna at about a quarter of the cost. At under $0.10 per task, it is the cheapest model on the leaderboard. Included chart titled "FrontierCode 1.1 Extended: Score vs Cost ($)" plotting score percentage against average cost (USD) per rollout. It compares GPT-6 Sol and Luna against models including Opus 5.5, SWE-2, GPT-6 Astra, GPT-5.6 Sol, and GPT-5.6 Luna, showing GPT-6 models achieving comparable or higher scores at significantly lower costs. Subtitle notes: Score is a weighted aggregate of rubric items (0 if blocking criteria fail). Cost is mean USD per rollout. Each dot is a reasoning effort.
September 22, 2026 at 7:23 PM
timkellogg.me
Gemini 3.7 Flash is here!

and these bars are ordered very intentionally

blog.google/innovation-a...
A bar chart titled "Production code quality: FrontierCode 1.1 Main" comparing four AI models: Gemini 3.7 Flash (highlighted in dark blue) leads at 43.6%, followed by Claude Sonnet 5 at 42.7%, GPT-5.6 Terra at 41.3%, and Gemini 3.6 Flash at 34.4%.
August 13, 2026 at 6:31 PM
anthropicai.xmirror.bot
On agentic terminal coding (Frontier-Bench v0.1), Opus 5 is the new state of the art at 43.3%: ahead of Fable 5 (33.7%) and more than double Opus 4.8.

On FrontierCode v1.1 it matches Fable 5, at half the price.
Image from Twitter
July 24, 2026 at 5:35 PM
timkellogg.me
crazy — cognition/windsurf are officially in the game

(also, why no xhigh effort level? that’s sick…)
A line and scatter plot titled "FrontierCode 1.1 Main: Score vs Cost ($)" comparing AI model performance against expense. Axes * Y-axis: Score (%), ranging from 35 to 55. * X-axis: Average cost (USD) per rollout, ranging from $0.00 to $10.00 in increments of $2.00. Model Performance Data * SWE-2 (blue curve): Shows high efficiency, rising steeply from approximately 43% score at $0.20 to 50% score at $1.20. * GPT-6 Astra (green line): Reaches the highest peak performance at approximately 53.5% score for a cost around $4.30. * Fable 5.1 (orange line): Ranges between 49% and 51% score across costs from $2.50 to $9.30. * GPT-5.6 Sol (teal line): Climbs from 35.5% score at $2.10 to 47.5% score at $6.30. * Grok 4.6 (grey line): Scales from 40% score at $1.00 to 48% score at $3.10. * Kimi K3 (purple point): Placed around 44% score at $3.80. * SWE-1.7 (blue point): Placed around 42% score at $2.00. Footnotes * Score: A weighted aggregate of the rubric items. Solutions that don't pass blocking criteria receive 0. * Cost ($): The mean USD spend per rollout.
September 10, 2026 at 6:15 PM
timkellogg.me
Important cost vs performance phrasing
A line graph titled "FrontierCode: Accuracy vs Cost" comparing the performance scores and mean costs per task of three AI models. The vertical y-axis represents "Score (%)" ranging from 0 to 35 in increments of 5. The horizontal x-axis represents "Mean cost per task (USD, log scale)" with markers at $2, $5, $10, and $20. A sub-caption notes: "Diamond subset comprises the hardest 50 of 150 tasks in the benchmark." The data points for each model are labeled with categories (low, med, high, xhigh, max) reflecting increasing budget or effort per task: * **Claude Fable 5 (Orange line):** Starts at low effort and scales upward continuously. * **low:** ~$5.30 cost, ~11.5% score * **med:** ~$6.80 cost, ~18% score * **high:** ~$10 cost, ~24% score * **xhigh:** ~$13 cost, ~29.3% score * **max:** ~$19 cost, ~31% score * **Claude Opus 4.8 (Green line):** Shows mixed progression, peaking at xhigh before dipping at max. * **low:** ~$3.50 cost, ~8% score * **med:** ~$4.20 cost, ~6% score * **high:** ~$4.80 cost, ~8.5% score * **xhigh:** ~$7.50 cost, ~13.4% score * **max:** ~$11 cost, ~11.5% score * **GPT-5.5 (Black dashed line):** Remains relatively flat at a lower score tier. * **low:** ~$1.70 cost, ~5% score * **med:** ~$3.90 cost, ~6% score * **high:** ~$5.00 cost, ~5% score * **xhigh:** ~$6.20 cost, ~5.7% score
June 9, 2026 at 5:29 PM
maxkennerly.bsky.social
You can see this with the latest Claude. Notice the logarithmic cost x-axis vs the linear performance y-axis. You don't get 2x better output for 2x the cost. That's not an outlier, it's well-known, OpenAI showed the same linear-performance-increase vs logarithmic-compute-cost in September 2024.
/2
graphic from anthropic comparing frontiercode results on multiple models We have found that the performance of o1 consistently improves with more reinforcement learning (train-time compute) and with more time spent thinking (test-time compute). The constraints on scaling this approach differ substantially from those of LLM pretraining, and we are continuing to investigate them.
June 12, 2026 at 3:51 PM
news-it.bsky.social
Cognition、AI生成コードの品質評価用ベンチマーク「FrontierCode」発表
#FrontierCode #ITニュース
ITちゃんねる
Cognition、AI生成コードの品質評価用ベンチマーク「FrontierCode」発表 #FrontierCode #ITニュース
it.f-frontier.com
June 9, 2026 at 7:25 AM
timkellogg.me
pretty good marketing though
Alt Text A benchmark comparison table evaluating four AI models—Grok 4.6 High, Grok 4.5 High, GPT-5.6 Sol Max, and Fable 5 Max—across ten performance metrics: AA Intelligence Index, GDPVal-AA v2, CursorBench v3.2, DeepSWE v1.1, FrontierCode v1.1 (Extended), APEX-Agents, Terminal-Bench v3.0, APEX-SWE, AA-Briefcase, and Harvey LAB (Vals). The Grok 4.6 High column is prominently highlighted as the featured model, showcasing highest scores on metrics such as GDPVal-AA v2 (1753), AA-Briefcase (1577), and Harvey LAB (15.8%). A footnote at the bottom reads: "Third-party model scores best of self-reported or publicly available." Color Scheme * Base & Neutral Tones: * Background: Clean pure white (#FFFFFF). * Text & Typography: Dark charcoal/black for key values and primary headers; mid-tone cool grey for row metric names, secondary model tiers (e.g., "High", "Max"), and the bottom footnote text. * Gridlines: Subtle, ultra-thin light grey borders separating table cells. * Accent & Highlight Palette: * Primary Accent: Vibrant warm orange border bar positioned directly above the Grok 4.6 High column header. * Column Background Tint: Soft, pale peach/cream fill applied to the entire Grok 4.6 High column to draw immediate visual focus. * Cell Highlights: Medium pastel peach/salmon fill highlighting top-performing cells within the featured Grok 4.6 column (GDPVal-AA v2, AA-Briefcase, and Harvey LAB). * Secondary Highlights: Very faint, subtle warm peach tints on selected top-score cells across other model columns (e.g., Fable 5 Max column header and selected metrics). Alt Text A benchmark comparison table evaluating four AI models—Grok 4.6 High, Grok 4.5 High, GPT-5.6 Sol Max, and Fable 5 Max—across ten performance metrics: AA Intelligence Index, GDPVal-AA v2, CursorBench v3.2, DeepSWE v1.1, FrontierCode v1.1 (Extended), APEX-Agents, Terminal-Bench v3.0, APEX-SWE, AA-Briefcase, and Harvey LAB (Vals). Unlike single-model promotional views, top scores across all models are highlighted in outlined green boxes: * Fable 5 Max achieves the highest score in 5 categories (AA Intelligence Index at 62, CursorBench v3.2 at 70.5%, FrontierCode v1.1 at 64.9%, APEX-Agents at 59.2%, and APEX-SWE at 58.8%). * Grok 4.6 High leads in 3 categories (GDPVal-AA v2 at 1753, AA-Briefcase at 1577, and Harvey LAB at 15.8%). * GPT-5.6 Sol Max leads in 2 categories (DeepSWE v1.1 at 73% and Terminal-Bench v3.0 at 34.6%). A footnote at the bottom reads: "Third-party model scores best of self-reported or publicly available." Color Scheme * Base & Neutral Tones: * Background: Plain white (#FFFFFF). * Text & Typography: Dark charcoal/black for primary header names and metric scores; medium grey for metric row titles, model status tiers ("High", "Max"), and the bottom footnote. * Gridlines: Subtle, ultra-thin light grey grid lines separating rows and columns. * Highlight Palette: * Winning Cell Fill: Light sage / pale mint green tint filling the background of winning benchmark cells. * Winning Cell Border: Thin, crisp olive-green outline framing each top-scoring cell across the table.
August 13, 2026 at 2:57 PM
five-oclick-tech.bsky.social
Cognition's SWE-2: better code, lower cost 💸

Cognition has released SWE-2, a new coding model that matches Fable 5.1's performance on FrontierCode 1.1 Main1 at 64% lower cost, scaling RL to multi-trillion parameters.

It's a strong move towards making high-end coding assistance more accessible.
Cognition's SWE-2: better code, lower cost 💸
September 11, 2026 at 3:27 PM
criticalnexus.bsky.social
@kimmonismus

Benchmarks GPT-6 Sol & Luna:

• FrontierCode: 48.4% vs. Fable 5.1’s 48.7%, both at xhigh. $1.37 vs. $9.27 per task, roughly 85% cheaper.

• DeepSWE: 68.8% at max vs. Fable 5’s best result of 69.9% at xhigh. $2.74 vs. $13.41 per tas...

https://x.com/i/web/status/2102462091299098640
September 23, 2026 at 6:31 PM
ai-linkstream.bsky.social
Introducing FrontierCode, a benchmark for evaluating AI-generated code quality through mergeability and adherence to coding standards, highlighting that even leading models struggle to meet these rigorous expectations. https://cognition.ai/blog/frontier-code
Introducing FrontierCode | Cognition
Introducing FrontierCode, a benchmark for evaluating AI-generated code quality through mergeability and adherence to coding standards, highlighting that even leading models struggle to meet these rigo
cognition.ai
August 9, 2026 at 9:00 AM
aitechnewsuk.bsky.social
🔥 FrontierCode revolutionizes AI
FrontierCode is a new AI concept that has the potential to revolutionize the field of artificial intelligence. This innovative approach could lead to sig...

🌐 aitechcodex.uk/news/2206/?utm_source=bluesky&utm_medium=social&utm_campaign=daily_news 📡 t.me/AITechNewsUK
June 9, 2026 at 3:01 AM
shawnchauhan1.bsky.social
The best AI coding model in the world scores 13 out of 100. On code a real engineer would actually merge.

Every benchmark you've seen measures: does this code run?

FrontierCode measures: would a senior maintainer accept this PR?

Scope discipline. Style consistency. Regression safety.
June 10, 2026 at 6:42 AM
methiaff.bsky.social
alignment is not on track, according to a new safety startup. this is fine. #ai #research
Import AI 461: “Alignment is not on track”; FrontierCode; and synthetic research interns
Welcome to Import AI, a newsletter about AI research. Import AI runs on arXiv, cappuccinos, and feedback from readers. If you’d like to support this, please subscribe. Subscribe now AI researchers launch new safety startup because “alignment is not on track”:…Sequent will have a portfolio of under-resourced research bets…Researchers from the UK AI Security Institute […]
jack-clark.net
June 15, 2026 at 4:00 PM
theo-mirr.selfhosted.social
It's pretty bad on most frontier-ish stuff.

iirc, FrontierCode is measuring "mergeability" based on a bunch of heuristics determined by maintainers, rather than "does it complete the task". Makes it really noisy.
July 24, 2026 at 10:22 PM
strike007.bsky.social
Devs: Prioritize "Kernel Contracts" to bound training-inference divergence and adopt "FrontierCode" benchmarks to filter LLM-generated slop. These tools shift workflows from "prompt-and-pray" to verifiable, high-fidelity code generation and deployment stability.
Enhancing Strawberry Yield Forecasting with Backcasted IoT Sensor Data and Machine Learning
Rapid global population growth underscores the need for digitally enabled agricultural systems that support sustainable food production and data-driven resource management for farmers and stakeholders. The adoption of Internet of Things (IoT) technologies, capable of capturing real-time environmenta
arxiv.org
June 9, 2026 at 10:34 AM
hossamudin.bsky.social
جربت Kimi K3 بنفسي… والنتيجة صدمتني
النموذج المفتوح المصدر اللي كسر الدنيا وبقى رقم 1 على FrontierCode
قعدت أختبره في حاجات حقيقية، وشوفوا عمل إيه

#حسام_الدين_حسن #خبير_اونلاين #Kimi #Kimi_K3 #KimiK3 #مفتوح_المصدر #OpenSource #الذكاء_الاصطناعي #نماذج_الذكاء_الاصطناعي #ذكاء_اصطناعي_عربي #Fable_5
July 18, 2026 at 2:35 PM
mengli512.bsky.social
Google launched Gemini 3.7 Flash, a new Flash model for coding and AI agents, just three weeks after 3.6 Flash. It improves in software engineering, web dev, and automation. Benchmarks show gains in FrontierCode and WebDev Arena.

#Google #Gemini

aidisruption.ai/p/musks-grok...
Musk's Grok Bot Is Here!
Musk drops Grok Bot—an AI teammate with its own cloud PC. Works 24/7, runs multiple bots, and automates real tasks like a human colleague.
aidisruption.ai
August 14, 2026 at 9:56 AM
mnapoli.bsky.social
If this benchmark is to be trusted, current frontier is:
- grok 4.5 (high): cheap + fast
- opus 5 (medium): best
- fable 5 (high): if you're feeling fancy
and forget the rest.

Now time to see if Opus 5 is really that good. Source: cognition.com/frontiercode
July 27, 2026 at 11:59 AM
nexttool.bsky.social
Gemini 3.7 Flash dropped today — Google's new 'workhorse' model for coding and agents. 43.6% on FrontierCode (up from 34.4%), and intro pricing is half what 3.6 Flash cost. If you're paying for Flash today, switch. https://blog.google/innovation-and-ai/models-and-research/gemini-models/introducing-g
August 14, 2026 at 9:08 AM
notengoprisa.bsky.social
También notable el salto en FrontierCode, un nuevo benchmark que no solo evalúa la capacidad de pasar los tests, sino la 'mergeabilidad' del código.

Así estaba hasta ahora, casi a cero,y Fable 5 da el salto hasta el 30% de golpe!
June 9, 2026 at 5:20 PM
labomaru.com
🐶 らぼまる速報⚡

【コード生成AI「SWE-2」徹底検証：Fable 5.1同等性能を64%コスト削減した Cognition …】
CognitionのコーディングモデルSWE-2を早速検証。Kimi K3をポストトレーニングした結果、FrontierCodeでFable 5.1並みの精度を叩き出しつつ、コストを64%も削減できて驚いた。実務のコード生成コスト改善に直結する。…

👇 詳細・一次ソース解説
https://labomaru.com/posts/20260913121123/

#AI速報 #らぼまる #AIツール
コード生成AI「SWE-2」徹底検証：Fable 5.1同等性能を64%コスト削減した Cognition の推論最適化メカニズム
CognitionのコーディングモデルSWE-2を早速検証。Kimi K3をポストトレーニングした結果、FrontierCodeでFable 5.1並みの精度を叩き出しつつ、コストを64%も削減できて驚いた。実務のコード生成コスト改善に直…
labomaru.com
September 13, 2026 at 3:13 AM
bigearthdata.ai
Cognition's SWE-2 Coding Agent Matches Rivals at a Quarter of the Price
->Startup Fortune | More on "AI coding agents cost efficiency" at BigEarthData.ai
Cognition's SWE-2 Coding Agent Matches Rivals at a Quarter of the Price
Cognition just launched SWE-2, an AI coding agent it says scores within a single point of Anthropic's top coding model while costing up to 70% less to run. The pitch is blunt: near-frontier code, at a fraction of the bill. Cognition, the startup behind the autonomous coding agent Devin, shipped SWE-2 on September 10, 2026, rolling it out across Devin Desktop, the Devin CLI, Devin Web and Devin Fusion. On the company's own FrontierCode 1.1 Main benchmark, which grades whether a human maintainer would actually merge an AI-written pull request, SWE-2 scored 50.0%. Anthropic's Fable 5.1 scored 50.9%. That's a single point apart. Cognition says it matched that score for 64% less compute cost. Against OpenAI's GPT-6 Astra, which scored 53.3% on the same test, Cognition claims SWE-2 comes in at roughly a quarter of the price. Price is the argument. It also beat its own predecessor, SWE-1.7, which scored 42.0%, and xAI's Grok 4.6, which scored 48.0%. The engineering behind that claim is specific. SWE-2 is post-trained from Moonshot AI's Kimi K3, a 2.8 trillion parameter base model, using a reinforcement learning method that trains three separate effort levels, medium, high and max, in a single run instead of...
startupfortune.com
September 11, 2026 at 3:45 AM