An assistant can solve the wrong problem beautifully. In XYBench’s 8,115 questions, Claude Sonnet 4.6, GPT 5 and three other models often offered help while...
#BenchmarksEvaluation #LLM #SafetyAlignment #AI #AIPulse
An assistant can solve the wrong problem beautifully. In XYBench’s 8,115 questions, Claude Sonnet 4.6, GPT 5 and three other models often offered help while...
#BenchmarksEvaluation #LLM #SafetyAlignment #AI #AIPulse
CerS Path beat foundation models on 26 of 30 cervical pathology tasks. That result makes a case for building around the diagnostic work...
#BenchmarksEvaluation #ComputerVision #Healthcare #AI #AIPulse
CerS Path beat foundation models on 26 of 30 cervical pathology tasks. That result makes a case for building around the diagnostic work...
#BenchmarksEvaluation #ComputerVision #Healthcare #AI #AIPulse
Ai2 has given the planet 10 meter embeddings; its next challenge is deciding whose questions count. The OlmoEarth v1.3 release put precomputed annual features within...
#BenchmarksEvaluation #ClimateEnvironment #RAGEmbeddings #AI #AIPulse
Ai2 has given the planet 10 meter embeddings; its next challenge is deciding whose questions count. The OlmoEarth v1.3 release put precomputed annual features within...
#BenchmarksEvaluation #ClimateEnvironment #RAGEmbeddings #AI #AIPulse
The announcement is a list of names, which is the shape the event has taken for years. It also has a theme: the year the benchmark score stopped being the...
#EnterpriseAI #BenchmarksEvaluation #AIStartups #AI #AIPulse
The announcement is a list of names, which is the shape the event has taken for years. It also has a theme: the year the benchmark score stopped being the...
#EnterpriseAI #BenchmarksEvaluation #AIStartups #AI #AIPulse
A vehicle recognizer that scores 63.3% after the vehicle is removed is learning the scene. On ATRNet STAR’s 40 class split, ConvNeXt Small...
#BenchmarksEvaluation #ComputerVision #InferenceOptimization #AI #AIPulse
A vehicle recognizer that scores 63.3% after the vehicle is removed is learning the scene. On ATRNet STAR’s 40 class split, ConvNeXt Small...
#BenchmarksEvaluation #ComputerVision #InferenceOptimization #AI #AIPulse
DeltaHarness cut complete delivery latency by 30.0% on one video editing task. That is a meaningful win for execution policy, but it says nothing by itself about...
#InferenceOptimization #AIAgents #BenchmarksEvaluation #AI #AIPulse
DeltaHarness cut complete delivery latency by 30.0% on one video editing task. That is a meaningful win for execution policy, but it says nothing by itself about...
#InferenceOptimization #AIAgents #BenchmarksEvaluation #AI #AIPulse
Google has put Neon, element 10, into Gemini’s web app code, but a name is doing work that a model result should do. The gemini 4 neon identifier...
#GoogleDeepMind #BenchmarksEvaluation #SoftwareDevelopment #AI #AIPulse
Google has put Neon, element 10, into Gemini’s web app code, but a name is doing work that a model result should do. The gemini 4 neon identifier...
#GoogleDeepMind #BenchmarksEvaluation #SoftwareDevelopment #AI #AIPulse
In 90 minutes, a Kimi assisted analysis flagged 100 candidate genes from data on 3,696 mouse pancreatic cells. DP Technology estimated the same work...
#ScienceBiology #AIStartups #BenchmarksEvaluation #AI #AIPulse
In 90 minutes, a Kimi assisted analysis flagged 100 candidate genes from data on 3,696 mouse pancreatic cells. DP Technology estimated the same work...
#ScienceBiology #AIStartups #BenchmarksEvaluation #AI #AIPulse
Anthropic’s Claude Fable 5.2 used roughly 700,000 simulated brushstrokes to paint a parking garage. That feat captures both its promise and its burden: the model...
#Anthropic #BenchmarksEvaluation #Robotics #AI #AIPulse
Anthropic’s Claude Fable 5.2 used roughly 700,000 simulated brushstrokes to paint a parking garage. That feat captures both its promise and its burden: the model...
#Anthropic #BenchmarksEvaluation #Robotics #AI #AIPulse
MedGPT beat unassisted doctors on 660 of 750 scored lung cancer treatment plans, but its strongest result came when doctors used it. The Chinese model’s 88 to 12...
#Healthcare #ChineseAI #BenchmarksEvaluation #AI #AIPulse
MedGPT beat unassisted doctors on 660 of 750 scored lung cancer treatment plans, but its strongest result came when doctors used it. The Chinese model’s 88 to 12...
#Healthcare #ChineseAI #BenchmarksEvaluation #AI #AIPulse
A legal chess move can still corrupt an entire transcription. In oracle replay, Zugwise cut silent errors from 16 to 1 in one tournament section and from 50 to 13 in...
#Reasoning #BenchmarksEvaluation #SafetyAlignment #AI #AIPulse
A legal chess move can still corrupt an entire transcription. In oracle replay, Zugwise cut silent errors from 16 to 1 in one tournament section and from 50 to 13 in...
#Reasoning #BenchmarksEvaluation #SafetyAlignment #AI #AIPulse
Google has bought a narrow math lead at a very wide price. Gemini 4 Carbon tops MathArena at an expected 85.4%, against 80.6% for GPT 6.1 Sol, but the tested runs cost $15.70...
#BenchmarksEvaluation #GoogleDeepMind #LLM #AI #AIPulse
Google has bought a narrow math lead at a very wide price. Gemini 4 Carbon tops MathArena at an expected 85.4%, against 80.6% for GPT 6.1 Sol, but the tested runs cost $15.70...
#BenchmarksEvaluation #GoogleDeepMind #LLM #AI #AIPulse
A 5.3 fold increase in API cost bought GPT 6 Astra almost no improvement in predicting wet experiments. On UniPat AI’s SPU benchmark, higher reasoning effort also...
#BenchmarksEvaluation #Reasoning #ScienceBiology #AI #AIPulse
A 5.3 fold increase in API cost bought GPT 6 Astra almost no improvement in predicting wet experiments. On UniPat AI’s SPU benchmark, higher reasoning effort also...
#BenchmarksEvaluation #Reasoning #ScienceBiology #AI #AIPulse
OpenAI is reorganizing its robotics staff around a 74.8 percent success rate on a simulated robot. In Nvidia’s study, GPT 6 Astra beat the best specialized models, which...
#OpenAI #Robotics #BenchmarksEvaluation #AI #AIPulse
OpenAI is reorganizing its robotics staff around a 74.8 percent success rate on a simulated robot. In Nvidia’s study, GPT 6 Astra beat the best specialized models, which...
#OpenAI #Robotics #BenchmarksEvaluation #AI #AIPulse
The finding that judges correlate weakly with general ability is the interesting one, and the variation with the objective is the useful part of it. A...
#BenchmarksEvaluation #BiasFairness #LLM #AI #AIPulse
The finding that judges correlate weakly with general ability is the interesting one, and the variation with the objective is the useful part of it. A...
#BenchmarksEvaluation #BiasFairness #LLM #AI #AIPulse
The hardware is the interesting half of the claim. An NVIDIA Vera Rubin NVL72 system with 64 GPUs in compatibility mode correlates a 10 TB dataset of radio...
#HardwareChips #NVIDIA #BenchmarksEvaluation #AI #AIPulse
The hardware is the interesting half of the claim. An NVIDIA Vera Rubin NVL72 system with 64 GPUs in compatibility mode correlates a 10 TB dataset of radio...
#HardwareChips #NVIDIA #BenchmarksEvaluation #AI #AIPulse
The comparison that sells it short is the honest one. Hydrogen is a fast general purpose model meant for everyday coding and agentic workflows, built to replace the earlier...
#LLM #SoftwareDevelopment #BenchmarksEvaluation #AI #AIPulse
The comparison that sells it short is the honest one. Hydrogen is a fast general purpose model meant for everyday coding and agentic workflows, built to replace the earlier...
#LLM #SoftwareDevelopment #BenchmarksEvaluation #AI #AIPulse
The trouble teams have always described is not the agent itself, but the question that follows deployment: whether it finished the task the user asked, stayed...
#BenchmarksEvaluation #AIAgents #EnterpriseAI #AI #AIPulse
The trouble teams have always described is not the agent itself, but the question that follows deployment: whether it finished the task the user asked, stayed...
#BenchmarksEvaluation #AIAgents #EnterpriseAI #AI #AIPulse
IBM's Nighthawk r3 announcement is a statement about what a processor can do rather than what it actually does. It targets 10,000 operations...
#HardwareChips #BenchmarksEvaluation #EnergyCompute #AI #AIPulse
IBM's Nighthawk r3 announcement is a statement about what a processor can do rather than what it actually does. It targets 10,000 operations...
#HardwareChips #BenchmarksEvaluation #EnergyCompute #AI #AIPulse
The goal is not to change the model but to change the pipeline, and the recipe is the point. A general purpose 3D model like TRELLIS.2 produces a decent...
#InferenceOptimization #ModelTraining #BenchmarksEvaluation #AI #AIPulse
The goal is not to change the model but to change the pipeline, and the recipe is the point. A general purpose 3D model like TRELLIS.2 produces a decent...
#InferenceOptimization #ModelTraining #BenchmarksEvaluation #AI #AIPulse
The claim is that a single model can learn to perform a wide range of robot tasks with few demonstrations of the robot itself. The demonstration path is the interesting...
#Robotics #ModelTraining #BenchmarksEvaluation #AI #AIPulse
The claim is that a single model can learn to perform a wide range of robot tasks with few demonstrations of the robot itself. The demonstration path is the interesting...
#Robotics #ModelTraining #BenchmarksEvaluation #AI #AIPulse
The number that matters is not the accuracy, which beats its rivals across the board: 92.2 percent on one benchmark, 79.9 on another,...
#BenchmarksEvaluation #InferenceOptimization #OpenAI #AI #AIPulse
The number that matters is not the accuracy, which beats its rivals across the board: 92.2 percent on one benchmark, 79.9 on another,...
#BenchmarksEvaluation #InferenceOptimization #OpenAI #AI #AIPulse
The study's finding is that the majority of messages in these closed loop synchrotron experiments involved interpretation and judgment rather than formal...
#ScienceBiology #BenchmarksEvaluation #AIAgents #AI #AIPulse
The study's finding is that the majority of messages in these closed loop synchrotron experiments involved interpretation and judgment rather than formal...
#ScienceBiology #BenchmarksEvaluation #AIAgents #AI #AIPulse
The claim is a sharp one: 34 percent full plan completion on 100 multi step tasks, beating the competition and leaving MiniMax H3 trailing on single actions and human...
#BenchmarksEvaluation #Reasoning #Robotics #AI #AIPulse
The claim is a sharp one: 34 percent full plan completion on 100 multi step tasks, beating the competition and leaving MiniMax H3 trailing on single actions and human...
#BenchmarksEvaluation #Reasoning #Robotics #AI #AIPulse
The numbers are the most important part of the announcement. Self reported scores on the web CTFs, with 93 and 95 out of 100 in the same cohort, and 81 out of 104...
#BenchmarksEvaluation #AIAgents #OpenSource #AI #AIPulse
The numbers are the most important part of the announcement. Self reported scores on the web CTFs, with 93 and 95 out of 100 in the same cohort, and 81 out of 104...
#BenchmarksEvaluation #AIAgents #OpenSource #AI #AIPulse