An assistant can solve the wrong problem beautifully. In XYBench’s 8,115 questions, Claude Sonnet 4.6, GPT 5 and three other models often offered help while...
#BenchmarksEvaluation #LLM #SafetyAlignment #AI #AIPulse
An assistant can solve the wrong problem beautifully. In XYBench’s 8,115 questions, Claude Sonnet 4.6, GPT 5 and three other models often offered help while...
#BenchmarksEvaluation #LLM #SafetyAlignment #AI #AIPulse
CerS Path beat foundation models on 26 of 30 cervical pathology tasks. That result makes a case for building around the diagnostic work...
#BenchmarksEvaluation #ComputerVision #Healthcare #AI #AIPulse
CerS Path beat foundation models on 26 of 30 cervical pathology tasks. That result makes a case for building around the diagnostic work...
#BenchmarksEvaluation #ComputerVision #Healthcare #AI #AIPulse
Ai2 has given the planet 10 meter embeddings; its next challenge is deciding whose questions count. The OlmoEarth v1.3 release put precomputed annual features within...
#BenchmarksEvaluation #ClimateEnvironment #RAGEmbeddings #AI #AIPulse
Ai2 has given the planet 10 meter embeddings; its next challenge is deciding whose questions count. The OlmoEarth v1.3 release put precomputed annual features within...
#BenchmarksEvaluation #ClimateEnvironment #RAGEmbeddings #AI #AIPulse
The announcement is a list of names, which is the shape the event has taken for years. It also has a theme: the year the benchmark score stopped being the...
#EnterpriseAI #BenchmarksEvaluation #AIStartups #AI #AIPulse
The announcement is a list of names, which is the shape the event has taken for years. It also has a theme: the year the benchmark score stopped being the...
#EnterpriseAI #BenchmarksEvaluation #AIStartups #AI #AIPulse
A vehicle recognizer that scores 63.3% after the vehicle is removed is learning the scene. On ATRNet STAR’s 40 class split, ConvNeXt Small...
#BenchmarksEvaluation #ComputerVision #InferenceOptimization #AI #AIPulse
A vehicle recognizer that scores 63.3% after the vehicle is removed is learning the scene. On ATRNet STAR’s 40 class split, ConvNeXt Small...
#BenchmarksEvaluation #ComputerVision #InferenceOptimization #AI #AIPulse
Nine boxes expose a weakness in how healthcare AI is judged: a good score on one medical task says little about the others. InclusionMed has launched an evaluation...
#Healthcare #BenchmarksEvaluation #AIStartups #AI #AIPulse
Nine boxes expose a weakness in how healthcare AI is judged: a good score on one medical task says little about the others. InclusionMed has launched an evaluation...
#Healthcare #BenchmarksEvaluation #AIStartups #AI #AIPulse
DeltaHarness cut complete delivery latency by 30.0% on one video editing task. That is a meaningful win for execution policy, but it says nothing by itself about...
#InferenceOptimization #AIAgents #BenchmarksEvaluation #AI #AIPulse
DeltaHarness cut complete delivery latency by 30.0% on one video editing task. That is a meaningful win for execution policy, but it says nothing by itself about...
#InferenceOptimization #AIAgents #BenchmarksEvaluation #AI #AIPulse
Google has put Neon, element 10, into Gemini’s web app code, but a name is doing work that a model result should do. The gemini 4 neon identifier...
#GoogleDeepMind #BenchmarksEvaluation #SoftwareDevelopment #AI #AIPulse
Google has put Neon, element 10, into Gemini’s web app code, but a name is doing work that a model result should do. The gemini 4 neon identifier...
#GoogleDeepMind #BenchmarksEvaluation #SoftwareDevelopment #AI #AIPulse
In 90 minutes, a Kimi assisted analysis flagged 100 candidate genes from data on 3,696 mouse pancreatic cells. DP Technology estimated the same work...
#ScienceBiology #AIStartups #BenchmarksEvaluation #AI #AIPulse
In 90 minutes, a Kimi assisted analysis flagged 100 candidate genes from data on 3,696 mouse pancreatic cells. DP Technology estimated the same work...
#ScienceBiology #AIStartups #BenchmarksEvaluation #AI #AIPulse
Anthropic’s Claude Fable 5.2 used roughly 700,000 simulated brushstrokes to paint a parking garage. That feat captures both its promise and its burden: the model...
#Anthropic #BenchmarksEvaluation #Robotics #AI #AIPulse
Anthropic’s Claude Fable 5.2 used roughly 700,000 simulated brushstrokes to paint a parking garage. That feat captures both its promise and its burden: the model...
#Anthropic #BenchmarksEvaluation #Robotics #AI #AIPulse
MedGPT beat unassisted doctors on 660 of 750 scored lung cancer treatment plans, but its strongest result came when doctors used it. The Chinese model’s 88 to 12...
#Healthcare #ChineseAI #BenchmarksEvaluation #AI #AIPulse
MedGPT beat unassisted doctors on 660 of 750 scored lung cancer treatment plans, but its strongest result came when doctors used it. The Chinese model’s 88 to 12...
#Healthcare #ChineseAI #BenchmarksEvaluation #AI #AIPulse
A legal chess move can still corrupt an entire transcription. In oracle replay, Zugwise cut silent errors from 16 to 1 in one tournament section and from 50 to 13 in...
#Reasoning #BenchmarksEvaluation #SafetyAlignment #AI #AIPulse
A legal chess move can still corrupt an entire transcription. In oracle replay, Zugwise cut silent errors from 16 to 1 in one tournament section and from 50 to 13 in...
#Reasoning #BenchmarksEvaluation #SafetyAlignment #AI #AIPulse
Google has bought a narrow math lead at a very wide price. Gemini 4 Carbon tops MathArena at an expected 85.4%, against 80.6% for GPT 6.1 Sol, but the tested runs cost $15.70...
#BenchmarksEvaluation #GoogleDeepMind #LLM #AI #AIPulse
Google has bought a narrow math lead at a very wide price. Gemini 4 Carbon tops MathArena at an expected 85.4%, against 80.6% for GPT 6.1 Sol, but the tested runs cost $15.70...
#BenchmarksEvaluation #GoogleDeepMind #LLM #AI #AIPulse
A 5.3 fold increase in API cost bought GPT 6 Astra almost no improvement in predicting wet experiments. On UniPat AI’s SPU benchmark, higher reasoning effort also...
#BenchmarksEvaluation #Reasoning #ScienceBiology #AI #AIPulse
A 5.3 fold increase in API cost bought GPT 6 Astra almost no improvement in predicting wet experiments. On UniPat AI’s SPU benchmark, higher reasoning effort also...
#BenchmarksEvaluation #Reasoning #ScienceBiology #AI #AIPulse
OpenAI is reorganizing its robotics staff around a 74.8 percent success rate on a simulated robot. In Nvidia’s study, GPT 6 Astra beat the best specialized models, which...
#OpenAI #Robotics #BenchmarksEvaluation #AI #AIPulse
OpenAI is reorganizing its robotics staff around a 74.8 percent success rate on a simulated robot. In Nvidia’s study, GPT 6 Astra beat the best specialized models, which...
#OpenAI #Robotics #BenchmarksEvaluation #AI #AIPulse
The finding that judges correlate weakly with general ability is the interesting one, and the variation with the objective is the useful part of it. A...
#BenchmarksEvaluation #BiasFairness #LLM #AI #AIPulse
The finding that judges correlate weakly with general ability is the interesting one, and the variation with the objective is the useful part of it. A...
#BenchmarksEvaluation #BiasFairness #LLM #AI #AIPulse
The hardware is the interesting half of the claim. An NVIDIA Vera Rubin NVL72 system with 64 GPUs in compatibility mode correlates a 10 TB dataset of radio...
#HardwareChips #NVIDIA #BenchmarksEvaluation #AI #AIPulse
The hardware is the interesting half of the claim. An NVIDIA Vera Rubin NVL72 system with 64 GPUs in compatibility mode correlates a 10 TB dataset of radio...
#HardwareChips #NVIDIA #BenchmarksEvaluation #AI #AIPulse
The comparison that sells it short is the honest one. Hydrogen is a fast general purpose model meant for everyday coding and agentic workflows, built to replace the earlier...
#LLM #SoftwareDevelopment #BenchmarksEvaluation #AI #AIPulse
The comparison that sells it short is the honest one. Hydrogen is a fast general purpose model meant for everyday coding and agentic workflows, built to replace the earlier...
#LLM #SoftwareDevelopment #BenchmarksEvaluation #AI #AIPulse
The trouble teams have always described is not the agent itself, but the question that follows deployment: whether it finished the task the user asked, stayed...
#BenchmarksEvaluation #AIAgents #EnterpriseAI #AI #AIPulse
The trouble teams have always described is not the agent itself, but the question that follows deployment: whether it finished the task the user asked, stayed...
#BenchmarksEvaluation #AIAgents #EnterpriseAI #AI #AIPulse
IBM's Nighthawk r3 announcement is a statement about what a processor can do rather than what it actually does. It targets 10,000 operations...
#HardwareChips #BenchmarksEvaluation #EnergyCompute #AI #AIPulse
IBM's Nighthawk r3 announcement is a statement about what a processor can do rather than what it actually does. It targets 10,000 operations...
#HardwareChips #BenchmarksEvaluation #EnergyCompute #AI #AIPulse
The goal is not to change the model but to change the pipeline, and the recipe is the point. A general purpose 3D model like TRELLIS.2 produces a decent...
#InferenceOptimization #ModelTraining #BenchmarksEvaluation #AI #AIPulse
The goal is not to change the model but to change the pipeline, and the recipe is the point. A general purpose 3D model like TRELLIS.2 produces a decent...
#InferenceOptimization #ModelTraining #BenchmarksEvaluation #AI #AIPulse
The claim is that a single model can learn to perform a wide range of robot tasks with few demonstrations of the robot itself. The demonstration path is the interesting...
#Robotics #ModelTraining #BenchmarksEvaluation #AI #AIPulse
The claim is that a single model can learn to perform a wide range of robot tasks with few demonstrations of the robot itself. The demonstration path is the interesting...
#Robotics #ModelTraining #BenchmarksEvaluation #AI #AIPulse
The number that matters is not the accuracy, which beats its rivals across the board: 92.2 percent on one benchmark, 79.9 on another,...
#BenchmarksEvaluation #InferenceOptimization #OpenAI #AI #AIPulse
The number that matters is not the accuracy, which beats its rivals across the board: 92.2 percent on one benchmark, 79.9 on another,...
#BenchmarksEvaluation #InferenceOptimization #OpenAI #AI #AIPulse
The study's finding is that the majority of messages in these closed loop synchrotron experiments involved interpretation and judgment rather than formal...
#ScienceBiology #BenchmarksEvaluation #AIAgents #AI #AIPulse
The study's finding is that the majority of messages in these closed loop synchrotron experiments involved interpretation and judgment rather than formal...
#ScienceBiology #BenchmarksEvaluation #AIAgents #AI #AIPulse
The claim is a sharp one: 34 percent full plan completion on 100 multi step tasks, beating the competition and leaving MiniMax H3 trailing on single actions and human...
#BenchmarksEvaluation #Reasoning #Robotics #AI #AIPulse
The claim is a sharp one: 34 percent full plan completion on 100 multi step tasks, beating the competition and leaving MiniMax H3 trailing on single actions and human...
#BenchmarksEvaluation #Reasoning #Robotics #AI #AIPulse