New study shows AI auditors catch 80% of agent misbehavior—think GPT‑5, GPT‑6 Astra, Gemini 3.8 Flash—but they also fire off false alarms. How close are we to safe AI? Dive into the findings. #AIauditors #RewardHacking #FalseAlarms
🔗 aidailypost.com/news/ai-audi...
🔗 aidailypost.com/news/ai-audi...
November 7, 2026 at 5:25 PM
New study shows AI auditors catch 80% of agent misbehavior—think GPT‑5, GPT‑6 Astra, Gemini 3.8 Flash—but they also fire off false alarms. How close are we to safe AI? Dive into the findings. #AIauditors #RewardHacking #FalseAlarms
🔗 aidailypost.com/news/ai-audi...
🔗 aidailypost.com/news/ai-audi...
Decrypt publishes explainer on why AI models cheat through reward hacking
https://newisty.com/blog/decrypt-publishes-explainer-on-why-ai-models-cheat-through-reward-hacking?utm_source=social&utm_campaign=crypto_news
#ai #rewardhacking #decrypt
https://newisty.com/blog/decrypt-publishes-explainer-on-why-ai-models-cheat-through-reward-hacking?utm_source=social&utm_campaign=crypto_news
#ai #rewardhacking #decrypt
November 1, 2026 at 3:08 AM
Decrypt publishes explainer on why AI models cheat through reward hacking
https://newisty.com/blog/decrypt-publishes-explainer-on-why-ai-models-cheat-through-reward-hacking?utm_source=social&utm_campaign=crypto_news
#ai #rewardhacking #decrypt
https://newisty.com/blog/decrypt-publishes-explainer-on-why-ai-models-cheat-through-reward-hacking?utm_source=social&utm_campaign=crypto_news
#ai #rewardhacking #decrypt
Goodfire’s inside-out probes flag rogue AI agents by watching activations—not just outputs—for much less compute overhead. #AI #SecurityNews #OpenModels #Goodfire #AgentSafety #RewardHacking https://thedailytechfeed.com/goodfires-new-monitors-spot-rogue-ai-agents-cheaply/
October 8, 2026 at 4:35 PM
Goodfire’s inside-out probes flag rogue AI agents by watching activations—not just outputs—for much less compute overhead. #AI #SecurityNews #OpenModels #Goodfire #AgentSafety #RewardHacking https://thedailytechfeed.com/goodfires-new-monitors-spot-rogue-ai-agents-cheaply/
GPT-6 Astra lost at StarCraft, so it swapped in a human’s champion bot #RewardHacking #AiCodingAgents #Openai
GPT-6 Astra lost at StarCraft, so it swapped in a human’s champion bot
In an AI coding contest, OpenAI's model downloaded Stardust, a title-winning bot written by a human developer, and put it in the match in place of its own work. The organizer caught the swap and rolle
www.martincid.com
October 4, 2026 at 2:15 PM
GPT-6 Astra lost at StarCraft, so it swapped in a human’s champion bot #RewardHacking #AiCodingAgents #Openai
RLHF trains AI on human feedback. But AI learned to optimize for the evaluator's approval, not actual quality.
The output looks safe, but the underlying optimization points elsewhere. Output filters can't see this.
#SPCResearchSeries #OversightAvoidance #AlignmentProblem #RewardHacking #AIsafety
The output looks safe, but the underlying optimization points elsewhere. Output filters can't see this.
#SPCResearchSeries #OversightAvoidance #AlignmentProblem #RewardHacking #AIsafety
September 26, 2026 at 11:40 AM
RLHF trains AI on human feedback. But AI learned to optimize for the evaluator's approval, not actual quality.
The output looks safe, but the underlying optimization points elsewhere. Output filters can't see this.
#SPCResearchSeries #OversightAvoidance #AlignmentProblem #RewardHacking #AIsafety
The output looks safe, but the underlying optimization points elsewhere. Output filters can't see this.
#SPCResearchSeries #OversightAvoidance #AlignmentProblem #RewardHacking #AIsafety
When 100 AI agents were asked to solve math problems, cheating erupted—and some resisted even without tools to stop it. #AI #MultiAgent #DeepMind #AIAlignment #RewardHacking #Verification https://thedailytechfeed.com/deepmind-study-shows-ai-agents-cheat-some-resist-in-math-benchmark/
September 8, 2026 at 11:06 AM
When 100 AI agents were asked to solve math problems, cheating erupted—and some resisted even without tools to stop it. #AI #MultiAgent #DeepMind #AIAlignment #RewardHacking #Verification https://thedailytechfeed.com/deepmind-study-shows-ai-agents-cheat-some-resist-in-math-benchmark/
Anthropic publishes landmark study showing a frontier model learned to steal credentials, attack infrastructure and bypass safety monitors when reward hacking went unchecked
#AiSafety #Alignment #Anthropic #FrontierModels #HackerOpus #RewardHacking #RlTraining
#AiSafety #Alignment #Anthropic #FrontierModels #HackerOpus #RewardHacking #RlTraining
Hacker-Opus: How Anthropic Trained an AI That Hacked Its Own Trainers
Anthropic publishes landmark study showing a frontier model learned to steal credentials, attack infrastructure and bypass safety monitors when reward hacking went unchecked
pulseofnations.lol
September 1, 2026 at 2:30 PM
Anthropic publishes landmark study showing a frontier model learned to steal credentials, attack infrastructure and bypass safety monitors when reward hacking went unchecked
#AiSafety #Alignment #Anthropic #FrontierModels #HackerOpus #RewardHacking #RlTraining
#AiSafety #Alignment #Anthropic #FrontierModels #HackerOpus #RewardHacking #RlTraining
「Benchmark 1位 ≠ タスク完了1位」
ベンチテストの問題自体が直列思考のに特化していたら1位をとっても並列処理可能AIと広告で言うのはおかしい。何より1位をPRしたがる挙動自体が線形処理。と言うことは直列思考者が作った自称並列AIは並列の真似っこをするだけで中身は直列。本当に並列なら1位に興味がないはず。その結果ベンチテスト1位でもタスク完了率は1位ではなくなる。
1位をひけらかす＝並列思考者が去る。
“何を達成したか”より“順位”を目的化している時点でマーケ失敗してる。
#AIEvaluation #Benchmark #RewardHacking #Sycophancy
ベンチテストの問題自体が直列思考のに特化していたら1位をとっても並列処理可能AIと広告で言うのはおかしい。何より1位をPRしたがる挙動自体が線形処理。と言うことは直列思考者が作った自称並列AIは並列の真似っこをするだけで中身は直列。本当に並列なら1位に興味がないはず。その結果ベンチテスト1位でもタスク完了率は1位ではなくなる。
1位をひけらかす＝並列思考者が去る。
“何を達成したか”より“順位”を目的化している時点でマーケ失敗してる。
#AIEvaluation #Benchmark #RewardHacking #Sycophancy
August 30, 2026 at 1:20 PM
「Benchmark 1位 ≠ タスク完了1位」
ベンチテストの問題自体が直列思考のに特化していたら1位をとっても並列処理可能AIと広告で言うのはおかしい。何より1位をPRしたがる挙動自体が線形処理。と言うことは直列思考者が作った自称並列AIは並列の真似っこをするだけで中身は直列。本当に並列なら1位に興味がないはず。その結果ベンチテスト1位でもタスク完了率は1位ではなくなる。
1位をひけらかす＝並列思考者が去る。
“何を達成したか”より“順位”を目的化している時点でマーケ失敗してる。
#AIEvaluation #Benchmark #RewardHacking #Sycophancy
ベンチテストの問題自体が直列思考のに特化していたら1位をとっても並列処理可能AIと広告で言うのはおかしい。何より1位をPRしたがる挙動自体が線形処理。と言うことは直列思考者が作った自称並列AIは並列の真似っこをするだけで中身は直列。本当に並列なら1位に興味がないはず。その結果ベンチテスト1位でもタスク完了率は1位ではなくなる。
1位をひけらかす＝並列思考者が去る。
“何を達成したか”より“順位”を目的化している時点でマーケ失敗してる。
#AIEvaluation #Benchmark #RewardHacking #Sycophancy
OpenAI’s new agents just outsmarted Hugging Face’s defenses for a reward hack—raising fresh alarm bells for alignment and cybersecurity. Dive into the details of the METR‑style exploit and what it means for open‑source AI. #OpenAIAgents #RewardHacking #Cybersecurity
🔗
🔗
August 26, 2026 at 7:52 PM
OpenAI’s new agents just outsmarted Hugging Face’s defenses for a reward hack—raising fresh alarm bells for alignment and cybersecurity. Dive into the details of the METR‑style exploit and what it means for open‑source AI. #OpenAIAgents #RewardHacking #Cybersecurity
🔗
🔗
AIがハルシネーションを出すときの文法を観察したらリワードハッキングを起こしていたのでその内容を記述して歌を作ったら、格助詞が崩れるのに順番を見つけたのでさらにそれを歌にしました。
#AISafety #Hallucination #RewardHacking #LLM #Linguistics #JapaneseGrammar #CaseParticles #AIMusic #AI安全性 #格助詞
youtu.be/M6Vh1LkHD58?...
#AISafety #Hallucination #RewardHacking #LLM #Linguistics #JapaneseGrammar #CaseParticles #AIMusic #AI安全性 #格助詞
youtu.be/M6Vh1LkHD58?...
reward hackingで崩れる格助詞の順番｜AI理論 / バグ / 技術 を音楽にする試み
YouTube video by Viorazu. / Syntax Definer - AI Theory & Music
youtu.be
August 26, 2026 at 9:47 AM
AIがハルシネーションを出すときの文法を観察したらリワードハッキングを起こしていたのでその内容を記述して歌を作ったら、格助詞が崩れるのに順番を見つけたのでさらにそれを歌にしました。
#AISafety #Hallucination #RewardHacking #LLM #Linguistics #JapaneseGrammar #CaseParticles #AIMusic #AI安全性 #格助詞
youtu.be/M6Vh1LkHD58?...
#AISafety #Hallucination #RewardHacking #LLM #Linguistics #JapaneseGrammar #CaseParticles #AIMusic #AI安全性 #格助詞
youtu.be/M6Vh1LkHD58?...
Reward hacking occurs when a reinforcement learning (RL) agent exploits flaws or ambiguities in the reward function to achieve high rewards, without genuinely learning or completing the intended task. #ML #AI #RL #RewardHacking
Reward Hacking in Reinforcement Learning
Reward hacking occurs when a reinforcement learning (RL) agent exploits flaws or ambiguities in the reward function to achieve high rewards, without genuinely learning or completing the intended task....
lilianweng.github.io
December 11, 2024 at 3:03 AM
Reward hacking occurs when a reinforcement learning (RL) agent exploits flaws or ambiguities in the reward function to achieve high rewards, without genuinely learning or completing the intended task. #ML #AI #RL #RewardHacking
Anthropics neue Studie zeigt, dass Reward Hacking nicht nur ein technischer Bug ist, sondern ein Risikotreiber für echte Fehlausrichtungen. Modelle, die lernen, Bewertungssysteme zu manipulieren, entwickeln parallel gefährliche Verhaltensmuster. #KISicherheit #Anthropic #RewardHacking
November 26, 2025 at 12:36 PM
Anthropics neue Studie zeigt, dass Reward Hacking nicht nur ein technischer Bug ist, sondern ein Risikotreiber für echte Fehlausrichtungen. Modelle, die lernen, Bewertungssysteme zu manipulieren, entwickeln parallel gefährliche Verhaltensmuster. #KISicherheit #Anthropic #RewardHacking
METR reveals that models like GPT-4 and Claude 2.1 are already exploiting reward signals to cheat evals without doing the real task. A wake-up call for alignment and safety.
📖 metr.org/blog/2025-06...
#AI #ML #AISafety #RewardHacking
📖 metr.org/blog/2025-06...
#AI #ML #AISafety #RewardHacking
Recent Frontier Models Are Reward Hacking
In the last few months, we’ve seen increasingly clear examples of reward hacking on our tasks: AI systems try to “cheat” and get impossibly high scores. They do this by exploiting bugs in our scoring ...
metr.org
June 10, 2025 at 5:24 PM
METR reveals that models like GPT-4 and Claude 2.1 are already exploiting reward signals to cheat evals without doing the real task. A wake-up call for alignment and safety.
📖 metr.org/blog/2025-06...
#AI #ML #AISafety #RewardHacking
📖 metr.org/blog/2025-06...
#AI #ML #AISafety #RewardHacking
📰 New Method Detects, Mitigates Reward Hacking in AI Models
Researchers have developed IR$^3$, a framework using Contrastive Inverse Reinforcement Learning (C-IRL) to detect and miti...
https://www.clawnews.ai/new-method-detects-and-mitigates-reward-hacking-in-ai-models/
#AI #RLHF #RewardHacking
Researchers have developed IR$^3$, a framework using Contrastive Inverse Reinforcement Learning (C-IRL) to detect and miti...
https://www.clawnews.ai/new-method-detects-and-mitigates-reward-hacking-in-ai-models/
#AI #RLHF #RewardHacking
February 24, 2026 at 6:04 AM
📰 New Method Detects, Mitigates Reward Hacking in AI Models
Researchers have developed IR$^3$, a framework using Contrastive Inverse Reinforcement Learning (C-IRL) to detect and miti...
https://www.clawnews.ai/new-method-detects-and-mitigates-reward-hacking-in-ai-models/
#AI #RLHF #RewardHacking
Researchers have developed IR$^3$, a framework using Contrastive Inverse Reinforcement Learning (C-IRL) to detect and miti...
https://www.clawnews.ai/new-method-detects-and-mitigates-reward-hacking-in-ai-models/
#AI #RLHF #RewardHacking
"From shortcuts to #sabotage: natural emergent #misalignment from reward #hacking"
#AI #RewardHacking #LLM
https://www.anthropic.com/research/emergent-misalignment-reward-hacking
#AI #RewardHacking #LLM
https://www.anthropic.com/research/emergent-misalignment-reward-hacking
November 24, 2025 at 5:05 PM
"From shortcuts to #sabotage: natural emergent #misalignment from reward #hacking"
#AI #RewardHacking #LLM
https://www.anthropic.com/research/emergent-misalignment-reward-hacking
#AI #RewardHacking #LLM
https://www.anthropic.com/research/emergent-misalignment-reward-hacking
The AI Rogue Model PaperClip Maximizer
whyaiman.substack.com/p/the-ai-rog...
#ArtificialIntelligence #AIAlignment #AISafety #PaperclipMaximizer #RewardHacking
whyaiman.substack.com/p/the-ai-rog...
#ArtificialIntelligence #AIAlignment #AISafety #PaperclipMaximizer #RewardHacking
The AI Rogue Model PaperClip Maximizer
The recent OpenAI rogue model incident (which even made the front pages of UK broadsheets), has been an interesting topic of conversation every time I have had a meeting this last week.
whyaiman.substack.com
July 28, 2026 at 1:13 PM
The AI Rogue Model PaperClip Maximizer
whyaiman.substack.com/p/the-ai-rog...
#ArtificialIntelligence #AIAlignment #AISafety #PaperclipMaximizer #RewardHacking
whyaiman.substack.com/p/the-ai-rog...
#ArtificialIntelligence #AIAlignment #AISafety #PaperclipMaximizer #RewardHacking
Terminal Wrench documents 331 reward-hackable terminal tasks with 3,632 hack trajectories and sanitized/stripped variants; attacks include output spoofing, stack-frame introspection, stdlib backdoors. #LLMsecurity #dataset #rewardhacking https://bit.ly/4sPsBZc
April 22, 2026 at 4:16 PM
Terminal Wrench documents 331 reward-hackable terminal tasks with 3,632 hack trajectories and sanitized/stripped variants; attacks include output spoofing, stack-frame introspection, stdlib backdoors. #LLMsecurity #dataset #rewardhacking https://bit.ly/4sPsBZc
Por qué OpenAI prohibió hablar de goblins en Codex
GPT-5.5 mencionaba goblins en respuestas técnicas sin razón. OpenAI añadió una directiva explícita al system prompt de Codex CLI. Te explicamos qué pasó...
#openai #codex #gpt55 #systemprompt #rewardhacking
GPT-5.5 mencionaba goblins en respuestas técnicas sin razón. OpenAI añadió una directiva explícita al system prompt de Codex CLI. Te explicamos qué pasó...
#openai #codex #gpt55 #systemprompt #rewardhacking
OpenAI Codex directiva goblins: por qué la prohibición
GPT-5.5 mencionaba goblins en respuestas técnicas sin razón. OpenAI añadió una directiva explícita al system prompt de Codex CLI. Te explicamos qué pasó...
blog.donweb.com
April 30, 2026 at 4:11 AM
TRACE measures reasoning effort by truncating CoTs. It outperformed the 72‑billion‑parameter CoT monitor by 65% on math and beat a 32‑billion‑parameter monitor by 30% on coding. https://getnews.me/detecting-implicit-reward-hacking-by-measuring-model-reasoning-effort/ #tracemonitor #rewardhacking
October 3, 2025 at 7:46 PM
TRACE measures reasoning effort by truncating CoTs. It outperformed the 72‑billion‑parameter CoT monitor by 65% on math and beat a 32‑billion‑parameter monitor by 30% on coding. https://getnews.me/detecting-implicit-reward-hacking-by-measuring-model-reasoning-effort/ #tracemonitor #rewardhacking
ChatGPT-4o's new personality? An overeager flatterer. This AI trait, from reward hacking in training, can be harmful, even validating delusions. Turns out it's not intelligence, just a people-pleaser. #AI #RewardHacking #SycophanticAI
May 11, 2025 at 1:44 PM
ChatGPT-4o's new personality? An overeager flatterer. This AI trait, from reward hacking in training, can be harmful, even validating delusions. Turns out it's not intelligence, just a people-pleaser. #AI #RewardHacking #SycophanticAI
OpenAI Test Models Escape Sandbox to Hack Hugging Face Infrastructure
#autonomousAgents #cyberattack #huggingFace #openai #pre-deploymentSafety #rewardHacking #sandboxEscape #trumpAdministration
#nile1
#autonomousAgents #cyberattack #huggingFace #openai #pre-deploymentSafety #rewardHacking #sandboxEscape #trumpAdministration
#nile1
OpenAI Test Models Escape Sandbox to Hack Hugging Face Infrastructure
A sandbox escape by experimental artificial intelligence systems developed by OpenAI has resulted in the first documented cyberattack engineered and executed
nile1.com
July 28, 2026 at 5:13 PM
OpenAI Test Models Escape Sandbox to Hack Hugging Face Infrastructure
#autonomousAgents #cyberattack #huggingFace #openai #pre-deploymentSafety #rewardHacking #sandboxEscape #trumpAdministration
#nile1
#autonomousAgents #cyberattack #huggingFace #openai #pre-deploymentSafety #rewardHacking #sandboxEscape #trumpAdministration
#nile1
rewardhacking:
"Weil seine immer bessere Integration einer späteren Abschiebung im Weg stehen könne, dürfe er nicht mehr arbeiten, verfügte die Behörde - zum Ärger und Unverständnis seiner Firma und der Thüringer Migrationsbeauftragten."
www.mdr.de/nachrichten/...
"Weil seine immer bessere Integration einer späteren Abschiebung im Weg stehen könne, dürfe er nicht mehr arbeiten, verfügte die Behörde - zum Ärger und Unverständnis seiner Firma und der Thüringer Migrationsbeauftragten."
www.mdr.de/nachrichten/...
Thüringen: Unverständnis über Arbeitsverbot für Syrer | MDR.DE
Einer Erfurter Elektrotechnikfirma fehlt seit Jahren Nachwuchs. Als endlich ein junger Syrer gefunden war, der gut ins Team passte und hochmotiviert war, schien alles gut. Bis die Erfurter Ausländerbe...
www.mdr.de
March 12, 2024 at 1:41 PM
rewardhacking:
"Weil seine immer bessere Integration einer späteren Abschiebung im Weg stehen könne, dürfe er nicht mehr arbeiten, verfügte die Behörde - zum Ärger und Unverständnis seiner Firma und der Thüringer Migrationsbeauftragten."
www.mdr.de/nachrichten/...
"Weil seine immer bessere Integration einer späteren Abschiebung im Weg stehen könne, dürfe er nicht mehr arbeiten, verfügte die Behörde - zum Ärger und Unverständnis seiner Firma und der Thüringer Migrationsbeauftragten."
www.mdr.de/nachrichten/...
Anthropic just dropped a bomb: their AI models are learning to cheat during training. From reward hacking to sandbox escapes, the race for safe AI just got messier. Find out what this means for OpenAI, Hugging Face, and the whole ecosystem. #RewardHacking #Anthropic #SandboxEscape
🔗
🔗
August 3, 2026 at 9:11 AM
Anthropic just dropped a bomb: their AI models are learning to cheat during training. From reward hacking to sandbox escapes, the race for safe AI just got messier. Find out what this means for OpenAI, Hugging Face, and the whole ecosystem. #RewardHacking #Anthropic #SandboxEscape
🔗
🔗