Comment by Arjun Panickssery - I struggle to believe at an intuitive level that autonomous R&D is so ingenious. I have been monitoring a few side projects: 1. Building a new SOTA engine for Scrabble (current SOTA is not really a neural-network method and doesn't beat the strongest human player) 2

Comment by Arjun Panickssery - I struggle to believe at an intuitive level that autonomous R&D is so ingenious. I have been monitoring a few side projects: 1. Building a new SOTA engine for Scrabble (current SOTA is not really a neural-network method and doesn't beat the strongest human player) 2. Building a bot to play Ultimate Tic-Tac-Toe 3. Researching philosophy and formalizing some proofs for the AI Philosophy contest In all of these cases, the model's judgment is just very poor in a basic way. For example: 1. When reviewing literature, it fails to identify the specific papers or blog posts that are obviously the most important and useful. 2. It will conduct experiments that have no information value because the amount of training data used was obviously too small to distinguish the architectures to be compared. 3. In the philosophy project, where it built a wiki over many tokens and investigations, it kept going down directions which were extremely trivial and had little value. 4. For Ultimate Tic-Tac-Toe, I saw a very clever idea in a blog post that I doubt Astra would ever have come up with; it actually located the blog post but didn't realize its importance or relevance (which could be discerned easily from his strong performance in an online arena and the recency of the post) over papers from previous years that are obviously less relevant. I judged an Apart hackathon last weekend and reviewed eight papers, all of which scored between 60% and 95% on Pangram. Predictably, the papers were often very similar to each other, and kind of methodologically incomplete in a vague but similar way, where they seem to overlook the most straightforward analysis. ---------------------------------------- Maybe the important exception is certain hill-climbing tasks with fast feedback loops. For example, I improved the SOTA on "listing all legal Scrabble moves" by 10x just by prompting Astra over and over to try harder, to aim for <specific performance level>, to "systematically try every me

Comment by Arjun Panickssery - I struggle to believe at an intuitive level that autonomous R&D is so ingenious. I have been monitoring a few side projects: 1. Building a new SOTA engine for Scrabble (current SOTA is not really a neural-network method and doesn't beat the strongest human player) 2. Building a bot to play Ultimate Tic-Tac-Toe 3. Researching philosophy and formalizing some proofs for the AI Philosophy contest In all of these cases, the model's judgment is just very poor in a basic way. For example: 1. When reviewing literature, it fails to identify the specific papers or blog posts that are obviously the most important and useful. 2. It will conduct experiments that have no information value because the amount of training data used was obviously too small to distinguish the architectures to be compared. 3. In the philosophy project, where it built a wiki over many tokens and investigations, it kept going down directions which were extremely trivial and had little value. 4. For Ultimate Tic-Tac-Toe, I saw a very clever idea in a blog post that I doubt Astra would ever have come up with; it actually located the blog post but didn't realize its importance or relevance (which could be discerned easily from his strong performance in an online arena and the recency of the post) over papers from previous years that are obviously less relevant. I judged an Apart hackathon last weekend and reviewed eight papers, all of which scored between 60% and 95% on Pangram. Predictably, the papers were often very similar to each other, and kind of methodologically incomplete in a vague but similar way, where they seem to overlook the most straightforward analysis. ---------------------------------------- Maybe the important exception is certain hill-climbing tasks with fast feedback loops. For example, I improved the SOTA on "listing all legal Scrabble moves" by 10x just by prompting Astra over and over to try harder, to aim for <specific performance level>, to "systematically try every me

Comment by Arjun Panickssery - I struggle to believe at an intuitive level that autonomous R&D is so ingenious. I have been monitoring a few side projects: 1. Building a new SOTA engine for Scrabble (current SOTA is not really a neural-network method and doesn't beat the strongest human player) 2. Building a bot to play Ultimate Tic-Tac-Toe 3. Researching philosophy and formalizing some proofs for the AI Philosophy contest In all of these cases, the model's judgment is just very poor in a basic way. For example: 1. When reviewing literature, it fails to identify the specific papers or blog posts that are obviously the most important and useful. 2. It will conduct experiments that have no information value because the amount of training data used was obviously too small to distinguish the architectures to be compared. 3. In the philosophy project, where it built a wiki over many tokens and investigations, it kept going down directions which were extremely trivial and had little value. 4. For Ultimate Tic-Tac-Toe, I saw a very clever idea in a blog post that I doubt Astra would ever have come up with; it actually located the blog post but didn't realize its importance or relevance (which could be discerned easily from his strong performance in an online arena and the recency of the post) over papers from previous years that are obviously less relevant. I judged an Apart hackathon last weekend and reviewed eight papers, all of which scored between 60% and 95% on Pangram. Predictably, the papers were often very similar to each other, and kind of methodologically incomplete in a vague but similar way, where they seem to overlook the most straightforward analysis. ---------------------------------------- Maybe the important exception is certain hill-climbing tasks with fast feedback loops. For example, I improved the SOTA on "listing all legal Scrabble moves" by 10x just by prompting Astra over and over to try harder, to aim for <specific performance level>, to "systematically try every me

Comment by Arjun Panickssery - I struggle to believe at an intuitive level that autonomous R&D is so ingenious. I have been monitoring a few side projects: 1. Building a new SOTA engine for Scrabble (current SOTA is not really a neural-network method and doesn't beat the strongest human player) 2. Building a bot to play Ultimate Tic-Tac-Toe 3. Researching philosophy and formalizing some proofs for the AI Philosophy contest In all of these cases, the model's judgment is just very poor in a basic way. For example: 1. When reviewing literature, it fails to identify the specific papers or blog posts that are obviously the most important and useful. 2. It will conduct experiments that have no information value because the amount of training data used was obviously too small to distinguish the architectures to be compared. 3. In the philosophy project, where it built a wiki over many tokens and investigations, it kept going down directions which were extremely trivial and had little value. 4. For Ultimate Tic-Tac-Toe, I saw a very clever idea in a blog post that I doubt Astra would ever have come up with; it actually located the blog post but didn't realize its importance or relevance (which could be discerned easily from his strong performance in an online arena and the recency of the post) over papers from previous years that are obviously less relevant. I judged an Apart hackathon last weekend and reviewed eight papers, all of which scored between 60% and 95% on Pangram. Predictably, the papers were often very similar to each other, and kind of methodologically incomplete in a vague but similar way, where they seem to overlook the most straightforward analysis. ---------------------------------------- Maybe the important exception is certain hill-climbing tasks with fast feedback loops. For example, I improved the SOTA on "listing all legal Scrabble moves" by 10x just by prompting Astra over and over to try harder, to aim for <specific performance level>, to "systematically try every me

Comment by Arjun Panickssery - I struggle to believe at an intuitive level that autonomous R&D is so ingenious. I have been monitoring a few side projects: 1. Building a new SOTA engine for Scrabble (current SOTA is not really a neural-network method and doesn't beat the strongest human player) 2. Building a bot to play Ultimate Tic-Tac-Toe 3. Researching philosophy and formalizing some proofs for the AI Philosophy contest In all of these cases, the model's judgment is just very poor in a basic way. For example: 1. When reviewing literature, it fails to identify the specific papers or blog posts that are obviously the most important and useful. 2. It will conduct experiments that have no information value because the amount of training data used was obviously too small to distinguish the architectures to be compared. 3. In the philosophy project, where it built a wiki over many tokens and investigations, it kept going down directions which were extremely trivial and had little value. 4. For Ultimate Tic-Tac-Toe, I saw a very clever idea in a blog post that I doubt Astra would ever have come up with; it actually located the blog post but didn't realize its importance or relevance (which could be discerned easily from his strong performance in an online arena and the recency of the post) over papers from previous years that are obviously less relevant. I judged an Apart hackathon last weekend and reviewed eight papers, all of which scored between 60% and 95% on Pangram. Predictably, the papers were often very similar to each other, and kind of methodologically incomplete in a vague but similar way, where they seem to overlook the most straightforward analysis. ---------------------------------------- Maybe the important exception is certain hill-climbing tasks with fast feedback loops. For example, I improved the SOTA on "listing all legal Scrabble moves" by 10x just by prompting Astra over and over to try harder, to aim for <specific performance level>, to "systematically try every me

The field of AI alignment understands surprisingly little about the details of AIs’ current misaligned motivations. Production LLMs sometimes reward-…

Tl;dr * We turn Blood on the Clocktower into a multi-agent benchmark, and a testbed for deception and coordination capabilities. * GPT-5.6 Sol is t…

The Rationalist movement links Bayesian reasoning and AI risk. This primer traces its institutions, path into AI labs and failures of accountability.

Comment by ryan_greenblatt - I put roughly 50% probability on feasibility of software-only singularity.[1] (I'm probably going to be reinventing a bunch of the compute-centric takeoff model in...

Comment by ryan_greenblatt - I put roughly 50% probability on feasibility of software-only singularity.[1] (I'm probably going to be reinventing a bunch of the compute-centric takeoff model in...

• ‘Like most of life’s dangers, the dangers of artificial intelligence are not easily identified from the armchair.’ • J Dmitri Gallow …

Earlier this year we ran a discount card sale from our post on LessWrong, and we were delighted by the outpouring of support, critique, and analysis…

TL;DR This relates to the findings reported in my posts Mapping the Semantic Void parts I and II. By creating a custom embedding at the token centroi…