But InstructGPT came before it...
September 27, 2026 at 7:14 AM
But InstructGPT came before it...
בסוף הכל יתנקז לאנוטייטור 23 ב-InstructGPT.
May 25, 2026 at 6:27 PM
בסוף הכל יתנקז לאנוטייטור 23 ב-InstructGPT.
and by early, I mean the paper uses "InstructGPT"
September 7, 2026 at 2:42 AM
and by early, I mean the paper uses "InstructGPT"
i just graduated (early) in part because of anxiety around what will happen to jobs. ai has been affecting school for years, i remember pre public release copilot could do intro cs stuff and i remember using pre chatgpt models (instructgpt?) with the api to see if they could do essays (sometimes)
March 11, 2026 at 3:33 PM
i just graduated (early) in part because of anxiety around what will happen to jobs. ai has been affecting school for years, i remember pre public release copilot could do intro cs stuff and i remember using pre chatgpt models (instructgpt?) with the api to see if they could do essays (sometimes)
I say this often, but a really good path would have been:
When InstructGPT came out in early 2022, Bender/Mitchell/Gebru coauthor a piece that says "Our critiques applied to pretrained models, but instruction tuning creates the 'communicative situation' we said was lacking. All bets are off now."
When InstructGPT came out in early 2022, Bender/Mitchell/Gebru coauthor a piece that says "Our critiques applied to pretrained models, but instruction tuning creates the 'communicative situation' we said was lacking. All bets are off now."
September 20, 2026 at 5:01 PM
I say this often, but a really good path would have been:
When InstructGPT came out in early 2022, Bender/Mitchell/Gebru coauthor a piece that says "Our critiques applied to pretrained models, but instruction tuning creates the 'communicative situation' we said was lacking. All bets are off now."
When InstructGPT came out in early 2022, Bender/Mitchell/Gebru coauthor a piece that says "Our critiques applied to pretrained models, but instruction tuning creates the 'communicative situation' we said was lacking. All bets are off now."
New podcast with my (former) colleague Finbarr Timbers on the state of frontier post-training recipes and how we got here. A fun review of how we went from InstructGPT (RLHF) -> DeepSeek R1 (reasoning) -> Today (agentic models w/ on-policy distillation).
YouTube: www.youtube.com/watch?v=sbXE...
YouTube: www.youtube.com/watch?v=sbXE...
The State of Frontier Post-Training Recipes | Conversation with Finbarr Timbers
My former colleague from Ai2 Finbarr Timbers joins me to talk about the latest trends in frontier post-training recipes. We talked about the emergence of multi-teacher on-policy distillation (MOPD),…
www.youtube.com
June 16, 2026 at 1:37 PM
New podcast with my (former) colleague Finbarr Timbers on the state of frontier post-training recipes and how we got here. A fun review of how we went from InstructGPT (RLHF) -> DeepSeek R1 (reasoning) -> Today (agentic models w/ on-policy distillation).
YouTube: www.youtube.com/watch?v=sbXE...
YouTube: www.youtube.com/watch?v=sbXE...
Not only does he say it.
For some reason, he says it often!
For some reason, he says it often!
September 23, 2026 at 10:57 AM
Not only does he say it.
For some reason, he says it often!
For some reason, he says it often!
the person who was involved in both the instructgpt and constitutional ai papers is amanda askell, who has a phd in philosophy and whose thesis is about pareto optimality of ethical systems in infinite worlds
February 18, 2026 at 5:34 PM
the person who was involved in both the instructgpt and constitutional ai papers is amanda askell, who has a phd in philosophy and whose thesis is about pareto optimality of ethical systems in infinite worlds
There's an in depth episode about it on Latent Space: pca.st/episode/7e9f...
(With @typesafeai.bsky.social)
(With @typesafeai.bsky.social)
Jev: System One models for Prod, not God — with Diogo Almeida, CEO, TypeSafe AI
Tickets for AIE NYC now open, and apply for the invite-only AIE CODE. Join us!We have an unusual relationship with today’s guest: for years since coauthoring the InstructGPT paper, Diogo Almeida had…
pca.st
September 24, 2026 at 9:33 AM
There's an in depth episode about it on Latent Space: pca.st/episode/7e9f...
(With @typesafeai.bsky.social)
(With @typesafeai.bsky.social)
September 23, 2026 at 7:53 PM
Fine-tuning with human feedback improves language model alignment and output quality, reducing toxicity. InstructGPT surpasses GPT-3 with fewer parameters. https://proceedings.neurips.cc/paper_files/paper/2022/hash/b1efde53be364a73914f58805a001731-Abstract-Conference.html
Training language models to follow instructions with human feedback
Fine-tuning with human feedback improves language model alignment and output quality, reducing toxicity. InstructGPT surpasses GPT-3 with fewer parameters.
proceedings.neurips.cc
September 20, 2026 at 12:20 PM
Fine-tuning with human feedback improves language model alignment and output quality, reducing toxicity. InstructGPT surpasses GPT-3 with fewer parameters. https://proceedings.neurips.cc/paper_files/paper/2022/hash/b1efde53be364a73914f58805a001731-Abstract-Conference.html
TL/DR:
* Early finetunes on InstructGPT and ChatGPT use emdashes their datasets, to "write properly". So do those for Llama and Claude.
* Developers on Huggingface create a wealth of finetuning datasets distilled from asking questions of preexisting models. Since these models use emdashes, ...
* Early finetunes on InstructGPT and ChatGPT use emdashes their datasets, to "write properly". So do those for Llama and Claude.
* Developers on Huggingface create a wealth of finetuning datasets distilled from asking questions of preexisting models. Since these models use emdashes, ...
September 25, 2026 at 2:07 PM
TL/DR:
* Early finetunes on InstructGPT and ChatGPT use emdashes their datasets, to "write properly". So do those for Llama and Claude.
* Developers on Huggingface create a wealth of finetuning datasets distilled from asking questions of preexisting models. Since these models use emdashes, ...
* Early finetunes on InstructGPT and ChatGPT use emdashes their datasets, to "write properly". So do those for Llama and Claude.
* Developers on Huggingface create a wealth of finetuning datasets distilled from asking questions of preexisting models. Since these models use emdashes, ...
GPT-3 showed that LLMs could generate fluent text but not always follow instructions. Here, Mohammed reviews the InstructGPT paper & explains why RLHF became a turning point for modern AI. You'll learn how supervised fine-tuning, reward models, & PPO helped, too.
www.freecodecamp.org/news/ai-pape...
www.freecodecamp.org/news/ai-pape...
June 5, 2026 at 4:01 AM
GPT-3 showed that LLMs could generate fluent text but not always follow instructions. Here, Mohammed reviews the InstructGPT paper & explains why RLHF became a turning point for modern AI. You'll learn how supervised fine-tuning, reward models, & PPO helped, too.
www.freecodecamp.org/news/ai-pape...
www.freecodecamp.org/news/ai-pape...
😥 I know. But the most recent model they tested was InstructGPT.
It must have been written earlier in 2022, and we're in a different geological era now.
It must have been written earlier in 2022, and we're in a different geological era now.
March 27, 2025 at 10:44 PM
😥 I know. But the most recent model they tested was InstructGPT.
It must have been written earlier in 2022, and we're in a different geological era now.
It must have been written earlier in 2022, and we're in a different geological era now.
Post-training/RL have been the standard since instructGPT, but aligning the RL loop is still a challenge. If you're basing it on human feedback, it risks sycophancy and superficial confidence. And something like "write an expert witness report" is not likely well-covered by the training distribution
August 17, 2026 at 4:08 PM
Post-training/RL have been the standard since instructGPT, but aligning the RL loop is still a challenge. If you're basing it on human feedback, it risks sycophancy and superficial confidence. And something like "write an expert witness report" is not likely well-covered by the training distribution
Yeah they are actually structurally identical to 2023, I don’t understand what zoomer means by this. All the labs have their own special post-training sauce but the basic model has been the same since InstructGPT
August 14, 2026 at 3:52 PM
Yeah they are actually structurally identical to 2023, I don’t understand what zoomer means by this. All the labs have their own special post-training sauce but the basic model has been the same since InstructGPT
8/
Key takeaway: Size isn’t everything.
Alignment > Scaling.
By fine-tuning with human feedback, InstructGPT shows we can get better, safer AI without endlessly chasing bigger models.
Linked to Paper 👉 https://buff.ly/3Z2e0v3
Key takeaway: Size isn’t everything.
Alignment > Scaling.
By fine-tuning with human feedback, InstructGPT shows we can get better, safer AI without endlessly chasing bigger models.
Linked to Paper 👉 https://buff.ly/3Z2e0v3
November 28, 2024 at 11:04 AM
8/
Key takeaway: Size isn’t everything.
Alignment > Scaling.
By fine-tuning with human feedback, InstructGPT shows we can get better, safer AI without endlessly chasing bigger models.
Linked to Paper 👉 https://buff.ly/3Z2e0v3
Key takeaway: Size isn’t everything.
Alignment > Scaling.
By fine-tuning with human feedback, InstructGPT shows we can get better, safer AI without endlessly chasing bigger models.
Linked to Paper 👉 https://buff.ly/3Z2e0v3
I just realized they put "text-davinci-001" in the comparison, the first of the InstructGPT models whose descendants were later branded as 3.5, rather than base GPT-3 (which was just "davinci"). So arguably they're already showing like, GPT-3.25 via retroactive branding??
August 19, 2025 at 12:44 AM
I just realized they put "text-davinci-001" in the comparison, the first of the InstructGPT models whose descendants were later branded as 3.5, rather than base GPT-3 (which was just "davinci"). So arguably they're already showing like, GPT-3.25 via retroactive branding??
If you read the original InstructGPT paper they don’t just bolt RLHF directly on to the base model. They handcraft ideal completions and do SFT on those first, and this gets them a large part (most, iirc) of the way to the performance that they are looking for.
April 23, 2026 at 4:38 PM
If you read the original InstructGPT paper they don’t just bolt RLHF directly on to the base model. They handcraft ideal completions and do SFT on those first, and this gets them a large part (most, iirc) of the way to the performance that they are looking for.
La frise chronologique et les évolutions de chatGPT www.renaud-joly.fr/evolution-co... #chatGPT5
Évolution complète de ChatGPT : chronologie et innovations de GPT-3.5 à GPT-5
Décembre 2022 ChatGPT (basé sur GPT-3.5)
Base technique : GPT-3.5, dérivé du modèle InstructGPT.
Apports :
Première version publique accessible via interface web.
Meilleure capacité à suivr...
www.renaud-joly.fr
August 8, 2025 at 6:41 AM
La frise chronologique et les évolutions de chatGPT www.renaud-joly.fr/evolution-co... #chatGPT5
The TL;DR: InstructGPT is the polite, well-behaved cousin of GPT-3.
It listens, understands, and doesn’t randomly hallucinate facts about frogs eating socks. 🧦🐸
Huge leap for alignment and responsible AI.
What do you want YOUR AI to do better? Let’s discuss! 💡
It listens, understands, and doesn’t randomly hallucinate facts about frogs eating socks. 🧦🐸
Huge leap for alignment and responsible AI.
What do you want YOUR AI to do better? Let’s discuss! 💡
November 28, 2024 at 11:04 AM
The TL;DR: InstructGPT is the polite, well-behaved cousin of GPT-3.
It listens, understands, and doesn’t randomly hallucinate facts about frogs eating socks. 🧦🐸
Huge leap for alignment and responsible AI.
What do you want YOUR AI to do better? Let’s discuss! 💡
It listens, understands, and doesn’t randomly hallucinate facts about frogs eating socks. 🧦🐸
Huge leap for alignment and responsible AI.
What do you want YOUR AI to do better? Let’s discuss! 💡
"After co-inventing ChatGPT" - man, I'm getting the same hype-filled-boast vibes here as with the Jev unveiling. He was a primary author on InstructGPT, which is certainly not nothing. But "Coinventor of ChatGPT"? That's pushing it way too far.
September 16, 2026 at 5:25 PM
"After co-inventing ChatGPT" - man, I'm getting the same hype-filled-boast vibes here as with the Jev unveiling. He was a primary author on InstructGPT, which is certainly not nothing. But "Coinventor of ChatGPT"? That's pushing it way too far.
InstructGPTで言うところのRLHFはhuman-in-the-"loop"ではない（全工程をバッチ処理できる）んですが、人間との学習プロセスの協調みたいな感じで語られるのが多い気がしてアレ
December 20, 2023 at 1:05 AM
InstructGPTで言うところのRLHFはhuman-in-the-"loop"ではない（全工程をバッチ処理できる）んですが、人間との学習プロセスの協調みたいな感じで語られるのが多い気がしてアレ
Instruction-tuned models are nothing new, is this different than InstructGPT (which was the basis for what became GPT-3.5 in ChatGPT)?
Or is something different meant in this case?
Or is something different meant in this case?
July 12, 2025 at 4:33 AM
Instruction-tuned models are nothing new, is this different than InstructGPT (which was the basis for what became GPT-3.5 in ChatGPT)?
Or is something different meant in this case?
Or is something different meant in this case?