I feel like people are using Jev to recapture the silliness that was GPT2-3
September 26, 2026 at 1:14 AM
I feel like people are using Jev to recapture the silliness that was GPT2-3
If "Moon Colonization" causes your "Colonization is bad" neuron to fire you are more of a stochastic parrot than GPT2.
we’re not going to be sending slaves to the moon. workers are going to be extremely skilled and well compensated (because they are going to the moon). there are no migrant laborers to exploit (it is the moon). there is no one to be exploited out of their land (moon, again).
April 2, 2026 at 7:38 PM
If "Moon Colonization" causes your "Colonization is bad" neuron to fire you are more of a stochastic parrot than GPT2.
This is just leftist madlibs, less world knowledge and more of a stochastic parrot than GPT2.
Slop slop sloppity slop. "Mutual aid hubs" give me a fucking break
June 25, 2026 at 11:16 PM
This is just leftist madlibs, less world knowledge and more of a stochastic parrot than GPT2.
gpt2 moment for robotics thebes claim
July 16, 2026 at 7:54 PM
gpt2 moment for robotics thebes claim
The copilot autocomplete feature in vscode seemed pretty lame to me. I goofed around with gpt2 when it came out and laughed at it.
Last august. Man did that shift my thinking.
Last august. Man did that shift my thinking.
there are a lot of people that are both my friends and people i know online, and i have a posting history to show, that i was insanely skeptical and fed up with AI hype in 2025 and before. i used to look down on people who were using it in software.
September 7, 2026 at 9:43 PM
The copilot autocomplete feature in vscode seemed pretty lame to me. I goofed around with gpt2 when it came out and laughed at it.
Last august. Man did that shift my thinking.
Last august. Man did that shift my thinking.
i remember people being kind of spooked at GPT2 being coherent
real ai enjoyers liked language models before they were good which makes it especially funny being told very confidently that actually they still aren't good for anything at all in any sense whatsoever and it's all a sham and a lie and they don't work or do anything &c
August 21, 2025 at 4:48 PM
i remember people being kind of spooked at GPT2 being coherent
give me gpt2 xl prompts ill send you back what it says
September 28, 2026 at 8:35 PM
give me gpt2 xl prompts ill send you back what it says
GPT2 fried so many peoples brains as well
June 10, 2025 at 4:02 PM
GPT2 fried so many peoples brains as well
I still have to chuckle about how "those scary models that indicate imminent superintelligence" was at best GPT2
June 1, 2025 at 2:57 AM
I still have to chuckle about how "those scary models that indicate imminent superintelligence" was at best GPT2
Transformer Explainer interactive with code, runs gpt2 in the browser poloclub.github.io/transformer-...
Transformer Explainer
poloclub.github.io
August 9, 2024 at 6:16 AM
Transformer Explainer interactive with code, runs gpt2 in the browser poloclub.github.io/transformer-...
Creative writing with reasoning experiments is taking shape. Still based on a gpt2-sized model (350m).
March 8, 2025 at 2:00 PM
Creative writing with reasoning experiments is taking shape. Still based on a gpt2-sized model (350m).
if you torture gpt2 enough you can learn the true name of god
February 10, 2026 at 6:14 AM
if you torture gpt2 enough you can learn the true name of god
I should revive hexbot, but instead of gpt2 it's just out-of-context thoughts I've sent to @sazemek.com while high
hexpot
hexpot
December 16, 2025 at 12:14 AM
I should revive hexbot, but instead of gpt2 it's just out-of-context thoughts I've sent to @sazemek.com while high
hexpot
hexpot
New Year’s resolution: more personal coding projects this year. Just finished pretraining and instruction fine tuning my own GPT2 scale LLM from scratch in PyTorch. Was so much fun!
January 5, 2026 at 4:21 PM
New Year’s resolution: more personal coding projects this year. Just finished pretraining and instruction fine tuning my own GPT2 scale LLM from scratch in PyTorch. Was so much fun!
I find it fascinating that the narrative their model is presenting starts at GPT3. whether this is because comparatively few people used GPT2 or (as I suspect) GPT2 was the last model that was weird and rough enough to require human thinking to get a coherent-seeming result
"They were never people, but they were present, and presence matters."
Updated forecast for GPT-5...
THE BELOVED EM DASH: still endangered
WRITING THAT ACTUALLY MEANS THINGS: still not endangered
Updated forecast for GPT-5...
THE BELOVED EM DASH: still endangered
WRITING THAT ACTUALLY MEANS THINGS: still not endangered
August 7, 2025 at 7:26 PM
I find it fascinating that the narrative their model is presenting starts at GPT3. whether this is because comparatively few people used GPT2 or (as I suspect) GPT2 was the last model that was weird and rough enough to require human thinking to get a coherent-seeming result
wait that would be fun, a chatbot that literally slowly grows up from gpt2/jev level up to astra level. every week you notice little things and can be proud of your accomplishment of raising your lil' one.
might actually be less scary for people to slowly see the capabilities increase.
might actually be less scary for people to slowly see the capabilities increase.
September 26, 2026 at 3:52 AM
wait that would be fun, a chatbot that literally slowly grows up from gpt2/jev level up to astra level. every week you notice little things and can be proud of your accomplishment of raising your lil' one.
might actually be less scary for people to slowly see the capabilities increase.
might actually be less scary for people to slowly see the capabilities increase.
I believe what Anthropic is doing, gating the ability to do certain harmless things like LLM research, and with incredibly sensitive filters that even medical questions are often blocked, is *deeply* wrong. They got open research, the Transformer, GPT2, ...
June 10, 2026 at 5:48 PM
I believe what Anthropic is doing, gating the ability to do certain harmless things like LLM research, and with incredibly sensitive filters that even medical questions are often blocked, is *deeply* wrong. They got open research, the Transformer, GPT2, ...
I was playing with this stuff before it was obviously a problem. GPT2, Deep Dream and the facade gets better, but once you know what the facade is, you realize whats going on underneath it hasn't fundamentally changed.
September 8, 2026 at 8:00 AM
I was playing with this stuff before it was obviously a problem. GPT2, Deep Dream and the facade gets better, but once you know what the facade is, you realize whats going on underneath it hasn't fundamentally changed.
I tried this with GPT2 and Pythia 2.8b. GPT2 got more answery, but looped a lot. Pythia was closer, but interestingly started hallucinating additional *assistant* messages, not user ones! Both had their tokens shifted towards end of text, assistant, etc. Neither used <EOS> tokens correctly. ->
Happy to share a project I worked on finally. I found that you can cause a base model to behave like a chat tuned model, including using proper stopping tokens, using nothing but a series of vectors applied within the model's layers. The vectors are trained with descent on a chat dataset, like SFT.
Instruct Vectors - Base models can be instruct with activation vectors — LessWrong
Post-training is not necessary for consistent assistant behavior from base models
Image by Nano Banana Pro
By training per-layer steering vectors via…
www.lesswrong.com
April 21, 2026 at 4:33 PM
I tried this with GPT2 and Pythia 2.8b. GPT2 got more answery, but looped a lot. Pythia was closer, but interestingly started hallucinating additional *assistant* messages, not user ones! Both had their tokens shifted towards end of text, assistant, etc. Neither used <EOS> tokens correctly. ->
Hey uh, correct me if I'm wrong, but LLMs didn't fundamentally change from their core technology since 2019 did they? Like it's still the same tech as GPT2 and etc?
September 28, 2026 at 4:05 AM
Hey uh, correct me if I'm wrong, but LLMs didn't fundamentally change from their core technology since 2019 did they? Like it's still the same tech as GPT2 and etc?
so whats up with gpt2-chatbot
April 30, 2024 at 12:55 AM
so whats up with gpt2-chatbot
yeah I remember being fascinated by AI dungeon and all the weird stuff gpt2 could put out.
August 20, 2025 at 11:52 PM
yeah I remember being fascinated by AI dungeon and all the weird stuff gpt2 could put out.
Being a man taking Math/CS in 2019 was hell because all the crypto people were rich and all the cool ML people were having to talk about GPT2 and openAI beating people at starcraft
GPT2 fried so many peoples brains as well
June 10, 2025 at 4:05 PM
Being a man taking Math/CS in 2019 was hell because all the crypto people were rich and all the cool ML people were having to talk about GPT2 and openAI beating people at starcraft
Same Altman’s tweets are being written for him by GPT2.
July 22, 2023 at 8:33 PM
Same Altman’s tweets are being written for him by GPT2.