pretraining
posttraining
post-posttraining
neo training
training revival
trainingwave
posttraining
post-posttraining
neo training
training revival
trainingwave
March 16, 2026 at 7:13 PM
pretraining
posttraining
post-posttraining
neo training
training revival
trainingwave
posttraining
post-posttraining
neo training
training revival
trainingwave
who called it adulthood instead of posttraining
September 22, 2026 at 5:00 AM
who called it adulthood instead of posttraining
its prevalence noticeably increases in posttraining, despite the fact that it is not in the data used for that purpose.
March 22, 2026 at 11:49 PM
its prevalence noticeably increases in posttraining, despite the fact that it is not in the data used for that purpose.
i also think that the profits should be allocated according to the mean importance-weighted score in posttraining over the corpus of the owner
"models cannot be owned and operation of one requires disgorgement of virtually all your attributable profits into a trust for journalism, science, and the arts."
June 3, 2026 at 10:42 PM
i also think that the profits should be allocated according to the mean importance-weighted score in posttraining over the corpus of the owner
yeah, this is the basic problem. what Anthropic is selling, to a substantial extent, is downstream of the floating point number representing estimated factuality in posttraining.
And, of note, facts are specifically excluded from being “owned” in the sense of IP (for very obvious reasons.)
June 2, 2026 at 4:50 AM
yeah, this is the basic problem. what Anthropic is selling, to a substantial extent, is downstream of the floating point number representing estimated factuality in posttraining.
this is literally just "posttraining ranks papers by their citation density"
9. Here's a surprise: controlling for the other covariates (correct me if I have that wrong, Kyle), we see the *most* LLM use in the high impact journals, not low impact journals.
June 4, 2026 at 5:59 AM
this is literally just "posttraining ranks papers by their citation density"
lesswrong post idea against behaviorist posttraining send skeet
November 24, 2025 at 7:51 PM
lesswrong post idea against behaviorist posttraining send skeet
New model introspection research! They find evidence that “introspective awareness” emerges in posttraining, in particular DPO
arxiv.org/abs/2603.21396
arxiv.org/abs/2603.21396
Mechanisms of Introspective Awareness
Recent work has shown that LLMs can sometimes detect when steering vectors are injected into their residual stream and identify the injected concept -- a phenomenon termed "introspective awareness." W...
arxiv.org
April 15, 2026 at 6:09 PM
New model introspection research! They find evidence that “introspective awareness” emerges in posttraining, in particular DPO
arxiv.org/abs/2603.21396
arxiv.org/abs/2603.21396
This might be enough to say that RL & posttraining are now significantly more expensive than pretraining
May 14, 2026 at 10:57 AM
This might be enough to say that RL & posttraining are now significantly more expensive than pretraining
undistilled multi trillion param model probably. they said they "scaled up pretraining and posttraining"
February 27, 2025 at 10:44 PM
undistilled multi trillion param model probably. they said they "scaled up pretraining and posttraining"
also unless he taught himself how to do posttraining (lol) and has a room full of a few million dollars worth of H100s he is full of shit and either ended up with a bot that won't answer questions, or is actually just asking a commercial bot to pretend to be them
August 10, 2026 at 2:48 PM
also unless he taught himself how to do posttraining (lol) and has a room full of a few million dollars worth of H100s he is full of shit and either ended up with a bot that won't answer questions, or is actually just asking a commercial bot to pretend to be them
I agree with norvid (potentially cyclically depending on the nature of the splash damage) that this is primarily a posttraining issue.
February 16, 2026 at 7:13 PM
I agree with norvid (potentially cyclically depending on the nature of the splash damage) that this is primarily a posttraining issue.
Und immer wieder nach dem Training die kurze Überlegung, ob ich meine verschwitzten Söckchen vielleicht verkaufen sollte.
#FeetLovers
#PostTraining
#FeetLovers
#PostTraining
January 6, 2025 at 6:21 PM
Und immer wieder nach dem Training die kurze Überlegung, ob ich meine verschwitzten Söckchen vielleicht verkaufen sollte.
#FeetLovers
#PostTraining
#FeetLovers
#PostTraining
for context in my professional life i personally have pretrained small models from scratch, run RL posttraining for specialized task-focused objectives, and have built frameworks for steering language models directly on their activations using trained SAEs and contrastive activation addition.
October 4, 2025 at 3:53 AM
for context in my professional life i personally have pretrained small models from scratch, run RL posttraining for specialized task-focused objectives, and have built frameworks for steering language models directly on their activations using trained SAEs and contrastive activation addition.
Truly bizarre stuff happening to the writing styles simultaneously with coding getting better. Tempting to conclude it's the result of a common posttraining mechanism but not sure there's real evidence of that yet.
the writing style of Opus 4.8 is more obnoxious than ever before, writing tics piled atop each other such that even the most milquetoast functional documents are grating to read
June 2, 2026 at 1:27 AM
Truly bizarre stuff happening to the writing styles simultaneously with coding getting better. Tempting to conclude it's the result of a common posttraining mechanism but not sure there's real evidence of that yet.
"needed to challenge the information provided" my sense from timeline splash damage from the various LLM whisperers is that's an RLHF posttraining for agreeableness OCEAN issue, not a lack of capability
February 15, 2026 at 4:36 PM
"needed to challenge the information provided" my sense from timeline splash damage from the various LLM whisperers is that's an RLHF posttraining for agreeableness OCEAN issue, not a lack of capability
i am well aware of this, but you cannot get anything out of posttraining which was not in pretraining -- KL penalties go to infinity as the probability of the continuation goes to zero, and in the limit this is also true of sequences.
June 4, 2026 at 5:18 AM
i am well aware of this, but you cannot get anything out of posttraining which was not in pretraining -- KL penalties go to infinity as the probability of the continuation goes to zero, and in the limit this is also true of sequences.
An interesting thing about LLMs in Python is that they seem to broadly push code towards some kind of conventional wisdom about best practices, as judged maybe by whoever is setting up the posttraining recipes (I say "in Python" mostly because I notice that more strongly in Python).
June 12, 2026 at 2:57 PM
An interesting thing about LLMs in Python is that they seem to broadly push code towards some kind of conventional wisdom about best practices, as judged maybe by whoever is setting up the posttraining recipes (I say "in Python" mostly because I notice that more strongly in Python).
no body knows this yet but you can ask insane questions like “how does changing the pretraining data corpus affect posttraining performance in LLMs” and do bizarre linguistic analysis on particular bodies of texts. feed it on borges and check out the loudest words it attends to
October 25, 2025 at 7:34 PM
no body knows this yet but you can ask insane questions like “how does changing the pretraining data corpus affect posttraining performance in LLMs” and do bizarre linguistic analysis on particular bodies of texts. feed it on borges and check out the loudest words it attends to
I kinda want to pool some money to train a cheap model like deepseek to see what it's like before posttraining because all the big LLM people who've interacted with them say that it's like talking to a monster from a psychological horror movie.
thinking about this article from 3 years ago about what is probably the first guy to ever get chat bot psychosis
Google AI is real, says fired engineer | CNN Business
Google has fired an engineer who claimed an unreleased AI system had become sentient, the company confirmed, saying he violated employment and data security policies.
www.cnn.com
August 14, 2025 at 4:05 PM
I kinda want to pool some money to train a cheap model like deepseek to see what it's like before posttraining because all the big LLM people who've interacted with them say that it's like talking to a monster from a psychological horror movie.
(Although this is just an approximation of base model behavior, since the weight changes from posttraining are still in effect)
December 9, 2025 at 11:54 AM
(Although this is just an approximation of base model behavior, since the weight changes from posttraining are still in effect)
i also kind of think that at least Terra isn’t distilled at all. e.g. Terra did Sol’s posttraining run, implying that it was done before Sol was so Sol couldn’t have been used to distill
July 11, 2026 at 11:11 AM
i also kind of think that at least Terra isn’t distilled at all. e.g. Terra did Sol’s posttraining run, implying that it was done before Sol was so Sol couldn’t have been used to distill
