In this framing, NHST is just parlor tricks. Ok Revvy B was Merlin. I like it.
November 12, 2026 at 1:02 PM
In this framing, NHST is just parlor tricks. Ok Revvy B was Merlin. I like it.
teaching nhst always makes me glad to be bayesian
November 8, 2026 at 2:42 PM
teaching nhst always makes me glad to be bayesian
The trick to win in NHST is to never decide.
November 11, 2026 at 8:05 PM
The trick to win in NHST is to never decide.
Yes, exactly. It's the combination of expectations:
O + NHST + low FPR
that really does for us.
FPR = false positive rate.
O + NHST + low FPR
that really does for us.
FPR = false positive rate.
November 9, 2026 at 9:20 AM
Yes, exactly. It's the combination of expectations:
O + NHST + low FPR
that really does for us.
FPR = false positive rate.
O + NHST + low FPR
that really does for us.
FPR = false positive rate.
This is exactly how I teach NHST and p-values (the emphasis on statistics through simulation is the organizing principle for my data analysis course), and the interactive tool is great!
November 8, 2026 at 10:09 AM
This is exactly how I teach NHST and p-values (the emphasis on statistics through simulation is the organizing principle for my data analysis course), and the interactive tool is great!
Because every different view I tried to apply basically leads me to a meaning such as "if the scientific theory and/or derived hypothesis is strong/plausible enough, then ignore NHST".
November 10, 2026 at 7:40 AM
Because every different view I tried to apply basically leads me to a meaning such as "if the scientific theory and/or derived hypothesis is strong/plausible enough, then ignore NHST".
You never ignore NHST. If you can't use the best way to generate scientific j owledge (which is hypothesis testing, with a sensible null) then you can use the second best - have subjective priors.
November 10, 2026 at 9:39 AM
You never ignore NHST. If you can't use the best way to generate scientific j owledge (which is hypothesis testing, with a sensible null) then you can use the second best - have subjective priors.
It may well make sense for a discipline to be mainly exploring widely, daft wild ideas, but only so long as we understand that this is a high false positive strategy, especially if we are using NHST. And we judge the value of work on that understanding - there's a high false positive rate.
November 8, 2026 at 8:32 PM
It may well make sense for a discipline to be mainly exploring widely, daft wild ideas, but only so long as we understand that this is a high false positive strategy, especially if we are using NHST. And we judge the value of work on that understanding - there's a high false positive rate.
I may be too dumb to get it because I feel I need to see this formalised. Treating parameters as random variables and the lack of a decision rule (such as NHST) are, formally, all there is to the Bayesian-frequentist difference.
November 2, 2026 at 9:38 PM
I may be too dumb to get it because I feel I need to see this formalised. Treating parameters as random variables and the lack of a decision rule (such as NHST) are, formally, all there is to the Bayesian-frequentist difference.
90% of the problems with NHST in a few dozen words.
October 30, 2026 at 9:15 AM
90% of the problems with NHST in a few dozen words.
Pons : l’adjoint au commerce veut « retrouver un dynamisme collectif »
🔗https://l.sudouest.fr/nHsT
🔗https://l.sudouest.fr/nHsT
October 27, 2026 at 8:44 AM
Pons : l’adjoint au commerce veut « retrouver un dynamisme collectif »
🔗https://l.sudouest.fr/nHsT
🔗https://l.sudouest.fr/nHsT
Side comment while I catch up on this.
For a purist, NHST doesn't work. It says, we can model a world where there is *only* the null and find out how probable our data would be. It then seeks to reject the null, ignoring that we just asserted that there is *only* the null.
For a purist, NHST doesn't work. It says, we can model a world where there is *only* the null and find out how probable our data would be. It then seeks to reject the null, ignoring that we just asserted that there is *only* the null.
October 22, 2026 at 8:23 AM
Side comment while I catch up on this.
For a purist, NHST doesn't work. It says, we can model a world where there is *only* the null and find out how probable our data would be. It then seeks to reject the null, ignoring that we just asserted that there is *only* the null.
For a purist, NHST doesn't work. It says, we can model a world where there is *only* the null and find out how probable our data would be. It then seeks to reject the null, ignoring that we just asserted that there is *only* the null.
#statstab #637 Evaluate What Is Claimed to Be Confirmed: Initial Version of a Falsification Assessment Form (FAF)
Thoughts: This seems like a faf🙃
#falsification #nhst #hypothesis #hypothesistesting #assumptions #causalinference
#OpenScience #auxiliary
meth.psychopen.eu/index.php/me...
Thoughts: This seems like a faf🙃
#falsification #nhst #hypothesis #hypothesistesting #assumptions #causalinference
#OpenScience #auxiliary
meth.psychopen.eu/index.php/me...
Evaluate What Is Claimed to Be Confirmed: Initial Version of a Falsification Assessment Form (FAF)| Methodology
meth.psychopen.eu
October 14, 2026 at 10:59 AM
#statstab #637 Evaluate What Is Claimed to Be Confirmed: Initial Version of a Falsification Assessment Form (FAF)
Thoughts: This seems like a faf🙃
#falsification #nhst #hypothesis #hypothesistesting #assumptions #causalinference
#OpenScience #auxiliary
meth.psychopen.eu/index.php/me...
Thoughts: This seems like a faf🙃
#falsification #nhst #hypothesis #hypothesistesting #assumptions #causalinference
#OpenScience #auxiliary
meth.psychopen.eu/index.php/me...
We then did resampling (corridor of stability) analyses (after Marek et al 2022) and synthetic data simulation analyses to find that normative samples most likely will not sustain stable estimates or nominal error rates if you do NHST. You do need bigger samples like ours, requiring a field shift
October 12, 2026 at 7:51 AM
We then did resampling (corridor of stability) analyses (after Marek et al 2022) and synthetic data simulation analyses to find that normative samples most likely will not sustain stable estimates or nominal error rates if you do NHST. You do need bigger samples like ours, requiring a field shift
But they can get hollowed out. Local NHST has closed down one of wards with its 24 beds in our community hospital in last few years, citing the expansion of community-based intermediate care and rehabilitation- virtual wards
October 10, 2026 at 7:02 AM
But they can get hollowed out. Local NHST has closed down one of wards with its 24 beds in our community hospital in last few years, citing the expansion of community-based intermediate care and rehabilitation- virtual wards
#statstab #624 Severe Testing:
Error Statistics versus Bayes Factor Tests
Thoughts: This seems to be the framework most NHST researchers are adopting.
#nhst #hypothesis #hypothesistesting #severetestting #equivalence #sesoi #methods #frequentist
doi.org/10.1086/736950
Error Statistics versus Bayes Factor Tests
Thoughts: This seems to be the framework most NHST researchers are adopting.
#nhst #hypothesis #hypothesistesting #severetestting #equivalence #sesoi #methods #frequentist
doi.org/10.1086/736950
Severe Testing: Error Statistics versus Bayes Factor Tests | The British Journal for the Philosophy of Science: Vol 0, No ja
doi.org
September 24, 2026 at 6:25 PM
#statstab #624 Severe Testing:
Error Statistics versus Bayes Factor Tests
Thoughts: This seems to be the framework most NHST researchers are adopting.
#nhst #hypothesis #hypothesistesting #severetestting #equivalence #sesoi #methods #frequentist
doi.org/10.1086/736950
Error Statistics versus Bayes Factor Tests
Thoughts: This seems to be the framework most NHST researchers are adopting.
#nhst #hypothesis #hypothesistesting #severetestting #equivalence #sesoi #methods #frequentist
doi.org/10.1086/736950
😂 That's so close to triggering another watt-rant. Luckily postponed for now...no-one wants to know what I think about NHST.
ps. emojis are quite useful really.
ps. emojis are quite useful really.
September 20, 2026 at 2:59 PM
😂 That's so close to triggering another watt-rant. Luckily postponed for now...no-one wants to know what I think about NHST.
ps. emojis are quite useful really.
ps. emojis are quite useful really.
The thing about this is they are all trying to express vibes with math, failing, and then inventing increasingly dangerous ways to explain that away.
And it’s not even productive like the racists where you get ideas like NHST or correlation out of it. It’s just dumb.
And it’s not even productive like the racists where you get ideas like NHST or correlation out of it. It’s just dumb.
Leopold is, allegedly, an effective altruist and not a rationalist, but you can see how this gets very difficult to keep straight. In what year were you one and not the other? Was your organization, at the time, financially tied directly to Yudkowsky or no?
September 17, 2026 at 5:15 AM
The thing about this is they are all trying to express vibes with math, failing, and then inventing increasingly dangerous ways to explain that away.
And it’s not even productive like the racists where you get ideas like NHST or correlation out of it. It’s just dumb.
And it’s not even productive like the racists where you get ideas like NHST or correlation out of it. It’s just dumb.
#AcademicSky
#PsychSciSky is fixated on what's broken in the field, from WEIRD Ps, to QRPs, to NHST vs Bayesian etc
But I'm working on review paper. And it's SO evident that authors do not describe, in sufficient details, what they did and in what order. METHODS.
#PsychSciSky is fixated on what's broken in the field, from WEIRD Ps, to QRPs, to NHST vs Bayesian etc
But I'm working on review paper. And it's SO evident that authors do not describe, in sufficient details, what they did and in what order. METHODS.
September 16, 2026 at 4:18 PM
#AcademicSky
#PsychSciSky is fixated on what's broken in the field, from WEIRD Ps, to QRPs, to NHST vs Bayesian etc
But I'm working on review paper. And it's SO evident that authors do not describe, in sufficient details, what they did and in what order. METHODS.
#PsychSciSky is fixated on what's broken in the field, from WEIRD Ps, to QRPs, to NHST vs Bayesian etc
But I'm working on review paper. And it's SO evident that authors do not describe, in sufficient details, what they did and in what order. METHODS.
I believe it comes from t(df>30) being approximately normal for the purpose of NHST at alpha=.05, and since for simple cases df≈N, this "approaches normality" got mixed with CLT (a wholey different "approaches normality"), and... here we are.
September 12, 2026 at 5:16 PM
I believe it comes from t(df>30) being approximately normal for the purpose of NHST at alpha=.05, and since for simple cases df≈N, this "approaches normality" got mixed with CLT (a wholey different "approaches normality"), and... here we are.
sofuckingannoyinggg like ok nhst bc YOU havent unpacked your problems completely doesnt mean u need to turn against me. 😭
September 11, 2026 at 6:51 AM
sofuckingannoyinggg like ok nhst bc YOU havent unpacked your problems completely doesnt mean u need to turn against me. 😭
Null findings are usually dismissed ("Well, you'd see it if you looked at it some other way"), while any supportive finding is taken as evidence in favor of the hypothesis.
Meehl talked about this years ago. This is what happens when theories are vague and people rely on NHST.
Meehl talked about this years ago. This is what happens when theories are vague and people rely on NHST.
September 2, 2026 at 3:45 PM
Null findings are usually dismissed ("Well, you'd see it if you looked at it some other way"), while any supportive finding is taken as evidence in favor of the hypothesis.
Meehl talked about this years ago. This is what happens when theories are vague and people rely on NHST.
Meehl talked about this years ago. This is what happens when theories are vague and people rely on NHST.
Across previous studies, we saw three main concerns:
1️⃣ Breakpoints from curve-fitting without clear physiological meaning
2️⃣ Vague or poorly specified hypotheses
3️⃣ Correlations & NHST used where equivalence is the real question
Here’s what we did differently 👇 3/8
1️⃣ Breakpoints from curve-fitting without clear physiological meaning
2️⃣ Vague or poorly specified hypotheses
3️⃣ Correlations & NHST used where equivalence is the real question
Here’s what we did differently 👇 3/8
August 26, 2026 at 8:18 AM
Across previous studies, we saw three main concerns:
1️⃣ Breakpoints from curve-fitting without clear physiological meaning
2️⃣ Vague or poorly specified hypotheses
3️⃣ Correlations & NHST used where equivalence is the real question
Here’s what we did differently 👇 3/8
1️⃣ Breakpoints from curve-fitting without clear physiological meaning
2️⃣ Vague or poorly specified hypotheses
3️⃣ Correlations & NHST used where equivalence is the real question
Here’s what we did differently 👇 3/8