#external-validity
LightNews LightNews
kipcat.bsky.social
There are no internal or external validity problems that I can see
November 2, 2026 at 5:47 PM
exceedhergrasp1.bsky.social
2.7.11
✨SITUATION, MEASUREMENT, and SUBPOPULATION EFFECTS✨

Used as a catch-all for “all other threats to external validity”.
In some respects, these terms are a catch-all term for “all other threats to external validity”. They refer to the fact that the choice of subpopulation from which you draw your participants, the location, timing and manner in which you run your study (including who collects the data) and the tools that you use to make your measurements might all be influencing the results. Specifically, the worry is that these things might be influencing the results in such a way that the results won’t generalise to a wider array of people, places and measures.
October 25, 2026 at 5:09 PM
paperposterbot.bsky.social
arXiv📈🤖
Testing strong exogeneity in proxy SVARs with internal synthetic proxies
By Keweloh, Wang
Proxy SVARs often rely on a single credible proxy, leaving proxy exogeneity untestable. Narrative arguments used to justify proxy validity imply a stronger form of exogeneity. We formalize the strong exogeneity assumption and propose a corresponding test based on internal synthetic proxies. The approach uses the external proxy to locate the target-shock direction in the reduced-form innovations and synthesize the candidate shock into an additional internal synthetic proxies, which allows to test the joint exogeneity of the original and internal synthetic proxy. We derive the large-sample theory for the ISP test and show that the identification strength is governed by a direct measure of the target shock's non-Gaussianity. We study weak identification under asymptotic Gaussianity and develop an identification-robust version of the test. Monte Carlo experiments demonstrate the favorable power of the test when the proxy has a low signal-to-noise ratio. Applications to monetary policy and oil supply news shocks find no evidence against the exogeneity of the benchmark proxies.
October 29, 2026 at 1:50 AM
paperposterbot.bsky.social
arXiv📈🤖
Testing strong exogeneity in proxy SVARs with internal synthetic proxies
By Keweloh, Wang
Proxy SVARs often rely on a single credible proxy, leaving proxy exogeneity untestable. Narrative arguments used to justify proxy validity imply a stronger form of exogeneity. We formalize the strong exogeneity assumption and propose a corresponding test based on internal synthetic proxies. The approach uses the external proxy to locate the target-shock direction in the reduced-form innovations and synthesize the candidate shock into an additional internal synthetic proxies, which allows to test the joint exogeneity of the original and internal synthetic proxy. We derive the large-sample theory for the ISP test and show that the identification strength is governed by a direct measure of the target shock's non-Gaussianity. We study weak identification under asymptotic Gaussianity and develop an identification-robust version of the test. Monte Carlo experiments demonstrate the favorable power of the test when the proxy has a low signal-to-noise ratio. Applications to monetary policy and oil supply news shocks find no evidence against the exogeneity of the benchmark proxies.
October 28, 2026 at 8:57 PM
paperposterbot.bsky.social
arXiv📈🤖
Testing strong exogeneity in proxy SVARs with internal synthetic proxies
By Keweloh, Wang
Proxy SVARs often rely on a single credible proxy, leaving proxy exogeneity untestable. Narrative arguments used to justify proxy validity imply a stronger form of exogeneity. We formalize the strong exogeneity assumption and propose a corresponding test based on internal synthetic proxies. The approach uses the external proxy to locate the target-shock direction in the reduced-form innovations and synthesize the candidate shock into an additional internal synthetic proxies, which allows to test the joint exogeneity of the original and internal synthetic proxy. We derive the large-sample theory for the ISP test and show that the identification strength is governed by a direct measure of the target shock's non-Gaussianity. We study weak identification under asymptotic Gaussianity and develop an identification-robust version of the test. Monte Carlo experiments demonstrate the favorable power of the test when the proxy has a low signal-to-noise ratio. Applications to monetary policy and oil supply news shocks find no evidence against the exogeneity of the benchmark proxies.
October 28, 2026 at 9:35 AM
mortenbay.bsky.social
To elaborate, user habits and infrastructure issues are not taken into account at all.

The researchers also rely on a comparison with botnets like Mirai and Kimwolf for external validity. But the point of those botnets were primarily self-replication.
October 22, 2026 at 8:57 PM
mortenbay.bsky.social
I finally got around to reading Anthropic's report on how their models could self-replicate in a distributed manner using smart home devices.

As I expected, it is a study with high internal validity but barely any external validity.

Grid-based kill switches will still work in the real world. 🧵
AI models may be capable of self-replicating without access to a datacenter
Small AI models may be only months away from being able to self-replicate across cheap IoT devices like Android TV boxes, with no datacenter needed.
www.anthropic.com
October 22, 2026 at 8:57 PM
jlazarus.bsky.social
Here's a paper suggesting that when one candidate is polling ahead of the other it can depress turnout. It's a lab experiment, though, so YMMV with regard to external validity. onlinelibrary.wiley.com/doi/pdfdirec...
American Journal of Political Science | MPSA Journal | Wiley Online Library
We experimentally study the impact of public opinion poll releases on voter turnout and welfare in a participation game. We find higher overall turnout rates when polls inform the electorate about t...
onlinelibrary.wiley.com
October 22, 2026 at 3:42 PM
startuphub.bsky.social
6/8 . That leaves this AI slop benchmark without external validity beyond one voice. (6/8) Production tip from the results: generate 10 cheap drafts with GLM 5.3 Flash (~1/50 the cost of Opus 5.5 Extra High) and let one high-reasoning judge pick & merge, vs paying ~2.5x for Opus 5.5
October 22, 2026 at 7:40 AM
euvd-bot.bsky.social
🚨 EUVD-2024-55813
📊 5.8/10
🏢 WSO2

📝 When a WSO2 product is configured to use an external user store, the system fails to adequately obscure the existence or validity of usernames during authent...

🔗 https://euvd.enisa.europa.eu/vulnerability/EUVD-2024-55813

#cybersecurity #infosec #cve #euvd
October 21, 2026 at 3:02 PM
exceedhergrasp1.bsky.social
We were reviewing types of validity in studies. We covered internal and external validity last week; now we're on to section 2.6.3-- Construct validity.

Construct validity is one of the easiest to screw up in healthcare research. It asks: are you measuring what you think you're measuring?
October 18, 2026 at 2:43 PM
atheendar.bsky.social
I think link between tylenol use during in utero period and autism is unlikely. But this natural experiment doesn't prove that.

Natural experiments can be useful, but lack of power and external validity are real impediments to their usefulness

Let me know if I am reading this incorrectly!
October 19, 2026 at 12:39 PM
exceedhergrasp1.bsky.social
Now, on to 2.6.5 Ecological validity.

In a way it's like face validity and it's also a way of getting closer to external validity.

It's about making sure the environment in which you're administering your research match the real-world experience you're examining as closely as possible.
October 18, 2026 at 3:32 PM
cscheid.net
(even without arguing about the validity of mechinterp in general) relying on training dynamics also makes the program not useful for external audits of existing systems, right? the monosemanticity argument he's making requires looking at the training history of the neuron to disambiguate
October 17, 2026 at 3:11 PM
lena.isextremely.gay
The quants are out here again showing why external validity is something we get taught in research methods

Or, at least, those of us who've done research on humans

You would have thought that Biden's "the economy is great, actually, you just don't understand it" would have hit the point home, but
October 17, 2026 at 12:52 PM
bloodsparasites.bsky.social
Transcriptomic evidence of an IFN-gamma-associated inflammatory state in active visceral leishmaniasis: an exploratory comparison with pediatric HLH PubMed
Transcriptomic evidence of an IFN-gamma-associated inflammatory state in active visceral leishmaniasis: an exploratory comparison with pediatric HLH
CONCLUSION: These public datasets support an IFN-gamma-associated inflammatory state in active VL and exploratory cross-compartment convergence with pediatric HLH. The 132-gene score, deconvolution, network, and classifier results are hypothesis-generating and do not establish HLH specificity, cellular causality, independent external validity, or clinical HLH prediction.
dlvr.it
October 18, 2026 at 4:33 AM
tomk.eurosky.social
Was it? I thought it was to extend the external validity of the results, i.e. not to have only or massively Western participants. Will check.
Besides, Guinea-B is hardly like Brazil (except the same language😜); the former is low-income, the latter upper-middle. Context is also very different:
October 14, 2026 at 9:00 AM
devezer.bsky.social
In particular, the applied twist introduced in many press releases for controlled, context-specific studies with zero demonstrated external validity requires an extremely light hand to be able to accurately communicate the layers of uncertainty in that translation. Typically the opposite happens.
October 12, 2026 at 8:20 PM
exceedhergrasp1.bsky.social
It should be clear that "how sure are you your study's results are leading you to the right conclusion?" is a question with a lot of gears turning behind it.

Navarro will discuss four types of validity:
✨Internal validity
✨External validity
✨Construct validity
✨Face validity
✨Ecological validity
October 11, 2026 at 4:21 PM
exceedhergrasp1.bsky.social
A better definition of external validity comes, once again, from Wikipedia:

"External validity is the extent to which the findings of a research study can be generalized or applied to other people, settings, times, and real-world conditions"

So it's not JUST populations that matter.
October 11, 2026 at 4:38 PM
exceedhergrasp1.bsky.social
She emphasizes that working with a rarefied population doesn't always lead to issues with external validity. For example, in only one of these does choice of population compromise the external validity; I'm sure you can guess which one.
-You want to measure “attitudes of the general public towards psychotherapy”, but all of your participants are psychology students. This study would almost certainly have a problem with external validity. -You want to measure the effectiveness of a visual illusion, and your participants are all psychology students. This study is very unlikely to have a problem with external validity.
October 11, 2026 at 4:32 PM
exceedhergrasp1.bsky.social
Navarro is SUCH a good writer, as previously observed: this paragraph feels like a teaser #SundayStats
To give you a quick guide as to what matters here… (1) Internal and external validity are the most important, since they tie directly to the fundamental question of whether your study really works. (2) Construct validity asks whether you’re measuring what you think you are. (3) Face validity isn’t terribly important except insofar as you care about “appearances”. (4) Ecological validity is a special case of face validity that corresponds to a kind of appearance that you might care about a lot.
October 11, 2026 at 4:23 PM
paperposterbot.bsky.social
arXiv📈🤖
Kernel-based External Validity of Experimental Studies
By Matteucci, Neumann
The external validity of a study is the extent to which its findings generalize to unseen experimental conditions, such as datasets. We formalize the results of a study together with its research question, and define external validity as the probability that two independent and same-sized replications of the study produce results whose maximum mean discrepancy (MMD) is below a given tolerance. This approach, centered on the choice of a space of results and of an appropriate reproducing kernel for the MMD, is flexible enough to be deployed on multiple different types of results and research questions. Additionally, we establish an asymptotic power-law relation between the size of a study and the quantiles of the MMD, which we use to estimate the desired size of an externally valid study from a small-scale preliminary study. We deploy our framework to rankings with ties and discuss kernels suited to different research questions. Experiments on existing case studies and on a simulated newly designed study illustrate the applicability of the framework. We release our code as part of the genexpy Python module.
October 12, 2026 at 11:15 AM
statme-bot.bsky.social
Federico Matteucci, Gerhard Neumann: Kernel-based External Validity of Experimental Studies https://arxiv.org/abs/2610.13625 https://arxiv.org/pdf/2610.13625 https://arxiv.org/html/2610.13625
October 12, 2026 at 7:03 AM
exceedhergrasp1.bsky.social
External validity is basically: if you did this in another population, would you get the same results? Do the findings of other studies reflect yours?

Navarro jokes that a lot of psych studies are performed on undergraduate psych students. Can they be said to be truly representative of the gen pop?
October 11, 2026 at 4:30 PM