#MeasurementError
LightNews LightNews
klauspforr.eurosky.social
Measurement error research has reached Inception level www.nber.org/system/files... #measurementerror #happiness
March 11, 2024 at 8:42 AM
gordonhodsonphd.bsky.social
When drafted into NHL, he was pick #14

Later took Canadiens to 6 (!) Stanley Cups as goalie.

Academics, especially those adjudicating awards, a reminder about #MeasurementError & difficulty in predicting "stars" early in a career.
gordonhodsonphd.bsky.social Gordon Hodson @gordonhodsonphd.bsky.social · Sep 6
Canadian icon Ken Dryden has passed away

As a child he influenced me in an usual way

I was astounded that an NHL star had a law degree.

(even more amazing, he took a break from NHL to article and get law degree!)

Really stressed, to me, the importance of #education

www.cbc.ca/sports/hocke...
Former Canadiens star goaltender Ken Dryden dies of cancer at age 78 | CBC Sports
Ken Dryden, the Hall of Fame goaltender whose long resumé in and out of hockey included six Stanley Cup victories and helping backstop Canada's generation-defining victory at the 1972 Summit Series, d...
www.cbc.ca
September 6, 2025 at 6:21 PM
oacarah.bsky.social
This new body of work on using proxy measures for confounding control should be seen in the context of the earlier literature on #measurementerror in (de)confounders and replicate measurements such as:

#statsky #episky #psychsky
Measurement bias and effect restoration in causal inference on JSTOR
MANABU KUROKI, JUDEA PEARL, Measurement bias and effect restoration in causal inference, Biometrika, Vol. 101, No. 2 (JUNE 2014), pp. 423-437
www.jstor.org
November 26, 2024 at 4:39 AM
paperposterbot.bsky.social
arXiv📈🤖
Efficient Inference for Noisy LLM-as-a-Judge Evaluation
By Chen, Lu, Li et al
Large language models (LLMs) are increasingly used as automatic evaluators of generative AI outputs, a paradigm often referred to as "LLM-as-a-judge." In practice, LLM judges are imperfect predictions for the underlying truth and can exhibit systematic, non-random errors. Two main approaches have recently been proposed to address this issue: (i) direct measurementerror correction based on misclassification models such as Rogan-Gladen-style estimators, and (ii) surrogate-outcome approaches such as prediction-powered inference (PPI), which correct bias by calibrating prediction residuals on a small set of gold-standard human labels. In this paper, we systematically study the performance of these two approaches for estimating mean parameters (e.g., average benchmark scores or pairwise win rates). Leveraging tools from semiparametric efficiency theory, we unify the two classes of estimators by deriving explicit forms of efficient influence function (EIF)-based efficient estimators and characterize conditions under which PPI-style estimators attain strictly smaller asymptotic variance than measurement-error corrections. We verify our theoretical results in simulations and demonstrate the methods on real-data examples. We provide an implementation of the benchmarked methods and comparison utilities at https://github.com/yiqunchen/debias-llm-as-a-judge.
January 12, 2026 at 4:56 PM
jrboehnke.bsky.social
Useful paper investigating the precision of various #reliability and #MeasurementError parameters under different conditions and study designs:
link.springer.com/article/10.1...

It comes with an #RStats shiny tool to explore some of these oneself:
iriseekhout.shinyapps.io/ICCpower/

#Psychometrics
Sample size recommendations for studies on reliability and measurement error: an online application based on simulation studies - Health Services and Outcomes Research Methodology
Simulation studies were performed to investigate for which conditions of sample size of patients (n) and number of repeated measurements (k) (e.g., raters) the optimal (i.e., balance between precise a...
link.springer.com
July 14, 2025 at 4:14 PM
davdittrich.economicscience.net
random with respect to the characteristics of the subjects. Training simpler supervised models on high-quality human annotations with flexible coding leads to less #measurementError and bias than #LLM annotations"

via https://fediscience.org/@davdittrich/111403259435538469 2/2
March 6, 2024 at 12:38 PM
mzloteanu.bsky.social
#statstab #281 Correcting Cohen’s d for Measurement Error (A Method!)

Thoughts: Scale reliability can be incorporated into effect size computation (i.e., remove attenuation)

#measurementerror #effectsize #cohend #reliability #scales

rpubs.com/JLLJ/RPBD
RPubs - Classical Reliability Correction for Cohen's d and the Point-Biserial
rpubs.com
February 17, 2025 at 11:21 AM
klauspforr.eurosky.social
"Careless respondents" aka straight-liners etc can cause spurious effects. This makes this measurement quality research worthwile beyond survey methodology #surveymethods #surveymethodology #measurementerror journals.sagepub.com/doi/epub/10....
Sage Journals: Your gateway to world-class research journals
Subscription and open access journals from Sage, the world's leading independent academic publisher.
journals.sagepub.com
March 18, 2024 at 9:06 AM
prebeiro.bsky.social
7. Further, we assessed sensitivity to #MeasurementError of the exposure and #Exogeneity (unmeasured confounding) by excluding those states w/ more generous #Medicaid income eligibility (>100% of #FPL) prior to #Medicaid expansion under the #ACA: AZ, CA, DE, DC, HI, NY, & VT...
November 11, 2024 at 5:10 PM