Ben Van Calster
@benvancalster.bsky.social
Medical Statistician at KU Leuven. My brain is like a snail but it gets there in the end (or not).
Regarding metric- vs content-based science, I see 3 general positions:
- Idealists: contents matter 100%; highly needed but most disappear (frustration, don't get promotions)
- Pragmatists: focus on content but use metrics to survive
- Conformists: metrics matter 100%
- Idealists: contents matter 100%; highly needed but most disappear (frustration, don't get promotions)
- Pragmatists: focus on content but use metrics to survive
- Conformists: metrics matter 100%
May 26, 2026 at 9:17 AM
Regarding metric- vs content-based science, I see 3 general positions:
- Idealists: contents matter 100%; highly needed but most disappear (frustration, don't get promotions)
- Pragmatists: focus on content but use metrics to survive
- Conformists: metrics matter 100%
- Idealists: contents matter 100%; highly needed but most disappear (frustration, don't get promotions)
- Pragmatists: focus on content but use metrics to survive
- Conformists: metrics matter 100%
Happy to see this in print!
doi 10.1146/annurev-statistics-042324-123749
@maartenvsmeden.bsky.social @laurewynants.bsky.social @vanamsterdam.bsky.social and Ewout Steyerberg
doi 10.1146/annurev-statistics-042324-123749
@maartenvsmeden.bsky.social @laurewynants.bsky.social @vanamsterdam.bsky.social and Ewout Steyerberg
March 10, 2026 at 8:40 AM
Happy to see this in print!
doi 10.1146/annurev-statistics-042324-123749
@maartenvsmeden.bsky.social @laurewynants.bsky.social @vanamsterdam.bsky.social and Ewout Steyerberg
doi 10.1146/annurev-statistics-042324-123749
@maartenvsmeden.bsky.social @laurewynants.bsky.social @vanamsterdam.bsky.social and Ewout Steyerberg
Reposted by Ben Van Calster
Doug Altman was an internationally renowned statistician who served as The BMJ’s chief statistical adviser.
Read about life and work that made this statistician a "citation millionaire"
#BMJChristmas
www.bmj.com/content/391/...
Read about life and work that made this statistician a "citation millionaire"
#BMJChristmas
www.bmj.com/content/391/...
December 17, 2025 at 4:13 PM
Doug Altman was an internationally renowned statistician who served as The BMJ’s chief statistical adviser.
Read about life and work that made this statistician a "citation millionaire"
#BMJChristmas
www.bmj.com/content/391/...
Read about life and work that made this statistician a "citation millionaire"
#BMJChristmas
www.bmj.com/content/391/...
Our guidance regarding performance measures for medical AI models is finally out!
- Stop bashing AUROC, although it does not settle things
- Calibration and clinical utility are key
- Show risk distributions
- Classification statistics (e.g. F1) are improper
www.thelancet.com/journals/lan...
- Stop bashing AUROC, although it does not settle things
- Calibration and clinical utility are key
- Show risk distributions
- Classification statistics (e.g. F1) are improper
www.thelancet.com/journals/lan...
Evaluation of performance measures in predictive artificial intelligence models to support medical decisions: overview and guidance
Numerous measures have been proposed to illustrate the performance of predictive artificial
intelligence (AI) models. Selecting appropriate performance measures is essential
for predictive AI models i...
www.thelancet.com
December 13, 2025 at 2:04 PM
Our guidance regarding performance measures for medical AI models is finally out!
- Stop bashing AUROC, although it does not settle things
- Calibration and clinical utility are key
- Show risk distributions
- Classification statistics (e.g. F1) are improper
www.thelancet.com/journals/lan...
- Stop bashing AUROC, although it does not settle things
- Calibration and clinical utility are key
- Show risk distributions
- Classification statistics (e.g. F1) are improper
www.thelancet.com/journals/lan...
Reposted by Ben Van Calster
Expertise is having fucked up in enough different ways that you become able to anticipate it.
September 4, 2025 at 3:33 AM
Expertise is having fucked up in enough different ways that you become able to anticipate it.
In our latest work, we show that risk estimates for patients are HUGELY uncertain due to model, data, and population uncertainty. Even for well performing models (c statistic, calibration, utility) based on large N.
@laure_wynants @ESteyerberg @lasaibarrenada.bsky.social
arxiv.org/abs/2506.17141
@laure_wynants @ESteyerberg @lasaibarrenada.bsky.social
arxiv.org/abs/2506.17141
The fundamental problem of risk prediction for individuals: health AI, uncertainty, and personalized medicine
Background: Clinical prediction models for a health condition are commonly evaluated regarding performance for a population, although decisions are made for individuals. The classic view relates uncer...
arxiv.org
August 29, 2025 at 8:27 AM
In our latest work, we show that risk estimates for patients are HUGELY uncertain due to model, data, and population uncertainty. Even for well performing models (c statistic, calibration, utility) based on large N.
@laure_wynants @ESteyerberg @lasaibarrenada.bsky.social
arxiv.org/abs/2506.17141
@laure_wynants @ESteyerberg @lasaibarrenada.bsky.social
arxiv.org/abs/2506.17141
Universities love open science
Small print: unless money is involved
Small print: unless money is involved
August 26, 2025 at 10:15 AM
Universities love open science
Small print: unless money is involved
Small print: unless money is involved
Reposted by Ben Van Calster
Huge variability documented in how publishers respond when informed about a problematic body of work by a research group. www.jclinepi.com/article/S089...
#publishers #retractions
#publishers #retractions
August 24, 2025 at 3:35 PM
Huge variability documented in how publishers respond when informed about a problematic body of work by a research group. www.jclinepi.com/article/S089...
#publishers #retractions
#publishers #retractions
Reposted by Ben Van Calster
Multiple Imputation of Missing Covariates When Using the Fine–Gray Model. Edouard F. Bonneville, Jan Beyersmann, Ruth H. Keogh, Jonathan W. Bartlett, Tim P. Morris, Nicola Polverelli, Liesbeth C. de Wreede, Hein Putter. Statistics in Medicine. onlinelibrary.wiley.com/doi/10.1002/...
Multiple Imputation of Missing Covariates When Using the Fine–Gray Model
The Fine–Gray model for the subdistribution hazard is commonly used for estimating associations between covariates and competing risks outcomes. When there are missing values in the covariates includ...
onlinelibrary.wiley.com
July 25, 2025 at 7:50 AM
Multiple Imputation of Missing Covariates When Using the Fine–Gray Model. Edouard F. Bonneville, Jan Beyersmann, Ruth H. Keogh, Jonathan W. Bartlett, Tim P. Morris, Nicola Polverelli, Liesbeth C. de Wreede, Hein Putter. Statistics in Medicine. onlinelibrary.wiley.com/doi/10.1002/...
Reposted by Ben Van Calster
Seconded.
Any time someone uses this term, make sure they explain exactly what they mean. If they can't, they are obviously trying to bullshit you.
Any time someone uses this term, make sure they explain exactly what they mean. If they can't, they are obviously trying to bullshit you.
July 14, 2025 at 8:29 AM
Seconded.
Any time someone uses this term, make sure they explain exactly what they mean. If they can't, they are obviously trying to bullshit you.
Any time someone uses this term, make sure they explain exactly what they mean. If they can't, they are obviously trying to bullshit you.
Reposted by Ben Van Calster
“Volume is a bad driver,” [Sir Mark Walport] said. “The incentive should be quality, not quantity. It’s about re-engineering the system in a way that encourages good research from beginning to end.”
www.theguardian.com/science/2025...
www.theguardian.com/science/2025...
Quality of scientific papers questioned as academics ‘overwhelmed’ by the millions published
Widespread mockery of AI-generated rat with giant penis in one paper brings problem to public attention
www.theguardian.com
July 13, 2025 at 3:31 PM
“Volume is a bad driver,” [Sir Mark Walport] said. “The incentive should be quality, not quantity. It’s about re-engineering the system in a way that encourages good research from beginning to end.”
www.theguardian.com/science/2025...
www.theguardian.com/science/2025...
Reposted by Ben Van Calster
What is common knowledge in your field, but shocks outsiders?
Data isn't objective and researchers have innumerable ways to put their thumbs on the scale. Many don't understand statistics well enough to realize they're doing it.
Data isn't objective and researchers have innumerable ways to put their thumbs on the scale. Many don't understand statistics well enough to realize they're doing it.
What is common knowledge in your field, but shocks outsiders?
Almost all of the bugs and problems and breakage in the software you use is known to the engineers, we just aren't allowed to fix it. Gotta ship new features.
Almost all of the bugs and problems and breakage in the software you use is known to the engineers, we just aren't allowed to fix it. Gotta ship new features.
What is common knowledge in your field, but shocks outsiders?
The number of parents that don't care much about their kids and/or have very thinly veiled resentment against them is much higher than most people think.
The number of parents that don't care much about their kids and/or have very thinly veiled resentment against them is much higher than most people think.
June 17, 2025 at 4:01 PM
What is common knowledge in your field, but shocks outsiders?
Data isn't objective and researchers have innumerable ways to put their thumbs on the scale. Many don't understand statistics well enough to realize they're doing it.
Data isn't objective and researchers have innumerable ways to put their thumbs on the scale. Many don't understand statistics well enough to realize they're doing it.
Reposted by Ben Van Calster
**New Lancet DH paper**
"Importance of sample size on the quality & utility of AI-based prediction models for healthcare"
- for broad audience
- explains why inadequate SS harms #AI model training, evaluation & performance
- pushback to claims SS irrelevant to AI research
👇
tinyurl.com/yrje52fn
"Importance of sample size on the quality & utility of AI-based prediction models for healthcare"
- for broad audience
- explains why inadequate SS harms #AI model training, evaluation & performance
- pushback to claims SS irrelevant to AI research
👇
tinyurl.com/yrje52fn
Importance of sample size on the quality and utility of AI-based prediction models for healthcare
Rigorous study design and analytical standards are required to generate reliable findings in healthcare from artificial intelligence (AI) research. On…
www.sciencedirect.com
June 2, 2025 at 3:18 PM
**New Lancet DH paper**
"Importance of sample size on the quality & utility of AI-based prediction models for healthcare"
- for broad audience
- explains why inadequate SS harms #AI model training, evaluation & performance
- pushback to claims SS irrelevant to AI research
👇
tinyurl.com/yrje52fn
"Importance of sample size on the quality & utility of AI-based prediction models for healthcare"
- for broad audience
- explains why inadequate SS harms #AI model training, evaluation & performance
- pushback to claims SS irrelevant to AI research
👇
tinyurl.com/yrje52fn
I recently had a paper in a journal where the fee for open access was lower than the fee for closed access. Is that common? @grahamkendall.bsky.social
May 11, 2025 at 4:17 PM
I recently had a paper in a journal where the fee for open access was lower than the fee for closed access. Is that common? @grahamkendall.bsky.social
Reposted by Ben Van Calster
Oh, there's an English version as well! DW reports about misconduct at Max Planck Institutes: www.youtube.com/watch?v=n5nE...
March 13, 2025 at 9:35 AM
Oh, there's an English version as well! DW reports about misconduct at Max Planck Institutes: www.youtube.com/watch?v=n5nE...
Reposted by Ben Van Calster
I will be presenting our recent work on individual risk estimation uncertainty at ENAR in New Orleans. Come say hi!
Work with @benvancalster.bsky.social @laurewynants.bsky.social #DoranneThomassen #EwoutSteyerberg
Work with @benvancalster.bsky.social @laurewynants.bsky.social #DoranneThomassen #EwoutSteyerberg
March 23, 2025 at 3:35 PM
I will be presenting our recent work on individual risk estimation uncertainty at ENAR in New Orleans. Come say hi!
Work with @benvancalster.bsky.social @laurewynants.bsky.social #DoranneThomassen #EwoutSteyerberg
Work with @benvancalster.bsky.social @laurewynants.bsky.social #DoranneThomassen #EwoutSteyerberg
Quousque tandem?
@kuleuvenuniversity.bsky.social @fwovlaanderen.bsky.social
A lot of publications in MDPI journals also in Flanders, despite it being listed on predatoryjournals.org.
@kuleuvenuniversity.bsky.social @fwovlaanderen.bsky.social
A lot of publications in MDPI journals also in Flanders, despite it being listed on predatoryjournals.org.
March 16, 2025 at 10:23 AM
Quousque tandem?
@kuleuvenuniversity.bsky.social @fwovlaanderen.bsky.social
A lot of publications in MDPI journals also in Flanders, despite it being listed on predatoryjournals.org.
@kuleuvenuniversity.bsky.social @fwovlaanderen.bsky.social
A lot of publications in MDPI journals also in Flanders, despite it being listed on predatoryjournals.org.
Reposted by Ben Van Calster
🚨 Preprint! We combine our recent open dataset of #APC prices with the article counts per journal-year from #OpenAlex to estimate how much the academic community has paid in APCs over the last 5 years.
A. $8.349 billion ($8.968 B in 2023 USD)
$2.5B in 2023 alone.
arxiv.org/abs/2407.16551 #metasci
A. $8.349 billion ($8.968 B in 2023 USD)
$2.5B in 2023 alone.
arxiv.org/abs/2407.16551 #metasci
Estimating global article processing charges paid to six publishers for open access between 2019 and 2023
This study presents estimates of the global expenditure on article processing charges (APCs) paid to six publishers for open access between 2019 and 2023. APCs are fees charged for publishing in some ...
arxiv.org
November 18, 2024 at 4:16 PM
🚨 Preprint! We combine our recent open dataset of #APC prices with the article counts per journal-year from #OpenAlex to estimate how much the academic community has paid in APCs over the last 5 years.
A. $8.349 billion ($8.968 B in 2023 USD)
$2.5B in 2023 alone.
arxiv.org/abs/2407.16551 #metasci
A. $8.349 billion ($8.968 B in 2023 USD)
$2.5B in 2023 alone.
arxiv.org/abs/2407.16551 #metasci
Reposted by Ben Van Calster
my life would radically improve if i weren’t subjected to microsoft products..,. like my days would feel a 1000 x better
March 14, 2025 at 10:58 AM
my life would radically improve if i weren’t subjected to microsoft products..,. like my days would feel a 1000 x better
Reposted by Ben Van Calster
NEW PREPRINT 📊: We propose 3 methods to obtain flexible calibration plots while accounting for clustering:
1. Clustered Group Calibration (CG-C)
2. Two-Stage Meta-Analysis Calibration (2MA-C)
3. Mixed Model Calibration (MIX-C)
Ready-to-use R code included!
1. Clustered Group Calibration (CG-C)
2. Two-Stage Meta-Analysis Calibration (2MA-C)
3. Mixed Model Calibration (MIX-C)
Ready-to-use R code included!
March 12, 2025 at 12:07 PM
NEW PREPRINT 📊: We propose 3 methods to obtain flexible calibration plots while accounting for clustering:
1. Clustered Group Calibration (CG-C)
2. Two-Stage Meta-Analysis Calibration (2MA-C)
3. Mixed Model Calibration (MIX-C)
Ready-to-use R code included!
1. Clustered Group Calibration (CG-C)
2. Two-Stage Meta-Analysis Calibration (2MA-C)
3. Mixed Model Calibration (MIX-C)
Ready-to-use R code included!
We tried to look at ways to obtain flexible calibration plots in clustered (e.g. multicenter) validation studies.
Work with @lasaibarrenada.bsky.social @laurewynants.bsky.social @bavodccampo.bsky.social
arxiv.org/abs/2503.08389
Work with @lasaibarrenada.bsky.social @laurewynants.bsky.social @bavodccampo.bsky.social
arxiv.org/abs/2503.08389
Clustered Flexible Calibration Plots For Binary Outcomes Using Random Effects Modeling
Evaluation of clinical prediction models across multiple clusters, whether centers or datasets, is becoming increasingly common. A comprehensive evaluation includes an assessment of the agreement betw...
arxiv.org
March 12, 2025 at 8:43 AM
We tried to look at ways to obtain flexible calibration plots in clustered (e.g. multicenter) validation studies.
Work with @lasaibarrenada.bsky.social @laurewynants.bsky.social @bavodccampo.bsky.social
arxiv.org/abs/2503.08389
Work with @lasaibarrenada.bsky.social @laurewynants.bsky.social @bavodccampo.bsky.social
arxiv.org/abs/2503.08389
It is a good day today, I got the reviewer comment "please consult a statistician" again.
March 10, 2025 at 8:10 AM
It is a good day today, I got the reviewer comment "please consult a statistician" again.
Reposted by Ben Van Calster
'Onze universiteit roept op tot evidence-based beleid, maar zelf volgt de rector zijn buikgevoel'
www.veto.be/onderzoek/on...
www.veto.be/onderzoek/on...
'Onze universiteit roept op tot evidence-based beleid, maar zelf volgt de rector zijn buikgevoel'
Professor wetenschapsfilosofie Andreas De Block zat acht jaar lang de rectorale denktank rond basisfinanciering voor. Hij blikt teleurgesteld terug op het afgelegde traject: 'De uitkomst moest altijd ...
www.veto.be
March 4, 2025 at 4:14 PM
'Onze universiteit roept op tot evidence-based beleid, maar zelf volgt de rector zijn buikgevoel'
www.veto.be/onderzoek/on...
www.veto.be/onderzoek/on...