Reviewers: Prepare for a tsunami of manuscripts made from low-quality [or even made-up] silicon samples in social science!
#ResearchIntegrity #ReproducibilityCrisis
#ResearchIntegrity #ReproducibilityCrisis
Can large language models stand in for human participants?
Many social scientists seem to think so, and are already using "silicon samples" in research.
One problem: depending on the analytic decisions made, you can basically get these samples to show any effect you want.
THREAD 🧵
Many social scientists seem to think so, and are already using "silicon samples" in research.
One problem: depending on the analytic decisions made, you can basically get these samples to show any effect you want.
THREAD 🧵
The threat of analytic flexibility in using large language models to simulate human data: A call to attention
Social scientists are now using large language models to create "silicon samples" - synthetic datasets intended to stand in for human respondents, aimed at revolutionising human subjects research. How...
arxiv.org
September 21, 2025 at 10:23 AM
Reviewers: Prepare for a tsunami of manuscripts made from low-quality [or even made-up] silicon samples in social science!
#ResearchIntegrity #ReproducibilityCrisis
#ResearchIntegrity #ReproducibilityCrisis
UNRELIABLE: Bias, Fraud, and the Reproducibility Crisis in Biomedical Research by Csaba Szabo / Columbia University Press
-
Design: #HenrySeneYee;
AD: Julia Kushnirsky
-
#csabaSzabo #BiomedicalResearch #ReproducibilityCrisis #STEMEducation #BiomedicalScience #bookcoverdesign
-
Design: #HenrySeneYee;
AD: Julia Kushnirsky
-
#csabaSzabo #BiomedicalResearch #ReproducibilityCrisis #STEMEducation #BiomedicalScience #bookcoverdesign
March 21, 2025 at 4:01 PM
UNRELIABLE: Bias, Fraud, and the Reproducibility Crisis in Biomedical Research by Csaba Szabo / Columbia University Press
-
Design: #HenrySeneYee;
AD: Julia Kushnirsky
-
#csabaSzabo #BiomedicalResearch #ReproducibilityCrisis #STEMEducation #BiomedicalScience #bookcoverdesign
-
Design: #HenrySeneYee;
AD: Julia Kushnirsky
-
#csabaSzabo #BiomedicalResearch #ReproducibilityCrisis #STEMEducation #BiomedicalScience #bookcoverdesign
Germany signed an agreement with MDPI, while Finland and Norway downgraded its journals.
Allegations prioritising quantity over quality include:
- Self-citations
- Focus on special issues (up to 9/day)
- Point system rewarding publication over rejection
tinyurl.com/MDPI-DE
#ReproducibilityCrisis
Allegations prioritising quantity over quality include:
- Self-citations
- Focus on special issues (up to 9/day)
- Point system rewarding publication over rejection
tinyurl.com/MDPI-DE
#ReproducibilityCrisis
Germany faces questions over publishing agreement with MDPI
Landmark deal should raise questions about publisher’s practices on peer review, citations and special issues, say research sleuths
tinyurl.com
January 18, 2025 at 9:53 AM
Germany signed an agreement with MDPI, while Finland and Norway downgraded its journals.
Allegations prioritising quantity over quality include:
- Self-citations
- Focus on special issues (up to 9/day)
- Point system rewarding publication over rejection
tinyurl.com/MDPI-DE
#ReproducibilityCrisis
Allegations prioritising quantity over quality include:
- Self-citations
- Focus on special issues (up to 9/day)
- Point system rewarding publication over rejection
tinyurl.com/MDPI-DE
#ReproducibilityCrisis
LAB CHRONICLE #14
-Trial note-
Name: Drop 37B
Mission: Observe the Leidenfrost effect on thermally uneven substrates
Status: Actively exploring, but refuses to behave
Philosophy: If water could make decisions, it’d probably be leaving the lab by now
#RebelliousDrop #ReproducibilityCrisis #PlanBefore
-Trial note-
Name: Drop 37B
Mission: Observe the Leidenfrost effect on thermally uneven substrates
Status: Actively exploring, but refuses to behave
Philosophy: If water could make decisions, it’d probably be leaving the lab by now
#RebelliousDrop #ReproducibilityCrisis #PlanBefore
August 27, 2025 at 8:37 AM
LAB CHRONICLE #14
-Trial note-
Name: Drop 37B
Mission: Observe the Leidenfrost effect on thermally uneven substrates
Status: Actively exploring, but refuses to behave
Philosophy: If water could make decisions, it’d probably be leaving the lab by now
#RebelliousDrop #ReproducibilityCrisis #PlanBefore
-Trial note-
Name: Drop 37B
Mission: Observe the Leidenfrost effect on thermally uneven substrates
Status: Actively exploring, but refuses to behave
Philosophy: If water could make decisions, it’d probably be leaving the lab by now
#RebelliousDrop #ReproducibilityCrisis #PlanBefore
"Among the world’s most cited biomedical scientists, with 'more than thirty years of direct experience of biomedical research,' Szabo blows the lid off his field."—Christopher Lane, Psychology Today buff.ly/K8ItxJi #ResearchFraud #ResearchEthics #ReproducibilityCrisis #BiomedicalResearch
Unreliable Biomedicine: A Book Review
A new book on biomedical research sounds the alarm on its replication crisis.
buff.ly
March 10, 2025 at 3:32 PM
"Among the world’s most cited biomedical scientists, with 'more than thirty years of direct experience of biomedical research,' Szabo blows the lid off his field."—Christopher Lane, Psychology Today buff.ly/K8ItxJi #ResearchFraud #ResearchEthics #ReproducibilityCrisis #BiomedicalResearch
With calibrated filtering on modification probability, stoichiometry, and coverage, false positives drop sharply.
High confidence sites remain.
Without controls, most low stoichiometry m6A calls do not hold up.
#DataQuality #MethodsMatter #ReproducibilityCrisis #ResearchIntegrity #RNAbiology
High confidence sites remain.
Without controls, most low stoichiometry m6A calls do not hold up.
#DataQuality #MethodsMatter #ReproducibilityCrisis #ResearchIntegrity #RNAbiology
May 13, 2026 at 6:04 PM
With calibrated filtering on modification probability, stoichiometry, and coverage, false positives drop sharply.
High confidence sites remain.
Without controls, most low stoichiometry m6A calls do not hold up.
#DataQuality #MethodsMatter #ReproducibilityCrisis #ResearchIntegrity #RNAbiology
High confidence sites remain.
Without controls, most low stoichiometry m6A calls do not hold up.
#DataQuality #MethodsMatter #ReproducibilityCrisis #ResearchIntegrity #RNAbiology
Art reproducibility≠ Art reproduction
Do we really need to reinvent the wheel to establish a dictionary of usage in open research? A blind repeatability does not fully capture the meaning of reproducibility, nor does accountability, translatability etc.
#openresearchinart#reproducibilitycrisis
Do we really need to reinvent the wheel to establish a dictionary of usage in open research? A blind repeatability does not fully capture the meaning of reproducibility, nor does accountability, translatability etc.
#openresearchinart#reproducibilitycrisis
January 20, 2025 at 10:17 AM
Art reproducibility≠ Art reproduction
Do we really need to reinvent the wheel to establish a dictionary of usage in open research? A blind repeatability does not fully capture the meaning of reproducibility, nor does accountability, translatability etc.
#openresearchinart#reproducibilitycrisis
Do we really need to reinvent the wheel to establish a dictionary of usage in open research? A blind repeatability does not fully capture the meaning of reproducibility, nor does accountability, translatability etc.
#openresearchinart#reproducibilitycrisis
2/ Why does replication code matter? Providing replication code is key to computational #reproducibility (same data + same code = same results), fostering trust in science, reducing errors, and enabling others to build on published research. Yet, how common is this practice? 🤔 #ReproducibilityCrisis
November 29, 2024 at 2:59 PM
2/ Why does replication code matter? Providing replication code is key to computational #reproducibility (same data + same code = same results), fostering trust in science, reducing errors, and enabling others to build on published research. Yet, how common is this practice? 🤔 #ReproducibilityCrisis
Science, particularly the biomedical branch, suffers from a #reproducibilitycrisis. Elizabeth Bik from https://microbiomedigest.com/ who is one of the best known science sleuths (and alas still on X). Has received the Einstein Award - along with #pubpeer […]
Original post on fediscience.org
fediscience.org
November 18, 2024 at 10:14 AM
Science, particularly the biomedical branch, suffers from a #reproducibilitycrisis. Elizabeth Bik from https://microbiomedigest.com/ who is one of the best known science sleuths (and alas still on X). Has received the Einstein Award - along with #pubpeer […]
Science #ReproducibilityCrisis due to dropping statistical standards, eroding public trust @NatureNews http://www.nature.com/news/reproducibility-don-t-cry-wolf-1.17859?WT.mc_id=TWT_NatureNews HT @deevybee
Reproducibility: Don't cry wolf - Nature
Tighten the requirements for declaring physics breakthrou...
www.nature.com
December 6, 2024 at 8:06 AM
Science #ReproducibilityCrisis due to dropping statistical standards, eroding public trust @NatureNews http://www.nature.com/news/reproducibility-don-t-cry-wolf-1.17859?WT.mc_id=TWT_NatureNews HT @deevybee
"[UNRELIABLE] is a disturbing but compelling exploration of the causes of irreproducibility—from hypercompetition and honest errors, to statistical shenanigans and outright fraud."—Asher Mullard, C&EN buff.ly/EgN7nLn #BiomedicalResearch #ReproducibilityCrisis #ScienceIntegrity #ResearchEthics
‘Why is it that nobody can reproduce anybody else’s findings?’
Pharmacologist Csaba Szabo discusses his upcoming book, <i>Unreliable</i>
buff.ly
March 9, 2025 at 9:57 PM
"[UNRELIABLE] is a disturbing but compelling exploration of the causes of irreproducibility—from hypercompetition and honest errors, to statistical shenanigans and outright fraud."—Asher Mullard, C&EN buff.ly/EgN7nLn #BiomedicalResearch #ReproducibilityCrisis #ScienceIntegrity #ResearchEthics
Small differences in computational analyses can drastically alter findings, even with identical data. Parag Mallick and Andreas Huhmer dive into means of combating this component of the #reproducibilityCrisis on the latest episode of Translating Proteomics.
www.nautilus.bio/blog/transla...
www.nautilus.bio/blog/transla...
Translating Proteomics Episode 15: Combating the Reproducibility Crisis in Computational Proteomics
Discover new ways to improve the reproducibility of proteomic and multiomic analyses.
www.nautilus.bio
January 23, 2025 at 5:30 PM
Small differences in computational analyses can drastically alter findings, even with identical data. Parag Mallick and Andreas Huhmer dive into means of combating this component of the #reproducibilityCrisis on the latest episode of Translating Proteomics.
www.nautilus.bio/blog/transla...
www.nautilus.bio/blog/transla...
Limitations in fair benchmarking & performance comparison:
* same problem formulated differently
* datasets annotated inconsistently
* models cannot be tested on multiple benchmarks
* missing details about training, testing, and deployment conditions in the experiments
#ReproducibilityCrisis
* same problem formulated differently
* datasets annotated inconsistently
* models cannot be tested on multiple benchmarks
* missing details about training, testing, and deployment conditions in the experiments
#ReproducibilityCrisis
September 10, 2026 at 9:19 AM
Limitations in fair benchmarking & performance comparison:
* same problem formulated differently
* datasets annotated inconsistently
* models cannot be tested on multiple benchmarks
* missing details about training, testing, and deployment conditions in the experiments
#ReproducibilityCrisis
* same problem formulated differently
* datasets annotated inconsistently
* models cannot be tested on multiple benchmarks
* missing details about training, testing, and deployment conditions in the experiments
#ReproducibilityCrisis
Limitations in fair benchmarking & performance comparison:
* same problem formulated differently
* datasets annotated inconsistently
* models cannot be tested on multiple benchmarks
* missing details about training, testing, and deployment conditions in the experiments
#ReproducibilityCrisis
* same problem formulated differently
* datasets annotated inconsistently
* models cannot be tested on multiple benchmarks
* missing details about training, testing, and deployment conditions in the experiments
#ReproducibilityCrisis
September 10, 2026 at 9:01 AM
Limitations in fair benchmarking & performance comparison:
* same problem formulated differently
* datasets annotated inconsistently
* models cannot be tested on multiple benchmarks
* missing details about training, testing, and deployment conditions in the experiments
#ReproducibilityCrisis
* same problem formulated differently
* datasets annotated inconsistently
* models cannot be tested on multiple benchmarks
* missing details about training, testing, and deployment conditions in the experiments
#ReproducibilityCrisis
We may lose ideas and public engagement by raising statistical standards says @anjahuja http://on.ft.com/1S38xKF #ReproducibilityCrisis
Physicists brought low by thinking big
Science is a messy affair peopled by characters ready to ...
on.ft.com
December 6, 2024 at 8:11 AM
We may lose ideas and public engagement by raising statistical standards says @anjahuja http://on.ft.com/1S38xKF #ReproducibilityCrisis
What can be done about the #reproducibilitycrisis in science? @ConversationUK http://theconversation.com/the-science-reproducibility-crisis-and-what-can-be-done-about-it-74198?utm_source=twitter&utm_medium=twitterbutton
The science ‘reproducibility crisis’ – and what can be do...
We asked three experts for their takes.
theconversation.com
December 6, 2024 at 6:43 AM
What can be done about the #reproducibilitycrisis in science? @ConversationUK http://theconversation.com/the-science-reproducibility-crisis-and-what-can-be-done-about-it-74198?utm_source=twitter&utm_medium=twitterbutton
No publication without confirmation - 'pre-clinical trials' as a way to fix #reproducibilitycrisis http://www.nature.com/news/no-publication-without-confirmation-1.21509 @NatureNews
No publication without confirmation - Nature
Jeffrey S. Mogil and Malcolm R. Macleod propose a new kin...
www.nature.com
December 6, 2024 at 6:38 AM
No publication without confirmation - 'pre-clinical trials' as a way to fix #reproducibilitycrisis http://www.nature.com/news/no-publication-without-confirmation-1.21509 @NatureNews
Useful guidance (in US context) from @AmstatNews on fixing the #reproducibilitycrisis in science http://www.amstat.org/asa/files/pdfs/POL-ReproducibleResearchRecommendations.pdf
December 6, 2024 at 6:33 AM
Useful guidance (in US context) from @AmstatNews on fixing the #reproducibilitycrisis in science http://www.amstat.org/asa/files/pdfs/POL-ReproducibleResearchRecommendations.pdf
Big @BBCr4today feature tomorrow on the #reproducibilitycrisis in science - see links above on some of the debate @TomFeilden @SMC_London
December 6, 2024 at 6:28 AM
Big @BBCr4today feature tomorrow on the #reproducibilitycrisis in science - see links above on some of the debate @TomFeilden @SMC_London
Is psychology really in crisis? #Reproducibilitycrisis https://theconversation.com/is-psychology-really-in-crisis-60869 @ConversationUK
Is psychology really in crisis?
Only 1% of published psychology research papers are ever ...
theconversation.com
December 6, 2024 at 6:23 AM
Is psychology really in crisis? #Reproducibilitycrisis https://theconversation.com/is-psychology-really-in-crisis-60869 @ConversationUK
#Reproducibilitycrisis- the statistical counterattack @signmagazine http://onlinelibrary.wiley.com/doi/10.1111/j.1740-9713.2015.00827.x/full
December 6, 2024 at 6:23 AM
#Reproducibilitycrisis- the statistical counterattack @signmagazine http://onlinelibrary.wiley.com/doi/10.1111/j.1740-9713.2015.00827.x/full
Good to see funders such as @The_MRC thinking about how to deal with the #reproducibilitycrisis in science http://www.mrc.ac.uk/news/browse/funding-reproducible-robust-research/
Medical Research Council (MRC)
MRC funds research at the forefront of science to prevent...
www.mrc.ac.uk
December 6, 2024 at 6:17 AM
Good to see funders such as @The_MRC thinking about how to deal with the #reproducibilitycrisis in science http://www.mrc.ac.uk/news/browse/funding-reproducible-robust-research/
And the more we fix the #reproducibilitycrisis the more WEIRD science we'll get @rcolvile
December 6, 2024 at 6:17 AM
And the more we fix the #reproducibilitycrisis the more WEIRD science we'll get @rcolvile
There's a bit of a #reproducibilitycrisis in science - should we worry? asks @TimHarford http://on.ft.com/1Ye7Nsv
The dubious power of power poses
‘Many notable results in psychology are now being questio...
on.ft.com
December 6, 2024 at 6:12 AM
There's a bit of a #reproducibilitycrisis in science - should we worry? asks @TimHarford http://on.ft.com/1Ye7Nsv
Pleased to see @TheEconomist continue to cover #reproducibilitycrisis in science http://www.economist.com/news/science-and-technology/21690020-reproducibility-should-be-sciences-heart-it-isnt-may-soon
Let’s just try that again
Reproducibility should be at science’s heart. It isn’t. B...
www.economist.com
December 6, 2024 at 6:07 AM
Pleased to see @TheEconomist continue to cover #reproducibilitycrisis in science http://www.economist.com/news/science-and-technology/21690020-reproducibility-should-be-sciences-heart-it-isnt-may-soon