#Metascience2025
So, I started keeping a list of empirical metascience questions I wished we had better answers to, especially where there was strong policy relevance.

After going to #Metascience2025 last week, I decided to clean up the list and share it publicly.
July 4, 2025 at 11:00 AM
As one of my favourite colleagues Etienne Roesch just whispered to me in response to a #MetaScience2025 speaker suggesting AI could act as an additional grant reviewer:

A👏I👏is👏not👏an👏analytic👏tool
July 2, 2025 at 2:51 PM
Outstanding commitment by @marialep.bsky.social president of @erc.europa.eu to provide data to researchers who want to assess ERC policies

(And also passionate defence of incremental research 👏)

#MetaScience2025
July 2, 2025 at 1:16 PM
In the Qs, @alexrushforth.bsky.social makes the great point that scientific societies can help research assessment reform by moving away from individual awards and towards awards for teams and infrastructure contributions

(@improvingpsych.org does a great job of this)

#MetaScience2025
July 2, 2025 at 12:04 PM
I’m hoping we can get papercheck developed to the level soon that it can appear in this company of services #MetaScience2025

scienceverse.github.io/papercheck/a...
Which eval services eval/when? Authors have to disclose if alternative outputs are available and share them (if not, why not). These #openscience outputs then help determine who reviews/what. The more they submit the more eval they get = more robust review. #LifeCycleJournal

#metascience2025
July 2, 2025 at 11:42 AM
@alexrushforth.bsky.social speaking about the Dutch national initiative to reform research assessment
#MetaScience2025
July 2, 2025 at 11:10 AM
@lizziegadd.bsky.social from @coarassessment.bsky.social speaks on how you assess the success of a reform movement #MetaScience2025
July 2, 2025 at 10:58 AM
Bregt Saenen presents data from a European Survey of Strategic Approaches to, and Research Assessment of, Open Science
#MetaScience2025

www.scienceeurope.org/our-resource...
July 2, 2025 at 10:54 AM
I’m delighted to see an old colleague, Kelly Cobey, as co-chair of @dorassessment.bsky.social at #MetaScience2025 presenting A Practical Guide to Implementing Responsible Research Assessment at Research Performing Organizations

zenodo.org/records/1500...
A Practical Guide to Implementing Responsible Research Assessment at Research Performing Organizations
"A Practical Guide to Implementing Responsible Research Assessment at Research Performing Organizations" provides action-oriented guidance, resources, and illustrative examples, for research performin...
zenodo.org
July 2, 2025 at 10:43 AM
Emmanuel Boakye, founder of the African Reproducibility Network is building bridges between communities in Africa
#MetaScience2025
July 2, 2025 at 8:38 AM
Anastasia Garnick from @convergentresearch.bsky.social presents the “gap map” and how Focussed Research Organisations (FROs) can help #MetaScience2025

www.gap-map.org
Gap Map
Explore R&D Gaps and their related Foundational Capabilities.
www.gap-map.org
July 2, 2025 at 8:29 AM
I ❤️🧡💛💚💙💜🩷🤍🩵🤎🖤 this take on the tension caused by involving big corporate sponsors in #MetaScience2025. I fall pretty firmly in the “dismantle existing power systems” camp for both, but can also see the wisdom in strategic working with power
I’ve been thinking a lot about the interaction yesterday at the #metascience2025 panel on AI and more generally how I’ve felt about the conference and how I fit into the metascience community, and here’s a potentially useful analogy… 🌈

thread👇
July 2, 2025 at 8:22 AM
There was no bad intent, but it reduces credibility if you say you “asked GPT” and then recite a hallucination that @psysciacc.bsky.social was essential in resourcing the ManyLabs studies (which happened before the PSA was founded)

#MetaScience2025
July 2, 2025 at 8:17 AM
Two useful perspectives on the comment/question we’ll all be talking about after the #MetaScience2025 AI panel:
July 1, 2025 at 4:34 PM
So nice to see the results of a @improvingpsych.org hackathon matured to a paper!

Jane Herbert presents a Bestiary of Questionable Research Practices in Psychology

osf.io/preprints/ps...

#MetaScience2025
OSF
osf.io
July 1, 2025 at 11:51 AM
Both optimistic and pessimistic conclusions from @tomhardwicke.bsky.social #MetaScience2025

Open practices are increasing, but not very fast.

doi.org/10.1177/2515...
July 1, 2025 at 11:46 AM
Key takeaways from @davidvalenta.bsky.social

• Al-assisted teams do not outperform human-only teams
• Some Al-led teams successfully assessed #ComputationalReproducibility
• Al is not reliable - a significant human component is still required

#MetaScience2025
July 1, 2025 at 11:08 AM
Conclusions from a systematic replication of 30 highly cited AI studies (Odd Erik Gundersen)
#MetaScience2025

arxiv.org/abs/2412.17859
July 1, 2025 at 10:53 AM
The Papercheck unconference workshop will be in the SOAS Gallery Lecture theatre at 14:00

#MetaScience2025
July 1, 2025 at 10:40 AM
The question I’d love to ask this morning’s #MetaScience2025 panel is:

How do we, as a community of metascientists, prevent grifters from weaponising the issues we discuss?
July 1, 2025 at 9:10 AM
First time presenting at a big conference: check!

So happy to see how the community of PYMS is growing over the last couple of days, and how people are constantly finding each other to hang out at #metascience2025. This is what PYMS is all about!
We had a great time at our session at #metascience2025 on Monday! We got a lot of new members and connections that joined our community over the last days.

Missed our talk, and want to join PYMS or learn more? Find one of our board members at the last day of the conference!
July 2, 2025 at 10:48 AM
For those who missed it at #Metascience2025, @katiecorker.bsky.social kindly recorded our session on "Transforming research communication: innovations in publishing, peer review, & curation" and you can check it out here: youtu.be/6d-LJYbNBmE
Transforming research communication: Innovations in publishing, peer review, and curation
YouTube video by ASAPbio
youtu.be
July 17, 2025 at 3:55 PM
Check out @rouhiroo.bsky.social posts for a great overview of our session on transforming scholarly communication in case you missed it at #metascience2025
Which eval services eval/when? Authors have to disclose if alternative outputs are available and share them (if not, why not). These #openscience outputs then help determine who reviews/what. The more they submit the more eval they get = more robust review. #LifeCycleJournal

#metascience2025
July 2, 2025 at 1:06 PM