#ProcessTracing
LightNews LightNews
byrdnick.com
I'm pumped to see others beat me to a re-analysis of our earliest think-aloud datasets.

Long live #openScience and #processTracing!

We're also developing AI-mediated, human-validated approaches to public decision transcripts, albeit with slightly different research questions. Promising methods!

🧵
jeremiebeucler.bsky.social Jérémie Beucler @jeremiebeucler.bsky.social · 3d
4/15

We reanalyzed vCRT think-aloud data from Byrd et al. (2023, J. Intell.). We transcribed audio using a speech-to-text model, and used LLMs to segment and score reasoning chunks. We validated the scores against human ratings before modeling their dynamics using generalized additive mixed models.
Analysis pipeline. Think-aloud recordings are transcribed by a speech-to-text model, segmented into reasoning chunks and scored by an LLM. Human raters score a subset for validation. Statistical models estimate how the reasoning processes change over time. The transcript, scores and curves illustrate the procedure.
September 22, 2026 at 2:21 PM
hildevanmeegdenburg.com
What better plans for the summer then joining
@methodsnet.bsky.social in Vienna? From 07.07 until 11.07, I will be offering a one week course on #ProcessTracing methods. 1/
April 28, 2025 at 11:38 AM
hildevanmeegdenburg.com
Gentle reminder that you can still sign up for my upcoming (online) #ProcessTracing course at #Komex2024 @komex.bsky.social. Details ⬇️

afww.uni-konstanz.de/en/microcred...
hildevanmeegdenburg.com Hilde van Meegdenburg 🟥 @hildevanmeegdenburg.com · Oct 10
Coming February I offer an one-week, online course on Process Tracing Methods at KOMEX--the Konstanz Methods Excellence Workshops. Registration is now open!

See ⬇️ for more information and registration.

afww.uni-konstanz.de/en/microcred...
November 1, 2023 at 9:31 AM
methodsnet.bsky.social
'Process Tracing Methods' by Derek Beach & @hildevanmeegdenburg.com Webinar is on TOMORROW!

Sign up for free and make sure you don't miss out on this exciting event!

#methods #methodstraining #webinar #methodnets #processtracing #qualitative
methodsnet.bsky.social MethodsNET @methodsnet.bsky.social · Feb 22
SAVE THE DATE! 📢
MethodsNET Webinar Series presents:

'Process Tracing Methods'
February 27, 2025
15:00-16:00 CET (including Q&A)
Instructors: Derek Beach & Hilde van Meegdenburg

Do not miss this session with two leading experts!
Register now: forms.gle/jabBpDaTwYfv...

#research #webinar
February 26, 2025 at 9:38 AM
methodsnet.bsky.social
Process Tracing in action! 🤩
Big thanks to Hilde van Meegdenburg and all participants for a vibrant MethodsNET course at CEU @weareceu.bsky.social last week.
Lots of learning, exchange and a creative “PT” hand sign in the front row! 💜
#methods #research #summerschool #ceu #methodsnet #processtracing
July 16, 2025 at 9:59 PM
hildevanmeegdenburg.com
Next week Thursday, Derek Beach and I will be hosting a #Free online webinar on #ProcessTracing Methods. We'll talk about what PT is and what it is that we are tracing. We'll also make sure there is time for questions. Join us!

📅: Thursday Feb 27
⏰: 15:00 CET
🌐: docs.google.com/forms/d/e/1F...
February 20, 2025 at 3:09 PM
hildevanmeegdenburg.com
June 17-21, I offer a #ProcessTracing course for the MethodsNET summer school.

Part of the Summer School in Social Research Methods (#3SRM), it is for those interested in qualitative case studies and studying "causal mechanisms".

short🧵.

ru.nl/en/education...
April 11, 2024 at 9:22 AM
hildevanmeegdenburg.com
With the end of the year approaching the count stands 3.466 downloads (or, at least, times opened). Who helps me reach 3.500 by the end of the year?
 
Process Tracing: An Analyticist Approach: hildevanmeegdenburg.files.wordpress.com/2023/01/vanm...
 
#Interpretivism #ProcessTracing #Narratives
December 19, 2023 at 10:10 AM
byrdnick.com
🤓 Stimulating DLPFC often impacted performance on NUMERIC cognitive reflection tests (including a base rate neglect task), but not performance on VERBAL cognitive reflection tests (N = 48):

doi.org/10.1016/j.he...

#neuroscience #processTracing #decisionScience #psychology #measurement #assessment
September 2, 2024 at 8:45 PM
michaelschulte.bsky.social
#processtracing #berlin #mpi #nicepeople ... what else?
January 17, 2024 at 12:36 PM
byrdnick.com
Do people vacillate more before or after they can decide?

In this paper, most vacillations occurred before people could decide about logic or moral problems, except maybe for moral dilemmas.

Check out the #processTracing method: doi.org/10.1017/jdm....

#xPhi #ethics #cogSci
Figure 6. Density plots of response times of the first, median, and last switches for each participant in 3 experiments. X-axis is time in seconds. The deliberation phase starts with the presentation of the problem in text format to participants at X = 0. The dashed vertical lines are at X = 60 seconds after which participants could report their final decision.
February 28, 2025 at 11:39 AM
byrdnick.com
Why can economic #inequality depress the #minimumWage?

An is-ought #fallacy?

From over 135,000 people in #protests, experiments, and #processTracing studies, scientists found that people seemed to infer what people OUGHT to earn from what they DO earn.

doi.org/10.1037/xge0...
Observational protest data Experimental results Mouse-tracking results Eye-tracking results and a visualization tool that "attenuated the gap between people’s minimum wage prescriptions" (N ≅ 2000).
June 12, 2025 at 10:32 AM
komex.bsky.social
Ready to push QCA past the basics?

Enhanced standard analysis, principled counterfactuals, robustness/sensitivity, clustering, theory evaluation + QCA×ProcessTracing. In R with modern toolchains.

👩‍🏫 @nenaoana.bsky.social

Online | 23–27 Feb, 2026| €440

🔗 afww.uni-konstanz.de/en/microcred...
November 19, 2025 at 3:13 PM
evalexpl.bsky.social
#ProcessTracing: "in-depth theory-based case study method that helps evaluators empirically trace the process through which an intervention produces a contribution. It looks not just at results, but at..mechanisms, interactions, & contextual factors that bring about those results" #Evaluation #MandE
November 28, 2025 at 2:39 AM
byrdnick.com
Do reflection test solutions actually involve reflection?

Our think-aloud studies found they usually do (doi.org/10.14264/0f1...), but Ryan Jesson found that solution-prompting insight is often unconscious or spontaneous.

doi.org/10.14264/0f1...

#ProcessTracing #psychometrics
Participants and data Procedure Results (1 of 2) Results (2 of 2)
September 23, 2025 at 10:27 AM
hildevanmeegdenburg.com
Today is your last chance to register for the @methodsnet.bsky.social summer school and my #ProcessTracing course.

Join me in Vienna and learn all the ins and outs of this method, mechanisms, and within-case analysis and inferences!

methodsnet.org/course/c01-0...

with @weareceu.bsky.social
C01-03PM: Process Tracing Methods: An Introduction – MethodsNET
methodsnet.org
May 13, 2025 at 7:19 AM
hildevanmeegdenburg.com
#KOMEX2025 just opened for registrations!

If you are looking to brush up your #ProcessTracing skills, come follow my 1wk online course.

Dates: 17-21 Feb 2025
Early bird registration: EUR 440.
All info: afww.uni-konstanz.de/en/microcred...

@komex.bsky.social
Microcredential ekomex: Introduction to Process Tracing Methods | Academy of Advanced Studies at the University of Konstanz
afww.uni-konstanz.de
September 23, 2024 at 11:37 AM
byrdnick.com
My final session at #SJDM 2024: Online #ProcessTracing!

Ian Krajbich gave the rationale.

Xiaozhi Yang demoed webcam #eyeTracking.

Nitisha Desai showed #mouseTracking code.

@adaaka.bsky.social advised on #coding scalable #textAnalysis with #LLMs.

Files on github.com/krajbichlab/...

#python #AI
Workshop outline (with names and affiliations in case that helps you find and cite the experts). Ian Krajbich and visual examples of online eye-tracking, mouse-tracking, and text one may want to analyze (e.g., online reviews) Nitisha Desai showing an example of mouse tracking patterns can reveal about decisions and judgments. Ada Aka explaining how think-aloud protocols and other forms of text process tracing matter: they get us inside the opaque box of people's reasoning! Did you know think-aloud protocols can also be done online? See Byrd et al 2023 below: Byrd, N., Joseph, B., Gongora, G., & Sirota, M. (2023). Tell Us What You Really Think: A Think Aloud Protocol Analysis of the Verbal Cognitive Reflection Test. Journal of Intelligence, 11(4). DOI: 10.3390/jintelligence11040076
November 25, 2024 at 5:23 PM
byrdnick.com
Many cognitive scientists change decisions; few look in the black box to see HOW.

Steph Smith demoed such #processTracing
...in #Qualtrics!
…without any coding!

Find everything you need at github.com/QMT-code/QMT

Follow her work at www.researchgate.net/profile/Step...

#eyeTracking #mouseTracking
Qualtrics Mouse Tracing (QMT) may be the only scalable, code-free way to track mouse activity during experiments, according to Steph Smith (U Chicago). "If you want complete and total flexibility for your study design, we have a step-by-step version with extensive documentation!" – Steph Smith (U Chicago) Does the search pattern predict choice? (Yes) • Outcome: Proportion of choices for higher expected-value option • Predictor: Payne Index {option-wise transitions - attribute-wise transitions} Steph Smith's collaborator is David Dolifka. Find their code, Qualtrics files, etc. on GitHub with the QR code or the URL in the post.
November 22, 2025 at 8:44 PM
byrdnick.com
#AI reasoning models may seem to reason reflectively when they say things like, "Let me rethink that".

But do these "reflective" phrases predict better reasoning performance?

Not in #Deepseek R1 Zero: doi.org/10.48550/arX...

#cogSci #decisionScience #processTracing #psychology
Some examples of the "aha" moments in allegedly "self-reflection" output from AI reasoning models. "An additional important question is whether self-reflection behaviors are associated with improved model performance after RL training. To investigate this, we host DeepSeek-R1-Zero and analyze its responses of the same questions from MATH dataset. While self-reflection behaviors occur more frequently in R1-Zero, we observe these behaviors are not necessarily imply higher accuracy. Detailed analysis can be found in App. D." Self-reflection does not necessarily imply higher accuracy. To investigate whether self-reflection behaviors are associated with model performance during the inference (acknowledging that self-reflection may improve exploration during training—a potential positive effect outside this section’s scope), we analyze questions that elicit at least one response with self-reflection from DeepSeek-R1-Zero across eight trials. For each question, we sample 100 responses and divide them into two groups: those with self-reflection and those without. We then compute the accuracy difference between these two groups for each question. As shown in Fig. 13, the results indicate that nearly half responses with self-reflection do not achieve higher accuracy than those without self-reflection, suggesting that self-reflection does not necessarily imply higher inference-stage accuracy for DeepSeek-R1-Zero. What keywords or phrases count as "self-reflection"? Here's what the paper reports: "terms like “wait” and “try again” frequently result in false positive detections. To reduce false positives, we maintain a small, highly selective keyword pool consisting of terms that are strongly indicative of self-reflection. In our experiment, the keyword pool is limited to: recheck, rethink, reassess, reevaluate, re-evaluate, reevaluation, re-examine, reexamine, reconsider, reanalyze, double-check, check again, think again, verify again, and go over the steps."
June 24, 2025 at 10:24 AM
eakoebele.bsky.social
Now out in an issue in @journalpa.bsky.social with Aaron Deslatte and the newly minted Dr. Adam Wiechman:

“Embracing the ambiguity: Tracing climate response diversity in urban water management”

#urbanwater #ratemaking #institutionalgrammar #processtracing

onlinelibrary.wiley.com/doi/10.1111/...
Embracing the ambiguity: Tracing climate response diversity in urban water management
Climate change is a management and governance challenge requiring diverse potential responses. This article highlights the critical role public managers play in navigating the response diversity of s...
onlinelibrary.wiley.com
March 11, 2025 at 4:02 PM
ecpr.bsky.social
🕵️ Need a hands-on introduction to #ProcessTracing methods for your research? ✅ Sign up for our online course this March: https://buff.ly/3Q5pfAo

You'll cover:
❇️ Causal mechanisms
❇️ Evidence collection & analysis
❇️ Case selection
❇️ Practical applications

👀 Sneak peek below
ECPR Course Taster: Process Tracing Methods
Livestreamed at the ECPR 2024 General Conference, Wouter Wolfs delivers a taster session for his course, 'Process Tracing', as part of our Methods School.Rea...
buff.ly
February 7, 2025 at 11:55 AM
hildevanmeegdenburg.com
***Registration deadline coming up!***

June 17-21, I offer a hashtag#ProcessTracing course for the MethodsNET summer school.

Registration deadline: 1st of May.
Venue: Radboud University, Nijmegen, the NL.
Course dates: 17-21 June, 2024
Costs: €699

www.ru.nl/en/education...
Process Tracing Methods - Introduction (RSS1.07) | Radboud University
This one-week Radboud Summer School course offers a thorough but hands-on introduction to PT and applies the insights to participants’ own projects.
www.ru.nl
April 25, 2024 at 7:46 AM
byrdnick.com
Next I presented papers about how better reflection tests and manipulations as well as #processTracing #surveyMethods challenge/refine #DualProcessTheory.

Most of the papers are online. To find them and get alerts when others are online: scholar.google.com/citations?us...

More in today's poster.
Byrd, N., & Conway, P. (2019). Not all who ponder count costs: Arithmetic reflection predicts utilitarian tendencies, but logical reflection predicts both deontological and utilitarian tendencies. Cognition, 192. https://doi.org/10.1016/j.cognition.2019.06.007 Byrd, N. (2023). Great Minds do not Think Alike: Philosophers’ Views Predicted by Reflection, Education, Personality, and Other Demographic Differences. Review of Philosophy and Psychology, 14(2), 647–684. https://doi.org/10.1007/s13164-022-00628-y Byrd, N., Joseph, B., Gongora, G., & Sirota, M. (2023). Tell Us What You Really Think: A Think Aloud Protocol Analysis of the Verbal Cognitive Reflection Test. Journal of Intelligence, 11(4). https://doi.org/10.3390/jintelligence11040076 Byrd, N., Chapkovski, P., & Michalska, K. J. (2024, November). Experiments In Reflective Equilibrium Using The Socrates Platform. Society for Judgment and Decision Making, New York City. https://bsky.app/profile/byrdnick.com/post/3lbry6q5xas2x
January 10, 2025 at 12:22 PM
polcomm.bsky.social
Online first: The Dynamic #ProcessTracing Environment (DPTE) as a Tool for Studying #PolComm, by David J. Andersen @IowaStateU, @DavidRedlawsk @UDelaware & Richard R. Lau @RutgersU https://doi.org/10.1080/10584609.2019.1579771 #PoliSci
November 18, 2024 at 2:33 AM