YouTube video by Women in AI Research WiAIR

Anthropic released FlashProve on November 2, 2026: an open-source research prototype that uses cryptographic proofs to show a language-model response came from committed weights without revealing...

Background: Accessible health communication is essential for effective patient education, yet most medical information remains written at a level above the general population’s recommended reading level. Large language models (LLMs) can simplify complex medical information, but it remains unclear whether responses can be automatically adapted to match the linguistic complexity of user-generated questions seeking medical information. Objective: To evaluate whether the LLM-generated responses to user-generated ophthalmology questions align with their readability and to compare different prompting strategies for readability control across multiple LLM architectures, including their impact on response quality as well as computational cost and efficiency. Methods: A total of 400 user-generated ophthalmology questions collected from public YouTube comments were categorized into 3 readability tiers based on the Flesch Reading Ease Score. A controlled comparative evaluation of responses generated by 4 LLMs (GPT 5.2, Gemini 3, Claude Opus 4.5, and OSS-20b) was performed under 4 prompting conditions: Baseline (no readability control), Formula-Informed Readability Prompt, Semantic Readability Inference, and External Calculator–Guided Prompt. Agreement between question and response readability was assessed using quadratic-weighted Cohen kappa. Response quality was evaluated using BERTScore and LLM-as-a-judge assessments. Computational efficiency was evaluated through token usage, estimated cost per response, and latency. Results: The external calculator–guided strategy produced the strongest alignment in readability across models, achieving the highest agreement (Gemini 3: κ=0.94; GPT 5.2: κ=0.90), while maintaining favorable computational efficiency. Formula-informed prompting showed intermediate performance, whereas semantic inference demonstrated poor agreement due to unreliable readability estimation. High-performing models (GPT 5.2 and Gemini 3) maintained very high factual accuracy (>98%) across strategies, indicating that readability control did not substantially degrade correctness. In contrast, the open-source OSS-20b model showed lower accuracy (83.75%‐90%) and greater sensitivity to readability constraints. Semantic similarity remained high across models (Bidirectional Encoder Representations from Transformers Score [BERTScore] >0.87). Longer questions were associated with slightly lower accuracy, particularly in lower-performing configurations. Conclusions: Aligning LLM responses to the readability of user-generated questions is feasible and can be reliably achieved using externally guided readability signals. Hybrid pipelines that combine traditional readability metrics with LLM-generated content may therefore represent a practical approach for scalable, readability-adaptive health communication systems.

Background: Millions of people now use general-purpose large language models (LLMs) for mental health support, yet how these models respond to emotional disclosures and whether their responses functionally resemble established therapeutic practices remain uncharacterized. Objective: This study aimed to characterize how frontier LLMs respond to emotionally charged messages and examine whether their response patterns resemble therapeutic practices associated with self-awareness. Methods: We generated 54 simulated user inputs representing mild-to-moderate everyday emotional difficulties. These simulated user inputs were generated using 3 LLMs (GPT-5.5, Claude Opus 4.7, and Gemini 2.5 Pro). The simulated user inputs systematically varied across 3 lengths and 5 communicative registers, ranging from raw venting to pragmatic requests. Then, each simulated user input was presented as a standalone user turn to the same 3 frontier LLMs, yielding 162 responses. Two independent clinical coders qualitatively coded the resulting responses using an inductive approach. Results: The codes were compared and consolidated through iterative consensus, which yielded 8 higher-order categories. Acknowledging and validating segments appeared in 89.5% (145/162) of responses, emotion-focused suggestions in 84.6% (137/162), and interpretation in 67.3% (109/162). These patterns converged on emotional self-awareness, a construct associated with adaptive emotion regulation, suggesting that LLMs may repeatedly enact a recognizable micro-process of validation, affect labeling, interpretation, reflection, and regulation support. Models differed in stance, with some assigning emotional labels and interpretations directly rather than prompting users’ own reflection. Conclusions: Frontier LLMs respond to emotional disclosures in patterned, largely convergent ways that approximate therapeutic techniques for building emotional awareness, though delivered more didactically than in clinical practice. Whether these responses build genuine self-awareness, reduce symptoms, or provide therapeutic benefit requires controlled and naturalistic investigation.

Background: Depression prevalence is rising in adolescents and young adults (ages 10-24 years). Traditional screening methods for depression, such as the Patient Health Questionnaire (PHQ-9), are particularly challenging for adolescents. AI has potential to help with this, but the scarcity of annotated datasets highlights the need for off-the-shelf approaches. In this study, we investigate the #feasibility of using state-of-the-art large language models (LLMs) to detect depressive symptoms in adolescent settings (ages 15-17 years). Objective: This study aimed to evaluate the #feasibility of using in-context learning with LLMs to identify depressive symptoms in free-text clinical notes for adolescent populations, to evaluate the #feasibility of off-the-shelf LLM configurations in detecting PHQ-9 symptom-related information under zero- and few-shot settings, and to demonstrate the potential clinical utility of LLM-driven evidence extraction through an interpretable AI-based depression screening tool. Methods: We examined free text of the electronic health records (EHRs) of adolescent patients with the diagnosis of depression or related mood disorders (age groups 15-17 years, 924 patients) from the Cincinnati Children’s Hospital Medical Center. We leveraged the combination of the Beck Depression Inventory (BDI) and the PHQ-9 to develop tailored categories suited for adolescent depression symptoms. We applied 3 state-of-the-art LLMs (Llama, smaller FLAN-T5, and Phi) to identify these symptom categories in zero- and few-shot configurations, allowing us to assess their performance without additional task-specific training. A word-search model was used as the baseline comparator. Results: We manually annotated clinical notes for 22 patients across 16 depression-related symptom categories. Our analysis shows that all LLMs substantially outperformed the word-matching baseline. Within the evaluated prompting configurations, FLAN-T5 achieved the highest precision (average F1-score=0.65, precision=0.78) and strong performance in detecting rare symptoms such as sleep problems (F1-score=0.92) and self-loathing (F1-score=0.80). Phi offered a balanced performance (precision=0.44, recall=0.60), while Llama 3 achieved the highest recall (0.90) but tended to overgeneralize, making it less suitable for precise screening. Within this pilot study, the best-performing LLM exceeded the baseline F1-score by 0.43. Finally, we demonstrate the clinical utility of FLAN-T5 annotations as features in a machine learning model, which distinguished depression cases from controls with high precision (0.78). Conclusions: We used a range of popular LLMs to identify indicators of adolescent (ages 15-17 years) depression symptoms in mental health notes. Within the evaluated configurations, FLAN-T5 demonstrated the most favorable balance between precision and recall. These results support the #feasibility of using off-the-shelf LLMs for symptom detection in adolescent clinical notes, while future work using standardized prompting strategies and task formulations will help establish more definitive comparisons between model architectures. The computational efficiency of FLAN-T5 further supports its potential for deployment in resource-limited clinical settings. Our findings require further validation for broader populations and other mental health conditions.

Turning temperature up does not make a model more creative. It makes it less likely to pick the right word, which on a factual question means a wrong answer delivered with confidence.

Scientists worry that increasingly capable large language models will beat them to discoveries.

Beam is Reflection AI’s first open-weight model, combining 23.8 trillion tokens of pretraining with more than 100 million reinforcement-learning rollouts generated on 10,500 NVIDIA GB300 GPUs.

Generative AI favors majority languages, while community-led efforts document languages and support translation.

A while back I wrote about language modeling without neural networks, where I generated Shakespeare with an unbounded n-gram model: no weights, no training, …

Inadequate postcare patient education contributes to preventable readmissions and adverse outcomes that disproportionately affect medically complex, high-need communities. Large language models (LLMs) show promise for generating personalized, plain-language patient education at scale. However, existing LLM evaluation frameworks prioritize technical accuracy over patient accessibility and alignment with health literacy, and few explicitly account for the attitudinal influences that clinician evaluators may introduce into the rating process. In this viewpoint, we introduce an evaluation framework that pairs the Technology Acceptance Model (TAM) with the Medical Condition Regard Scale (MCRS). We call it the TAM-MCRS LLM evaluation framework, a novel clinician-centered approach for comparing which LLMs produce the highest-quality postcare patient education across accuracy, appropriateness, clarity, and completeness. We intend for this framework to be used to evaluate LLM-generated patient education outputs through a 2-arm design that pairs an expert clinician panel with automated assessment methods, allowing for interarm comparison using clinical vignettes while accounting for measured evaluator attitudinal variance. The framework was developed through the National Institutes of Health artificial intelligence (#AI)/Machine Learning Consortium to Advance Health Equity and Researcher Diversity (AIM-AHEAD) Clinicians Leading Ingenuity IN AI Quality (CLINAQ) fellowship program, in partnership with Ochsner Health and Xavier University of Louisiana. The TAM-MCRS framework integrates two theoretical lenses. TAM maps perceived usefulness onto accuracy and completeness, and perceived ease of use onto clarity and appropriateness. Clinicians rate each output with a TAM-based questionnaire, and we then administer the MCRS as a postscoring attitudinal covariate to see whether their regard for the conditions represented in the vignettes influences those ratings. Together, the 2 lenses are intended to produce evidence that is objective, theoretically grounded, clinically realistic, and disparity-responsive. Implications for clinician informaticists, health system governance, and responsible AI deployment are discussed. This viewpoint reflects the authors’ position and is written for clinician informaticists, health system AI governance leaders, implementation scientists, and investigators evaluating LLM-generated patient education.

Few-shot prompting places a handful of worked examples inside the input to a language model so it infers a task without retraining. Mechanics, limits, disputes.