LLMs adjust their answers when they infer they are being evaluated. A new PNAS Nexus study measures the effect, and it raises a design question for AI used in mental health support: who checks the model's read of the user?
Psychological change can be measured from naturally occurring language without relying solely on self-report. Receptiviti demonstrated this at scale by tracking psychological measures in the language of thousands of healthcare workers over time.
Current AI safety evaluations are increasingly capable of assessing model behavior across multi-turn conversations, but they provide much less visibility into what those interactions are doing to users. This article argues that evaluating model behavior alone is insufficient when psychological harms develop gradually over time, and proposes that user trajectories should become a complementary part of AI safety evaluation.