OpenAI has released MentalHealthBench, a public benchmark for testing how models handle mental health conversations across routine stress, high-acuity situations, and emergencies. The dataset and method are outlined in the OpenAI announcement.
The release includes 1,215 synthetic conversation prefixes and 5,262 expert-authored rubric criteria. OpenAI says more than 80 licensed psychiatrists and psychologists from 22 countries contributed, with coverage spanning 19 languages. Responses are scored with weighted rubric criteria, and the benchmark paper says GPT-5.6 Sol is used as the grader.
For teams evaluating assistants in sensitive domains, the benchmark is a new public reference point rather than a complete safety verdict. It is effectively single-turn, and the automated grader is its own dependency, so benchmark wins alone are not enough evidence for routing, policy, or compliance claims without internal checks.