Psy-LLM – Evaluation of the Use of Large Language Models for the Diagnosis of Psychiatric Disorders
Project Leads:
Dr. med. Stefanie Schreiter (Charité)
Prof. Dr. Daniel Fürstenau (FU Berlin)
Project Team Members:
Anne-Chloé Joblin (PhD Candidate, Charité), Luzie Lerche (Research Associate, FU Berlin)
Former team members: Dr. Constantin Volkmann (Study Physician, Charité), Dr. Moritz Schiltenwolf (Postdoctoral Researcher, FU Berlin), Zahraa Rezk (Intern, University of Lübeck)
Project Start:
05/2025
Project Duration:
initially 24 months
Project Description:
Large Language Models (or "LLMs" for short) are programs that have learned to understand and generate human-like language based on large amounts of text. Current research on Large Language Models demonstrates significant potential for their use in medicine, particularly in psychiatry, where a large amount of information is available in text form.
In the Psy-LLM project, anonymized interviews are used to explore whether and to what extent various current Large Language Models can generally be used to support psychiatric diagnostics, for example by identifying symptoms and generating summaries.
Key questions of the study concern the agreement between the diagnostic results produced by the LLMs and those of medical experts, as well as the quality of the summaries generated by the LLMs. The study also investigates which factors influence the diagnostic accuracy of the LLMs (e.g. mental disorder, prompt engineering).


