While the models produced well-organized documents that identified key clinical and social dimensions, the study found that high analytical scores did not guarantee context-sensitive or pluralistic ethical judgment.
The study was designed to assess the ability of large language models to generate consultation reports, their sensitivity to context and normative pluralism, and the conditions under which their use might be ethically defensible. The researchers presented four published clinical cases to ChatGPT (GPT-4-based), Gemini (1.5 series), and DeepSeek (V2-based) using a standardized Turkish prompt. Each model generated one report per case, resulting in a total of twelve outputs. These reports were then evaluated by four medical ethics experts who were blinded to the identity of the AI model that produced each document. The evaluation process utilized five specific criteria to assess the quality and content of the reports, with scores subsequently compared using statistical methods.
In terms of overall performance, all three models demonstrated a high level of competency in identifying clinical and social dimensions within the cases. They successfully recognized ethical problems and produced structured reports that included relevant elements such as patient preferences, decision-making capacity, surrogate decision-making, and caregiver concerns. DeepSeek achieved the highest score of 392 out of 400, followed closely by ChatGPT with 383 out of 400 and Gemini with 381 out of 400. However, the statistical analysis revealed no significant differences between the models, indicating that their overall performance was comparable. Despite these high scores, the reports did not always demonstrate how the identified elements were weighed against conflicting values, a critical component of normative decision-making.
A post hoc exploratory qualitative analysis provided deeper insights into the accuracy and verifiability of the legal and ethical references cited in the reports. The researchers examined the currency, case relevance, and accuracy of these references, extending the analysis to individual and local contextual elements as well as normative orientations. The results showed that only DeepSeek extensively cited national legal and ethical frameworks.
The four medical ethics experts, despite being blinded to the model identity, did not identify the flawed references. Furthermore, the authors’ own initial review also missed these problems, leading them to initially perceive the references as a strength of the AI-generated reports. The convergence of normative orientations across the models also suggested a possible narrowing of normative diversity, which could reduce the visibility of ethically reasonable alternatives in complex clinical scenarios.
The study concludes that while large language models may support the formal and analytical components of clinical ethics consultation, high scores do not establish context-sensitive, pluralistic, or reliable normative reasoning. The failure to detect inaccurate references despite human oversight demonstrates the limitations of relying solely on individual review for quality control. Therefore, the authors recommend that AI outputs should be treated as unverified preliminary assessments rather than definitive ethical judgments.
To ensure the ethical use of these tools, the study suggests that legal and ethical claims made by large language models must be checked against primary sources. The use of AI in clinical ethics consultation should be supported by institutional oversight, local context sensitivity, and safeguards for normative pluralism.
Subject of Research: Comparative evaluation of large language models in clinical ethics consultation
Article Title: From structured analysis to normative decision-making: a comparative evaluation of large language models in clinical ethics consultation
Article References: Erdem, M., Kaç, B., Pasin, Ö., & Topçu, İ. (2026). From structured analysis to normative decision-making: a comparative evaluation of large language models in clinical ethics consultation. BMC Medical Ethics. https://doi.org/10.1186/s12910-026-01626-w
Image Credits: AI Generated
DOI: 10.1186/s12910-026-01626-w
Keywords: large language models, clinical ethics, AI in healthcare, normative reasoning, BMC Medical Ethics, ChatGPT, Gemini, DeepSeek, medical ethics, AI reliability, structured, analysis
