AI is rapidly becoming an integral part of modern medicine, but are healthcare professionals prepared to handle the ethical challenges it brings? There’s no doubt that AI offers immense potential for improving diagnostics, decision-making, and patient outcomes, but the lack of clear ethical guidelines for healthcare professionals (HCPs) remains a significant concern.
A new study, “Developing Professional Ethical Guidance for Healthcare AI Use (PEG-AI): An Attitudinal Survey Pilot,” published in AI & Soc (2025), explores this pressing issue. By gathering insights from healthcare practitioners, academics, and patients, it uncovers the gaps in existing professional guidelines and proposes a unified ethical framework for AI use in clinical settings.
Why AI in healthcare needs ethical guidelines
AI is increasingly present in hospitals and clinics, aiding in everything from diagnostic imaging to patient risk assessments. However, despite its growing use, there is no standardized professional guidance to help HCPs navigate the ethical challenges AI introduces. Current regulations tend to focus on AI developers and purchasers, leaving end-users – the doctors, nurses, and clinicians who rely on AI-driven insights – without a clear rulebook.
This lack of guidance creates a dangerous gray area where professionals must make AI-related decisions without knowing how much responsibility they hold, how to manage biases, or when to challenge AI-generated recommendations. The study emphasizes the urgent need for ethical oversight to prevent AI from compromising patient safety, fairness, and professional accountability.
What should ethical AI guidance include?
To address these concerns, the study surveyed 42 participants, including healthcare professionals, academics, and patients. Respondents reviewed six core ethical themes and 15 specific guidelines that should be part of an official Professional Ethical Guidance for AI (PEG-AI).
1. Preventing patient harm
One of the strongest concerns among participants was that AI must not lower professional standards or compromise patient safety. Many worried that HCPs might over-rely on AI systems without fully understanding their limitations. Training was seen as essential to ensure that HCPs critically evaluate AI outputs before acting on them.
2. Ensuring fairness, inclusiveness, and equity
Bias in AI systems is a well-documented issue, particularly in areas like dermatology, where training datasets often fail to represent diverse patient populations. The study highlights the need for AI tools to be tested for fairness before deployment to prevent the widening of existing healthcare inequalities.
3. Protecting patient autonomy
Many respondents felt that patients should have the right to know when AI is being used in their care and even the option to refuse AI-driven decisions. However, others noted that, as AI becomes more integrated into clinical workflows, refusing its use might become impractical.
4. Preserving healthcare professionals’ autonomy
The study found mixed opinions on how much control HCPs should relinquish to AI. While some believed AI should serve only as a decision-support tool, others raised concerns that excessive reliance on AI could erode clinical judgment over time.
5. Accountability and responsibility
A key issue in AI ethics is who is responsible when things go wrong – the HCP, the AI developers, or the healthcare institution? The study emphasizes that HCPs should remain accountable for AI use, ensuring they understand and justify the decisions AI supports. However, clear legal frameworks are needed to define liability in AI-assisted care.
6. Transparency and consent
The study found that not all patients are aware when AI is used in their diagnosis or treatment. Many respondents felt that transparency is crucial, but opinions varied on whether explicit consent should be required. Some argued that AI is just another medical tool – akin to an MRI machine – while others believed patients should always have a choice.
Challenges in implementing ethical AI guidelines
While respondents generally agreed on the need for professional AI guidance, the study also highlights several obstacles to implementing it. These include:
- High Variability in AI Performance : AI systems can perform well in controlled environments but struggle in real-world settings where patient data varies. How should HCPs handle AI recommendations when models are inconsistent?
- Data Privacy Concerns: AI systems rely on large datasets, often containing sensitive patient information. Clear policies on data security and ethical AI training are needed.
- Conflicting Regulations: Different healthcare regulators may develop their own AI guidelines, leading to fragmented and inconsistent standards. A unified, cross-specialty framework is essential.
- Training and Education: Many HCPs lack AI literacy, making it difficult for them to critically evaluate AI-generated recommendations. The study suggests that AI ethics should be integrated into medical education and ongoing professional training.
Path forward: A unified ethical framework
The study proposes the development of a universal ethical framework for AI in healthcare. This guidance would serve as a foundation for regulators, helping to:
- Set clear accountability rules for AI-driven decisions.
- Ensure AI tools undergo fairness and safety testing before deployment.
- Educate HCPs on AI’s risks, limitations, and biases.
- Protect patient rights by ensuring transparency in AI-driven care.
- Harmonize regulations across different medical fields and specialties.
While this research is only a pilot study, the findings lay the groundwork for larger-scale consultations, expert discussions, and policy developments. The authors plan to expand this work by conducting interviews, workshops, and iterative consensus-building exercises to refine PEG-AI.
