Dr. Sanjay Kumar is an AI & Data Science Product Leader with 15+ yrs in AI, MLOps & cloud analytics driving enterprise innovation.
With agentic AI, AI is starting to make its own decisions, set its own goals and sometimes act completely autonomously, similar to a student testing the boundaries of independence.
The benefits of this development for society and industry could be significant, but we cannot ignore valid concerns. Who is responsible for monitoring these technologies? Who is accountable if something goes wrong? Is it reasonable to expect independent AI to act ethically?
To answer these questions, we need to create clear guidelines, maintain active oversight and approach agentic AI adoption with critical thinking. Without these steps, we risk allowing these systems to run unchecked, which could lead to unintended and widespread consequences.
Before diving into what ethical agentic AI oversight looks like, let’s consider a few of the major concerns:
Bias
Like with GenAI, if you feed AI agents bad information, it will produce flawed results. AI systems can, for instance, internalize old prejudices, affecting areas like hiring, loans and even court rulings. The stakes for these biases only rise when AI acts independently.
To address this, you need to diversify your data, check for fairness and continuously test the system for any unusual outputs.
Hallucinations
“Hallucinations” doesn’t mean that AI agents see things that aren’t there; it’s when AI invents information. In agentic systems, incorrect information can lead to serious mistakes. Imagine a robot giving misleading medical advice, or a chatbot misguiding a politician.
To keep AI on track, you need to verify its facts, assess its confidence and involve humans in the process.
Overreliance
People can become complacent. When they encounter a slick AI, they might start to trust this technology too much. That’s a problem. If people stop thinking for themselves, who will notice when the machine makes a mistake?
The goal should be to have people and machines collaborate, not to let technology take over human judgment.
Misuse
Unfortunately, there are always those who will exploit technology for nefarious purposes: scams, spying, manipulation and more. To maintain some control, we need to set up safety measures like strong ethics, secure features and constant monitoring.
Ethical And Responsible Agentic AI
Major tech companies and regulators both understand these concerns, so they are starting to create guidelines and regulations to ensure agentic systems can be implemented ethically and responsibly. While there’s no way to cover all of the frameworks and regulatory guidelines here, the two getting the most attention so far are:
1. The NIST AI Risk Management Framework (NIST AI RMF): This framework was developed by the U.S. National Institute of Standards and Technology (NIST) to provide guidelines for the safe and reliable use of AI. Its purpose is to help organizations identify, assess and manage potential AI risks—such as glitches, privacy breaches or bias. The framework guides users through recognizing possible issues, evaluating their likelihood and impact and implementing safeguards to mitigate them. For example, in healthcare, hospitals can use the AI RMF to evaluate whether a new AI diagnostic tool might introduce risks or lead to harmful outcomes before it is deployed.
2. The EU’s AI Act: Europe is taking a stricter approach to regulation. The EU’s AI Act categorizes AI into different levels of risk: minimal, limited, high or outright prohibited. For instance, the act is especially strict for law enforcement or critical infrastructure—no one wants a malfunctioning robot officer. To develop high-risk AI, you must meet strict requirements: Demonstrate your system works effectively, document everything and ensure humans remain in control when it counts.
Guidelines and regulations are only part of the story. Responsible AI means going beyond what’s required to ensure ethics is built into every stage of development. This starts by understanding a few key principles:
1. Fairness: AI systems must avoid bias and treat everyone equally.
2. Transparency: If an AI makes decisions, the company using it should understand the reasons and methods behind those decisions.
3. Accountability: Someone must take responsibility—whether AI gets something right or wrong. We can’t simply pass the blame to the machine.
4. Privacy: Personal information must be safeguarded, not exposed or misused.
5. Reliability: AI should perform consistently, not just on its best days.
Governance Of Ethical Autonomous AI
Without proper oversight, AI can deviate, reinforce biases or completely fail. Regular audits and bias checks are not just bureaucratic steps—they are practical ways to detect issues before they grow.
The “black box” issue is a significant barrier to trust, particularly in fields like finance and healthcare. In these areas, privacy and transparency are essential. People need clarity on how decisions are made. While providing explanations can enhance trust, it may also reveal the complexity and difficulty of these decision processes.
In short, agentic AI increases the stakes. These technologies can improve productivity but also carry serious risks. Active monitoring and regular evaluations are crucial to manage these risks effectively.
Forbes Technology Council is an invitation-only community for world-class CIOs, CTOs and technology executives. Do I qualify?
