AI is already playing multiple roles within HR departments at many companies. A recent survey of more than 250 HR leaders in the US found that 73% are using AI in recruitment and hiring processes. In an upcoming survey commissioned by Indeed, a staggering 8% of Canadian HR and talent acquisition leaders reported that their teams are not currently utilizing AI tools.
Today, AI tools can be used to help with reviewing resumes and scoring job candidates, sourcing talent for open roles, writing job descriptions, identifying opportunities to promote employees, and even sending automated messages to applicants.
“You name it, and there’s an AI tool being built today to work on it,” Trey Causey, head of Responsible AI and senior director of data science at Indeed, said.
AI has the potential to reduce human bias, particularly in hiring, creating better opportunities for workers while streamlining rote tasks so HR professionals can focus on the more human aspects of their roles. But AI can also perpetuate and even amplify inherent biases — and waste both money and time. Organizations should be aware of the risk spectrum involved in using AI and develop strategies for how to use it responsibly.
Here are four ways organizations can identify risks and ensure their use of AI is fair, ethical, and effective.
1. Evaluate the risks and rewards for your organization
AI systems can scale up processes, such as identifying and scoring more job candidates than could be processed manually.
However, “you can also scale up mistakes and errors, as no system is perfect,” Jey Kumarasamy, an associate at Luminos.Law, a law firm focused on AI, said. “Even if you have 90% accuracy, which is being generous, if you are processing thousands of applications, there’s going to be a sizable number of applications that were assessed incorrectly.”
The starting point for evaluating AI-powered HR tools should be an understanding that the tools are imperfect. “Biases are inevitable, so companies will need to figure out how they plan to address them, or accept that they are at risk,” Causey said.
While some companies accept the risk because of the productivity boost, others may feel that the potential margin of error compromises their values or creates too much complexity in the face of increased regulatory pressures.
If you move forward with AI, choose your tools wisely. AI that provides transcripts of interview conversations, for example, is typically a relatively low-risk application (although it can perform poorly when used with speech from non-native speakers). In contrast, AI that assesses and scores candidates based on their performance in video interviews “is probably the most problematic area because there are a lot of risks and ways it can go wrong,” Kumarasamy said..
Ultimately, AI should augment and improve human processes, not replace them. Before adopting AI tools, make sure your HR team is sufficiently staffed so that humans can review every step of any process that AI automates. Leave critical HR matters to people, such as final hiring decisions, promotions, and employee support.
2. Screen third-party vendors that provide AI-powered tools
Once you’ve decided what kind of AI tools are best for your organization’s needs, you might approach prospective vendors with specific questions, such as if they’re compliant with current and emerging regulations. “I spoke with a vendor last year and asked if they were compliant with a specific regulation, and they hadn’t heard of it before,” Causey said. “Not only was that a red flag, but it clearly, directly impacted their product.”
Asking third-party vendors if they’ll comply with any AI audits you conduct is also important. “When you do an AI audit, chances are you need a vendor to help — and that’s usually not the best time to find out that your vendor doesn’t want to cooperate with you or provide you with documentation or results,” Kumarasamy said.
These are other questions you should be asking yourself and third-party vendors that provide AI-powered tools:
- How do they audit their system? When was the last time it was tested, and what metrics were used?
- Was the testing done internally or by an external group?
- How is bias mitigated? If they claim their system has minimal bias, what does that mean, and how is that bias measured?
- Are there testing metrics available for you to review as a prospective client?
- If the model degrades in performance, do the vendors provide post-deployment services to help train your employees in configuring and maintaining the system?
3. Identify and monitor bias
AI algorithms are only as unbiased as the data used to train them. While employers can’t modify how algorithms are developed, they may run the risk of opening themselves to potential liability under Canadian human rights laws. These prohibit discrimination in employment based on certain protected grounds, such as race, ethnic origin, gender identity, and age. There are ways to test out the tools before implementing them, such as conducting third-party bias auditing before launching AI for hiring purposes.
As you implement AI systems, make sure to continuously monitor them to identify and correct any discriminatory patterns when they emerge, while staying apprised of developing research on data science and AI. “When you have humans making decisions, it’s difficult to know if they’re biased,” Causey said. “You can’t get into someone’s brain to ask about why they said yes to this candidate but no to that candidate; whereas with a model, we can do that.”
There’s no standard suite of tests to evaluate HR tools for bias. At a minimum, employers should clearly understand how AI is being used within the organization, which could include keeping an inventory of all AI models in use. Organizations should document which tools were provided by which vendor, along with the use cases for each tool.
4. Stay ahead of evolving legislation
The potential risks of automated HR tools are not just reputational and financial — they’re legal, too. Legislation is quickly emerging in response to the proliferation of AI in the workplace.
In the European Union, the proposed AI Act aims to assign risk levels to AI applications based on their potential to be unsafe or discriminatory, and then regulate them based on their ranking. The current proposal considers AI applications that scan resumes and CVs to be high-risk applications that would be subject to strict compliance requirements.
In Canada, the government took the first step towards regulating AI when they tabled the Artificial Intelligence and Data Act (AIDA) in 2022. Similar to the EU, the act centers on risk regulation in relation to the design, development, and commercialization of AI technologies in Canada.
Additionally, many existing laws, such as PIPEDA, apply to employment decisions made by AI. “There is a misconception that if a law doesn’t directly address AI systems, it doesn’t affect an AI system,” Kumarasamy said. “That’s not true, especially when we’re talking about employment.” Whether your employment outcome is attributed to a human being or an AI system, the organization is liable for any bias.
While audits are a good starting point, the best way to prepare for emerging regulatory requirements and ensure that your AI is operating effectively and equitably is to build out a larger AI governance program.
Governance systems document the organization’s principles with respect to AI and create processes for continually assessing tools, detecting issues, and rectifying any problems. For example, Indeed has developed and publicly published its own principles for the ethical and beneficial use of AI at the company. Indeed has also created a cross-functional AI ethics team that builds tools, systems, and processes to help ensure that technology is used responsibly.
Even with safeguards, the new generation of AI tools are complex and fallible. However, putting in the effort to use them responsibly opens the door to building better processes. AI can help humans be more efficient and less biased, but only if humans provide the necessary oversight. There’s a real potential for leveling the playing field with AI for job seekers,” Causey said. Skills-based hiring can be less biased than relying on school or company names — something that AI can be tuned into in a way humans might not be.
Learn more about Indeed’s matching and hiring platform here.
This post was created by Indeed with Insider Studios.
