Further to our recent article on the new regulatory regime that will apply to artificial intelligence (AI) systems, the Equality and Human Rights Commission (EHRC) has now issued new guidance as to how public bodies, such as local authorities, can avoid AI bias in relation to the delivery of public services.
Public sector bodies have a duty to adhere to the public sector equality duty (PSED) with respect to the delivery of public services, including preventing discrimination against those with protected characteristics under the Equality Act 2010.
AI Bias
There have been mounting concerns for some time that the algorithms that underpin AI systems can produce AI bias in terms of treatment and outcomes that are less favourable to some, especially those with protected characteristics under the Equality Act 2010 (e.g. race, sex, disability, age, etc). For instance, the Information Commissioners Office (ICO) recently launched a formal inquiry into AI bias arising out of the use of AI systems in the recruitment sector, including the algorithms used to screen candidates.
A spokesperson for the ICO stated: “We will be investigating concerns over the use of algorithms to sift recruitment applications, which could be negatively impacting employment opportunities of those from diverse backgrounds. We will also set out our expectations through refreshed guidance for AI developers on ensuring that algorithms treat people and their information fairly.”
The ICO investigation into AI bias comes in the wake of a number of high profile incidents. For example, Estée Lauder recently settled with and paid out 3 members of staff who were made redundant as a result of an automated video interview, and the use of software underpinned by an algorithm. One of the staff members told the BBC that: “I literally thought we would be videoed and someone would mark it after. I found out that wasn’t the case. Nobody saw the video, it was all algorithms.”
Moreover, back in 2018, Amazon had to drop its use of algorithm based AI in its recruitment processes, following a trial of the software which produced AI bias. This was because they found that the software was not gender neutral, and favoured male candidates. According to Reuters, “Amazon’s computer models were trained to vet applicants by observing patterns in resumes submitted to the company over a 10-year period. Most came from men, a reflection of male dominance across the tech industry. In effect, Amazon’s system taught itself that male candidates were preferable."
Public Sector Equality Duty (PSED)
The guidance issued by the EHRC will focus upon eliminating AI bias from systems used by public bodies, in terms of their delivery of public services and their duties arising out of the PSED.
Under the PSED, public bodies are obliged to conduct their functions in accordance with the objectives listed in s149 of the Equality Act 2010, namely to "(a) eliminate discrimination, harassment, victimisation and any other conduct that is prohibited by or under this Act; (b) advance equality of opportunity between persons who share a relevant protected characteristic and persons who do not share it; [and] (c) foster good relations between persons who share a relevant protected characteristic and persons who do not share it."
As AI systems are increasingly being used by public bodies to deliver public services and to carry out their functions, any AI bias in the algorithms underpinning those systems would represent a breach of the PSED. Hence, the guidance issued by EHRC, who from October 2022, will now be actively monitoring the use of AI systems by public bodies
New EHRC Guidance
To assist with eliminating AI bias, the EHRC have produced a checklist within its guidance, which it recommends that public bodies "think about [in terms of] what they need to do to comply with the PSED."
The checklist recommends that public bodies:-
- Identify what AI systems are being used and how PSED applies
- Gather evidence relating to equality issues and any AI bias
- Review how the AI system used could potentially impact those with protected characteristics
- Assess the evidence gathered and the impact the AI system has had in relation to the public bodies obligations under PSED
- Review the outcome of the assessment to determine what changes, if any, need to be made in terms of the future use of the AI system given any evidence of AI bias
- Maintain good record keeping
- To maintain transparency, publish results
- Ensure staff are well trained
- Maintain good procedures for monitoring the impact of the AI system
Commenting upon the guidance produced on eliminating AI bias, the chief executive of the EHRC, Marcial Boo, stated: "While technology is often a force for good, there is evidence that some innovation, such as the use of artificial intelligence, can perpetuate bias and discrimination if poorly implemented....It’s vital for organisations to understand these potential biases and to address any equality and human rights impacts. As part of this, we are monitoring how public bodies use technology to make sure they are meeting their legal responsibilities, in line with our guidance published today."
