AI Bias Mitigation Resources Your Whole Team Will Love [Technical and Multidisciplinary]

Many deep technical problems across AI applications are becoming more manageable. Simultaneously the potential for AI to impact the lives of humans (in good and bad ways) is growing. This leads to a shifting focus for many data and machine learning teams.

Technical requirements are becoming inter-disciplinary concerns. Many non-technical leaders are increasing the sophistication of their understanding of AI. Simultaneously, many technical leaders are pulled into conversations around ethics, trust, system impact on decision making, and consumer perception.

For AI systems that play a role in opportunities for humans, the potential for bias is always top of mind. A growing list of bias auditing regulations ensures large organizations will need to ensure technically rigorous bias mitigation built on diverse perspectives.

At Holistic, we’ve helped many leading AI adopters on their journeys toward more trustworthy AI, and below are some of our favorite resources for upskilling interdisciplinary leaders in AI GRC.

Why bias occurs and why we need to mitigate it

AI bias can occur at any stage of the AI development lifecycle, from data collection to model training to deployment. As systems become more complex and ingest more varied data, bias tracking and mitigation becomes harder. This has led to the need for proactive anti-bias mechanisms throughout the AI product lifecycle.

Algorithmic bias can significantly impact people's lives. An AI system that is biased against certain racial or ethnic groups, for example, could deny them loans or jobs. There is, unfortunately, a growing list of real-world examples of how bias has led to real-world harm.

The consequences of biased AI are wide-ranging, including:

• Impact on Decision-Making: Research has shown that AI systems are increasingly used in decision-making processes. For instance, in hiring, it's estimated that over 83% of employers use AI to screen resumes before being seen by a human. There are even guides on how best to communicate with AI to get through this stage of hiring. If these systems are biased, they can unfairly disadvantage certain groups of applicants.
• Healthcare Disparities: In healthcare, AI is used in a progressively wider array of applications, from diagnosing illnesses, to suggesting treatments, to predicting health risks. A 2023 report from the NHS Office of Minority Health found widespread inaccuracy in birth success rates among minority women, leading to unnecessary C-sections in these groups. After removing ethnicity and race as algorithmic factors, algorithmic equity improved.
• Financial Services: In financial services, AI is used for credit scoring. A report by the University of California, Berkeley, found that both online and face-to-face lenders charge higher interest rates to minority borrowers, suggesting potential biases in algorithms.
• ‍Public Trust: A Pew Research Center survey found that 58% of Americans believe AI needs to be carefully managed. This indicates a public concern over potential biases and the ethical use of AI.
• ‍Gender Bias in AI: A study by the University of Virginia revealed that certain AI systems had a 34% error rate in identifying the gender of darker-skinned women, compared to a 0.8% error rate for lighter-skinned men, highlighting significant gender and racial biases.
• ‍Legal Implications: With the growing awareness of AI biases, there's an increasing legal focus on AI fairness. For instance, the EU AI Act includes provisions for algorithmic accountability and fairness, which can have legal repercussions for non-compliance.
• ‍Economic Impact: Biased AI can lead to missed opportunities and economic losses. For example, if an AI system in a financial institution incorrectly assesses credit risk based on biased data, it could lead to either risky investments or missed opportunities with creditworthy clients.
• ‍Consumer Perception: A Capgemini report found that 62% of consumers would place higher trust in a company whose AI interactions they perceived as ethical, indicating the business value of unbiased AI.

Additionally, bias audits are increasingly required by regulation in many jurisdictions and AI use cases. Most notably the pending EU AI Act as well as the NYC Bias Audit governing AI in hiring.

To address concerns about algorithmic harms related to bias and lack of fairness, several focus areas are commonly employed.

• Data cleaning and balancing involves removing or adjusting biased data points to ensure it adequately represents the intended population
• A growing focus on algorithmic fairness applies fairness as a set of metrics to compare and develop algorithms that are less prone to bias
• A focus on inventorying and tracking dependencies and their potential for bias throughout the AI lifecycle has grown over time
• Structure around the continued role of human oversight in AI systems has developed, with experts tasked with reviewing AI systems and the output of different stages, as well as monitoring broader news and regulation
• The implementation of “guardrails” around the input and output of opaque generative AI systems has grown more prevalent

Learn how to mitigate bias with Holistic AI and the Alan Turing Institute

We’ve distilled many of our top learnings on AI Governance, Risk, and Compliance into two technical courses offered in conjunction with the Alan Turing Institute.

The first, Assessing and Mitigating Bias and Discrimination in AI, is a comprehensive exploration of bias in AI systems, equipping learners with both the foundational knowledge and practical tools to identify, understand, and address bias in machine learning, catering to both beginners and those with coding experience in Python.

• Audience: all team members with an interest in mitigating AI bias
• Prerequisites: none for Milestones 1-2, basic Python and data manipulation skills for Milestones 3-5

The second, Assessing and Mitigating Bias and Discrimination in AI: Beyond Binary Classification, gives technical professionals the tools and strategies to address fairness concerns, expanding on foundational concepts and diving deeper into multiclass classification, regression, recommender systems, and clustering, while integrating robustness, privacy, and explainability considerations.

• Audience: data and machine learning team members and leaders
• Prerequisites: basic linear algebra, machine learning and Python

Together, these courses underscore Holistic AI and the Turing Institute's shared commitment to fostering both technological prowess and ethical responsibility in the realm of AI.

Open-source tools and resources to identify, understand and mitigate bias

As awareness of the need to tackle bias has grown, so too has the repository of open-source tools available to streamline this process.

Some of the most effective tools include:

‍Holistic AI Library [Python Bias Toolkit]

‍This python library is designed to assess and enhance the trustworthiness of AI systems.

It provides AI researchers and practitioners with techniques to measure and mitigate bias in various tasks. Its long-term objective is to present methods for addressing AI risks across five key areas: Bias, Efficacy, Robustness, Privacy, and Explainability.

Presently there are methods for measuring bias in algorithms by type including:

• Classification algorithms
• Regression algorithms
• Clustering algorithms
• Recommender algorithms
• And multiclass tasks

Our mitigation module helps in mitigating bias in algorithms focused on use cases in pre-processing, in-processing, and post-processing tasks.

Finally, our plotting module provides a series of pre-made plots related to bias assessment.

This approach ensures a comprehensive evaluation of AI systems. Additionally, the aim of the Holistic AI Library is to minimize risks associated with AI and data projects by introducing risk mitigation roadmaps, which are guides designed to help users navigate and counteract prevalent AI risks.

IBM AI Fairness 360‍

This is a toolkit that provides a variety of metrics and visualizations for assessing the fairness of AI systems.

It provides AI developers and data scientists with a suite of state-of-the-art tools and metrics to ensure fairness throughout the machine learning pipeline, from data training to prediction.

By incorporating comprehensive metrics and algorithms, the toolkit addresses biases in datasets and models, making it an essential tool for those aiming to uphold fairness in AI systems.

LinkedIn Fairness Toolkit

‍This Scala/Spark library is tailored to assess and address biases in large-scale machine learning processes.

It enables users – often data scientists, engineers and researchers – to measure fairness across datasets and models, pinpointing significant disparities in model performances across varying subgroups.

The library also incorporates post-processing techniques that adjust model scores to ensure equality of opportunity in rankings, without altering the existing model training framework.

You may also want to consider resources supplied by the likes of The Partnership on AI, a collaboration between leading technology companies, academic institutions, civil society groups, and media organizations. The Partnership on AI has, among other assets, compiled a set of principles for responsible AI development. Its multidisciplinary approach makes it ideal for a diverse audience.

Addressing bias in AI is an ongoing, collective effort. For those seeking to delve deeper into this critical issue, Holistic AI’s policy and data science teams have created an extensive repository of white papers, blogs, and webinars which, like the resources above, balance technical depth with ethical considerations. Your engagement in this area, whether through education or other means, helps shape a more equitable future for AI.

Focus areas in bias mitigation

While the resources above present a general foundation and toolkits in bias mitigation, application areas and underlying architecture types are expanding in AI. This leads to many areas where teams may need to bolster their understanding of specifics.

Bias in Data Sampling: This type of bias occurs during the data collection phase, where errors in the sampling process lead to overrepresentation of one group and underrepresentation of another. This can happen intentionally or accidentally, resulting in a skewed model that disproportionately reflects certain characteristics. Ideally, data sampling should be either completely random or accurately reflective of the population being modeled.

Bias in Measurement: Measurement bias arises when the data collected is not measured or recorded accurately. For instance, using salary as a metric might introduce bias due to unaccounted factors like bonuses or regional salary variations. Other forms of measurement bias include using incorrect units, inappropriate data normalization, or calculation errors.

Exclusion Bias: Similar to sampling bias, exclusion bias occurs when data is improperly omitted from the dataset. In the process of handling large volumes of data, selecting a smaller subset for training might inadvertently leave out important data, leading to a biased dataset. This bias can also occur when removing duplicates that are, in fact, unique data points.

Observer Bias: This bias happens during the data recording process. The observer or experimenter might selectively record data, omitting certain instances. For example, a machine learning model based on sensor data might miss crucial information if the sampling is not continuous. Additionally, the act of observation or recording itself can influence the data, potentially altering behavior and introducing bias.

Prejudicial Bias: This form of bias is linked to human prejudices, where data selection is influenced by biases against certain communities. Particularly in historical data used for training models in areas with a history of prejudice, it's crucial to ensure that these biases are not perpetuated in new models.

Confirmation Bias: This bias occurs when data is selected to support pre-existing beliefs or hypotheses, disregarding data that might contradict these notions. As a result, the dataset becomes skewed, favoring information that aligns with these preconceived ideas.

Bandwagon Effect: This type of bias emerges when trends in data or communities influence data collection. As a trend gains popularity, the data supporting it may become overrepresented. However, such trends can be fleeting, leading to a temporary and potentially misleading emphasis in the data.

Comprehensive bias mitigation

Regulators and lawmakers are ramping up their efforts to regulate AI, both in your industry and beyond.

Make sure you are equipped to navigate existing and emerging legislation with Holistic AI.

Our 360 AI Governance Platform and suite of innovative solutions are trusted by global companies.

Schedule a call with one of our AI policy experts to enable your financial AI systems with heightened efficacy, adoption, and trust.