{"id":83582,"date":"2023-05-23T14:01:36","date_gmt":"2023-05-23T14:01:36","guid":{"rendered":"https:\/\/www.globallogic.com\/uk\/?post_type=insightsection&p=83582"},"modified":"2025-01-20T07:01:45","modified_gmt":"2025-01-20T07:01:45","slug":"mlops-principles-part-two-model-bias-and-fairness","status":"publish","type":"insightsection","link":"https:\/\/www.globallogic.com\/uki\/insights\/blogs\/mlops-principles-part-two-model-bias-and-fairness\/","title":{"rendered":"MLOps Principles Part Two: Model Bias and Fairness"},"content":{"rendered":"
Welcome back to the second instalment of our two-part series \u2013 MLOps (Machine Learning Operations) Principles. If you missed part one, which focused on the importance of model monitoring, it can be found here<\/a>.<\/p>\n

This blog explores the various forms that model bias can take, whilst delving into the challenges of detecting and mitigating bias, and the ethical implications of using biased models.<\/p>\n

Not only will we provide readers with a better understanding of the problem of model bias, we will introduce tools which can help to empower Data Scientists to make more informed decisions when developing and deploying machine learning models.<\/p>\n

 <\/p>\n

Model Bias<\/h4>\n

In recent years, the use of machine learning models has become increasingly prevalent in a wide range of applications. From self-driving cars to facial recognition technology, these models are being used to make important decisions that can have significant consequences on people’s lives.<\/p>\n

However, one important issue that has come to light is the bias in these models. Model bias occurs when a model’s predictions are systematically skewed in favour of, or against, certain groups of people. This can lead to discriminatory and inaccurate decisions.<\/p>\n

There have been several real-life examples of machine learning model bias affecting certain groups of people negatively. Here are just a couple of examples:<\/p>\n

In 2018, it was revealed that Amazon’s AI-driven recruitment tool<\/a> was biased against women. The algorithm was designed to analyse resumes and rank applicants based on their qualifications. However, since the training data consisted predominantly of male applicants’ resumes, the AI developed a preference for male applicants, disadvantaging women.<\/p>\n

The Correctional Offender Management Profiling for Alternative Sanctions (COMPAS) algorithm<\/a>, used in the United States to predict the likelihood of re-offending, was shown to be biased against African Americans. A 2016 investigation by ProPublica found that the algorithm was more likely to falsely label African American defendants as high-risk, while white defendants were more likely to be labeled as low-risk, despite similar criminal records.<\/p>\n

The examples above show that it is essential we appropriately address any potential model bias to reduce the risk of deploying unfair digital systems into society.<\/p>\n

There are different types of model bias which we need to account for and they can originate in all the stages of our model development lifecycle.<\/p>\n

 <\/p>\n

Types of Model Bias<\/h4>\n

There are four different places in the machine learning lifecycle where bias can appear:<\/p>\n

\"\"<\/p>\n

Figure 1 \u2013 Bias in the data science lifecycle<\/em><\/p>\n

<\/h4>\n
Bias originating from the world:<\/h6>\n

Bias originating from the world (also known as historical bias) can still occur with accurate data sampling. Even if the data represents the world perfectly, it can still inflict harm on a certain population \u2013 such as reinforcing a stereotype on a particular group.<\/p>\n

 <\/p>\n

Bias from data generation:<\/h6>\n

There are three main types of bias which can arise from the data generation stage of a data science lifecycle: sampling bias, measurement bias and annotation bias. Each bias essentially comes from what data is recorded from the real world into a useable format \u2013 for example, sampling bias often occurs when data has not been sampled correctly, which can cause underrepresentation of some part of a population and subsequently cause problems when used to generalise a whole population.<\/p>\n

 <\/p>\n

Bias from learning:<\/h6>\n

Learning bias exists if modelling choices amplify performance disparities across different examples in the data. For example, the objective function of an ML algorithm (e.g., cross-entropy loss for classification problems, or mean squared error for regression problems) that is used to optimise the model during training might unintentionally lead to a model with more false positives than desirable. Therefore, it is important to consider potential bias when making modelling choices and prioritise objectives (e.g., overall accuracy) in a way that does not damage another objective (e.g., disparate impact).<\/p>\n

 <\/p>\n

Bias from evaluation:<\/h6>\n

Evaluation bias can occur when the data which is being used to benchmark the model does not accurately represent the use population, therefore the metrics which are calculated in the testing phases of model production are not accurate to its performance in real life. This bias can further be exacerbated by the choice of metrics used to define model performance \u2013 for example, certain metrics such as accuracy could hide subgroup performance which may impact the prediction performance on a particular subgroup of the population.<\/p>\n

 <\/p>\n

Types of harms:<\/h4>\n

There are many different types of harms; below defines the most common types which can occur when using a ML system which inherently has some type of bias in it.<\/p>\n