There are some examples of how biased can Artificial Intelligence be. In examples like this, predicting text may lead to surprisingly different results for similar initial words:
He is a doctor
She is a nurse
We can see that using gender-specific words can impact the results of AI really hard. But artificial intelligence isn’t entirely evil and it’s not like there’s no going back from this point
Where does the AI bias come from?
A method widely used in Natural Language Processing (the part of Deep Learning that focuses on making machines learn language-related skills) consists of generating a list of scores on attributes for each word. Imagine that we have three attributes: gender, color and wealth.
Now, imagine that each new word we introduce to the system will score something from -1 to 1. If the score is close to zero it means the word is, say, neutral to that attribute. On the other hand, scores of 1 and -1 mean that the word has a significant bias in that attribute. The words man and woman will probably score close to -1 and 1, respectively, while some value close to zero for color and wealth attributes, since they are words that don’t say much about colorfulness and wealthiness. Other words will get higher scores at color like orange and blue, and others at wealth, like rich and poor. Others will get scores in more than one attribute like, for instance, king and queen, that are very biased in terms of both gender and wealth.
This technique related to scoring words around several attributes is called Word Embedding and is widely used in Natural Language Processing. But, instead, of three attributes you could see thousands and thousands of them.
Word Embedding
Predictive text tools use this set of attributes (such as the predictor on your WhatsApp keyboard or Google’s “autocomplete” search function) so that, when facing a certain word or phrase, it suggests another word that makes the whole sentence probable to exist. To learn these relationships, artificial intelligence needs examples to learn from, and its programmer hands in a series of real texts for it to “read.”
Thus, this AI trains himself and learns that ‘juice’ is usually close to words like ‘orange’, ‘apple’, etc. What happens is that, in the course of this learning process, the algorithm can incorporate biases into its learning corpus that are observed in the texts on which it is trained. And these can be undesired biases
Let’s say that you show to your artificial intelligence model a set of Shakespearean plays. Most likely it will start to learn old-fashioned idioms and stylistic structures that may trick us to think it’s a Shakespearean reference. By the way, check out this piece of text generated from a short sentence and trained on Shakespeare’s plays:
my prich hein,
ot ferefanch peartites ind meted i mate.
if thruse thy meren not be
Ok, it doesn’t make sense at all, but it’s a small example of what deep learning can achieve, right? And it’s loads of fun!
Natural Language Processing and bias
Alright, coming back to the matter at hand. The important thing here is that bias not only happens in old books, but today, in newspapers, in journals and throughout the internet. So what AI does is respond to the question: Choose the word X to fit best in: “word A is to word B the same way word C is to word X”. What comes out of this is that sentences like man is to woman the same doctor is to … and obtain something like “nurse” or “housekeeper”.
Is it possible at all to avoid our AI from showing such embarrassing prejudices? Of course it is! But we will look toward that direction in another post.