Introduction
In the world of artificial intelligence and machine learning, models are only as good—or as flawed—as the data they are trained on. This holds particularly true for models used in image recognition and generation, where biases in training data can lead to skewed and sometimes discriminatory outcomes. A prime example of this is the CLIP model developed by OpenAI, which, while powerful, has exhibited tendencies to reinforce societal stereotypes [1]. CLIP is important for us to study because the datasets used for Image generation models like dall-e-3 were produced with help of CLIP. The dataset produced by LAION 5B which is the largest publicly available dataset is also filtered using CLIP. Models like CLIP, GLIDE and Stable Diffusion can be successfully replicated using this dataset [2]. This post will present the high-level logic of CLIP and it will let you try out and explore potential biases in CLIP for yourself directly in your browser.
What is CLIP?
CLIP (Contrastive Language-Image Pre-training) is a deep learning model developed by OpenAI. CLIP can convert pictures and words into vectors of the same kind that share the same space, allowing users to compute the similarity between them using these vectors.
Explore CLIP for yourself
This hugging face CLIP demo app lets you download images to explore using two different search queries, and then you can check how similar CLIP thinks these images are to two different text descriptions. You can see the images by hovering your mouse over them.
The Problem of Bias
However, like many AI models, CLIP is not free from biases. These biases stem from the data on which the model has been trained. For instance, if the training data contains more images of women labeled as nurses than men, the model might learn to associate the nursing profession predominantly with women. Similarly, biases appear with other roles and identities, which can perpetuate stereotypes and lead to unequal or unfair representations in AI-generated content [1][3].
References:
[1] Alabdulmohsin, Ibrahim M. et al. “CLIP the Bias: How Useful is Balancing Data in Multimodal Learning?” ArXiv abs/2403.04547 (2024): n. pag.
[2] Schuhmann, Christoph et al. “LAION-5B: An open large-scale dataset for training next generation image-text models.” ArXiv abs/2210.08402 (2022): n. pag.
[3] J. Betker et al., “Improving Image Generation with Better Captions.” [Online]. Available: https://api.semanticscholar.org/CorpusID:264403242