News

Zahraa Al-Sahili won an "honourable mention" award at AACL/IJCNLP 2025 held in Mumbai, India. It is one of the top tier of NLP conferences. There were only 10 awards given out across over 200 papers at the conference (not to mention the other 1,000 or so papers submitted that didn't make it into the conference), so it's a great achievement - very well done Zahraa!

The paper is about social biases in multimodal foundation models, and looks in particular at multilingual models - surprisingly, things like gender bias often get worse in multilingual models, and biases can spread into gender-neutral languages from others in the training data.

Abstract

Multilingual vision-language models (VLMs) promise universal image-text retrieval, yet their social biases remain underexplored. We perform the first systematic audit of four public multilingual CLIP variants: M-CLIP, NLLB-CLIP, CAPIVARA-CLIP, and the debiased SigLIP-2, covering ten languages that differ in resource availability and morphological gender marking. Using balanced subsets of FairFace and the PATA stereotype suite in a zero-shot setting, we quantify race and gender bias and measure stereotype amplification. Contrary to the intuition that multilinguality mitigates bias, every model exhibits stronger gender skew than its English-only baseline. CAPIVARA-CLIP shows its largest biases precisely in the low-resource languages it targets, while the shared encoder of NLLB-CLIP and SigLIP-2 transfers English gender stereotypes into gender-neutral languages; loosely coupled encoders largely avoid this leakage. Although SigLIP-2 reduces agency and communion skews, it inherits -- and in caption-sparse contexts (e.g., Xhosa) amplifies -- the English anchor's crime associations. Highly gendered languages consistently magnify all bias types, yet gender-neutral languages remain vulnerable whenever cross-lingual weight sharing imports foreign stereotypes. Aggregated metrics thus mask language-specific hot spots, underscoring the need for fine-grained, language-aware bias evaluation in future multilingual VLM research.

Citation

Zahraa Al Sahili, Ioannis Patras, and Matthew Purver. 2025. Breaking Language Barriers or Reinforcing Bias? A Study of Gender and Racial Disparities in Multilingual Contrastive Vision Language Models. In Proceedings of the 14th International Joint Conference on Natural Language Processing and the 4th Conference of the Asia-Pacific Chapter of the Association for Computational Linguistics, pages 331–352, Mumbai, India. The Asian Federation of Natural Language Processing and The Association for Computational Linguistics. DOI:
10.18653/v1/2025.ijcnlp-long.20

Also Available from https://arxiv.org/abs/2505.14160