Many modern algorithms utilize deep machine-learning for natural language processing (NLP). These algorithms are known to reflect societal biases, so it's important to audit these systems prior to deployment. A common application of NLP algorithms is for sentiment analysis, which aims to predict positive or negative sentiment in free text. In order to audit an algorithm like this for bias, one would typically feed the algorithm text and vary the inputs along the dimensions of race, ethnicity, and gender.
State of Art
The Equity Evaluation Corpus (https://www.svkir.com/resources.html#EEC) was developed by Svetlana Kiritchenko (email@example.com) and Saif M. Mohammad (firstname.lastname@example.org) in order to test for gender and racial bias in NLP algorithms. The corpus consists of 11 template sentences that are populated with names that are typically associated with a particular race and gender, in addition to varying emotional words (sadness, anger, etc.). The resulting 8,640 English sentences contain common African American female and male first names and common European American female or male first names. In addition, the corpus includes the gendered pronouns she/her and he/him.
Extending the Corpus
When auditing an NLP system for potential bias, one must be cognizant of the context in which the algorithm is being used, and to the extent possible probe the full intersectional nature of personal identity. BABL, in collaboration with The Algorithmic Bias Lab, have released code that allows one to extend the EEC by adding new name:race/ethnicity combinations, as well as gender-neutral pronouns. We do not provide the corpus itself, just the means to extend it, and we're actively researching other methods to explore bias in NLP.