Trust, Equity, Fairness and Bias Reduction in Assessments

Tests are gateways to opportunity. An opportunity to get mental health treatment and receive learning supports, to be selected for special academic programming. To get hired, coached, or promoted, and to guide decisions regarding risk, levels of supervision, and treatment in criminal justice settings. They are a gateway to life-changing decisions.

Yet, we know that psychological tests have not historically opened doors to universal opportunity. Systemic practices and tools actualizing the myth of the superiority of some groups over others have resulted in “…racism, racial discrimination, and denigration” of communities of color (APA, 2021). In 2021, the American Psychological Association issued an apology for their actions in perpetuating systematic racism where “…Psychologists created, promoted, and administered psychological tests, intelligence tests, educational assessments, and interventions that were normed on White samples, culturally-biased, and discriminated against people of color” (APA, 2021).

The industry must and can do better. To this end, MHS strives for fairness and equity in the development and deployment of the products we make, the solutions we provide, and in the people we lead. Through our strategic focus on Fairness, Equity, and Bias Reduction, MHS is committed to the responsible development and deployment of tests that ensure fair and unbiased decision-making for everyone, regardless of their race, ethnicity, sex, gender, culture, or other status. Unbiased decision-making tools are a gateway to fair and equitable opportunities for all test takers in the intended test population.

A multifaceted Fairness Framework, made up of four key pillars, operationalizes this strategic priority and underlies and guides our product development efforts:

• Research Design and Test Content (Is the test content fair?)
• Psychometric Evidence (Are the test scores fair?)
• Test Deployment (Is the test designed and administered, and available in a way that is fair?)
• Interpretation and Training (Is the test used and interpreted fairly)?

Research Design and Test Content

Ensuring fairness of test content is a multifaceted process where every item we develop is formulated in the context of many considerations to maximize the universality of test content across groups. Is the item using gender-inclusive language? Is the item relevant across different cultural or linguistic backgrounds? Do certain life opportunities, or experiences differentiate how respondents are able to respond? Is the language free of idioms and colloquialisms? Are the test stimuli representative and varied in terms of age, gender, race, and ethnicity? These questions are addressed by rigorous data collection that relies on representative samples. A fair test does not advantage or disadvantage particular individuals or groups because of factors that are irrelevant to the intended construct to be measured.

Psychometric Evidence

Qualitative and quantitative research are crucial to validate and calibrate the carefully constructed and reviewed content. Several external experts with varied backgrounds qualitatively review the test content for cultural sensitivity and applicability across diverse groups. Extensive quantitative research and psychometric analyses are also employed to ensure the scores are not influenced by factors that are not relevant for the construct we want to measure, to establish invariance between groups, and to assess equality in scores and outcomes. A test should measure what it is meant to measure, and that should be true for all people taking the test—the test scores should not favor one group over another based on factors such as age, gender, race, education level, and/or language spoken.

Test Deployment

Accessibility of test content is a crucial component of equity, fairness, and bias reduction in testing. Accessible test deployment refers to the goal of removing any barriers from getting a fair measurement of all people. In particular, wherever possible, accessibility for people with visual, hearing, motor, or cognitive impairment or people from potentially disadvantaged groups (e.g., historically marginalized communities or with low socioeconomic status) is considered, and the test is designed and deployed in a way that maximizes access across groups. When accessibility is not possible (e.g., a visually impaired person unable to take a test with visual stimuli), we clearly articulate the accessibility limitations and the options to accommodate special needs.

Interpretation and Training

A fair test can still be used in an unfair way. Ultimately, fair test usage is the responsibility of the test administrator and interpreter, but as the test developer, MHS provides guidance, resources, and training to support professionals in fair interpretations. When used correctly, assessments provide objective data and insight that can reduce implicit bias or subjective decision-making.

Our Commitment

Trust in our products and solutions is fundamental to our business and our commitment to our customers and the people they serve. Potential should have no barriers. No effort is spared in our continued focus on expanding, developing, and adapting our products and solutions to open doors for equitable opportunity.

Learn more about MHS’ Fairness Framework here.