The Problems with Personality Tests (2024)

Many AI tools assessing “personality” and “cultural fit” make big claims that they provide accurate identification of personality traits such as openness, conscientiousness, extroversion, emotional stability, adaptability, assertiveness, responsiveness, intensity, optimism, sociability, and grit. However, these assessments aren’t actually based on scientific methods. It is often unclear what these tests assess—and there is significant evidence that they work particularly poorly for underrepresented groups like people with disabilities.

Do AI-enabled personality tests work?

Personality tests—whether AI-enabled or not—use indirect measures to predict the success of an employee by attempting to identify personality traits in candidates and assess them based on the personality traits sought for the job. These connections are weak and unproven. For instance, up to half of people who take personality tests more than once receive different results each time.

Such tests are also most likely to fail when used with outlier candidates including people with disabilities, which could negatively impact an organization’s diversity, equity, and inclusion efforts. Consider that neurodiversity hiring initiatives are gaining rapid traction as organizations seek to diversify their workforce with untapped sources of innovative talent. Using these tests could promote bias and discrimination because an autistic person may score poorly on a generic commercial personality test. And because of the great stigma and negative stereotyping they often face, autistic candidates may be unlikely to disclose before a job offer.

Similarly, an AI system that purports to measure “friendliness” based on voice tone raises a number of questions. Can you be sure the system has been properly tested with a range of Deaf voices? Has the test administrator considered how to avoid unfairly excluding people who communicate by means other than verbal speech (e.g., sign language or text communication)? Can this correlation between friendliness and job success be accurately made to begin with?

Why job seeker personality tests are flawed

Researchers agree that while personality traits are valuable for analyzing group trends, they can’t be used to predict individual behavior. Why? Because human behavior is context-based. People exhibit different personalities in different situations, and do not have fixed personality traits throughout our lifetimes. We change, we learn, and we grow.

Any employee might find success working with one manager, but have the opposite experience with another, even when working in the same position. They might perform differently in different team roles, or in difficult organizational cultures. For example, a person with a disability might perform strongly with an organization with a vibrant culture of inclusion, even if their performance has been subpar in roles where they faced physical, attitudinal, or programmatic barriers in their work.

Good recruiting should focus on matching individuals to optimal contexts. It should focus on matching an individual to a job environment where they can thrive. Is there a role for AI in that? Maybe! Personality tests, on the other hand, assume that human behaviors are static and fixed.

Do behavioral assessments have any scientific validity? They certainly can between a patient and medical professional. Personality tests that are scientifically vetted are generally based on the Five Factors model. But even in the case of scientifically-vetted tests, any attempt to analyze and apply the test results to determine if someone would be a good fit for the job relies on unproven assumptions about which combinations of personality factors would make someone a well-qualified employee.

Personality tests may collect unlawful data

Under the Americans with Disabilities Act (ADA), a company cannot ask job candidates to take a medical exam or make a disability-related inquiry (questions likely to elicit information about a candidate’s disability) prior to making a conditional job offer. Personality tests that ask questions likely to prompt disclosures of psychological, cognitive, or emotional medical conditions might be considered a disability-related inquiry. For this reason, personality testing has sparked ADA-related lawsuits for companies.

Title VII, the Age Discrimination in Employment Act, and the ADA also prohibit the use of procedures that have a disparate impact or disproportionally exclude protected classes, unless the selection procedure is job-related and consistent with business necessity. For a company to administer a personality test that results in a disparate impact, they must prove that the data is measuring factors that are directly related to job performance to avoid exposure to legal risk. The company should also ensure that the test it is using is the least discriminatory alternative available to measure the relevant data; in other words, the company should ensure that there is no equally valid personality test that would tend to have a lesser impact on a protected class.

Mental health and wellness

As the nature of work has shifted in recent years, companies increasingly list worker mental health and wellness as a top priority for maximizing retention and employee success. Up to 94 percent of US and UK workers feel stressed at work, with almost a third saying their stress levels are “high” to “unsustainably high.” Companies are increasingly investing in mental health initiatives to drive their business goals through support of each employee’s individual well-being.

In addition to the legal risks discussed above, using personality tests may be particularly risky for removing good candidates with mental health conditions from consideration. An AI assessment may determine that a candidate with depression or anxiety is a bad fit, even if they have individual strategies for management, and if the organization they are applying to has strong mental health supports in place.

“Cultural Fit” can undermine diversity goals

Cultural fit and diversity are not incompatible, but could be in the hands of an AI tool. If your organization is seeking to increase diversity in the workplace, you don’t want AI tools to pick the same type of candidate that you’ve hired in the past. Because most AI predictors work by comparing candidates’ personality traits with those of past successful hires, they can learn to discriminate by picking people who are similar to past hires in terms of factors such as race, disability, and gender identity. This is true even when demographic data is removed.

Continue to Civil Rights Principles for Hiring Assessment Technologies