A group of researchers claims that AI can use a photo of you to predict what salary you will earn
LinkedIn login, PHOTO: Shutterstock
A picture is worth a thousand words or, perhaps, a substantial increase in salary. Academic researchers claim that personality traits inferred by analyzing photographs with the help of artificial intelligence (AI) can predict how the people depicted will fare in the job market, The Register reports.
The group of researchers from the United States and Israel emphasize that they do not advocate such a practice, because extracting personality from facial images is fundamentally discriminatory. Even so, they say, personality assessment is already widespread in HR departments, and AI tools that provide personality assessments are being rapidly adopted.
Therefore, they argue that an academic evaluation of this technology is needed.
In a paper published this month titled “AI Personality Extraction from Faces: Labor Market Implications,” the authors describe how they used LinkedIn facial images of more than 96,000 MBAs to extract their personality traits. They focused on the so-called “Big Five” traits of the respective individuals – Openness, Conscientiousness, Extraversion, Agreeableness and Neuroticism.
The study conducted by the researchers is not without controversy
The machine learning algorithm used by the researchers was originally described in an article published in 2020 in the journal Scientific Reports, entitled “Assessing the Big Five personality traits using real-life static facial images”.
It was one of about 24 articles cited as examples of “pseudoscience washed up by Machine Learning” in a 2024 study titled “The reanimation of pseudoscience in machine learning and its ethical repercussions.”
But according to the authors of the new paper, the algorithm “uses facial features to predict self-reported personality, not others' perceptions of personality based on visual appearance.”
In other words, the authors are trying to say that their model does not mimic human “shallow judgement,” but instead claims to identify correlations between facial features and actual, self-reported psychological profiles—which is not without controversy and is scientifically debatable.
The authors say their algorithm can even infer personality traits from photos
What is certain is that by applying this algorithm, the authors say they found “that personality traits inferred from facial features provide substantial incremental predictive power for labor market outcomes.”
The researchers concluded that using machine learning to infer personality traits from facial images produced accurate predictions about the university rank the person in the photo received their degree from, starting salary level, salary trajectory and career transitions.
If an HR department were to use a similar technique to assess the personality of candidates for managerial positions, the result could serve as a forecast of the candidate's future performance in the labor market—however biased it may be.
And researchers say this is already happening, in the US and other countries, at least.
Researchers say employers are already using such AI-based tools
Marina Niessner, an associate professor of finance at Indiana University and one of the study's authors, told The Register that players like banks are already using personality quizzes in hiring and promotion decisions, and AI-based recruiting firms are starting to use technologies like personality trait analysis in video call interviews.
“The regulatory environment, as you probably know, is very uncertain,” Niessner pointed out. “So we don't think that's necessarily a valid way of doing things [sau] that companies should proceed this way. But I think it's extremely important to have an academic evaluation of these methodologies if there's going to be even a regulatory discussion around them,” she added.
The other authors of the study are researchers from the Wharton School of Business at the University of Pennsylvania, Yale University and Reichman University in Israel.
