Using Facial Recognition Software to Measure Emotions
Researchers in the Eller College of Management at the University of Arizona are leading efforts to determine how facial recognition software can be effectively used to measure emotions in the workplace.
Tamar Kugler, Associate Professor of Management and Organizations
Tamar Kugler, associate professor of management and organizations, Charles Noussair, Eller Professor of Economics and director of the Economic Science Laboratory, and Bohan Ye, PhD candidate in economics, have teamed with Daphna Motro ’16 PhD (Management), assistant professor of management and entrepreneurship at Hoftstra University, in their research on emotions and particularly the strong relationship between disgust and trust.
In research published in Social Psychological and Personality Science in 2019, they found that the traditional methods of measuring emotions, which use different forms of self-reporting, are not always accurate because participants may withhold, misidentify, or misrepresent their emotions, or the process of self-reporting itself may change their emotional state.
Enter facial recognition software, which according to an article published by the researchers in MIT Sloan Management Review on November 20, 2019, “provides an immediate, unobtrusive, and objective reading of real-time expressions of anger, fear, anxiety, sadness and happiness.”
Charles Noussair, Eller Professor of Economics and Director, Economic Science Laboratory
Previous analysis of facial recognition results has benefited the fields of psychology, economics and finance. The research by Kugler, Noussair, Ye and Motro builds on these previous studies with a particular focus on management, including issues such as trust, negotiations and deviance, they say. Their study is also the first to “employ an immersive virtual reality environment to induce emotions,” pioneering new technology both for inducing emotions and for measuring them.
For example, in one of their studies using the Noldus FaceReader—which, through algorithms, finds and analyzes “ 500 key points in the face to detect emotions objectively and unobtrusively, using facial movements from photos and videos recorded in real time”—the researchers found that participants “experiencing disgust judged other people in the study as less trustworthy and were less likely to risk lending them money.” Feelings of disgust are very likely antithetical to the building of trust, a conclusion bolstered by the use of facial recognition software.
Bohan Ye, PhD Candidate in Economics
Indeed, the software can be used not only for research but also to assist with the identification of deviant and unethical behavior by employees, as well as in marketing and employee training.
Yet the use of facial expression software is not without questions of ethics and privacy, which the researchers acknowledge in the MIT Sloan Management Review: “We must be vigilant in using this software responsibly, keeping in mind that the primary goal is to help employees deal with negative emotions in a constructive way.”
Even with a cautious approach, Kugler, Noussair, Ye and Motro find that facial recognition software provides “a revolutionary approach” to measuring emotions effectively—both in research and the workplace.
Header image courtesy Noldus Information Technology.