Using deep learning to predict ideology from facial photographs: expressions, beauty, and extra-facial information

Stig Hebbelstrup Rye Rasmussen, Steven G. Ludeke, Robert Klemmensen*

*Corresponding author for this work

Research output: Contribution to journalJournal articleResearchpeer-review

45 Downloads (Pure)

Abstract

Deep learning techniques can use public data such as facial photographs to predict sensitive personal information, but little is known about what information contributes to the predictive success of these techniques. This lack of knowledge limits both the public's ability to protect against revealing unintended information as well as the scientific utility of deep learning results. We combine convolutional neural networks, heat maps, facial expression coding, and classification of identifiable features such as masculinity and attractiveness in our study of political ideology in 3323 Danes. Predictive accuracy from the neural network was 61% in each gender. Model-predicted ideology correlated with aspects of both facial expressions (happiness vs neutrality) and morphology (specifically, attractiveness in females). Heat maps highlighted the informativeness of areas both on and off the face, pointing to methodological refinements and the need for future research to better understand the significance of certain facial areas.

Original languageEnglish
Article number5257
JournalScientific Reports
Volume13
Number of pages8
ISSN2045-2322
DOIs
Publication statusPublished - 31. Mar 2023

Keywords

  • Beauty
  • Deep Learning
  • Face/anatomy & histology
  • Facial Expression
  • Female
  • Humans
  • Male
  • Masculinity
  • Neural Networks, Computer

Fingerprint

Dive into the research topics of 'Using deep learning to predict ideology from facial photographs: expressions, beauty, and extra-facial information'. Together they form a unique fingerprint.

Cite this