Using deep learning to predict ideology from facial photographs: expressions, beauty, and extra-facial information

Stig Hebbelstrup Rye Rasmussen, Steven G. Ludeke, Robert Klemmensen*

*Kontaktforfatter

Publikation: Bidrag til tidsskriftTidsskriftartikelForskningpeer review

53 Downloads (Pure)

Abstract

Deep learning techniques can use public data such as facial photographs to predict sensitive personal information, but little is known about what information contributes to the predictive success of these techniques. This lack of knowledge limits both the public's ability to protect against revealing unintended information as well as the scientific utility of deep learning results. We combine convolutional neural networks, heat maps, facial expression coding, and classification of identifiable features such as masculinity and attractiveness in our study of political ideology in 3323 Danes. Predictive accuracy from the neural network was 61% in each gender. Model-predicted ideology correlated with aspects of both facial expressions (happiness vs neutrality) and morphology (specifically, attractiveness in females). Heat maps highlighted the informativeness of areas both on and off the face, pointing to methodological refinements and the need for future research to better understand the significance of certain facial areas.

OriginalsprogEngelsk
Artikelnummer5257
TidsskriftScientific Reports
Vol/bind13
Antal sider8
ISSN2045-2322
DOI
StatusUdgivet - 31. mar. 2023

Bibliografisk note

Publisher Copyright:
© 2023. The Author(s).

Fingeraftryk

Dyk ned i forskningsemnerne om 'Using deep learning to predict ideology from facial photographs: expressions, beauty, and extra-facial information'. Sammen danner de et unikt fingeraftryk.

Citationsformater