Major study on facial recognition – humans versus computers

Dr Eilidh Noyes was part of a major project to confirm whether forensic facial examiners and people with innate recognition skills – ‘super-recognisers’ – could outperform the latest computer algorithms.  The results came as a surprise.

ACCURATE facial identification from images is increasingly vital as an aid to criminal investigation and to prevent miscarriages of justice.  

However, despite the rise of facial recognition technology, the final decision on facial identifications in criminal and security situations currently resides with humans. 

For the first time, a major study has investigated the merits of ‘man versus machine’ to establish the benefits and shortcomings of both when it comes to facial identification.

Forensic facial examiners – trained experts in facial recognition – are believed to be best at making identity decisions.  The decisions they make are often used as evidence in court.  Gifted individuals or “super-recognisers” – people with a natural ability for facial recognition – are also expected to perform well on identification tasks. 

But the burning question now is could the latest computer algorithms out-perform their human counterparts?

A lecturer at the University of Huddersfield has taken part in major research projects that have probed the issue.

Dr Eilidh Noyes joined the University as a lecturer in cognitive psychology and face recognition is a key area of her research and teaching. 

While based at the University of Texas in Dallas, she took part in the most comprehensive examination so far that has appraised identification performance among groups of humans – including specially-trained experts – and by state-of-the-art face recognition algorithms.

The result… a draw!  Despite huge strides in algorithm performance in recent years, the outcome was more or less a stalemate with the newest computer algorithm performing at similar levels of accuracy to the average score of the forensic examiners.  There was also no difference in the performance of forensic facial examiners and super-recognisers, reassuringly, both groups outperformed the control groups (students and finger print examiners) on the identification task. 

However, the startling result was that when man and machine were combined and worked together, the accuracy attainment level shot up and achieved far better results even than when two facial examiners worked together.

Face recognition accuracy

The findings have been published in the PNAS (Proceedings of the National Academy of Sciences of the United States of America) in an article entitled Face recognition accuracy of forensic examiners, superrecognizers, and face recognition algorithms.

The project, leading to the article in PNAS, was global, with the researchers able to test trained forensic facial examiners based in five continents.  Also included in the project were “super-recognisers” – who had no professional training, but had demonstrated a strong innate ability to recognise faces. 

All of the participants were presented with 20 carefully-selected and challenging image pairs and the article includes tables showing the success rates of all the practitioners and technologies included in the project.

“There are all these ways to perform face identification and it is really important that we get the correct answers because of the consequences,” said Dr Noyes.  “So we performed the first comprehensive study to be able to directly compare performance of forensic examiners, super-recognisers, control participants and some of the state-of-the-art computer algorithms in face recognition.

“What we found is that the best results are achieved by combining the results of a forensic expert with our best-performing algorithm.  So it’s not the case that man has necessarily to compete with machine – we get the best results when they work together.”

“We think that humans are doing something a little bit different to the algorithm,” added Dr Noyes.  “So you can use the strengths of both to get the best results.”

Dr Noyes has also co-authored a recent article in Applied Cognitive Psychology that investigates the problems identifying faces in poor-quality images, such as CCTV footage.  And she is lead author of a 2018 article in Cognitive research: principles and implications that analyses body and biological motion recognition as relevant skills for person identification.