Chapter 8 – Facial Recognition

“Before integrating any new technologies into American life, we must be absolutely sure that those innovations are imbued with our values.” – U.S. Senator Edward Markey, 2018.

facial recognition
Copyright © 2016 Gaoli Sang et al. – Used under CC-BY

Overview

From an objective perspective, most technologies that we commonly encounter are “innocent”. It is only when they are employed in ways which conflict with our moral sensibilities that we would label them as a menace to our liberty, humanity, or community.

For example, it is not unusual to see people taking photos with their smartphones today. It has become so ubiquitous that an observation was made that the sound people make when watching Tiger Woods strike a golf ball has changed in the past 15+ years because of smartphones.

In 2002, they applauded after his shot. Today, everyone is taking a picture or video with their smartphone, so they shout instead of applaud.


But portable photographic technology was not always perceived as a normal and accepted presence in society. When the first Kodak portable cameras were sold to the public around 1888, a cry of outrage ensued about the breach of privacy that was being inflicted by “all those Kodakers” taking people’s pictures in public. Over time, however, society became accustomed to it while social norms adjusted to define what is considered acceptable behavior for individuals using camera technology.

Enter: facial recognition.

Several companies (see below) have developed facial recognition software systems that can be licensed to developers for use in law enforcement, customs enforcement, and passenger checks in airports by the Transportation Safety Administration (TSA).

However, strong objections have been levied by the ACLU, congressional leaders, privacy advocates, and others who feel that facial recognition is not only an invasion of privacy, but a potential weapon to be used to advance discrimination.

Despite these concerns, the presence of facial recognition technology has not unsettled Facebook users who upload 350,000,000 images a day – all processed in Facebook’s DeepFace facial recognition system to conveniently identify their friends in each image. Nor is there much concern from iPhone users about Apple’s FaceID feature for biometric authentication. And then there is Snapchat’s Face Lenses accessory for animating facial images which can acquire several dimensions of a face within a second.

In this chapter, we will examine the affordances of facial recognition systems, where they are used, and then explore some of the ethical issues surrounding their use.

Key Terms

Physiognomy – A pseudoscientific study of a person’s facial features or expression as indicative of character or ethnic origin; the supposed art of judging character from facial characteristics. Here is more on the history of physiognomy dating back to many centuries before classical Greek culture. Findings of these efforts have often been associated with proponents of eugenics theories (Valla, J. M., Ceci, S. J., & Williams, W. M., 2011).

    Required     How facial recognition it works

Below (fig. 1) is a graphical representation of a basic facial recognition system. Each phase of the process is defined below.

facial recognition process
Fig. 1 – AFER Process Flow – © Corneanu, C et al (2016)

 

Localization: The process of identifying the location of key facial features in an image, or in simpler terms, finding where the face is.

facial registration
Fig. 2 – Facial Registration Points – © Xuehan Xiong, Fernando De la Torre (2013)

Registration: The affixing of points on the localized face that correspond with key registration points on a standardized matrix (see Fig. 2).

Feature extraction: Once the registration points have been established, a set of features can be extracted from the matrix that corresponds to a model that has been “taught” to the computer through the input of examples. The section of the graphic referring to “other modalities” means that it is possible to combine facial data with data sets from other forms of acquisition, such as audio, posture, gait, or other physiological data.

Classification: A facial recognition system can be trained to classify input data into prescribes sets that correspond to patterns.  For example, a system can classify input into a set associated with stress and then flag an image if it matches the preset “stress” pattern.

Regression: A statistical process that attempts to produce a “best fit” trend to a set of data. In this case, it is used to provide a match to a database within a statistical degree of certainty, such as “we are 90% certain that the input image matches against the existing profile identified as [ person “X” ].”


Below is a link to a slideshow with a simplified visual representation of the facial recognition process. It is produced by the Electronic Frontier Foundation (EFF), a non-profit organization that advocates for privacy and legal issues related to emerging technologies.

Facial Recognition What is it and how does it work?” slideshow.

What should you be focusing on?

Your objectives in this module are:

  • Identify the current facial recognition systems in use today.
  • Identify the various contexts of deployment, by whom, for what purpose, and under what moral/ethical presumptions.
  • Explain how facial recognition might or might not be a valuable aspect of your project idea.

Readings & Media

Thematic narrative in this chapter

In the following readings and media, the authors will present the following themes:

  1. The history of facial recognition and its uses span to the extremes: from enhancing one’s social media experience to catching international terrorists.
  2. Facial recognition quietly and discretely keeps us all safe.
  3. Facial recognition is genie that cannot be put back in its bottle. Once it is in the hands of agents with bad intent, it will lead to oppression and tyranny.

    Required     Article: A brief history of facial recognition technology

FaceFirst, a corporation that develops facial recognition software and hardware systems, offers a brief description of the key points in the historical development of facial recognition technologies: “A brief history of facial recognition technology“.

Galton portraits
Francis Galton, Frontispiece from Inquiries into Human Faculties, 1883

Optional:  For a glimpse at a very early attempt to formulate archetypes of criminality, sickliness, and other general characteristics based on facial appearance, look at the work of Victorian era (late 1800s) statistician Francis Galton who postulated that by combining photographs of multiple individuals, a composite archetype would be revealed, such as a “pianistic face” of a piano player. He did not consider it successful, though the effort is noteworthy. Do you feel like averaging some faces like Galton? Try it out at FaceResearch.org‘s “Face Average” interactive demo.

 

    Required     Article: List of the most common uses of facial recognition

Alexis AliThe GlobalMe Blog – “Facing the Future: Facial Recognition Uses Today” (Oct 25, 2018). The GlobalMe Blog is produced by a commercial venture that specializes in language localization – a service that enables companies to present themselves in multiple languages. This involves analysis of massive data sets of speech patterns in a way that is similar to facial recognition.

Optional: NEC corporation published an easy-to-read whitepaper that explains common use cases for facial recognition with a few FAQ about the technology and data that is collected. This is a useful document to acquaint yourself with some of the basics, but keep in mind that it is a corporate publication – not an objective report.

Optional: A more scientific document can be accessed through the Granite State College online library entitled, “Survey on RGB, 3D, Thermal, and Multimodal Approaches for Facial Expression Recognition: History, Trends, and Affect-related Applications“.

“A more surveillant society is a safer one.”

    Required     Video (5:56): How Facial Recognition Technologies are Enhancing Australia’s Public Safety

NEC’s General Manager of Smart Systems, Paul Howie describes the various benefits of facial recognition.

    Required     List: Companies that develop facial recognition systems

FaceReader software is used to record data of people’s facial expressions for research purposes and emotional profiling.

NtechLab software “…detects and identifies people’s faces in live video streams and video footage addressing a wide range of business tasks, such as precise people count, demographic information, people flow and client behavior.” Coming soon: tracking a person’s path and recognizing a person’s ethnicity.

Faception is software that claims to detect personality types. Here are excerpts from their website:

Faception is a [technology] for profiling people and revealing their personality based only on their facial image.

Faception can analyze faces from video streams (recorded and live), cameras, or online/offline databases, encode the faces in proprietary image descriptors and match an individual with various personality traits and types with a high level of accuracy. We develop proprietary classifiers, each describing a certain personality type or trait such as an Extrovert, a person with High IQ, Professional Poker Player or a Terrorist. Ultimately, we can score facial images on a set of classifiers and provide our clients with a better understanding of their customers, the people in front of them or in front of their cameras.

“Coding bodies leads to discrimination.”

    Required     Essay: Chris Gilliard’s “Friction-Free Racism” from Real Life magazine

Gilliard’s essay recounts incidences where the perception of his racial identity caused people to react as if his presence was somehow out of place. He connects this experience to the reality that facial recognition is only marginally used for helpful things like unlocking smartphones or identifying a single criminal fugitive. Rather, he states, facial recognition is deployed substantially for the purpose of assigning a person to an identity category which, in a historical context, has served systems of discrimination.

He then projects a future use of facial recognition technology into a form of optics we can wear that, like augmented reality, to help identify “others” according to some identity algorithm so that we can avoid categories of people with whom we don’t want “friction”.

What to look for as you read:

  • What are the connections Gilliard makes between social constructs in a society and the technologies a society embraces?
  • Gilliard says that coding biometric difference – or defining “who is what” – is a form of “biometric determinism”, which is to say that “what you are classified as will determine what you will be or where you belong”. What motivates Gilliard’s concern about this?
  • Given the power we already have in SM systems to select and block people that we do not want to be engaged with, why is Gilliard more concerned that facial recognition system will allow us to avoid friction with people in the real world?

About Real Life magazine: Real Life publishes essays, arguments, and narratives about living with technology. It was founded and edited by Nathan Jurgenson.

Gillard, G. (2018, October 15). Friction-Free Racism [ Blog post ]. Retrieved from https://reallifemag.com/friction-free-racism/

    Required     Article: Letter to Jeff Bezos from Congress

BuzzFeed – “Bipartisan Lawmakers Want To Talk To Amazon About Its Facial Recognition Tech“. Davey Alba, a BuzzFeed reporter on technology issues, provides background surrounding the letter written to Amazon CEO Jeff Bezos by a bipartisan group of U.S. congressional lawmakers about the affordances and risks associated with Amazon’s Rekognition facial recognition technology. The actual letter is embedded at the bottom of the article. A PDF of the primary resource can be downloaded here.

What to look for as you read:

  • What do these lawmakers perceive as the risks of using facial recognition technology, and why?
  • What do they believe should be the solution for controlling the technology?

 



Optional: Supplemental resources related to facial recognition

TechCrunch:3D-printed heads let hackers – and cops – unlock your phone“.

Also worth noting:


References

Valla, J. M., Ceci, S. J., & Williams, W. M. (2011). The accuracy of inferences about criminality based on facial appearance. Journal of Social, Evolutionary, and Cultural Psychology5(1), 66.Corneanu, C., Oliu, M., Cohn, J. F., & Escalera, S. (2016). Survey on RGB, 3D, Thermal, and Multimodal Approaches for Facial Expression Recognition: History, Trends, and Affect-related Applications. Retrieved from http://search.ebscohost.com/login.aspx?direct=true&db=edsarx&AN=edsarx.1606.03237&site=eds-live

Xiong, X., & De la Torre, F. (2013). Supervised descent method and its applications to face alignment. In Proceedings of the IEEE conference on computer vision and pattern recognition (pp. 532-539).

License

Icon for the Creative Commons Attribution 4.0 International License

Trends in Digital & Social Media (V13) by Steve Covello is licensed under a Creative Commons Attribution 4.0 International License, except where otherwise noted.

Share This Book