Citation: Pann, V.; Lee, H.J. Effective
Attention-Based Mechanism for
Masked Face Recognition. Appl. Sci.
2022, 12, 5590. https://doi.org/
10.3390/app12115590
Academic Editors: Pedro Latorre-
Carmona, Samuel Morillas and
Filiberto Pla
Received: 10 May 2022
Accepted: 30 May 2022
Published: 31 May 2022
Publisher’s Note: MDPI stays neutral
with regard to jurisdictional claims in
published maps and institutional affil-
iations.
Copyright: © 2022 by the authors.
Licensee MDPI, Basel, Switzerland.
This article is an open access article
distributed under the terms and
conditions of the Creative Commons
Attribution (CC BY) license (https://
creativecommons.org/licenses/by/
4.0/).
Article
Effective Attention-Based Mechanism for Masked
Face Recognition
Vandet Pann and Hyo Jong Lee *
Division of Computer Science and Engineering, CAIIT, Jeonbuk National University, Jeonju 54896, Korea;
pvd.vandet@jbnu.ac.kr
* Correspondence: hlee@jbnu.ac.kr
Abstract:
Research on facial recognition has recently been flourishing, which has led to the introduc-
tion of many robust methods. However, since the worldwide outbreak of COVID-19, people have had
to regularly wear facial masks, thus making existing face recognition methods less reliable. Although
normal face recognition methods are nearly complete, masked face recognition (MFR)—which refers
to recognizing the identity of an individual when people wear a facial mask—remains the most
challenging topic in this area. To overcome the difficulties involved in MFR, a novel deep learning
method based on the convolutional block attention module (CBAM) and angular margin ArcFace
loss is proposed. In the method, CBAM is integrated with convolutional neural networks (CNNs) to
extract the input image feature maps, particularly of the region around the eyes. Meanwhile, ArcFace
is used as a training loss function to optimize the feature embedding and enhance the discriminative
feature for MFR. Because of the insufficient availability of masked face images for model training,
this study used the data augmentation method to generate masked face images from a common
face recognition dataset. The proposed method was evaluated using the well-known masked image
version of LFW, AgeDB-30, CFP-FP, and real mask image MFR2 verification datasets. A variety of
experiments confirmed that the proposed method offers improvements for MFR compared to the
current state-of-the-art methods.
Keywords:
facial recognition; convolutional neural network; deep learning; masked face recognition;
attention module
1. Introduction
Face recognition (FR) has represented one of the most important research topics for
many years. Many researchers [
1
–
6
] have introduced robust methods to solve the FR
problem. The trend of developing methods for FR appears to have almost reached its peak
at the time of this writing. Influenced by the convolutional neural networks (CNNs), the
current algorithms using deep learning methods [
1
–
6
] have achieved superior accuracy
for FR. Systems based on FR are widely used in many areas across the world including
airports, community gates, and healthcare; FR is also employed in some authentication
applications, such as face-to-face attendance monitoring and mobile payment systems
based on face profiles.
With the emergence of the COVID-19 pandemic, a viral infection caused by severe
acute respiratory syndrome [
7
] has spread globally and brought many major challenges
to daily human activities. To avoid COVID-19 infection, many people have worn and
continue to wear masks. Mask wearing affects current FR application systems because the
human face—the target of interest—is partially covered. In real-world FR applications,
face occlusion, particularly masked face occlusion, will significantly affect existing FR
performance and decrease re-identification accuracy [8].
Modern deep learning-based models are advanced enough to extract face features and
learn the important key features such as face edges, mouth, nose, and eyes [
9
]. However,
Appl. Sci. 2022, 12, 5590. https://doi.org/10.3390/app12115590 https://www.mdpi.com/journal/applsci
