Peace of mind on match day: facial recognition solution at the football stadium

Due to the aggressive behavior of just a limited number of heated supporters, regular fans and those with young family members are often scared away from going to the stadium to watch matches. In the summer of 2019, Panasonic introduced a facial recognition system at a football stadium in Denmark that was dealing with this problem. By entrusting the high-precision system to identify and stop known troublemakers from entering, a process which was conventionally done manually by security staff at the entrance gates, the stadium has become a safer place where anyone can watch matches with peace of mind.

Brøndby IF is a professional football club in the Danish Superliga. At Brøndby IF’s stadium, Panasonic has installed “FacePRO,” a deep-learning facial recognition system.

This system is based on Panasonic’s over 60 years of security system technology and 30 years of Panasonic developing image sensing technology, and is comprised of security cameras equipped with the “Best-Shot” function and the high-precision deep-learning FacePRO facial recognition software.

Individuals who have been caught breaking the stadium rules receive a ban from coming back to games, and are registered on a blacklist. Brøndby IF has an average home game attendance of roughly 14,000 people, and there can be up to approximately 100 people registered on the blacklist at any given time. In the past, the security staff at the entrance gates prevented them from entering the stadium through visual checks against the list, but this manual approach had its limits, and it was difficult to stop every person, who at times attempted to enter the stadium wearing disguises such as sunglasses or by changing their appearance.

Faced with this issue, Panasonic spent approximately two years working with the club to develop a facial recognition system that could meet the stadium’s needs. When the system matches the face at the gate to someone who has been registered on the blacklist, a real time notification is seen by staff in the stadium surveillance room, and once they have double checked the accuracy, a message is sent to the security staff at the entrance to stop the individual before they can proceed through the gates. Through this, a strong security procedure has been established with a multi-check system comprised of the facial recognition system and actual people.

With this system, Panasonic’s original technology has been adopted in the camera and in the software that analyzes the facial images that are transmitted by the camera.

On the camera side, the “Best-Shot” function is utilized to automatically select the best facial images for analysis, even in situations with insufficient light. Only the best images suitable for facial recognition are selected and sent to the server from the multiple facial images obtained of the same person. This enables a large number of faces to be processed without overloading the network, and contributes to cost reduction of the entire system.

On the software side, the world’s highest-level^*1 of facial recognition performance has been achieved with the facial image analysis. The software enables extremely precise checks even outdoors where the lighting tends to be extreme, or with angled views of the face, such as when the facial direction is 45 degrees to the right or left or 30 degrees up and down, as well as in situations where the face is difficult to distinguish with the human eye due to changes in the face over time (equivalent to 10 years), or even when the face is partially concealed by sunglasses or a mask.

Computer screen reading 'Deleting photo data'

Another issue that had to be addressed at a stadium where many visitors gather was the new GDPR*² regulations that were put into place to protect personal data within the EU. After repeated in-depth meetings with Brøndby IF, the risk of information leakage was prevented by developing a mechanism that both encrypted the data from the camera and that did not store images and data of people not registered on the blacklist after verifying a non-match, as well as by placing the list of registered banned individuals to a server blocked from the Internet and all other external systems.

With the introduction of this system, Brøndby IF’s family-friendly stadium has become an even safer place for fans of all ages to enjoy matches with peace of mind. Also, the reduced burden on the security staff has allowed them to divert resources to providing improved fan support services. Panasonic continues to work on developing new applications, including using facial recognition solutions for the monitoring of important public facilities, building entrance management, and passport control at airports, contributing to a safer and more secure society.

*1 Evaluation points from the official evaluation report of the National Institute of Standards and Technology (NIST) as of May 9, 2017: https://www.nist.gov/programs-projects/face-challenges (The IJB-A Face Verification Challenge Performance Report, The IJB-A Face Identification Challenge Performance Report). This technology has been published as a paper. https://arxiv.org/abs/1704.00438

*2 GDPR (General Data Protection Regulation): A regulation adopted by the European Parliament, European Council and European Commission intended to enhance and integrate data protection and privacy for all individual citizens of the European Union. Extensive and strict provisions are set in place regarding the processing and transfer of personal information.