Lightweight Multimedia Authentication at the Edge Using Environmental Fingerprint

Abstract

With the fast development of Fifth-/Sixth-Generation (5G/6G) communications and the Internet of Video Things (IoVT), a broad range of mega-scale data applications have emerged. The rapid deployment of IoVT devices in modern smart cities has enabled secure infrastructures with minimal human intervention. Accompanied by the proliferation of multimedia is the increasing number of attacks against the IoVT systems. With increased onboard computational resources and technological advances in machine learning (ML) models, attack vectors and detection techniques have evolved to use ML-based techniques more effectively, resulting in non-equilibrium dynamics. Modern Artificial Intelligence (AI) technology is integrated with many multimedia applications to help enhance its applications. However, the development of General Adversarial Networks (GANs) also led to DeepFake attacks resulting in the misuse of AI. Instead of engaging in an endless AI arms race “fighting fire with fire”, where new Deep Learning (DL) algorithms keep making fake media more realistic and inadvertently causing the spread of misinformation, this dissertation proposes a novel authentication approach leveraging the Electrical Network Frequency (ENF) signals embedded in the multimedia as an environmental fingerprint against visual layer attacks.

An ENF operates at a nominal frequency of 60 Hz/50 Hz based on the geographical location and maintains similar fluctuations throughout the power grid. The multimedia recordings can be verified using signal consistency and similarity by leveraging the time-varying nature of the ENF signal collected from both the Audio-Video recordings and simultaneous ground truth ENF. By studying the spectral estimation techniques STFT and MUSIC along with robust frequency enhancement techniques like Weighted Harmonic Combination and Robust Filtering Algorithm (RFA), the optimal ENF estimation workflow for online forgery detection is analyzed and presented. To improve ENF-based authentication performance, frame processing techniques like Selective Superpixel Masking (SSM) to minimize noise from moving subjects and Singular Spectrum Analysis (SSA) to minimize false alarms in detection are introduced.

Equipped with reliable and robust algorithms, AI-generated media like DeepFake can be detected leveraging the ENF-based authentication system proposed as DeFakePro. Furthermore, the synchronous random fluctuations of ENF throughout the grid are integrated as part of a distributed consensus mechanism for IoVT devices, where the proposed LEFC system can detect faulty nodes with fake media broadcast in the network. Along with online detection capabilities, the DEMA system exploits the media metadata information to verify the source based on the time and location of recording using the reference ENF database from all interconnects. The experimental results show that the proposed systems designed using ENF as the environmental fingerprint can enable effective detection and authentication measures against visual layer attacks.