Fast Video Deduplication and Localization With Temporal Consistence Re-Ranking

The use of social media networks and mobile devices has experienced tremendous growth in recent years. This has led to a surge in the number of videos recorded and uploaded to social media platforms like TikTok and YouTube. However, this increase has also resulted in the rise of illegal duplicate videos, which are essentially the same as the original videos but with minor editing effects and variations in coding. In addition, the large number of duplicate videos is a major storage and communication efficiency issue. The task of finding duplicate videos from a large repository is referred to as video deduplication. Video deduplication is a crucial task for applications like saving storage space and detecting copyright infringement. This work proposes a fast and robust location-aware video deduplication system capable of retrieving duplicate videos from a large repository extremely quickly. In addition, the proposed system has the ability to find the precise location of the query video in the retrieved videos. To identify and localize short video clips against large video repositories, we utilize robust image-level features from keypoint aggregation and deep learning along with an efficient KNN search of query frames with a multiple k-d tree setup, giving us a set of candidate video clips. Then, a fast temporal consistence pruning algorithm re-ranks the clip-level candidates and identifies the matching clip along with its temporal location in a sequence in an efficient way. The system was tested on 1 million frame/145 hour and 4.5 million frame/636 hour repositories generated via the large-scale FIVR-200K and VCSL datasets, respectively. The proposed system achieves a recall of 98.8% and 94.1% for the FIVR-200K and VCSL datasets, respectively. A query frame is searched as fast as 83.96ms and 462.59ms from a 1 million frame/ 145 hour and a 4.5 million frame/636 hour repository, respectively. These experimental results demonstrate that our system is highly accurate and that the time consumption is extremely low for retrieving video along with its timestamp information from large-scale repositories.

I. Introduction

In recent years, the easy accessibility to mobile devices and increasing usage of social media platforms like YouTube,1 Facebook,2 and TikTok3 have led to explosive growth in the amount of video content shared online. For instance, as of May 2019, the rate of video content uploaded to YouTube surpassed 500 hours per minute.4 As a result, an increase in the amount of illegal pirated video content shared has been witnessed. These illegal pirate videos essentially contain content identical to the original videos. However, these videos are edited by adding slight variations to evade detection by copy detection systems. These variations are usually added by transformations such as modifying the aspect ratio, changing color, changing frame rate, padding, overlaying text, flipping, etc. Such videos are known as duplicates or near-duplicates. In addition, storing this huge amount of video content is a challenging issue that further worsens the problem.¹

https://www.youtube.com

²

https://www.facebook.com

³

https://www.tiktok.com

⁴

https://en.wikipedia.org/wiki/YouTube