Loading [MathJax]/extensions/MathMenu.js
Inpainting of Wide-Baseline Multiple Viewpoint Video | IEEE Journals & Magazine | IEEE Xplore
Subscribe
ADVANCED SEARCH
Journals & Magazines >IEEE Transactions on Visualiz... >Volume: 26 Issue: 7

Inpainting of Wide-Baseline Multiple Viewpoint Video

PDF
Andrew Gilbert; Matthew Trumble; Adrian Hilton; John Collomosse
All Authors

Abstract:

We describe a non-parametric algorithm for multiple-viewpoint video inpainting. Uniquely, our algorithm addresses the domain of wide baseline multiple-viewpoint video (MV...Show More

Metadata

Abstract:

We describe a non-parametric algorithm for multiple-viewpoint video inpainting. Uniquely, our algorithm addresses the domain of wide baseline multiple-viewpoint video (MVV) with no temporal look-ahead in near real time speed. A Dictionary of Patches (DoP) is built using multi-resolution texture patches reprojected from geometric proxies available in the alternate views. We dynamically update the DoP over time, and a Markov Random Field optimisation over depth and appearance is used to resolve and align a selection of multiple candidates for a given patch, this ensures the inpainting of large regions in a plausible manner conserving both spatial and temporal coherence. We demonstrate the removal of large objects (e.g., people) on challenging indoor and outdoor MVV exhibiting cluttered, dynamic backgrounds and moving cameras.
Published in: IEEE Transactions on Visualization and Computer Graphics ( Volume: 26, Issue: 7, 01 July 2020)
Page(s): 2417 - 2428
Date of Publication: 24 December 2018

ISSN Information:

PubMed ID: 30582545
DOI: 10.1109/TVCG.2018.2889297

Funding Agency:

Author image of Andrew Gilbert
Centre for Vision Speech and Signal Processing, University of Surrey, Guildford, United Kingdom
Andrew Gilbert is a senior lecturer with the University of Surrey. His current research interests include human pose estimation. inpainting, real-time visual tracking, human activity recognition, and Intelligent Surveillance. He is the author of more than 40 papers, journals, and book chapters. He is a member of the BMVA Executive committee and coordinates the national BMVA technical meetings.
Andrew Gilbert is a senior lecturer with the University of Surrey. His current research interests include human pose estimation. inpainting, real-time visual tracking, human activity recognition, and Intelligent Surveillance. He is the author of more than 40 papers, journals, and book chapters. He is a member of the BMVA Executive committee and coordinates the national BMVA technical meetings.View more
Author image of Matthew Trumble
Centre for Vision Speech and Signal Processing, University of Surrey, Guildford, United Kingdom
Mathew Trumble received the bachelor's degree in computer science from the University of Surrey, in 2014. He is working toward the PhD degree in CVSSP, University of Surrey. His research interests include deep learning, multi-view reconstruction, and human pose estimation.
Mathew Trumble received the bachelor's degree in computer science from the University of Surrey, in 2014. He is working toward the PhD degree in CVSSP, University of Surrey. His research interests include deep learning, multi-view reconstruction, and human pose estimation.View more
Author image of Adrian Hilton
Centre for Vision Speech and Signal Processing, University of Surrey, Guildford, United Kingdom
Adrian Hilton received the BSc (hons), DPhil, and CEng degrees. He is a professor and the director of CVSSP with the University of Surrey. His interest is in robust computer vision to model and understand real world scenes and his work in bridging-the-gap between real and computer generated imagery combines the fields of computer vision, graphics and animation. He leads several EU and UK/ industry projects and holds a 5-y...Show More
Adrian Hilton received the BSc (hons), DPhil, and CEng degrees. He is a professor and the director of CVSSP with the University of Surrey. His interest is in robust computer vision to model and understand real world scenes and his work in bridging-the-gap between real and computer generated imagery combines the fields of computer vision, graphics and animation. He leads several EU and UK/ industry projects and holds a 5-y...View more
Author image of John Collomosse
Centre for Vision Speech and Signal Processing (CVSSP), University of Surrey, Guildford, United Kingdom
Creative Intelligence Lab, Adobe Research, San Jose, USA
John Collomosse is a professor with CVSSP, University of Surrey and a visiting professor with Adobe Systems’ Creative Intelligence Lab. His research fuses computer vision, graphics and machine learning to tackle big data problems in visual media, with emphasis on post-production in the creative industries and intelligent algorithms for searching and rendering large visual media repositories. He is a member of the IEEE.
John Collomosse is a professor with CVSSP, University of Surrey and a visiting professor with Adobe Systems’ Creative Intelligence Lab. His research fuses computer vision, graphics and machine learning to tackle big data problems in visual media, with emphasis on post-production in the creative industries and intelligent algorithms for searching and rendering large visual media repositories. He is a member of the IEEE.View more
Contents

1 Introduction

Multiple viewpoint video is becoming commonplace in film and music video production, where shots are captured using multiple, wide-spaced, synchronised cameras for later editing. Frequently it is desirable to edit such footage to remove objects; e.g., to remove an actor in a motion-capture suit to be later replaced with an animated virtual character.

Author image of Andrew Gilbert
Centre for Vision Speech and Signal Processing, University of Surrey, Guildford, United Kingdom
Andrew Gilbert is a senior lecturer with the University of Surrey. His current research interests include human pose estimation. inpainting, real-time visual tracking, human activity recognition, and Intelligent Surveillance. He is the author of more than 40 papers, journals, and book chapters. He is a member of the BMVA Executive committee and coordinates the national BMVA technical meetings.
Andrew Gilbert is a senior lecturer with the University of Surrey. His current research interests include human pose estimation. inpainting, real-time visual tracking, human activity recognition, and Intelligent Surveillance. He is the author of more than 40 papers, journals, and book chapters. He is a member of the BMVA Executive committee and coordinates the national BMVA technical meetings.View more
Author image of Matthew Trumble
Centre for Vision Speech and Signal Processing, University of Surrey, Guildford, United Kingdom
Mathew Trumble received the bachelor's degree in computer science from the University of Surrey, in 2014. He is working toward the PhD degree in CVSSP, University of Surrey. His research interests include deep learning, multi-view reconstruction, and human pose estimation.
Mathew Trumble received the bachelor's degree in computer science from the University of Surrey, in 2014. He is working toward the PhD degree in CVSSP, University of Surrey. His research interests include deep learning, multi-view reconstruction, and human pose estimation.View more
Author image of Adrian Hilton
Centre for Vision Speech and Signal Processing, University of Surrey, Guildford, United Kingdom
Adrian Hilton received the BSc (hons), DPhil, and CEng degrees. He is a professor and the director of CVSSP with the University of Surrey. His interest is in robust computer vision to model and understand real world scenes and his work in bridging-the-gap between real and computer generated imagery combines the fields of computer vision, graphics and animation. He leads several EU and UK/ industry projects and holds a 5-year Royal Society Wolfson Research Merit Award. He is a member of the IEEE.
Adrian Hilton received the BSc (hons), DPhil, and CEng degrees. He is a professor and the director of CVSSP with the University of Surrey. His interest is in robust computer vision to model and understand real world scenes and his work in bridging-the-gap between real and computer generated imagery combines the fields of computer vision, graphics and animation. He leads several EU and UK/ industry projects and holds a 5-year Royal Society Wolfson Research Merit Award. He is a member of the IEEE.View more
Author image of John Collomosse
Centre for Vision Speech and Signal Processing (CVSSP), University of Surrey, Guildford, United Kingdom
Creative Intelligence Lab, Adobe Research, San Jose, USA
John Collomosse is a professor with CVSSP, University of Surrey and a visiting professor with Adobe Systems’ Creative Intelligence Lab. His research fuses computer vision, graphics and machine learning to tackle big data problems in visual media, with emphasis on post-production in the creative industries and intelligent algorithms for searching and rendering large visual media repositories. He is a member of the IEEE.
John Collomosse is a professor with CVSSP, University of Surrey and a visiting professor with Adobe Systems’ Creative Intelligence Lab. His research fuses computer vision, graphics and machine learning to tackle big data problems in visual media, with emphasis on post-production in the creative industries and intelligent algorithms for searching and rendering large visual media repositories. He is a member of the IEEE.View more

Contact IEEE to Subscribe
More Like This
Camera System for the Three-Dimensional Display Based on the Reconstruction of Parallax Rays

2006 World Automation Congress

Published: 2006

Many-Objective Deployment Optimization for a Drone-Assisted Camera Network

IEEE Transactions on Network Science and Engineering

Published: 2021

Show More

References

References is not available for this document.

IEEE Account

Purchase Details

Profile Information

Need Help?

A not-for-profit organization, IEEE is the world's largest technical professional organization dedicated to advancing technology for the benefit of humanity.
© Copyright 2025 IEEE - All rights reserved. Use of this web site signifies your agreement to the terms and conditions.