Given an input video sequence (top) and user-defined trimaps for the first and last frames, our method is able to efficiently interpolate a temporally coherent alpha matte (bottom) for the video.

A PatchMatch-based Approach for Matte Propagation in Videos

Marcos H. Backes mhbackes@inf.ufrgs.br

and

Manuel M. Oliveira oliveira@inf.ufrgs.br

Computer Graphics Forum.
Volume 38 (2019), Number 7, pp. 651-662. [DOI]

Despite considerable advances in natural image matting over the last decades, video matting still remains a difficult problem. The main challenges faced by existing methods are the large amount of user input required, and temporal inconsistencies in mattes between pairs of adjacent frames. We present a temporally-coherent matte-propagation method for videos based on PatchMatch and edge-aware filtering. Given an input video and trimaps for a few frames, including the first and last, our approach generates alpha mattes for all frames of the video sequence. We also present a user scribble-based interface for video matting that takes advantage of the efficiency of our method to interactively refine the matte results. We demonstrate the effectiveness of our approach by using it to generate temporally-coherent mattes for several natural video sequences. We perform quantitative comparisons against the state-of-the-art sparse-input video matting techniques and show that our method produces significantly better results according to three different metrics. We also perform qualitative comparisons against the state-of-the-art dense-input video matting techniques and show that our approach produces similar quality results while requiring only about 7% of the amount of user input required by such techniques. These results show that our method is both effective and user-friendly, outperforming state-of-the-art solutions.

Keywords

Video Matting Propagation; Alpha Matting; Video Compositing; PathMatch.

Paper

Pre-print Version (17.3 MB)

Eurographics Association Copyright Notice: Permission to make digital or hard copies of part or all of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than Eurographics must be honored.

This is the author's version of the work, and it is not for redistribution. The final publication is available via the Computer Graphics Forum Journal.

Code

Citation

Marcos H. Backes, Manuel M. Oliveira. A PatchMatch-based Approach for Matte Propagation in Videos, Computer Graphics Forum, 38(7), pp. 651-662, 2019.

BibTeX

@article{BackesOliveira_2019,
    author  = {Marcos H. Backes and Manuel M. Oliveira},
    title   = {A PatchMatch-based Approach for Matte Propagation in Videos},
    journal = {Computer Graphics Forum},
    volume  = {38},
    number  = {7},
    DOI     = {10.1111/cgf.13868},
    ISSN    = {1467-8659},
    pages   = {651--662},
    year    = {2019},
}
  

This work was sponsored by CNPq-Brazil (fellowships and grants 131965/2018-2, 312975/2018-0, 423673/2016-5).