(a) Frames from an input video. (b) Their motion and color-change magnified version (b). (c) and (d) show the time slices over his neck (small green line segment on (a) left) in the original and magnified videos, revealing the color changes and motion associated with the pulse in his coronary artery. The original video is from Wu et al. Eulerian video magnification for revealing subtle changes in the world. ACM Trans Graph 2012;31(4):1–8.

Simultaneous Magnification of Subtle Motions and Color Variations in Videos Using Riesz Pyramids

Thomas V. Fontanari tvfontanari@inf.ufrgs.br

and

Manuel M. Oliveira oliveira@inf.ufrgs.br

 

Computers & Graphics.
Volume 101 (2021) 35–45. [DOI]

Videos often contain subtle motions and color variations that cannot be easily observed. Examples include, for instance, head motion and changes in skin face color due to blood flow controlled by the heart pumping rhythm. A few techniques have been developed to magnify these subtle signals. However, previous techniques were targeted specifically towards magnification of either motion or color variations. Trying to magnify both aspects applying two of these techniques in sequence does not produce good results. We present a method for magnifying subtle motions and color variations in videos, which can be performed either separately or simultaneously. Our approach is based on the Riesz pyramid, which was previously used only for motion magnification. Besides modifying the local phases of the coefficients of this pyramid, we show how altering its local amplitudes and its residue can be used to magnify intensity/color variations. We demonstrate the effectiveness of our technique in multiple videos by magnifying both subtle motion and color variations simultaneously.

Keywords

Video Subtle Motion and Color Magnification; Eulerian Video Magnification; Riesz Pyramids.

Our simultaneous amplification of subtle motion and intensity/color variation pipeline. (a) The first stage decomposes each frame of the input video into a Riesz pyramid: phase, amplitude, and residue. (b) Per-pixel temporal filter applied to the phase, amplitude, and residue of the pyramid selects the frequencies bands to be magnified: φ_BM (for motion), and A_BC and I_R,B_C (for intensity/color). (c) The magnified video frames are recovered after scaling the selected frequency bands by factors α_M (motion) and α_C (intensity/color) and recombining them with the original Riesz pyramid elements.

Paper

Paper (pre-print, low resolution)

The final publication (high resolution, incorporating the feedback from the reviewing process) is available at ScienceDirect - Elsevier

Citation

Thomas V. Fontanari, Manuel M. Oliveira. Simultaneous Magnification of Subtle Motions and Color Variations in Videos Using Riesz Pyramids, Computers & Graphics, Volume 101, pp. 35-45, 2021.

BibTeX

@article{AraujoOliveira_2020a,
    author  = {Thomas V. Fontanari and Manuel M. Oliveira},
    title   = {Simultaneous Magnification of Subtle Motions and Color Variations in Videos Using Riesz Pyramids},
    journal = {Computers & Graphics},
    volume  = {101},
    DOI     = {10.1016/j.cag.2021.08.015},
    ISSN    = {0097-8493},
    pages   = {35--45},
    year    = {2021}
}
  

This work was sponsored by

	CNPq-Brazil fellowships and grants 312975/2018-0, 423673/2016-5.
	CAPES, Brazil Finance Code 001.