Color-Aware Deep Temporal Backdrop Duplex Matting System

verfasst von

Hendrik Hachmann, Bodo Rosenhahn

Abstract

Deep learning-based alpha matting showed tremendous improvements in recent years, yet, feature film production studios still rely on classical chroma keying including costly post-production steps. This perceived discrepancy can be explained by some missing links necessary for production which are currently not adequately addressed in the alpha matting community, in particular foreground color estimation or color spill compensation. We propose a neural network-based temporal multi-backdrop production system that combines beneficial features from chroma keying and alpha matting. Given two consecutive frames with different background colors, our one-encoder-dual-decoder network predicts foreground colors and alpha values using a patch-based overlap-blend approach. The system is able to handle imprecise backdrops, dynamic cameras, and dynamic foregrounds and has no restrictions on foreground colors. We compare our method to state-of-The-Art algorithms using benchmark datasets and a video sequence captured by a demonstrator setup. We verify that a dual backdrop input is superior to the usually applied trimap-based approach. In addition, the proposed studio set is actor friendly, and produces high-quality, temporal consistent alpha and color estimations that include a superior color spill compensation.

Organisationseinheit(en)

Institut für Informationsverarbeitung
Forschungszentrum L3S

Typ

Aufsatz in Konferenzband

Seiten

205-216

Anzahl der Seiten

12

Publikationsdatum

08.06.2023

Publikationsstatus

Veröffentlicht

Peer-reviewed

Ja

ASJC Scopus Sachgebiete

Computergrafik und computergestütztes Design, Mensch-Maschine-Interaktion, Software

Ziele für nachhaltige Entwicklung

SDG 12 – Verantwortungsvoller Konsum und Produktion

Elektronische Version(en)

https://doi.org/10.48550/arXiv.2306.02954 (Zugang: Offen)
https://doi.org/10.1145/3587819.3590973 (Zugang: Offen)