Utilizing Uncertainty in 2D Pose Detectors for Probabilistic 3D Human Mesh Recovery

verfasst von

Tom Wehrbein, Marco Rudolph, Bodo Rosenhahn, Bastian Wandt

Abstract

Monocular 3D human pose and shape estimation is an inherently ill-posed problem due to depth ambiguities, occlusions, and truncations. Recent probabilistic approaches learn a distribution over plausible 3D human meshes by maximizing the likelihood of the ground-truth pose given an image. We show that this objective function alone is not sufficient to best capture the full distributions. Instead, we propose to additionally supervise the learned distributions by minimizing the distance to distributions encoded in heatmaps of a 2D pose detector. Moreover, we reveal that current methods often generate incorrect hypotheses for invisible joints which is not detected by the evaluation protocols. We demonstrate that person segmentation masks can be utilized during training to significantly decrease the number of invalid samples and introduce two metrics to evaluate it. Our normalizing flow-based approach predicts plausible 3D human mesh hypotheses that are consistent with the image evidence while maintaining high diversity for ambiguous body parts. Experiments on 3DPW and EMDB show that we outperform other state-of-the-art probabilistic methods. Code is available for research purposes at github.com/twehrbein/humr.

Organisationseinheit(en)

Institut für Informationsverarbeitung

Externe Organisation(en)

Linkoping University

Typ

Aufsatz in Konferenzband

Seiten

5852-5862

Anzahl der Seiten

Publikationsdatum

26.02.2025

Publikationsstatus

Veröffentlicht

Peer-reviewed

ASJC Scopus Sachgebiete

Artificial intelligence, Angewandte Informatik, Maschinelles Sehen und Mustererkennung, Mensch-Maschine-Interaktion, Modellierung und Simulation, Radiologie, Nuklearmedizin und Bildgebung

Ziele für nachhaltige Entwicklung

SDG 3 – Gute Gesundheit und Wohlergehen

Elektronische Version(en)

https://doi.org/10.1109/WACV61041.2025.00571 (Zugang: Geschlossen)
https://doi.org/10.48550/arXiv.2411.16289 (Zugang: Offen)