Generative RGB-D face completion for head-mounted display removal
conference paper
Head-mounted displays (HMDs) are an essential display device for the observation of virtual reality (VR) environments. However, HMDs obstruct external capturing methods from recording the user’s upper face. This severely impacts social VR applications, such as teleconferencing, which commonly rely on external RGB-D sensors to capture a volumetric representation of the user. In this paper, we introduce an HMD removal framework based on generative adversarial networks (GANs), capable of jointly filling in missing color and depth data in RGB-D face images. Our framework includes an RGB based identity loss function for identity preservation and several components aimed at surface reproduction. Our results demonstrate that our framework is able to remove HMDs from synthetic RGB-D face images while preserving the subject’s identity
Topics
Artificial intelligenceComputer visionComputing methodologiesHuman computer interaction (HCI)Interaction paradigmsReconstructionHuman-centered computingVirtual realityAbstractingCell proliferationStreet traffic controlUser interfacesVirtual realityAdversarial networksFace imagesHead mounted displaysLoss functionsRgb-d sensorsSurface reproductionVolumetric representationVR applicationsHelmet mounted displays
TNO Identifier
967972
ISBN
9780738113678
Publisher
Institute of Electrical and Electronics Engineers Inc.
Article nr.
9419343
Source title
Proceedings - 2021 IEEE Conference on Virtual Reality and 3D User Interfaces Abstracts and Workshops, VRW 2021, 2021 IEEE Conference on Virtual Reality and 3D User Interfaces Abstracts and Workshops, VRW 2021, 27 March 2021 through 3 April 2021
Pages
109-116
Files
To receive the publication files, please send an e-mail request to TNO Repository.