High Quality Illumination of Virtual Objects Based on an Environment Estimation in Mixed Reality App

Visualizations of virtual objects in the real environment is often done by a simplified representation with simple surfaces and without reference to the surrounding environment. The seamless fusion of the virtual and real environment is, however, an essential factor in many areas, which is of particular importance when calculating lighting in mixed realities on mobile devices. Current approaches focus on approximations, which allow the calculation of diffuse lighting, whereby the rendering of glossy reflection properties is often neglected. The aim of this book is to enable the visualization of mirror-like reflective surfaces in mixed reality. In order to achieve this goal, various approaches are explored enabling high-quality visualization of virtual objects in realtime with a focus on the use of common hardware such as cameras, sensors in mobile devices, and partially depth sensors. Complete ambient lighting can be estimated, which enables detailed reflections. The results providea novel way to embed complex and simple geometric shapes with glossy surfaces in the real world which offers a higher level of detail in the reflections without using additional hardware.