Using AI to create real-time depth maps for occlusions in AR

November 26, 2019


In order to make augmented reality (AR) more realistic, we need to be able to apply AR effects both in front and behind objects in a scene. Most AR systems, however, are based on SLAM, which only tracks a few sparse point features. This allows for overlays in front of objects, but not behind them. We developed a method that increases the density of the depth from the sparse point track locations to all other pixels, creating dense depth maps for every frame. This method helps AR effects fully interact with a scene geometry, such as enabling occlusions by real objects in the scene.

Watch the video:

Read the full paper:

Fast Depth Densification for Occlusion-aware Augmented Reality