Planar Tracker for AR

Marker-based augmented reality (AR) is extremely popular nowadays. However, full user immersion is possible only in the case of robust real-time computer vision solutions working on the mobile device.

We have developed a custom hybrid tracking system based on local feature tracking and template-based matching. The tracking engine tracks the homography changes using optical flow algorithm and then refines the residual warp using the optimized template matcher.

Planar tracking - principle of operation. Picture 1

Planar tracking - principle of operation. Picture 2

Fig.1. Hybrid tracker principle

(A – tracking steps, B – homography refinement)

Developed tracker is robust to extremal angles, drift-free and very accurate.

Tracker is robust to extremal angles Example of reconstructed camera motion

Fig. 2. Tracking example and reconstructed camera motion

Finally, here is an example of how the tracker works on the mobile device in real conditions:

Buttons with models

CUBE with videos

Algorithms work at 60 FPS on Android and iOS.

Planar Tracker for Augmented Reality