Marker-based augmented reality (AR) is extremely popular nowadays. However, full user immersion is possible only in the case of robust real-time computer vision solutions working on the mobile device.
We have developed a custom hybrid tracking system based on local feature tracking and template-based matching. The tracking engine tracks the homography changes using optical flow algorithm and then refines the residual warp using the optimized template matcher.
Fig.1. Hybrid tracker principle
(A – tracking steps, B – homography refinement)
Developed tracker is robust to extremal angles, drift-free and very accurate.
Fig. 2. Tracking example and reconstructed camera motion
Finally, here is an example of how the tracker works on the mobile device in real conditions:
Buttons with models
CUBE with videos
Algorithms work at 60 FPS on Android and iOS.