For this chapter, we started with the 2D instruction manual project developed in Chapter 6, How to Change a Flat Tire, and extended it to include augmented reality graphics as a new media type. After preparing the project to use the Vuforia SDK, we first implemented a screen button to toggle AR Mode. The AR Model hides the 2D content panel and allows the camera video feed to show.

We set up the project to capture User Defined Targets. We added an AR Prompt panel with a cursor, circle registration guide, and a Capture button. When the button is pressed, the app uses the current view as the AR target and displays the AR content on it. We used Vuforia's Trackable Events to know when to hide AR Prompt or to show a red circle as an error prompt if the capture target is of poor quality.

With the AR mode and user-defined targets, we then integrated 3D annotations into the instruction manual. The graphics are prefabs we imported into the project, and one may be instantiated when the user navigates to a step in the instruction manual. We also reviewed how these prefabs can be created, edited, and exported. Lastly, we also brought a copy of the 2D instructions (text, images, and video) into the AR view so that the user does not need to toggle back and forth between 2D and AR modes.

We also showed another way to implement the AR mode on iOS mobile devices using Apple ARKit. Instead of registering the AR graphics in the real world using image targets, we enabled the ARKit's spatial anchors to determine a real-world 3D location. A similar approach can be used on Android using Google ARCore (see our GitHub repository for details).

The chapter also showed how to implement the project for the Microsoft HoloLens wearable AR device. In this version, everything is in world-space coordinates. We created a single hologram that includes the 2D instruction content canvas, the navigation buttons, and the 3D AR graphics. We added an input module so that the select gesture can control UI buttons and scroll bar. Then, we added Spatial Mapping for tap to place the hologram in real space.

In the next chapter, we will explore the spatial mapping in even more detail. In that project, we'll virtually decorate your room with framed photos and take a new approach. The chapters starts with a HoloLens implementation and then shows a version for mobile AR.