This chapter starts a new Extended Reality (XR) project. In the previous two chapters, you worked on creating a virtual reality project. In this chapter, we'll start work on an Augmented Reality (AR) game where you can tap and place virtual objects in the real world using either an Android or iOS device. We'll create the game using Unity's new Universal Render Pipeline (URP).
The project creation process is similar to past projects, with a small twist. We will be using a completely new rendering pipeline. One of the reasons we will use URP for this project is that as Unity moves to URP, you will want to know how to set up AR using URP. The flow of previous projects doesn't change much in URP, but there are notable differences with AR. This chapter can also be completed without using URP; just skip the Implementing AR in URP section.
With most projects, there's no getting away from project setup, and AR is no exception. We will need to install various plugins and packages, and these plugins can differ depending on your target device. Fortunately, once the device-specific setup is complete, we can write code that will work for Android and iOS using AR Foundation, which provides a cross-platform implementation for AR.
Unity provides several premade components that we will add to the scene to enable AR. By the end of this chapter, you will have created a basic AR project without having to write a single line of code.
In this chapter, we will cover the following topics:
This chapter assumes that you have not only completed the projects from the previous chapters but also have a good, basic knowledge of C# scripting generally, though not necessarily in Unity.
The completed project is included in the Chapter13/End folder.
This chapter uses an asset from the Unity Asset Store, which, along with the author details, can be found at https://assetstore.unity.com/packages/3d/characters/creatures/rpg-monster-duo-pbr-polyart-157762.
To run the project in this book, you require an iOS 11 or greater device, or an Android 7.0 device (or later). These are the minimum OS versions required to support AR. Due to the ever-evolving nature of AR, we recommend you follow along using Unity 2020.1, as this version has been confirmed to work with the project.
Before we proceed with the implementation details, we should briefly define a couple of terms that we touched on in the introduction, that is, AR (and AR Foundation) and the Universal Render Pipeline.
We won't go into too much detail about AR as you're most likely aware of it, even if you haven't used it yourself. Using AR, we can superimpose data (in our case, it will be GameObjects) onto the world around us.
Tip
For a sneak peek of what our AR project will look like by the end of the chapter, see Figure 13.34.
Occasionally AR can be confused with virtual reality, and they both fall under the umbrella of Extended Reality. However, where virtual reality creates an entirely new reality for the player to be immersed in, AR is used to extend, or take advantage of, the real world inhabited by the player.
AR has received a lot of attention in the tech world in the last few years (Pokémon GO, Apple's WWDC, Microsoft's conference, and so on). Unity has taken notice, and in 2019 released AR Foundation – an easy way to create multi-platform AR apps within Unity and the first time AR has been officially supported. AR Foundation is a set of classes and APIs created by Unity that enables us to develop cross-platform AR games. It provides the ability to track a device's position and orientation, detect horizontal and vertical planes, and track a face or image, among other things.
Tip
For more information on the features provided by AR Foundation, see https://docs.unity3d.com/Packages/[email protected]/manual/index.html.
The URP is a complete refresh of Unity's built-in render pipeline. It contains several tools to help you easily create optimized graphics on a wide range of mobile, desktop, and game consoles. URP provides control over how Unity renders a frame by writing C# code.
Tip
For more information on the URP, see https://docs.unity3d.com/Packages/[email protected]/manual/index.html.
We'll take advantage of some of what the URP has to offer in the next chapter, when we look at post-processing. In this chapter, you will learn how to create a URP project and configure settings specific to AR. On that note, it's time to get started developing in AR by creating a URP project.
In this section, we will create a new URP project and perform the necessary steps to prepare our project for AR. These steps include the following:
That's a lot of ground to cover, even before we start adding our own custom functionality. However, once you have an understanding of the process, you'll be up and running in your projects in no time.
Let's jump in by creating the project.
As touched on previously, we'll be using a different template for the AR project. As the URP has been described in detail in Introducing the project, we can jump right into the implementation:
As with all previously created projects, this will create the project and open the Unity Editor. There is one noticeable difference, however. With newly created 2D or 3D projects, you're typically greeted by an empty scene (save a camera and lighting); however, with the URP template, you'll be greeted with a pretty sample scene, showing off some of the features of the URP. A sample scene is shown in Figure 13.3:
Feel free to play around with the sample scene before moving onto the next step: installing the libraries required for AR.
With the project created, we need to make a few changes to get it ready for AR. One of the necessary precursors to developing in Unity AR is installing the required plugins. Luckily, Unity provides a graphical interface to do just that:
And that's all the plugins we require (although we will install additional packages shortly).
Tip
If you don't see either the iOS or Android tab, make sure you have the necessary modules installed. You can check installed modules in Unity Hub on the Installs tab.
With the plugins installed, we can move on to configuring the necessary device-specific settings. Both iOS and Android require specific settings to be adjusted so that AR will run on their platforms. We'll go through each OS separately, so whichever device you have, you'll soon be able to test the AR game.
There are a couple of tweaks to the settings that need to be made to ensure that our AR game runs smoothly. The settings are device-specific; therefore, you only need to follow along with the section specific to your OS of choice. We'll start with the iOS-specific settings before moving on to Android.
There are only a couple of small changes required for iOS:
Important note
As well as having iOS 11 as the minimum OS version, ARKit also only supports ARM64 processors. Although, it is likely that Apple will release an Apple Silicon version in the near future.
To configure the player settings for Android, do the following:
That's it for the project settings, and there's only one more step before our project setup is complete: we need to install the AR Foundation package.
The AR Foundation package allows us to develop multi-platform AR games. It hides the complexity of writing OS-specific code, and this project will be able to be run on iOS or Android. Sounds good to me! Let's install it now:
This will install all of the necessary components to develop multi-platform AR games.
Tip
You can check whether the import was successful by right-clicking in the Scene Hierarchy and checking whether you have an XR option.
We'll be using one other package in our project. Although, it's not directly related to AR; instead, it provides a monster asset we can place in the world:
We now have everything we need to start developing our AR project. While the setup may seem involved, it is actually a quick way to get up and running with AR, especially considering it will work for iOS and Android. In the next section, we'll add the objects required by an AR scene.
In this section, we'll add an AR Session and AR Session Origin object to a new scene. The AR Session component is necessary for any AR scene as it enables the AR experience, so let's start with that:
An AR Session is an integral part of an AR scene. It controls the life cycle of the AR experience by enabling it on startup and disabling AR if the component is disabled. There is only one session per game, so even if you have multiple AR Session objects in a scene, they will talk to the same session. Ideally, you should not just have one AR Session in a scene to prevent conflicts.
Tip
If the Attempt Update option on the AR Session object is checked, the game will try to install the required AR software on the device. This option is enabled by default.
Next, we'll add the AR Session Origin object. This object gives us a way to query the physical world around us and change the scale of the virtual world.
The AR Session Origin object will transform features, such as the surfaces onto which we'll be spawning the turtle, from session space into Unity space with the correct position, orientation, and scale. The session space is relative to the device's coordinate system, whereas Unity space is what we've been working in up to this point. Once the positional data has been transformed into Unity space, we can easily compare and place objects without performing any manual conversion from device space to Unity space.
Tip
You can scale AR objects by adjusting the scale of the AR Session Origin object. Larger values will make AR content appear smaller, and vice versa. For example, by setting the scale to 0.5, the turtle will appear twice its current size.
With those two objects created, our scene is ready for AR. Simple!
Tip
If you find the icons for AR objects in the Scene view distracting, you can turn them off by opening the Gizmos menu at the top of the Scene view.
In the next section, we'll create the Prefab that will be spawned in the real world (well, the AR world anyway). We'll take advantage of the Monster package we imported earlier and perform a few tweaks to update it for URP.
In our AR game, we'll spawn an object in the world whenever we tap the screen. This section will create the object to spawn a friendly (albeit spiky) turtle:
You'll quickly notice that it doesn't look quite right on account of it being completely pink. This shade of pink may be familiar to you if you've ever used a broken shader (or broken one yourself – I know I have!) or have forgotten to apply a material to a mesh. In this case, the issue is down to the material not being compliant with the URP. Luckily, we can fix this easily:
This will update the material, so it is compliant with the URP.
Tip
You can upgrade all materials in the project at the same time by selecting Edit | Render Pipeline | Universal Render Pipeline | Update Project Materials to UniversalRP Materials from the Application menu.
Now it looks correct, we can create a Prefab of the turtle:
Tip
If objects appear too large or small in your AR game, rather than scaling each object individually, you can scale the AR world by scaling the AR Origins object.
With the turtle sitting happily in the scene and a Prefab created so we can spawn it when required, we can move on to testing our current project.
Before we go any further, I think now is an excellent time to test whether we can display the turtle in AR using a real device. To do that, we will need to build and deploy the project. As you would imagine, the exact steps differ based on your chosen platform; however, before we get into platform-specific settings, we need to change a few platform-agnostic settings:
It's at this point the settings branch depending on your platform. We'll first look at testing on Android before moving on to iOS.
To test on an Android device, do the following:
Unity will perform a number of steps, including re-importing assets, and then your project will be ready to test on your device:
Next, we'll look at testing on an iOS device.
Testing on an iOS device is slightly more involved than its Android counterpart, but will become second nature fairly quickly:
This will open the project in Xcode. There are a few more steps required before you can run the game on your device. In brief, you will need to do the following:
Luckily, Xcode can take care of most of this for you by enabling the Automatically managing signing option.
Detailed steps for configuring the Xcode project are beyond the scope of this book, however, Unity provides step-by-step instructions here: https://learn.unity.com/tutorial/building-for-mobile#.
Important note
To run the project on an iOS device, you need Xcode. Xcode is an IDE used to develop Mac and iOS apps. If you don't have Xcode installed, you can find it for free on the Mac App Store.
Whether you're using Android or iOS, once you have successfully deployed the app to a device, you may be disappointed with what you see. Running the game at this stage may greet you with a black screen, and nothing more. Lucky we tested it early! This is to do with the URP and is easy enough to fix, as you'll see in the next section. You may have been lucky, and the AR project works as expected, however, I would still recommend following the next section as it has useful information on how to implement AR using the URP.
In the last section, we compiled and deployed the project to a device with disappointing results. When the game ran, we were greeted with a black screen. This is a common issue when using the URP and AR, and to resolve it, we first need to create a new rendering asset:
That will create two files for us: ARRenderer and ARRenderer_Renderer. By clicking on each asset in turn, we can view the configurable options in the Inspector:
Many of the options are self-explanatory and it's beyond the scope of this book to explain them in depth. For more information on these settings, see https://docs.unity3d.com/Packages/[email protected]/manual/universalrp-asset.html.
Next, let's take a look at the ARRenderer_Renderer file:
Near the bottom of the settings, you'll notice the heading Renderer Features. It's here that we'll add a specific feature to correctly display the AR background:
This feature enables displaying the background in a URP AR application for any project using these settings.
Important note
If you don't see an option to add AR Background Renderer Feature, then please update your AR Foundation package using the Package Manager. It is only available from the AR Foundation package version 3.0.1 or above.
Now we need to configure the project to use our new custom renderer settings:
It's here you can edit render settings for specific platforms.
With that done, you should be able to successfully deploy your game to your device.
Important note
You can set different render settings on a per camera basis by selecting a camera and configuring the Renderer option under the Rendering heading.
If you build and deploy the project to your device, it should ask for your permission to use the camera and you should see the following:
The turtle will most likely be floating in space as its initial position is determined by where your phone was in space when the game started. You may also notice that the turtle flies around the room as you move your phone. This is not ideal and something that we'll fix in the next chapter, when we look at plane detection.
Congratulations! You've created a cross-platform AR project without writing a single line of code. In this chapter, you've learned how to quickly create a new project, add the necessary packages, and create a new AR scene. By performing these steps, you now know how to get up and running in AR efficiently. You know which packages to install for which device, and you know how to create your first AR scene. On top of that, you've created a URP project, and updated the AR project to take advantage of this new render pipeline.
In the next chapter, we'll extend these concepts further as we look at detecting planes in the real world so we can place the turtle on flat surfaces (such as the floor or a table), and we'll also take advantage of the URP by adding postprocessing to our scene.
Q1. To develop AR games in Unity, you should install:
A. ARCore
B. ARKit
C. AR Foundation
D. All of the above
Q2. The minimum iOS version that supports ARKit is:
B. 13
C. 9
D. 10
Q3. The AR Session component:
A. Has an AR camera
B. Controls the life cycle of an AR session
C. Isn't required in an AR scene
D. Should be added to every AR object in a scene
Q4. URP stands for:
A. Unity Render Pipeline
B. Unity's Really Pretty
C. Universal Reality Pipeline
D. Universal Render Pipeline
Q5. To run AR on devices using the URP, you should add an:
A. AR Session
B. AR Session Origin
C. AR Background Renderer Feature
D. All of the above
For more information on AR in Unity, see the following links: