Unity for Oculus Rift and Hololens: Unity & AR

Setting Up Vuforia

Vuforia Engine is a software platform for creating Augmented Reality apps. Developers can easily add advanced computer vision functionality to any application, allowing it to recognize images and objects, and interact with spaces in the real world.

This guide was adapted from here. You might wish to read the entire article to familiarize yourself with Vuforia, but follow the steps below.

1. In Unity, go to Edit > Project Settings > Player. Find the XR Settings section and make sure "Vuforia Augmented Reality" is checked on.
2. Go to Window > Vuforia Configuration.
   1. For HoloLens only: Under the Digital Eyewear section, set "Device Type" to "Digital Eyewear" and set "Device Config" to "HoloLens."
   2. For Android/iOS only: Under the Digital Eyewear section, set "Device Type" to "Handheld."
3. While still in Vuforia Configuration, find the Device Tracker section. Check "Track Device Pose" and make sure "Tracking mode" is set to "POSITIONAL." This turns on Extended Tracking (see the guide linked above for more information).
4. For HoloLens only: Go back to Edit > Project Settings > Player. While in the Windows icon tab, find the Resolution and Presentation section. Set "Default Orientation" to "Landscape Left."
5. While still in Player Settings, find the Publishing Settings section. Under Capabilities, make sure “InternetClient,” “WebCam,” “Microphone,” and “SpatialPerception” are turned on.
6. Within the Scene Hierarchy, right click and select Vuforia > ARCamera to add the specific Vuforia Camera to the scene. You can right click and delete the Main Camera originally there.
7. Create a Vuforia account using this.
8. Follow the “How to Create a License Key” workflow here.
9. Then follow the “How to Add a License Key to a Unity app” workflow here.

Adding a Vuforia Image Target

This will allow our Unity app to recognize a 2D image and place a 3D hologram over it. See this for an example.

1. From the same page where you found your License Key on the Vuforia site, go to Target Manager.
2. Select “Add database” and your give your database a name, Select “Device” under Type.
3. Once your database is created, click on “Add target” and select “Single Image.” Find your image using Browse and give it a name.
   1. Here is a guide to creating good image targets. If you’re stuck, try searching for “Vuforia image target” in Google Images and selecting one of those.
4. There are differing opinions on this but set Width to 1.
5. Your Target Manager page should now look something like this: 
   1. You want your image to have a 5-star rating; this is indicative of how well Vuforia will be able to track it.
6. Select the target you would like to download (the image you just uploaded), and click on “Download Database”. You want the Unity development platform. This will download a file we will need to import into Unity.
7. To import the assets you just uploaded to Vuforia, go to Assets > Import package > Custom package… and select the database file we just downloaded.
8. Import all.
9. If you go to Window > Vuforia Configuration, you should see your database under Databases.
10. Add an image target to your scene by right-clicking within the scene hierarchy and selecting Vuforia > Image.
11. View it in the Inspector. Under Image Target Behaviour, select “Predefined” and then select your database and image you want as your image target.
12. Make sure your image target stays at the origin even if it looks weird in the context of your Unity scene. Resist editing the transform component.
13. Add to your scene hierarchy whatever object you want to appear in the HoloLens when the user looks at the image target. Make this GameObject a child of your image target by dragging it onto “ImageTarget” in the scene hierarchy. (If you would like to use a model you created, follow the steps under the tab "Oculus Rift & Unity" > "Importing your Model" > "Importing a Model with Texture" of this guide, then select that model from your "Assets" > "Models" folder in the Project window and drag and drop it into the Hierarchy Window, on top of the word "ImageTarget" - this should nest it under "ImageTarget").
14. Change the position, rotation, and scale of your newly added object so it has the correct placement/size relative to what your image target will look like in real life (for instructions on how to do this, check out "Oculus Rift & Unity" > "Importing your Model" > "Moving/Rotating/Scaling an Object" on this guide).
15. Build and deploy your app. If you look at a picture of the image target you made, your object should appear.

Adding a Vuforia Model Target

This will allow our Unity app to recognize a real 3D object and place a 3D hologram over it. See this for an example.

1. Download the Model Target Generator tool here.
   1. Make sure the version you download matches the version of Vuforia you are using in your project! You can check this by opening your Unity project and going to Window > Vuforia Configuration, and looking at "Vuforia Version." If the versions do not match, you might need to update the version of Vuforia you are using in your project.
2. Follow this guide to use the Model Target Generator tool to make a model target dataset. You should end up with a .unitypackage file.
3. We now want to import this package into our Unity project. In Unity, go to the Assets menu > Import Package > Custom Package... and select the .unitypackage file created in the previous step.
4. For HoloLens only: The HoloLens camera is a landscape-oriented display. Thus, it is recommended to specify "Landscape" when creating a guide view in the Model Target Generator.
5. Follow this guide to set up your model target in Unity.
   1. For HoloLens only: Click on the ModelTarget in your scene, and in the Inspector, change “Guide View Mode” to “Guide View 3D.”
   2. For Android/iOS only: Click on the ModelTarget in your scene, and in the Inspector, change “Guide View Mode” to “Guide View 2D.”
6. Now when you deploy your project, you should be able to look at an object in the real world and see a hologram placed over it.

Creating a Gaze Cursor

1. Download “HoloToolkit” from here and place it into your Unity project under Assets. You might have to do this using the file system, outside of Unity.
2. Go to Assets > HoloToolkit > Input > Prefabs. Find and drag InputManager into the scene hierarchy.
3. Then go to Assets > HoloToolkit > Input > Prefabs > Cursor. Find and drag Cursor into the scene hierarchy.
4. Now when you run the app, you should see a cursor in front of you. It will be a transparent circle when it’s not on anything, but will turn into an opaque donut shape when it is on top of something (as long as that object has a collider).

Adding Airtap Functionality to an Object (non-UI)

1. Because of the InputManager we added to our scene in the previous section, we are already detecting gestures like air taps. But nothing happens when we tap!
2. If you want an object (non-UI) to do something when it is tapped, add a script to it. Add the following code to the script (you can change the name of the script to whatever makes sense):

using HoloToolkit.Unity.InputModule;
using UnityEngine;

public class ToggleableView : MonoBehaviour, IInputClickHandler, IInputHandler
{
   public void OnInputClicked(InputClickedEventData eventData)
   { }

   public void OnInputDown(InputEventData eventData)
   { }

   public void OnInputUp(InputEventData eventData)
   { }
}

3. Place the code that you want to run when the object is tapped within OnInputClicked.
4. If you want your object to be “tappable,” make sure to add a collider(s) to it (if it doesn’t already have them)!

Toggle the View of Two Objects by Tapping

This will let us toggle between the view of two objects by tapping on the objects.

1. Create an empty game object called “ToggleableView” and add the script from the previous section to it using “Add Component.”
2. In the script, add two variables right before the functions:

public GameObject model1;
public GameObject model2;

3. Within OnInputClicked, add the following:

// toggle object views
model1.SetActive(!model1.activeSelf);
model2.SetActive(!model2.activeSelf);

4. Set up your hierarchy to look like this:

5. PanCoverAntefixModel and PanCoverModel are the two objects we want to toggle between. For the object you want to NOT be the starting view, select it, view it in the Inspector, and turn off the checkmark next to its name. This sets it to inactive at the start.
6. Click on ToggleableView and view it in the Inspector. Drag the two models to the spaces for “Model1” and “Model2” so it looks like this:

7. Now when you run the app and click on your starting model, it should toggle the view between the two models.