Skip to Main Content
Go to Penn Libraries homepage   Go to Guides homepage

Unity for Oculus Rift and Hololens: Unity & AR

This guide provides basic information about using Unity to bring your 3D models into the Oculus Rift and Hololens

Unity & AR

System Requirements

  1. For HoloLens:
    1. ‚ÄčA computer running at least Windows 8, preferably Windows 10. You need Windows 10 if you plan on using the HoloLens emulator.
  2. For Android:
    1. An Android device
  3. For iOS: 
    1. An apple device (iPhone, iPad, etc.) running iOS 9 or later
    2. A Mac running OS X 10.10.5 or later.
    3. Check your Mac for XCode. If you have it, make sure it is version 9.X. If you do not have it or you have an older version, go to the app store and download/update Xcode.
    4. Register as an Apple developer at https://developer.apple.com/ with your Apple ID. If you do not have an Apple ID, you must make one.

Installation

  1. Install Unity (this guide was written using Unity v. 2018.2.10). When installing, make sure you have Vuforia Augmented Reality selected. Vuforia is an augmented reality/computer vision SDK we will use to make our application.
    1. For HoloLens only: Also make sure both UWP options are selected. UWP stands for “Universal Windows Platforms” and lets you build to the HoloLens.
    2. For Android only: Also make sure the "Android Build Support" option is selected.
    3. For iOS only: Also make sure the "iOS Build Support" option is selected.
  2. Install Visual Studio 2017. Visual Studio is often installed with Unity, but it might not have the right SDKs. You might want to reinstall, making sure you include the “Game Development with Unity” workload. Search for “Visual Studio Installer” on your computer; if you have it, you can just install the new workloads from there.
    1. For HoloLens only: Make sure the “Universal Windows Platform Development” workload is also included.

Overview of Unity UI

Screenshot showing different windows in Unity UI

Setting Up/Building for the HoloLens

Follow this workflow to get started with HoloLens development. This will help you set up a base HoloLens project and learn how to build and deploy your project to the HoloLens from Unity using Visual Studio. **Note**: when given a choice between the HoloLens applications or mixed reality applications, follow the instructions for HoloLens applications.

Setting Up/Building for Android

Follow this guide to get started with Android Unity development. This will help you set up a base Android project and learn how to build and deploy your project to an Android device from Unity. The important sections to follow are: Java Development Kit, Android SDK Tools, and Connect the Android SDK and Java Development Kit to Unity. Once the rest of your project is made and you are ready to build and deploy, follow the Building Your Application to a Mobile Device section.

Setting Up/Building for iOS


Setting Up Unity for iOS

Screenshot of setup window

Settings screenshot

  1. Once you have your project in Unity, go to File > Build Settings
  2. Select the iOS platform. Then click “Switch Platform” in the bottom left of the Build Settings window. This might take a few minutes.
  3. Then click the “Player Settings” button in the bottom left. This will open up in the Inspector on the right.
  4. Click on the tab that looks like an iPhone. These are the iOS settings.
  5. Go to Other Settings and find the Identification section.
  6. Find the “Bundle Identifier” option and replace it with your identifier. This can be anything you want, but it must be of the form “com.XXX.YYY,” for example, “com.YourName.AppName.”

Building to an iOS Device

Screenshot of Unity-iphone tab

  1. In Unity, go to File > Build Settings and click “Build.” You will be prompted to make a folder that will hold your iOS build; name it whatever you want. The build will then start; this can take a few minutes.
  2. Once the build finishes, go to the folder that was just made. Inside it will be a .xcodeproj file. Double click to open the file in Xcode.
  3. Go to the upper left corner of Xcode and click on the folder symbol, then on the project name header (“Unity-iPhone” in the image below).
  4. In the middle of the screen, find the “Signing” section. Select “Automatically manage signing.”
  5. In the “Team” drop-down menu that appears, click “Add an Account...” (or use your own account if it already appears).
  6. Login with your Apple ID. You should then see your account as a selection in the drop-down list. Select it, then the red warning signs should be gone in “Status.”
  7. Connect your iOS device to your computer with a cable.
  8. Change the current device from Generic iOS Device to your own device.
  9. Click the play button in the top left of Xcode to deploy to your device.

Setting Up Vuforia

Vuforia Engine is a software platform for creating Augmented Reality apps. Developers can easily add advanced computer vision functionality to any application, allowing it to recognize images and objects, and interact with spaces in the real world.

This guide was adapted from here. You might wish to read the entire article to familiarize yourself with Vuforia, but follow the steps below.

  1. In Unity, go to Edit > Project Settings > Player. Find the XR Settings section and make sure "Vuforia Augmented Reality" is checked on.
  2. Go to Window > Vuforia Configuration.
    1. For HoloLens only: Under the Digital Eyewear section, set "Device Type" to "Digital Eyewear" and set "Device Config" to "HoloLens."
    2. For Android/iOS only: Under the Digital Eyewear section, set "Device Type" to "Handheld."
  3. While still in Vuforia Configuration, find the Device Tracker section. Check "Track Device Pose" and make sure "Tracking mode" is set to "POSITIONAL." This turns on Extended Tracking (see the guide linked above for more information).
  4. For HoloLens only: Go back to Edit > Project Settings > Player. While in the Windows icon tab, find the Resolution and Presentation section. Set "Default Orientation" to "Landscape Left."
  5. While still in Player Settings, find the Publishing Settings section. Under Capabilities, make sure “InternetClient,” “WebCam,” “Microphone,” and “SpatialPerception” are turned on.
  6. Within the Scene Hierarchy, right click and select Vuforia > ARCamera to add the specific Vuforia Camera to the scene. You can right click and delete the Main Camera originally there.
  7. Create a Vuforia account using this.
  8. Follow the “How to Create a License Key” workflow here.
  9. Then follow the “How to Add a License Key to a Unity app” workflow here.

Adding a Vuforia Image Target

This will allow our Unity app to recognize a 2D image and place a 3D hologram over it. See this for an example.

  1. From the same page where you found your License Key on the Vuforia site, go to Target Manager.
  2. Select “Add database” and your give your database a name, Select “Device” under Type.
  3. Once your database is created, click on “Add target” and select “Single Image.” Find your image using Browse and give it a name.
    1. Here is a guide to creating good image targets. If you’re stuck, try searching for “Vuforia image target” in Google Images and selecting one of those.
  4. There are differing opinions on this but set Width to 1.
  5. Your Target Manager page should now look something like this: 
    1. You want your image to have a 5-star rating; this is indicative of how well Vuforia will be able to track it.
  6. Select the target you would like to download (the image you just uploaded), and click on “Download Database”. You want the Unity development platform. This will download a file we will need to import into Unity.
  7. To import the assets you just uploaded to Vuforia, go to Assets > Import package > Custom package… and select the database file we just downloaded.
  8. Import all.
  9. If you go to Window > Vuforia Configuration, you should see your database under Databases.
  10. Add an image target to your scene by right-clicking within the scene hierarchy and selecting Vuforia > Image.
  11. View it in the Inspector. Under Image Target Behaviour, select “Predefined” and then select your database and image you want as your image target.
  12. Make sure your image target stays at the origin even if it looks weird in the context of your Unity scene. Resist editing the transform component.
  13. Add to your scene hierarchy whatever object you want to appear in the HoloLens when the user looks at the image target. Make this GameObject a child of your image target by dragging it onto “ImageTarget” in the scene hierarchy. (If you would like to use a model you created, follow the steps under the tab "Oculus Rift & Unity" > "Importing your Model" > "Importing a Model with Texture" of this guide, then select that model from your "Assets" > "Models" folder in the Project window and drag and drop it into the Hierarchy Window, on top of the word "ImageTarget" - this should nest it under "ImageTarget").
  14. Change the position, rotation, and scale of your newly added object so it has the correct placement/size relative to what your image target will look like in real life (for instructions on how to do this, check out "Oculus Rift & Unity" > "Importing your Model" > "Moving/Rotating/Scaling an Object" on this guide).
  15. Build and deploy your app. If you look at a picture of the image target you made, your object should appear.

 

Adding a Vuforia Model Target

This will allow our Unity app to recognize a real 3D object and place a 3D hologram over it. See this for an example.

  1. Download the Model Target Generator tool here.
    1. Make sure the version you download matches the version of Vuforia you are using in your project! You can check this by opening your Unity project and going to Window > Vuforia Configuration, and looking at "Vuforia Version." If the versions do not match, you might need to update the version of Vuforia you are using in your project.
  2. Follow this guide to use the Model Target Generator tool to make a model target dataset. You should end up with a .unitypackage file.
  3. We now want to import this package into our Unity project. In Unity, go to the Assets menu > Import Package > Custom Package... and select the .unitypackage file created in the previous step.
  4. For HoloLens only: The HoloLens camera is a landscape-oriented display. Thus, it is recommended to specify "Landscape" when creating a guide view in the Model Target Generator.
  5. Follow this guide to set up your model target in Unity.
    1. For HoloLens only: Click on the ModelTarget in your scene, and in the Inspector, change “Guide View Mode” to “Guide View 3D.”
    2. For Android/iOS only: Click on the ModelTarget in your scene, and in the Inspector, change “Guide View Mode” to “Guide View 2D.”
  6. Now when you deploy your project, you should be able to look at an object in the real world and see a hologram placed over it.

Creating a Gaze Cursor

  1. Download “HoloToolkit” from here and place it into your Unity project under Assets. You might have to do this using the file system, outside of Unity.
  2. Go to Assets > HoloToolkit > Input > Prefabs. Find and drag InputManager into the scene hierarchy.
  3. Then go to Assets > HoloToolkit > Input > Prefabs > Cursor. Find and drag Cursor into the scene hierarchy.
  4. Now when you run the app, you should see a cursor in front of you. It will be a transparent circle when it’s not on anything, but will turn into an opaque donut shape when it is on top of something (as long as that object has a collider).

Adding Airtap Functionality to an Object (non-UI)

  1. Because of the InputManager we added to our scene in the previous section, we are already detecting gestures like air taps. But nothing happens when we tap!
  2. If you want an object (non-UI) to do something when it is tapped, add a script to it. Add the following code to the script (you can change the name of the script to whatever makes sense):
using HoloToolkit.Unity.InputModule;
using UnityEngine;

public class ToggleableView : MonoBehaviour, IInputClickHandler, IInputHandler
{
   public void OnInputClicked(InputClickedEventData eventData)
   { }

   public void OnInputDown(InputEventData eventData)
   { }

   public void OnInputUp(InputEventData eventData)
   { }
}
  1. Place the code that you want to run when the object is tapped within OnInputClicked.
  2. If you want your object to be “tappable,” make sure to add a collider(s) to it (if it doesn’t already have them)!

Toggle the View of Two Objects by Tapping

This will let us toggle between the view of two objects by tapping on the objects.

  1. Create an empty game object called “ToggleableView” and add the script from the previous section to it using “Add Component.”
  2. In the script, add two variables right before the functions:
public GameObject model1;
public GameObject model2;
  1. Within OnInputClicked, add the following:
// toggle object views
model1.SetActive(!model1.activeSelf);
model2.SetActive(!model2.activeSelf);
  1. Set up your hierarchy to look like this:

  1. PanCoverAntefixModel and PanCoverModel are the two objects we want to toggle between. For the object you want to NOT be the starting view, select it, view it in the Inspector, and turn off the checkmark next to its name. This sets it to inactive at the start.
  2. Click on ToggleableView and view it in the Inspector. Drag the two models to the spaces for “Model1” and “Model2” so it looks like this:

  1. Now when you run the app and click on your starting model, it should toggle the view between the two models.

Resources

  • Accessing the Hololens - The Penn Libraries can provide access to a Microsoft Hololens upon request, contingent on availability. To learn more, contact David Toccafondi, Vitale Digital Media Lab Manager.

Penn Libraries Home Franklin Home
(215) 898-7555