Thursday, 26 March, 2020 UTC


Summary

Welcome to Augmented Reality (AR) experiences. In this article, I will guide you through how to build your first hologram with Microsoft HoloLens 2 (HL2). Even if you do not have a device you can still try this guide with HoloLens 2 emulator (See prerequisites to find the download link for HL2 emulator).
What are we going to do? We will create a hologram, with which a user can interact, as shown in Figure 1. Firstly, in this article, we will build an application as in Figure 2. Afterwards, you can go through my other very short article on How to build your AR Application to your HoloLens 2 and deploy this application (call it AppX) and experience it as in Figure 1.
Figure 1
If you do not have a HL2 then you use either HL 2 emulator (which I have not explained here) or use Unity to simulate this (as in Figure 2).
Figure 2

Prerequisites

  1. A Windows 10 computer
  2. Visual Studio 2019 community version installed with ‘Desktop development with C++’ and ‘Universal Windows Platform (UWP) development’. You can tick these two workloads during the installation process.
  3. A HoloLens 2 device (or you can use the emulator)
  4. Unity Hub with Unity 2019.3.X installed and the Universal Windows Platform (UWP). Again you can tick this module (UWP) during the installation process)
  5. Unity Mixed reality Tool Kit (MRTK) version 2.3.0 — Download from here.
Let’s get started.
Assuming that you installed Visual Studio 2019 and Unity Hub with Unity version 2019.3.X (I use 2019.3.5f1), let us get started. Optionally, you can install the HL2 emulator if you do not have access to a device; the only difference is that you have to choose the emulator instead of ‘Device’ when running the project through Visual Studio. If you have all these tools installed you are all set to follow the rest of the steps. Please leave a comment if you find any issue during the installation process, I will try and answer them.

Configuring Unity to start with MRTK

First, you need to open Unity Hub and create a new project. Make sure to select 3D and give a name to the project and select a location and click on ‘CREATE’. Keep in mind that Windows OS has a MAX_PATH limit of 255 characters. Therefore, store your Unity project as close to the root of the drive.
Figure 3
The first thing you have to do is changing the build setting. Go to File -> Build Settings… (Or press Ctrl+Shift+B). A window will pop up and select the Universal Windows Platform and select the Target device as HoloLens and Architecture as x64. (Refer to Figure 4 below)
Figure 4
Then, click on the Switch Platform button and close the popup window. Next, go to Edit -> Project settings… You will be presented with another window as in Figure 5. On the left-hand side of the window select Player.
Figure 5
Expand XR Settings and tick Virtual Reality Supported. Once you tick this you will see a ‘+’ mark appear. Click on it and select Windows Mixed Reality as shown in Figure 6 below.
Figure 6
Give it a few seconds to Unity to finish adding the SDK. After that, there are a few settings that need to be changed for optimization. Under the expanded Windows Mixed Reality section, you will see a set of settings appear as in Figure 7 below. Select 16-bit depth under Depth Format, tick Enable depth Buffer Sharing and select Single Pass Instanced for Stereo Rendering Mode. You can refer to Figure 7 below to see how the final settings should look like.
Figure 7
Before closing the ‘Player’ settings there is one more setting to tick. Collapse XR Settings and expand Publishing Settings. Scroll down until you find Capabilities list and tick SpacialPerception. This allows the visualization of the spatial mapping mesh on Windows Mixed Reality devices. See Figure 8 below to find the screen capture with the described setting. After this, close the Project Settings window.
Figure 8
Now, go to Window -> TextMeshPro -> Import TMP Essential Resources to import TMP packages. We are importing this package because it is required by Mixed Reality Toolkit’s UI elements. Again refer to Figure 9 below.
Figure 9
You will get the Unity package window as in Figure 10. Click on All and then on the Import button. Give it a few seconds to imports everything.
Figure 10
While Unity completes the import, download Microsoft.MixedReality.Toolkit.Unity.Foundation.2.3.0.unitypackage and Microsoft.MixedReality.Toolkit.Unity.Tools.2.3.0.unitypackage (Optional but recommended). Once the Unity finishe TMP package imports, go to Assets -> Import Packages -> Custom Packages. See Figure 11 below.
Figure 11
Then, select the Microsoft.MixedReality.Toolkit.Unity.Foundation.2.3.0.unitypackage. Refer Figure 12.
Figure 12
Again click All and then the Import buttons. Refer to Figure 13.
Figure 13
Let Unity install all the necessary packages. This might take a while. Maybe a few minutes depending on your computer’s specifications.
Figure 14
Once Unity finishes all the imports you will get an MRTK popup window as in Figure 15 below. Click on the Apply button.
Figure 15
If you did not get a popup window as in Figure 15, go to Mixed Reality Toolkit -> Utilities -> Configure Unity Project. This will provide the same window. Refer to Figures 16 and 17 below.
Figure 16
Click on the Apply button.
Figure 17
Now repeat the sets in Figure 11 to install Microsoft.MixedReality.Toolkit.Unity.Tools.2.3.0.unitypackage. This is optional but I find this helpful; you can eliminate this step if you choose. Go to Assets -> Import Packages -> Custom Packages and then select the download package (Microsoft.MixedReality.Toolkit.Unity.Tools.2.3.0.unitypackage) and click All and the Import buttons. Refer to Figure 18 below.
Figure 18
If you come this far, congratulations, you have completed the Unity configuration to work with Mixed reality Toolkit (MRTK 2.3.0).

Trending AR VR Articles:

1. Oculus Go, the Nintendo Switch of VR
2. Expert View: 3 ways VR is transforming Learning & Development
3. Ready Player One : How Close Are We?
4. Augmented Reality — with React-Native

Building the first Holo Cube Scene

Now let’s get started to set up our Holo cube as in Figure 1. First, we need to add the Mixed Reality Toolkit object to our current scene. Go to Mixed Reality Toolkit -> Add to Scene and Configure. See Figure 19.
Figure 19
Then you will see MixedRealityToolkit object in the Hierarchy panel with a Directional Light object and MixedRealityPlaySpace object.
Figure 20
Now let’s first hide the spatial awareness mesh. Otherwise, what you see will look like below as in Figure 21. What you see in the background with white lines with a bunch of triangles is called ‘spatial mesh’. It is very useful in AR but you can make it invisible to the user.
Figure 21
In order to make the spatial awareness mesh invisible, you have to edit Mixed Reality Toolkit Profiles. The default MRTK profiles are not editable. Thus, we have to clone these profiles in order to edit. There are several nested layers of profiles. Each time we have to clone and edit several profiles when configuring one or more settings.
Alright, let’s get started. Click on the MixedRealityToolkit object and while it is selected in the Hierarchy panel, in the Inspector window, change the Mixed Reality Toolkit Configuration Profile to DefaultHoloLens2ConfigurationProfile. Check Figure 22 below.
Figure 22
Then, click the Copy & Customize button to open the Clone Profile window as in Figure 23. Then click the Clone button on the window.
Figure 23
You will see in the Inspector Window under MixedReality Toolkit, New MixedRealityToolkitConfigurationProfile is created. Refer to Figure 24 below.
Figure 24
Now there are several nested layers of profiles we need to clone. Click on the Spatial Awareness tab and then tick the Enable Spatial Awareness System checkbox. After that clone DefaultMixedrealitySpatialAwarenesssystemProfile. Again another Clone Profile window will open. Click the clone button. Refer to Figure 25 below to see all the describe settings. Look at the Inspector tab on the right side of Figure 25.
Figure 25
Now you will see a New MixedrealitySpatialAwarenessSystemProfile. This newly created Spatial Awareness System Profile is now automatically assigned to our Configuration Profile (New MixedRealityToolkitConfigurationProfile).
Then, expand the Windows Mixed Reality Spatial Mesh Observer section, then we have to do another nested profile clone. You will see DefaultMixedrealitySpatialAwarenessMeshObserverProfile. Click the Clone button. Refer to Figure 26.
Figure 26
You will see another Clone Profile window opens. Click the clone button. Refer to Figure 27 below.
Figure 27
Hooray! We reached the final step to make the spatial awareness mesh invisible. No more profile cloning. As you can see in Figure 28 scroll down in the Spatial Mesh Observer Settings and change the Display Option to Occlusion. This will make the spatial mapping mesh invisible but it will still be functional.
Figure 28
Alright, time to create our cube now. Go to the Hierarchy panel and right-click on it, select 3D Object -> Cube. Refer to Figure 29 below.
Figure 29
Well, the cube you see should be similar to Figure 30 below. It is too big and placed at (0,0,0) so we need to change some of these properties first.
Figure 30
Now click on the Cube object on the Hierarchy panel and then go to Inspector panel while Cude is being selected. Enter the values as in Figure 31 (You are free to place the cube anywhere you like) to exactly get what I have shown in Figure 1.
Position: X:0 Y:0 Z:0.6, Rotation X:30 Y:60 Z:0, Scale X:0.1 Y:0.1 Z:0.1
Figure 31
Before moving forward let me add some directional lights so that sides of the cubes are clearly visible. Right-click on the Directional Light object in the Hierarchy panel and click Duplicate. Do this three times to add three directional lights (This is an optional step, not necessary for the functionality). Refer to Figure 32.
Figure 32
See the transform properties of each of these directional lights in figures 33,34 and 35. You can add the same values or change the directions as required. The position and scale of the directional light can be disregarded. Only copy the Rotation properties.
Directional Light(1) Rotation X:-50 Y:30 Z:0 | Directional Light(2) Rotation X:-140 Y:120 Z:0 | Directional Light(3) Rotation X:140 Y:-120 Z:0
Figure 33Figure 34Figure 34
Now your Unity screen should look like below in Figure 35 when selected the Cube object.
Figure 35
In order to bring some interactions to your cube and to make it grabbable, assign these two scripts: ManipulationHandler.cs and NearInteractionGrabbable.cs. How do we do that? Go to the Inspector panel while the cube is selected on the Hierarchy panel. Click on Add Component and type NearInteractionGrabbable and press enter to add the script. Tick Show Tether When Manipulating checkbox.
Figure 36
There’s another script you need to add. Again click on Add Component and type ManipulationHandler and press enter to add the script.
Figure 37
After that, under ManipulationHandler, click the small circle-like button which you will see in front of Host Transform. You will get a popup window and double click on the Cube. Refer to Figure 38 below.
Figure 38
Your cube Inspector should be similar to Figure 39 if you followed all the instructions up to this point.
Figure 39
That is it! Now do not forget to save your scene. Go to File -> Save and select Scenes directory/folder and give a proper name and save your scene. Refer to Figure 40 below.
Figure 40
Now it is all set. Go ahead hit the play button in Unity and you will see the following window (Refer to Figure 41)with the cube. Use the space bar to see the hand and use the mouse to click and drag the cube.
Figure 41
Well, congratulations! You just completed your first Hologram application. Now you are ready to deploy your first AppX (Mixed reality application) into a real HL2.
As this is a very long article I will split the deployment part into another article. However, I will be using this same application. Thus, read here a very short article on How to build your AR Application to your HoloLens 2.
Feel free to leave a comment if you face any issues when following this guide. I will try to answer them. Also, if someone else finds the solution they will do the same. That’s how we should learn, sharing knowledge. Cheers!

Don’t forget to give us your 👏 !

https://medium.com/media/1e1f2ee7654748bb938735cbca6f0fd3/href
Build your first HoloLens 2 Application with Unity and MRTK 2.3.0 was originally published in AR/VR Journey: Augmented & Virtual Reality Magazine on Medium, where people are continuing the conversation by highlighting and responding to this story.