Spatial Understanding on HoloLens Emulator

Many years ago my wife and I were driving down the road when we were rear-ended by a high school kid. It was a minor bump up. However, when I got out of the car to talk to him, he promptly told me that his brakes didn’t work.

Yeah, right.

Since there wasn’t any real damage done, I told him that he didn’t need to make excuses. I then suggested that he should follow at a safer distance and we went on our way.

Come to think of it, I probably let him off easy because I had a few bump ups when I was 16, but that is a story for another day…

This story reminded me of a quote by our very first president, George Washington.

It’s better to offer no excuse than a bad one.

What made me think about that high school kid with the bad excuse was that I’ve recently talked with some people about HoloLens development and I’ve heard things like…

“I’d be interested in HoloLens development but the hardware is too expensive.”

and…

“I don’t have a device, so I can’t really do much when it comes to HoloLens development.”

These are excuses, and they are bad ones at that. In this week’s YouTube video I show how you can use something as complex as spatial understanding in the HoloLens emulator itself.

With free tools, you can get 90% of your app or game done without an actual device. Having a device is definitely cool, but not having one shouldn’t deter you from getting started.

To help you get a jump start on your HoloLens development I’ve created the Learn HoloLens site. It is a members only site and enrollment will open next week. The membership includes a monthly master class discussing a HoloLens topic as well as a Live Q&A. One of the master classes is all about how to use the HoloLens emulator to its full potential so you can make great progress to see if HoloLens development is right for you.

Also, to help you get going I’ll also be providing some free webinars next week as well. Don’t let not having a device keep you from checking out HoloLens development. Many of the current members don’t have a device and have yet they have told me they have received a lot of value from the membership already.

To be notified about the webinars and when the membership opens it’s doors again, make sure you sign up on the notification list. I’ll send you the dates and times you can register for the webinar once I nail down the exact dates and time for them.

Ultimate HoloLens Resource List

If you find this Ultimate HoloLens Resource List helpful, make sure you subscribe to keep up to date as this resource list grows.

Tips | UI Design | Real World Use | How It Works | UWP | How TosUnity | Cool Demos | Apps

Tips
Performance recommendations for HoloLens Apps
Recommendations and guidelines when developing for the HoloLens

Side by Side HoloLens Recording Comparison (Streaming vs. On Device)
When WiFi is good, record via streaming instead of on the device directly

How to prepare a Unity project for Git
This stack overflow answer shows a great way to setup Unity for Git repository. Step 1 can be ignored for HoloLens development since it is using Unity 5.5.

Optimizing Shader Load Time in Unity
Shaders are small programs that execute on the GPU. Optimizing shaders are important for HoloLens development.

Activating and Deactivating Objects in Unity using Mixed Reality InputManager
This tip shows how to hide an object by air tapping on it.

HoloLens Spatial Understanding works in the Unity Editor
This post and video explains how Spatial Understand now works inside of the Unity Editor which decreases the development time

Tips | UI Design | Real World Use | How It Works | UWP | How TosUnity | Cool Demos | Apps

UI Design
Microsoft HoloLens: 3 HoloStudio UI and Interaction Design Learnings
This video discusses some best practices for 3D UI and interaction design.

Defining the best practices for HoloLens – Welcome to the wild, wild west
Join the discussion in a Slack channel to discuss best practices for HoloLens 3D UI and interaction design.

Unite Europe 2017 – AR prototyping for the HoloLens with Unity
This Unity Europe session shows AR prototyping for the HoloLens and Unity

Tips | UI Design | Real World Use | How It Works | UWP | How TosUnity | Cool Demos | Apps

Real World Use
Hololens shows up in cool Honda NSX promo video
Honda shows off a cool HoloLens demo to promote their NSX automobile.

Microsoft HoloLens and Lowe’s, working to redefine your next home renovation
Lowe’s, the home improvement store, brings customers a new way to experience full-scale design and appliance options while maximizing precious floor space in the store.

Ignite Innovation Keynote demoing the HoloLens Lowe’s app
A video demo of the Lowe’s HoloLens app that shows a new Kitchen remodeling app

Transforming the global elevator service industry with the Microsoft HoloLens
Using HoloLens, thyssenkrupp, a global elevator service company, will be able to empower more than 24,000 of the company’s service technicians to do their jobs more safely and efficiently.

Practical HoloLens: Lubricating a Mower
Something as simple as opening up a PDF in while working on a mower can really help productivity. There is even a suggestion for a future app…

Volvo is testing HoloLens to speed up car development
While this isn’t actually in place yet, Volvo is using the HoloLens to ultimately reduce the development time of a new car model from 30 months to 20 months.

Unite Europe 2017 – Making a business case for HoloLens: Dev & Sales
This is a talk discussing business cases for HoloLens development and sales in the real world

Holographic Design & Installation
Landscaping and Design using the HoloLens by one of my LearnHoloLens members

Tips | UI Design | Real World Use | How It Works | UWP | How TosUnity | Cool Demos | Apps

How It works
How HoloLens Displays Work – The Imaginative Universal
This is an in-depth look at the HoloLens display technology actually works.

Microsoft’s official HoloLens hardware page
Microsoft shows how the adjustable headband is designed and shows the different sensors and optics the HoloLens device has.

Microsoft HoloLens Specifications
This contains the full list of hardware, sensors, specs and more for the first-ever wearable HoloLens holographic computer system.

HoloLens Actual Field of View FOV Captured
Through what appears to be a fluke, a HoloLens records the actual FOV and not the entire 720p space.

Tips | UI Design | Real World Use | How It Works | UWP | How TosUnity | Cool Demos | Apps

UWP
Microsoft Mixed Reality Toolkit
The MixedRealityToolkit is a collection of scripts and components intended to accelerate the development of mixed reality applications targeting Windows Mixed Reality.

UrhoSharp – 3D/2D Engine – Xamarin
UrhoSharp is a cross-platform high-level 3D and 2D engine that can be used to create animated 3D and 2D scenes for your applications using geometries, materials, lights and cameras.

Launch a 3D Holograms app from a 2D XAML app and going back to it using UWP
This blog post shows how to create a 2D XAML app which will be able to display 3D holograms and go back to the 2D view.

How to deploy an App to the HoloLens Device – Step by Step
This article shows how to deploy a UWP app to the HoloLens step by step.

UWP App Model
Windows Mixed Reality uses the app model provided by the Universal Windows Platform (UWP), a model and environment for modern Windows apps.

Tips | UI Design | Real World Use | How It Works | UWP | How TosUnity | Cool Demos | Apps

How Tos
Install the tools
Get the tools you need to build apps for the Microsoft HoloLens.

Turn HoloLens off and on
Sometimes it is the simple things. This article shows how to turn the HoloLens on and off as well entering standby mode and even charging the battery.

Create a complete Mixed Reality Game from Start to Finish – Step by Step
This 13 part series goes through the entire process of setting up a mixed reality game that will run on the immersive headsets as well as the HoloLens.

Part 1: Setup
Part 2: Core Game Logic
Part 3: Game Controller
Part 4: Motion Controller Support / Input
Part 5: Keeping Score
Part 6: Hiding Items and Disabling Input
Part 7: Displaying Score and Strikes
Part 8: High Scores
Part 9: Adding Animation
Part 10: Implementing Animation
Part 11: Adding Sounds
Part 12: Adding Particles
Part 13: Deploying to Devices

Understanding the Gaze and Adding a Gaze Input Cursor into your Unity 3D Holographic App
This article shows how you can utilize the Gaze input on the HoloLens

Understanding the Gesture and Adding Air Tap Gesture into your Unity 3D Holographic App
This article explains how to integrate Gesture into your HoloLens app.

How To Setup a Simple Unity Project using the Vuforia Library
The Vuforia Unity extension allows Unity developers to easily use the Vuforia augmented reality library with the HoloLens.

Advanced HoloLens Emulator Input
This help page shows the different types of input as well as the different input control modes and the ways to use the emulator to control an app.

Holograms 100 – Getting started with Unity
This Microsoft tutorial shows how to get started with HoloLens development using the Unity game engine.

Holograms 101E – Deploy to HoloLens Emulator
This Microsoft tutorial shows how to create a complete HoloLens application and run it on the Emulator.

Holograms 101 – Getting started with the HoloLens
This Microsoft tutorial shows how to create a complete HoloLens application and run it on the HoloLens device.

Holograms 210 – Getting started with Gaze
This Microsoft tutorial gets the reader up to speed quickly with using Gaze with the HoloLens.

Holograms 211 – Getting started with Gestures
This Microsoft tutorial shows how to use Gestures to allow your users to interact with holograms on the HoloLens device.

Holograms 212 – Getting started with Voice
This Microsoft tutorial show how to use voice input on the HoloLens device.

Holograms 220 – Getting started with Spatial Sound
This Microsoft tutorial shows how to get started using spatial sound.

Holograms 230 – Getting started with Spatial Mapping
This Microsoft tutorial shows how to get started with spatial mapping. Spatial mapping combines the real world and virtual world together by teaching holograms about the environment.

Holograms 240 – Getting started with HoloLens Networking / Sharing
This Microsoft tutorial shows how multiple HoloLens devices can share the same interactive experience.

Getting the most out of the HoloLens Companion App
The HoloLens companion app is the best way to record what the HoloLens is seeing. It has plenty of other features which this article describes.

How to Record from your HoloLens TWO different ways
This article discusses both ways to record what your HoloLens is seeing – directly on the device and streamed through the HoloLens app.

HoloLens Development Tutorial Based on Talk
This is a group of tutorials from Cameron Vetter teaching different HoloLens functionality . Really worth checking out.

Connect Device Portal via USB
El Bruno explains how to use the developer portal when the HoloLens device is connected via USB

Hitchhiking the HoloToolkit-Unity
Mike Taulty has several posts working with the HoloLens that should be looked at

How to Install Mixed Reality Toolkit for Unity
This article shows how you can get going quickly with HoloLens development by installing the Mixed Reality Toolkit and using it inside of Unity.

How to Deploy to the HoloLens from Unity
This article shows you step by step how to deploy the app to the HoloLens and to the Emulator.

Holographic Bouncing Ball Tutorial
This tutorial series covers the Unity environment, deploying apps, voice recognition and spatial mapping.

Tips | UI Design | Real World Use | How It Works | UWP | How TosUnity | Cool Demos | Apps

Unity
Microsoft Mixed Reality Toolkit for Unity
The Mixed Reality Toolkit is a collection of scripts and components intended to accelerate the development of holographic applications targeting Windows Holographic. This is effectively part of the existing Mixed Reality Toolkit, but is the repo that will contain all Unity specific components.

Performance recommendations for Unity
This Microsoft documentation shows general Unity performance recommendations you want to use when developing for the HoloLens device.

How to Deploy a Unity Game to the HoloLens Device – Step by Step
This article shows how to deploy a Unity app to the HoloLens step by step.

Microsoft HoloLens Hand Tracking – Building Blocks
This article shows the important pieces of code need to have the HoloLens handle hand tracking.

50 Tips and Best Practices for Unity (2016 Edition)
This article shows 50 different tips and best practices from one developer’s point of view. It has valuable information for you to look at when doing Unity development for the HoloLens.

Runtime OBJ Importer
OBJ File importer that loads OBJ models both in the editor and at runtime.

Saving Spatial Meshes
SaveSpatialMapping Example in the Mixed Reality Toolkit for Unity

Tips | UI Design | Real World Use | How It Works | UWP | How TosUnity | Cool Demos | Apps

Cool Demos
HoloBeam Tech
This demo app uses a stereoscopic camera to create a 3D point cloud which can be streamed in real-time to a user wearing the HoloLens.

HoloLens Image Based Texture
This demo code (and video) uses the Hololens camera to generate textures for the spatial mapping meshes

Medical Imaging – DICOM Volume Rendering on HoloLens
This demo uses real medical imaging data to display the data in 3D

HoloLens-Aided Neurosurgery
Emergency neurosurgery might soon be a lot safer, with the help of augmented reality. Duke surgeons and engineers are experimenting with using the HoloLens goggles to project virtual, 3D images of patient’s brains onto their heads, with the goal of guiding neurosurgeons during operations.

HoloLens with Object Tracking
Iowa State University created a demo of the HoloLens working with the Kinect with Marker-less Object Tracking.

Image Recognition in HoloLens using Microsoft Cognitive Services
This video shows an demo of an app where the HoloLens is asked to scan an image. The HoloLens examines what it sees and uses the cognitive services to say what it think it saw.

Microsoft HoloLens Holo Raid
Holo Raid is a game for HoloLens similar to Robo Raid. Transform your living room into a space portal to shoot alien raiders.

Video: HoloLens Projection – Skeletons!
This proof of concept demo shows how to use the HoloLens to project augmented reality information onto a human body. An external sensor is used to track the body and sent to the HoloLens which projects a skeleton onto the person.

HoloLens Magic Window Demo
This is a demo of the Magic Window effect which has a virtual world just beyond some space. Sometimes the world can’t be reached, other times it can.

HoloLens Developer Challenge: Wizard Battle (Mage)
This demo shows voice recognition and particle effects for a developer challenge.

HoloLens Developer Challenge: Wizard Battle (Sorcerer)
This demo shows different particle effects for the same developer challenge.

HoloLens Bed Demo
This demo shows a photo realistic bed being placed in the room and then they style of bed being switched out.

Developer combines HoloLens and Vive for ‘shared reality’
Collaborating across a mixed reality space is surprisingly simple.

Portal in Augmented Reality with HoloLens
What’s better than the game portal? How about this cool Portal demo using the HoloLens.

VRLA Mixed Reality Easter Egg Hunt: Behind the Scenes
Mixed Reality Easter Egg Hunt in partnership with Microsoft HoloLens

HoloViveObserver
Using Vive Controllers with the HoloLens

High-tech White Cane for the blind using HoloLens
This video shows a prototype of how someone who can’t see could use the HoloLens to assist in hearing where walls are

Why use a HoloLens in science museum?
The projects shows how within a museum context mixed reality can be useful because the virtual and the physical realm mutually empower each other.

Red Bull Air Race MR
REWIND’s CEO Sol Rogers explains how the team managed to forge a world-first immersive experience for the Red Bull Air Race using the new design principles of Mixed Reality and working with the HoloLens.

HoloLens – Airport Command and Control Center
Demo created fo SITA Lab and Helsinki airport shows a visualization of an airport command and control.

HoloMages – Shared Concept Demo Using Azure Cloud
Preston McCauley & Ron Dagdag put together to experiment with using cloud shared service with the Hololens.

Super Mario Bros Recreated as Life Size Augmented Reality Game
Super Mario Bros meets Augmented Reality in this crazy life sized first person experience.

HoloLens and MapBox
Neat app showing San Diego map

Holographic Workstation for Citi Financial Trading
8ninths’ envisioned a mixed reality evolution of the trading workstation using HoloLens

HoloLens CityLab 4D Construction Demo
See what 4D construction looks like in the Hololens.

First Person Real Life Shooter Using HoloLens & ARKit
Demo showing turning an iPhone into a laser weapon that works with the HoloLens

Dynamically Placed Items with Spatial Understanding
This demo shows placing objects in the real world using Spatial Understanding

Having Fun with HoloLens Particles
This demo shows different particle effects running on the HoloLens.

Little Holographic Dragon
This demo shows a little dragon running around in the real world.

Tips | UI Design | Real World Use | How It Works | UWP | How TosUnity | Cool Demos | Apps

Apps
HoloPaintball
This app had it roots in a Unity tutorial and was modified to work on the HoloLens. The source code is also available for download.

Here is a list of Made for HoloLens Apps
This is a frequently updated list of apps made specifically for the HoloLens that are available in the Windows store. It also lists, separately, apps that work on the HoloLens, but weren’t made especially for the HoloLens.

Top 5 Apps in the HoloLens Store on Day 1
This video shows the top 5 apps that were in the HoloLens store on the very first day.

HoloLems
This video shows me having fun playing a HoloLens game called HoloLems

Ocean Treasure
Ocean Treasure Hidden Object HoloLens app

Nextcloud Downloader
NextCloud Downloader facilitates the retrieval of file assets from a local instance of Nextcloud server to HoloLens storage.

White Cane
Helping those with no vision to be able to detect surfaces with sound

HoloFlight
Visualize real flight data as 3D holograms

HoloValentine
This app was made for Valentines Day and the source code is available for download. The demo uses gaze, gestures, voice, spatial sound and spatial mapping.

If you find this Ultimate HoloLens Resource List helpful, make sure you subscribe to keep up to date as this resource list grows.

Tips | UI Design | Real World Use | How It Works | UWP | How TosUnity | Cool Demos | Apps

 

SIEGE Conference 2016 Recap

Giving a talk at the SIEGE Conference

I had the last minute opportunity to speak at the Southern Interactive Entertainment and Games Expo (SIEGE Conference) over the weekend. During the first part of the week, I was asked to present on the HoloLens. The reason for the last minute ask was because of Hurricane Matthew and previously scheduled speakers not being able to make it to the conference.

This was not my first time at the SIEGE conference. I’ve had the opportunity to speak at the conference in the past. I enjoyed the conference this year as well. I was able to meet some great people and have a lot of conversations around augmented reality and the HoloLens device in particular.

In attendance you will find both Indie game studios, one man teams, students, professionals, traditional game studios, investors, composers, and even lawyers. The keynotes are really great. The first year I attended I was able to meet Nolan Bushnell, co-founder of Atari.

This year, the creator of the infamous game Myst, Rand Miller was interviewed in the first keynote. It was a great session!

Also, Nick Laing, Senior Producer at Turn 10 (makers of Forza Motorsports), also gave a fantastic talk around personal leadership. While his talk was geared toward game production, much of what he talked about was very applicable to those who are starting their own business and are on the path to entrepreneurship.

True leadership, no matter where you are going to end up, it starts at the personal level and it’s a deeply personal endeavor.

Personal Leadership is the process of self-influence in which a person can aid and support themselves for accomplishment. This is responsibility for yourself, this is setting standards for yourself, this is holding yourself accountable.

If you are thinking of creating your own game as an indie game developer, this notion of personal leadership or self-discipline is super important. I’ve have the opportunity to work out of my house for the last 16 years. When I first started working from home, I couldn’t separate work from home.

This is a common struggle for many. For some, they can’t focus on the work that needs to be done because of distractions at home, or just the overwhelming feeling of everything at home that needs to be done. For others, and I fall into this group, the issue is more that they don’t work at home as much as they live at work. It is hard to set aside time to do things around the house because the stack of work is always there. Fortunately, I’ve gotten much better over the years at separating work and home.

It takes discipline to start work and to then stop work when it is appropriate. When you start your own company, or launch your own indie game studio, the number of tasks increase. Besides the technical aspect of work that will ultimately pay the bills (graphic design, software development, art creation, etc.), you also need to do the paperwork of the company, like submitting invoices, paying bills, doing the bookkeeping, paying taxes, etc.

I believe that time management doesn’t exist. It comes down to self-management. Time is what it is. We can’t manage it, but we can manage ourselves.

This notion of self-management is more than just self-discipline. Self-discipline is about doing what needs to be done even if it doesn’t feel good. Self-management is about knowing what needs to be done and prioritizing those important things to be done in an efficient manner.

I’ve mentioned the book Essentialism before, but I listened to the book again on the way back from the conference. When you run a business, you get a lot of opportunities and you have to decide which opportunities are worth doing. Where should your time be spent?

Self-management says we need to set a priority based on what needs to be done and then have self-discipline to actually do them in that order, regardless if we feel like it or not.

The SIEGE Conference live streamed my HoloLens session

The talk I gave this year at SIEGE was around spatial mapping for the HoloLens. I’ll be giving a similar talk at CodeMash in January of next year. So if you are close to Ohio you should check it out.

Also, I have a Master Class talking all about spatial mapping in a more concise manner (because of the power of video editing) in the Learn HoloLens membership site. So if you are interested in learning about the HoloLens you should check out the site. Enrollment for the LearnHoloLens.com membership site is only open a couple of times a year.

We did the alpha launch of LearnHoloLens.com the first week of September. I’ll be reopening enrollment the first week of November for our beta launch. If you want to be notified when membership opens, make sure you sign up on the waiting list.

The first month’s Master Class was all about using spatial mapping with the HoloLens. This month, the Master Class will be all around using the Emulator. This way you can get started immediately with HoloLens development even if you don’t have a device.

Augmented Reality and Mixed Reality are going to continue to grow. Make some time to get in on the ground floor to learn how to work with AR and MR using the HoloLens.

I provide free videos on YouTube every week, including this week’s video. Enjoy!

 

HoloLens Actiongram: Grim Reaper vs the Mime

grim-vs-mime

I took a break from developing demos for the HoloLens to play around with some of the apps currently available. This week’s video features the Actiongram app from Microsoft and the Land of the Dinosaurs app.

The Land of Dinosaurs app is a Virtual Reality type of app, much like the HoloTour.

The Actiongram app is still in beta, but it seems to be running pretty solidly. It lets you add a few pre-recorded items to a scene and you can change the scale, rotation, and position of each “actor”. You can also specify when it should start playing – immediately, on gaze, after X seconds, etc.

I enjoyed playing with the app, but definitely had to clean up and trim up the shots once I saved the videos. The videos record 30 seconds at a time.

Playing with Actiongram was a nice diversion. Did you notice the butterfly that landed on the flowers?

On a side note, if you are going to be in or around Atlanta for October 7th-9th let me know. I’ll be speaking at SiegeCon. I’ll be doing a talk on the HoloLens around spatial mapping.

If you have a minute, you should leave me a comment and let me know what topic, app or demo you’d like me to cover next.

Enjoy the video and have a great week!