Drone based mixed reality
HCI Engineering Interactive Technologies Final Project
In Fall of 2017, I was part of the first Engineering Interactive Technologies HCI class. The class was designed to push us to investigate new software and hardwaree interfaces. Me and my team of 3 worked on building a drone based mixed reality system.
A Drone’s Eye View is a system to bridge the physical and digital worlds. An autonomous drone with the ability to project interfaces around you.
Pick a surface, and watch it come to life.
Walk over to the workbench, and project a circuit design tool right next to your prototyping board.
Walk to the coffee table and project the presentation that you’re giving.
Head to the home theatre system, and an interface is on the wall right next to it.
With the flick of a hand gesture you can provide natural and intuitive inputs.
We imagined that we could use small drones to project interfaces onto different locations and different objects.
Rather than using displays for interfaces, a single drone could project the interfaces for different systems.
To prototype this, we used a parrot AR drone along with an Optitrack visual tracking system. We designed a tracking system in C# for the drone to autonomously navigate and follow a user around the room. The user wore an IR reflective wrist band.
On the drone was a small pico projector. We designed a custom music player application to run on a microcontroller connected to the projector. Lastly, a leap motion with a custom made wristband was used to send gestures as controls to change the interface.
