Leap Motion: Shortcuts

2014 - Augmenting Desktop Interfaces with Gesture

Media Tools
App Switcher


Shortcuts gives you quick access to common computer controls. Play your favorite music and videos, scroll through web pages and documents, or switch between apps with quick, simple gestures.


Media and volume controls. Point at the screen with your index finger to reveal media controls – play, previous track, next track, and a volume slider.

Scrolling. Pinch your fingers and drag to vertically scroll through web pages, applications, and documents.

Application switching. Flip your hand upside-down and raise your hand to reveal the Application Switcher, then grab and drag a window into the center of your screen to bring it to focus. To exit, simply drop your hand out of view.


My Role

Research, Design, Front-End Engineering

I have a multi-faceted roll on the Shortcuts team. Myself and a small team originally ran an extended contextual inquiry research and prototyping program to identify opportunities for gesture augmentation of desktop UIs. This was before Leap Motion transitioned into a sensor primary for Virtual and Augmented Reality. The Shortcuts team took some of the low-hanging fruit opportunities from our research and developed them into a full app.

During the production, I was both the interaction designer and front-end engineer. I built on top of our team's previous research and prototyping work to define a learnable, and trackable gesture set. To build the front-end interactions, I collaborated with our platform engineers, Walter Gray, Jonathan Marsden, and others who built out the underlying technologies allowing us to interact with and draw on top of the OS. They provided me with APIs to wrap the low-level OS interactions and visual effects and I built the font-end interaction on top. Shortcuts was developed for OSX and Windows in C++ and the AutoWiring Framework