Introduction to NUI

Natural User Interface, or NUI, is an extension of GraphWorX64 that makes it possible to use sensors for natural user interactions.

NUI enables users to control GraphWorX64 displays by using Skeletal Tracking (touchless) interactions and voice commands. It also provides the ability, when integrated with the necessary hardware, to turn any surface into a virtual Multi-touch display.

There are two type of interactions with a NUI device, through gestures and by using voice control.

For gestures, the NUI device has two detection modes:

The Skeletal Tracking mode detects and keeps track of the user’s body from a distance and allows the user to perform gestures by moving his or her arms around in the air, and by performing selection and clicking action by opening and closing his or her hands. Moreover, the user can associate voice commands with pick actions, and hence associate voice commands with virtually any command in GraphWorX64.
In the Multi-touch mode, the NUI device is aligned with a screen or surface. The screen or surface is mapped to the computer screen space and becomes a touch sensitive region. The NUI device detects anything that approaches the screen/surface within a certain range. As the user approaches to touch the surface, his or her finger can be tracked by the NUI device up to a certain low threshold. The device will keep track of finger position on top of the tracked surface, and use that to control cursor movement. If the finger reaches a minimum closer threshold, the finger will then trigger a touch action on the surface, which will be converted to a touch event, as if it was triggered from a touch screen.

The Natural User Interface help documentation is specific to NUI usage in GraphWorX64. Kinect hardware and connection issues are not covered within these help topics. For Kinect specific issues, please visit the Microsoft Kinect online help page: http://msdn.microsoft.com/en-us/library/hh855347.aspx.