Gesture is a powerful means for manipulating meaning in communication. In addition, gesture enables the manipulation and exploration of ones environment, which can in turn modify perception of that environment. Sketching, as one form of gesture, is a demonstrated aid in early idea formation as a way to externalise and rapidly manipulate ideas. In sketching the feedback is purely visual. But in gestural interaction with a design representation, feedback may also be haptic, auditory or kinaesthetic. It follows that gestural interaction might also serve to externalise and manipulate ideas as does sketching.

Current systems that employ gesture in computer-aided-design focus on gesture as a recognition and interpretation problem. Similarly, others require static, unnatural hand signs to operate task-specific commands, such as navigation through a virtual environment. This largely limits the potential of gesture as a means for idea stimulation through free exploration.

Gestural interaction offers the potential to explore different modes of concept formation according to the various external representations that can be manipulated using ones hands and tools as extensions of hands. Aside of sketching, we want to determine to represent gestural interaction so that it can be integrated into computational systems for designing.