Designing Graphical User Interfaces Integrating Gestures

The world of today and its new technologies like smartphones, tablets, or any flat interaction surface has increasing the need for graphical user interfaces integrating gestural interaction in which 2D pen-based gestures are properly used. Integrating this interac- tion modality in streamlined software development represents a significant challenge for designers or developers: it requires im- portant knowledge in gestures management, in deciding which gesture recognition algorithm should be used or refined for which types of gestures, or which usability knowledge should be used for supporting the development. These skills usually belong to ex- perts for gesture interaction and not actors usually involved in us- er interface design process. In this paper, we present a structured method for facilitating the integration of gestures in graphical user interfaces by describing the roles of the gesture specialist and oth- er stakeholders involved in the development life cycle, and the process of cooperation leading to the creation of a gesture-based user interface. The method consists of three pillars: a conceptual model for describing gestures on top of graphical user interfaces and its associated language, a step-wise approach for defining ges- tures depending on the end user's task, and a software that sup- ports this approach. This method is exemplified with a running example in the area of document navigation.