During my research I read some excerpts from the book “The Handbook of Multimodal-Multisensor Interfaces” to understand the basic principles of multimodal interface design and to familiarize myself with the vocabulary. The book was published in 2017 and deals intensively with various aspects of interface design and conception based on neuroscientific and psychological research. In my blog entries I would like to excerpt some passages and take a closer look at them.
One Major reason why multimodal-multisensor interfaces have dominated on mobile devices is their flexibility. They support the users’ ability to select a suitable input mode, or to shift among modalities as needed during the changing physical contexts and demands of continuous mobile use.” ( In this statement, the author concisely answers the cause of the increase in multimodal multisensory interfaces. It should be emphasized here that the further development of mobile devices in the aspect of multimodality and sensor technology has taken place globally for all target groups, regardless of language, age, technical knowledge or impairments.
Cohen Philip: The Handbook of Multimodal-Multisensor Interfaces, 1
The awareness of how much one can control the behaviour and further development of people to achieve a certain result or behaviour was only created by the increase of multicomponent physical tools. In The Handbook of Multimodal-Multisensor Interfaces it is even described as a renaissance in the design of physical devices, as we consider 3 levels in the design and conception of devices:
- The tool can/should be used in many ways.
- The tool should improve the desired performance.
- The tool should be easy to use.
Yusupov and Karpov specify in their paper “Multimodal Interfaces of Human-Computer Interaction” the principles in the interaction from human to machine on the following points:
naturalness (intuitiveness): interaction without the need to instruct the user how to work with the interface and to carry various auxiliary devices;
ergonomics: convenience of using an interface by various users;
friendliness: unobtrusive user tracking and avoiding wrong user actions;
reliability: error-free operation of interface components under various interactive conditions, i.e., assurance of the maximally possible accuracy and robustness of the functioning of interface recognition components
efficacy: a minimal number of operations on the part of the user and minimal time for performing the required task;
universality: applicability to various user categories, including, e.g., the physically disabled, elderly people, or expert operators who act under extreme conditions, such as weightlessness, underwater situations, etc., without the need for adaptation;
multimodality: providing simultaneously several means of communication: speech, gestures, text, etc., so that the user could choose from intelligible and convenient tools.
Karpov, Yusupov: Multimodal Interfaces of Human-Computer Interaction, 67
The renaissance of the possibilities of what input on a digital device can look like, calls for an increased stimulation of cognitive abilities through its flexibility and thus increases the desired performance. An impressive example of this is the Handbook of Multimodal Multi-Sensor Interfaces, in which the same students working on the same scientific problem find more solutions to the problem when using a digital pen compared to a simple keyboard input.
From a communications perspective, results have demonstrated that more expressively powerful input modalities, and multimodal combination of them, can directly facilitate our ability to think clearly and perform well on tasks.
Cohen Philip: The Handbook of Multimodal-Multisensor Interfaces, 5
Sources:
Cohen Philip: The Handbook of Multimodal-Multisensor Interfaces
Karpov, Yusupov: Multimodal Interfaces of Human–Computer Interaction
