Integration of the sensory system in the design of user experiences

The dream of visual dynamism is the same; to leave behind earthbound stasis and to fly into that liquid space of numerical architecture without gravity – John Whitney

This quote by John Whitney poetically describes the claim he made for himself as an animator, inventor and composer. The fascination in Whitney’s quote lies in the statement “to leave behind earthbound stasis” – herewith Whitney declares his vision as an animator to break out of the conventional and to get to the bottom of new possibilities.

In the most distant sense, this is also the claim or much more the subject I want to deal with. In the course of the Design & Research course, I would like to deal with the question of how the human sensory organs (Human-Sensory System) of a person can not only be integrated but also further developed through digital user experiences. In the context of this question I would like to question conventional and habitual information transfer and knowledge transfer and at the same time I would like to get to the bottom of new possibilities.

In a first step I will deal with what constitutes the sensory organs, what they need to be (further) developed and which analog and digital endeavors already exist in this respect.

In the next step, I will look at how knowledge transfer can take place in digital space by involving several sensory organs, using examples from data journalism, media art and education.

A further area of interest in my research will be what possible areas of application this opens up and what best cases are available.

Basic questions that are supported in each step can be found in “The Mental Notes Behaviour Cube”, which was created by Stephen P. Anderson under Creative Commons in 2013. Anderson tries to find out in the context of 5 questions what controls/shapes the behaviour.

  • What stimulates the brain?
    • Narrative/Story
    • Humor
    • Surprise
    • Delighters
    • Visual imagery
    • Judgement based on emotions / affect heuristics
    • Sensory attraction
  • What influences our decisions?
    • Rarity
    • Social proof
    • Authority
    • Gifts / Mutual recognition
    • Limited choice
    • Limited duration
    • Commitment and consistency
    • Owner profile
    • Loss aversion
    • Periodic events
    • Set completion
    • Status Quo Prejudice
    • need for certainty
  • What shapes memory and perception?
    • Pattern Recognition
    • Conceptual metaphor
    • feedback loops
    • Visual language
    • Narrative
    • Peak-End Rule
    • Preparation
    • Anchoring and adaptation
    • Framing
    • Runtime effects
    • Comparison
    • Doped effect on progress
    • Gauze Cueing
    • Composition
    • Uniform solidarity
    • recognition over recall
    • Series position effect
    • Value assignment
    • Contrast
    • Familiarity distortion
    • Aesthetic usability
    • Positive Imitation
  • What encourages and discourages us?
    • Achievement
    • Different rewards
    • feedback loops
    • Limited access
    • Collect
    • Regular events
    • Sequencing
    • Design
    • Trigger
  • What motivates/driving us?
    • Suitable challenges
    • Curiosity
    • Control / Autonomy
    • Fantasy
    • Competition
    • Cooperation
    • Reputation / Status
    • Self-portrayal
    • Limited access

All these questions play an important role in the reception, transmission and strengthening of knowledge.

Further Literature:

Alles spekulativ: Design, Fiktion und soziale Träume, Anthony Dunne

Das Handbuch der multimodalen Multisensor-Schnittstellen Band 1, ACM-Bücher

Wie man gewohnheitsbildende Produkte aufbaut, Nir Eyal

Interaktiver Journalismus: Hacker, Daten und Code, Nikki Usher

Der Smarter Screen: Überraschende Wege zur Beeinflussung und Verbesserung des Online-Verhaltens, Shlomo Benartzi