INTERACTION TYPES

 Instructing
 Issuing commands and selecting options

 Conversing
 Interacting with a system as if having a conversation

 Manipulating
INTERACTION  Interacting with objects in a virtual or physical space by
TYPES manipulating them

 Exploring
 Moving through a virtual environment or a physical space

 Responding
 The system initiates the interaction and the user chooses whether
to respond

WWW.ID-BOOK.COM
  Where users instruct a system and tell it what to do
 For example: Tell the time, print a file, or save a file

 Very common conceptual model underlying a diversity of

devices and systems
1. INSTRUCTING  For instance: Word processors, VCRs, and vending machines

 The main benefit is that instructing supports quick and

efficient interaction
 Good for repetitive kinds of actions performed on multiple objects

WWW.ID-BOOK.COM
 WHICH IS EASIEST AND WHY?

WWW.ID-BOOK.COM
  Underlying model of having a conversation with another
human

 Ranges from simple voice recognition menu-driven systems to

more complex ‘natural language’ dialogs

2. CONVERSING  Examples include timetables, search engines, advice-giving

systems, and help systems

 Also virtual agents, chatbots, toys, and pet robots designed

to converse with you

WWW.ID-BOOK.COM
 WOULD YOU TALK WITH ANNA?

WWW.ID-BOOK.COM 57
  Allows users, especially novices, to interact with a system in a
way that is familiar to them

PROS AND  Can make them feel comfortable, at ease, and less scared

CONS OF
CONVERSATION  Misunderstandings can arise when the system does not know
how to parse what the user says
AL MODEL
 For example, voice assistants can misunderstand what children say

WWW.ID-BOOK.COM
WWW.ID-BOOK.COM
  Involves dragging, selecting, opening, closing and zooming
actions on virtual objects

 Exploit’s users’ knowledge of how they move and

manipulate in the physical world

3. MANIPULATING  Can involve actions using physical controllers (for example,

Nintendo Wii) or air gestures (such as, Microsoft Kinect) to
control the movements of an on-screen avatar

 Tagged physical objects (for instance, balls) that are

manipulated in a physical world result in physical/digital
events (such as animation)

WWW.ID-BOOK.COM
  Ben Shneiderman (1983) coined the term DM

DIRECT  Three core properties:

 Continuous representation of objects and actions of interest
MANIPULATION  Physical actions and button pressing instead of issuing commands
(DM) 
with complex syntax
Rapid reversible actions with immediate feedback on object of
interest

WWW.ID-BOOK.COM
  Novices can learn the basic functionality quickly

 Experienced users can work extremely rapidly to carry out a

wide range of tasks−even defining new functions

BENEFITS OF  Intermittent users can retain operational concepts over time

DIRECT  Error messages rarely needed

MANIPULATION  Users can immediately see if their actions are furthering

their goals, and if not, do something else

 Users experience less anxiety

 Users gain confidence and mastery and feel in control

WWW.ID-BOOK.COM
  Some people take the metaphor of direct manipulation too
literally

 Not all tasks can be described by objects, and not all actions
can be done directly
DISADVANTAGES  Some tasks are better achieved through delegating, for
OF DM example, spell checking

 Can become screen space ‘gobblers’

 Moving a cursor using a mouse or touchpad can be slower

than pressing function keys to do the same actions

WWW.ID-BOOK.COM
  Involves moving through virtual or physical environments
 Users can explore aspects of a virtual 3D environment
 Physical environments can also be embedded with sensors that
when detect the presence of someone will trigger digital or
physical events to happen
4. EXPLORING  Many examples of virtual environments, including cities,
parks, buildings, rooms, and datasets
 Enable users to fly over them and zoom in and out of different
parts

WWW.ID-BOOK.COM
 EXPLORING
DATA IN VR

Image courtesy of Kalev Leetaru, National Center for Supercomputing Applications, University of
WWW.ID-BOOK.COM Illinois.
  System takes the initiative to alert user to something that it
“thinks” is of interest
 System does this by:

 Detecting the location and-or presence of someone in a

vicinity and notifies them on their phone or watch,
 What it has learned from their repeated behaviors
RESPONDING  Examples:

 Alerts the user of a nearby coffee bar where some friends

are meeting
 User’s fitness tracker notifies them of a milestone reached
 Automatic system response without any requests made by the
user

WWW.ID-BOOK.COM
 POTENTIAL  Can get tiresome or frustrating if too many notifications or
CONS OF the system gets it wrong

SYSTEM-  What does it do when it gets something wrong?

INITIATED 


Does it apologize?
Does it allow the user to correct the advice or information?
NOTIFICATIONS

WWW.ID-BOOK.COM
  Direct manipulation is good for ‘doing’ types of tasks,
for example, designing, drawing, flying, driving, or
sizing windows
CHOOSING AN  Issuing instructions is good for repetitive tasks, for
INTERACTION example, spell-checking and file management
TYPE  Having a conversation is good for certain services, for
instance, finding information or requesting music
 Hybrid conceptual models are good for supporting
multiple ways of carrying out the same actions

WWW.ID-BOOK.COM
 Interaction type:

DIFFERENCE
BETWEEN  A description of what the user is doing when interacting with a
system, for example, instructing, talking, browsing, or responding

INTERACTION
TYPES AND Interface style:

INTERFACE  The kind of interface used to support the interaction, for instance,
STYLES command, menu-based, gesture, or voice

WWW.ID-BOOK.COM
 1. Command
2. Graphical
3. Multimedia
4. Virtual reality
5. Web
6. Mobile
7. Appliance
20 INTERFACE
8. Voice
9. Pen TYPES COVERED
10. Touch
11. Gesture
12. Haptic
13. Multimodal
14. Shareable
15. Tangible
16. Augmented Reality
17. Wearables
18. Robots and drones
19. Brain–computer interaction
20. Smart

WWW.ID-BOOK.COM 70
WHICH INTERFACE?
 Which interface to use will depend on task, users, context, cost, robustness, and so on
 Is multimedia better than tangible interfaces for learning?

 Is speech as effective as a command-based interface?

 Is a multimodal interface more effective than a mono-modal interface?

 Will wearable interfaces be better than mobile interfaces for helping people to find information in foreign cities?

 Are virtual environments the ultimate interface for playing games?

 Are shareable interfaces better at supporting communication and collaboration compared with using networked
desktop PCs?

WWW.ID-BOOK.COM 75