HCI 2018 (7/10) HCI Engineering. UI Evaluation. Models

Master on Software Engineering :: Human-Computer Interaction
Dr. Sabin-Corneliu Buraga – profs.info.uaic.ro/~busaco/
Human-Computer Interaction
model-based user interaction
in the context of software engineering
labs.ideo.com/2014/09/19/digital-tools-for-design-research/

“If you are not embarrassed by the first version of
your product, you’ve launched too late.”
Reid Hoffman

How we can evaluate the usability (quality)
of interactive systems?

UI quality
Presentation of information
multiplicity of devices & representations

UI quality
Presentation of information
input/output reusability
use output produced by one action as input for another

UI quality
Ordering of task planning
multiplicity of user roles
regular user versus administrator

UI quality
multiplicity of execution paths
menu versus toolbox versus shortcuts

UI quality
non-preemptiveness
degree of freedom for users to decide what’s next

UI quality
reachability
possibility to navigate in the system
(e.g., undo, redo, breadcrumb pattern)

UI quality
observability versus browsability

UI quality
Adaption of interaction
reconfigurability
system ability to support user personalization

UI quality
reconfigurability
system ability to support user personalization
configuration versus personalization
affects system’s
function & performance
relevant to
a certain user

UI quality
adaptivity
system ability to support automated adaptation

UI quality
migrability
system ability to transfer responsibility
from one user to another,
among users,
among users and systems/platforms

UI quality
plasticity
system ability to adapt to the context of use
while preserving predefined usability properties

UI quality
Standards
ISO/IEC 9126-1 (2001) – quality model
ISO/IEC 9126-2 (2003) – external measures
ISO/IEC 9126-3 (2003) – internal measures
ISO/IEC 9126-4 (2004) – quality in use measures

UI quality
Quality in use is the user’s view of the quality of
the software product when it is used in a specific
environment and a specific context of use
ISO/IEC 9126-4 (2004)

UI quality
External quality is the totality of characteristics
of the software product from an external view
ISO/IEC 9126-2 (2003)

UI quality
External quality is the totality of characteristics
of the software product from an external view
measured and evaluated in the testing phase
within a simulated environment,
by using external metrics

UI quality
Internal quality is the totality of attributes of
a product that determine its ability to satisfy
stated and implied needs when used
under specified conditions
ISO/IEC 9126-3 (2003)

UI quality
Internal quality is the totality of attributes of
a product that determine its ability to satisfy
stated and implied needs when used
under specified conditions
remains unchanged until the system redesign

UI quality
For details, consult:
E. Law, E. Hvannberg & G. Cockton (Editors),
Maturing Usability. Quality in Software, Interaction
and Value, Springer, 2008

Criteria
Scapin & Bastien, 1997; Vanderdonckt, 1995
compatibility
consistency
work load
adaptation
dialog control
guidance
error management
Usability evaluation

Important aspects
a priori and/or a posteriori
design versus testing (evaluation)

Example:
the lack of explanatory messages in the case of Web links
Click here to go to the UAIC main page.
Click here to visit our HCI Website.
Click here for details about this event.

Example:
the lack of explanatory messages in the case of Web links
Click here to go to the UAIC main page.
Click here to visit our HCI Website.
Click here for details about this event.
“click here”
anti-pattern

Example:
adoption of different placement strategies
for content and navigational methods

“Optimal” placement of links for the main page
of a Website (Shaikh & Lenz, 2006)
case study

“Optimal” placement of links for the subsidiary pages
(Shaikh & Lenz, 2006)
case study

“Optimal” placement of the internal search tools
case study

“Optimal” placement of the advertisements
case study

One of the most used general technique:
eye tracking
30—60 minute interviews, in which users are asked
to complete real-life tasks online, while the eye tracking
monitor captures their eye movements
J. Nielsen, K. Pernice, Eyetracking Web Usability, New Riders, 2009
www.nngroup.com/reports/how-to-conduct-eyetracking-studies/

What areas of the page draw users’ attention?
www.evocinsights.com

Do users notice and use key navigation elements?

Do users notice key marketing elements
and do they recall them?

Are users successful in completing
a particular task?

Which pages, ads, videos, or images
do users prefer?

What do users expect to find
but overlook on the site?

One of the most used general technique:
eye tracking
several open-source software solutions:
EyeWriter – eyewriter.org/developer/
ExpertEyes – code.google.com/p/experteyes/
Gaze Tracking – sourceforge.net/projects/gazetrackinglib/
OGAMA (OpenGazeAndMouseAnalyzer) – www.ogama.net
PyGaze – www.pygaze.org

PUI – Plastic User Interfaces
adaptation to the context of use
while satisfying predefined usability properties of interest

Calitate
D. Thevenin, J. Coutaz & G. Calvary, 2004

PUI – Plastic User Interfaces
levels of adaptation:
lexical
sintactic
semantic
performed tasks
concerning the user goals

Accessibility
successful access to information and IT applications
by people having special needs
visual, hearing, motor, cognitive, seizure disorders
discussion

Accessibility
realities:
20% of US population have a certain type of disability
10% of persons are having severe problems
4% of world-wide humans have major sight problems
discussion

discussion
general usability
affects all users
physical barriers
affects only disabled people
variables
inconveniences for some
barriers for others

discussion

discussion
cognitive issues

Accessibility
offering alternative means in order to facilitate
the completion of users’ tasks for people having
temporary (on short/long term) or permanent problems
discussion

Accessibility
general strategies:
textual descriptions of multimedia content
(images, audio, animations, video, 3D)
discussion

Accessibility
general strategies:
a proper (logical) organization
of the content and navigational paths
discussion

Accessibility
general strategies:
support for keyboard-only interaction
discussion

Accessibility
general strategies:
using standardized formats
discussion

Accessibility
guidelines and tools – concerning Web applications:
www.w3.org/WAI/
www.webaim.org
discussion

discussion
WAVE (Web Accessibility Evaluation Tool)

Accessibility
see also Ire Aderinokun,
The Accessibility Cheatsheet (2015)
https://bitsofco.de/the-accessibility-cheatsheet/
discussion

Internationalization (I18N) and localization (L10N)
language
linguistic preferences of the users
locale
cultural preferences concerning number and date
formatting, currency, etc.
discussion

“If the user can’t read the description of the preference,
he/she doesn’t even have a chance to make a choice.”
Achim Ruopp, 2007
discussion

it is desirable to use complete phrases
in order to give a better translation
discussion

Web guidelines & resources:
www.w3.org/International/
www.globalbydesign.com/
discussion

discussion
avoid absolute width (texts in other
languages could have different lengths)
avoid pictures
(use translatable
text)

analytical evaluation
controlled experiment
field study
formative evaluation
heuristic evaluation
predictive evaluation
summative evaluation
usability laboratory
usability studies
user testing
Usability evaluation (Rogers, 2007)

Multiple existing heuristics
introduced by Jakob Nielsen (1994)
www.nngroup.com/articles/ten-usability-heuristics/
visibility
match
control
consistency
prevention
recognition
minimalism
flexibility recover
help

proposed by Bruce Tognazzini (revised in 2014)
asktog.com/atc/principles-of-interaction-design/
Aesthetics
Anticipation
Autonomy
Color
Consistency
Defaults
Discoverability
Efficiency of the User
Explorable Interfaces
Fitts’ Law
Human-Interface Objects
Latency Reduction
Learnability
Metaphors
Protect Users’ Work
Readability
Simplicity
State: Track it
Visible Interfaces

context: mobile computing (E. Bertini et al., 2006)
1. Visibility of system status & device findability
2. Match between system and the real world
3. Consistency and mapping
4. Good ergonomics & minimalist design
5. Ease of input, screen readability and glancability
6. Flexibility, efficiency of use and personalization
7. Aesthetic, privacy and social conventions
8. Realistic error management

The evaluator could not substitute the real users
an UI has usability problems
only if its final common users have these problems
code inspection vs. code testing
Usability evaluation (Rogers, 2007)

User testing
UI quality evaluation could be performed by users
user testing

User testing
Aspects of interest (J. Dumas & J. Fox, 2008)
the focus is on usability

User testing
the participants are (potential) end users

User testing
there is a real product/service to be evaluated

User testing
the participants perform tasks,
usually while thinking aloud

User testing
the data are recorded and further analyzed

User testing
a usability test plan HTML5 template
profs.info.uaic.ro/~stefan.negru/usability/

User testing
Considered metrics
effectiveness, efficiency & satisfaction

User testing
Considered metrics
effectiveness, efficiency & satisfaction
+
fun, challenge & stimulation

User testing: planning (Adaptive Path, 2001)
t –2 weeks Determine test audience, start recruiting immediately
t –2 weeks Determine feature set to be tested
t –1 week Write first version of guide, discuss with team, check on recruiting
t –3 days Write second version of guide, recruiting should be completed
t –2 days Complete guide, schedule practice test, set up and check equipment
t –1 day Do practice test in the morning, adjust guide/tasks as appropriate
t Test (usually 1-2 days, depending on scheduling)
t +1 day Discuss with observers, collect copies of all notes
t +3 days Watch all video recordings, take notes
t +1 week Combine notes, write analysis
t +1 week Present to team, discuss and note directions for further research

User testing
Methods:
formative evaluation
field study
controlled experiment

User testing
Formative evaluation
finding problems for next iteration of the design project

User testing
prototype/implementation is evaluated
within a controlled environment (lab),
with focus on specific tasks

User testing
users, facilitators, observers offers qualitative data
(usability issues)

User testing
Field study
tries to find problems with respect to a given context

User testing
Field study
evaluates the (preliminary) UI in a concrete context,
with focus on real tasks

User testing
Field study
evaluates the (preliminary) UI in a concrete context,
with focus on real tasks
offers qualitative annotations

User testing
Controlled experiment
to test a hypothesis
e.g., interface X is easier to be used than interface Y

User testing
could be used to evaluate a preliminary UI,
in lab rigorous conditions, with focus on specific tasks
has one or more conditions (independent variables)
and measures (dependent variables)

User testing
gives quantitative information:
time of reaction, error rate, user satisfaction,…
Ron Kohavi et al., “Practical Guide to Controlled
Experiments on the Web”, KDD 2007, ACM Press, 2007
exp-platform.com/Documents/GuideControlledExperiments.pdf

User testing
Advices + aspects of interest (Lukas Mathis, 2011)
do not influence the tester
avoid stressful situations
consider the ethics of conducting tests

User testing
A/B testing
comparing 2 versions of an UI element
or an entire Web page
in a time interval to see which performs better

User testing
A/B testing
comparing 2 versions of an UI element
or an entire Web page
in a time interval to see which performs better
performance criterion = conversion rate
(from visitors to goal achievers)

User testing
L. Swanson, “A Primer on A/B Testing”, A List Apart, 2011
www.alistapart.com/articles/a-primer-on-a-b-testing/
consult also elem.com/~btilly/effective-ab-testing/

User testing
A/B testing
multivariate testing – different versions of individual
parts of the design are tested at the same time

a multivariate test (Julian Gaviria, 2017)
discussion
julian.is/article/exit-intent-popups/

User testing
A/B testing
several tools:
Convert
Genetify – github.com/gregdingle/genetify/wiki
Optimizely
Unbounce
Vanity – vanity.labnotes.org
Visual Website Optimizer

User testing
real case studies:
vwo.com/resources/casestudy/

User testing
Pilot study
a trial run of an experimental procedure,
not expected to produce valid research data

User testing
Remote testing
by using a screen-sharing software
for details, read N. Bolt, “Quick and Dirty
Remote User Testing”, A List Apart, 2010:
alistapart.com/article/quick-and-dirty-remote-user-testing

User testing
Remote testing
the tester’s environment cannot be typically controlled
also, the tester might get distracted during the test

Predictive evaluation
Having a correct model of the interaction
between users and computers, we can predict
the usability of a system, without the need of
designing and/or testing a concrete UI

User testing could only relieve certain problems,
but can not explaining them

Classical models:
probabilistic – e.g., Bayes networks
GOMS (Goals, Operators, Methods, Selection rules)
CMP-GOMS (Cognitive-Motor-Perceptual)

task graph used by CMP-GOMS model
the critical path is the path of tasks
having the longest time of execution

Directions of research
objectives beyond productivity
hedonomics (Hancock et al., 2005)

Directions of research
RITE – Rapid Iterative Test and Evaluation
used by Microsoft (M. Medlock et al., 2002, 2005)
focused on fixing usability problems
rather than finding them

For further information, study:
G. Cockton, Usability Evaluation, “The Encyclopedia of
Human-Computer Interaction” (2nd Edition), 2015
www.interaction-design.org/encyclopedia/usability_evaluation.html
C. Bank & J. Cao, The Guide to Usability Testing,
UXPin, 2014
www.uxpin.com/studio/ebooks/guide-to-usability-testing/

How about designing interactivity
by using model-driven approaches?

Reality
Major differences between software engineering
and the effective UI development

Reality
Target-applications & domains of interest
personal productivity, business, entertainment, etc.

Reality
Notations and engineering tools
(formal) methods for software development

Reality
Multiple interaction paradigms
textual, graphical, multimedia, natural, 3D,…

Reality
Evolution of the interactive apps: context of use = (U, P, E)
Time
Platform
User(s)
Environment
Language
Jean Vanderdonckt, 2006

Reality
UI #12UI #11UI #10UI #9Application 3
Platform #4Platform #3Platform #2Platform #1
Multiple models to be considered (Abrams et al., 2001)

Reality
Platform #4Platform #3Platform #2Platform #1
Application 1
Application 2
Application 3
UI model #1
UI model #2
UI model #3
Platform #1
Platform #2
Platform #3
Platform #4
Platform model #1
Platform model #2
Platform model #3
Platform model #4
Multiple models to be considered (Abrams et al., 2001)

Model-based UI design
MDA (Model-Driven Architecture)
www.omg.org/mda/

Model(s) facilitate(s) an abstract view
of the interaction
separation of concerns, ability of correlation
parsability, editability
if possible, human readability

Models
explicitly capture knowledge about UI and
interactive applications with appropriate abstractions

Method
structures the definition and use of underlying models
in a stage-wise approach
example: agile usability (Scott Ambler, 2008)
www.agilemodeling.com/essays/agileUsability.htm

Supporting tools
support the use of the method by providing tools
for models and their related operations
ideally, one model should be supported by
at least one tool

Proiectarea bazată pe modele
Models used in the processes of UI design
(P. Forbrig et al., 2004)

Proiectarea bazată pe modele
Using notations to model tasks to be performed by users
(L. Marucci et al., 2004)

Calitate
Process Reference Framework
for the development of plastic user interfaces

Environment T
Final user
Interface T
Concrete user
Interface T
Task and
Domain T
Abstract user
Interface T
T = Target context of use
Concrete user
Interface S
Final user
Interface S
Task and
Domain S
Abstract user
Interface S
S = Source context of use
Reification
Abstraction
Reflection
Translation
User S Platform S Environment S Platform TUser T

Task &
Concepts
Abstract
UI
Concrete
UI
Final UI Reformating
Transcoding
Recoding
Respecification
Retasking
Restructuration
Program
understanding
Redocumentation
Reverse Engineering
Design recovery
Reengineering
Revamping
Bouillon (2006)

CADUI – Computer-Aided Design of User Interface
(formal) descriptions of interactive systems,
in terms of existing meta-models,
used to design and deploy multiple user interfaces

Interac-
tive
System
Model of
the IS
Model of
the IS’
Interac-
tive
System’
Transfor-
mation
Property
checking
Jean Vanderdonckt

At the abstract level,
the specification of the interactive system is given
by CIM (Computation-Independent Model)

mission statement, function reference tree, use cases
interaction requirements, concurrent tasks trees
Requirements
Model

see also “Requirements Engineering from an HCI Perspective” (A. Sutcliffe, 2015)
www.interaction-design.org/encyclopedia/requirements_engineering.html
Requirements
Model

The platform independent description is based on
PIM (Platform-Independent Model)

The platform independent description is based on
PIM (Platform-Independent Model)
object model
dynamic model
functional model
presentation model
Conceptual
Model

The development focused on a specific platform is
based on PSM (Platform-Specific Model)

The development focused on a specific platform is
based on PSM (Platform-Specific Model)
uses software tools of transformation (compilation)
based on an application model
Compilation
Model

The concrete implementation is using
CM (Code Model)

The concrete implementation is using
CM (Code Model)
concerns creation/generation of the source-code:
interface tier
application tier
persistence tier
integration

Task and
Domain
Abstract
User Interface
Concrete
User Interface
Final
User Interface
Abstract
User Interface
T1 RenderingT2
T3
Task and
Domain
Abstract
User Interface
Concrete
User Interface
Final
User Interface
Abstract
User Interface
T1 RenderingT2
T3

Task and
Domain
Abstract
User Interface
Concrete
User Interface 1
(2-D Desktop)
Final
User Interface
T1
Rendering
T2
T3 Concrete
User Interface 2
(2-D small display)
Concrete
User Interface 3
(auditory)
Final
User Interface
Final
User Interface
Final
User Interface
Concrete
User Interface
Task and
Domain
Abstract
User Interface
T4
Rendering
Rendering
Rendering
T5
T6 T7
Task and
Domain
Abstract
User Interface
Concrete
User Interface 1
(2-D Desktop)
Final
User Interface
T1
Rendering
T2
T3 Concrete
User Interface 2
(2-D small display)
Concrete
User Interface 3
(auditory)
Final
User Interface
Final
User Interface
Final
User Interface
Concrete
User Interface
Task and
Domain
Abstract
User Interface
T4
Rendering
Rendering
Rendering
T5
T6 T7

(Meta)languages for describing UI
MARIA
(Model-based lAnguage foR Interactive Applications)
UIML – User-Interface Modeling Language
UsiXML – USer Interface eXtensible Markup Language
chess.eecs.berkeley.edu/pubs/940/dreams-18-09-2012.pdf

(Meta)languages for describing UI
FXML (JavaFX)
Glade XML
JSX (React.js)
XAML – eXtensible Application Markup Language
XUL – eXtensible User-interface Language

Model to Model
Platform
Independent
Model (PIM)
Platform
Specific
Model (PSM)
Model to Code Source code
MDA (Model-Driven Architecture)
UsiXML
Computing
Independent
Model (CIM)
Model to Model
UsiXML model:
Abstract user
interface
UsiXML model:
Concrete user
interface
Rendering Final user
interface
UsiXML
models: task,
domain
Graph
transformations
Graph
transformations

designing the abstract UI of a computer game
user task modelling via a specific tool
(S. Tofan, A. Pradais, S. Buraga, 2009)

Support for multiple displays
(Grolaux & Vanderdonckt, 2005)
detachmigrateplastify
case study

case study

detachmigrateplastify
for migration, rules and/or design patterns could be used
case study

the use of rules – example:
 x  Ts : x = input and
(x.type = “text” or x.type = “password” or x.type = NULL)
 AddNode (“textComponent”, idText)
where idText = NodeValue (Tt)
case study

case study
substitution removal moving
examples of actions that could be performed for migration
and/or creating PUI – revisit responsive Web design

For other examples,
read G. Meixner, G. Calvary, J. Coutaz (Eds.),
Introduction to Model-Based User Interfaces,
W3C Working Group Note (2014)
www.w3.org/TR/mbui-intro/

148 Polytechnic University of Valencia – Doctoral Course, Valencia, November 2006
Target Applications,
Domains
Notations &
Engineering Tools
User Interface
Interaction Techniques
Invisible UI
No more programming: only models
All Applications near future
No Interaction TechniqueAutomated, batch systems Nothing
Character UIsBusiness applications Screen definitions
Graphical User
Interfaces
Information systems Entity-relationship
Attribute model
State-transition diagrams
Multi-platform
User Interfaces
Web, desktop,
mobile apps
Task model,
context model, UML,…
Virtual Reality
User Interfaces3D Applications Scene model
Mixed Reality
User Interfaces
Command &
control systems,
games
World models
Tangible UIs
Embodied UIsPhysical modelsEmbedded
systems
Palanque,2002&Vanderdonckt,2006

“Conclusion”
interactivity in the context of software engineering
UI quality, testing, models

next episode: data visualization

HCI 2018 (7/10) HCI Engineering. UI Evaluation. Models

More Related Content

What's hot (19)

Similar to HCI 2018 (7/10) HCI Engineering. UI Evaluation. Models (14)

More from Sabin Buraga (20)

Recently uploaded (20)

HCI 2018 (7/10) HCI Engineering. UI Evaluation. Models