Pervasive/Ubiquitous

Computing
Class 1

Name: Bhuwan Chandra Joshi

Email/Web: [email protected]
Ubiquitous computing: smart devices, environments and
interaction

Overview

Introduction
Modelling the Key Ubiquitous Computing
Properties
Ubiquitous System Environment
Interaction
HCI in pervasive computing

Architecture Elements of pervasive

computing
Ubiquitous computing: smart devices, environments and
interaction

Introduction
What Ubiquitous Computing is
Computing is so profound to disappear into
background of our lives

What Ubiquitous Computing isnt

virtual reality

Difference
virtual reality puts people inside a computergenerated world
ubiquitous computing forces the computer to
live out here in the world with people

Ubiquitous computing: smart devices, environments and

interaction

Introduction
Pervasive
Ubiquitous
Invisible

Computing

Processing of data/signals
Automatic decision making
Decision based on past experience
Artificial Intelligence

Ubiquitous computing: smart devices, environments and

interaction

Introduction.
What are the current technology trends in pervasive/ubiquitous
computing?

Ubiquitous computing: smart devices, environments and

interaction

Introduction..
Trend: smaller, higher resource devices
Ear/microphone
Communicator
Display

AVrecorder

Ubiquitous computing: smart devices, environments and

interaction

Introduction.
Weisers 3 waves of computing

Ubiquitous computing: smart devices, environments and

interaction

Introduction
Real Life Scenario of pervasive
computing

In
In
In
In

Home
Automobile
Personal Communication
Workplace

Ubiquitous computing: smart devices, environments and

interaction

Introduction..
Some More Scenarios(Personal Memories)

Ubiquitous computing: smart devices, environments and

interaction

Introduction..

How can we enhance the personal memories service using

UbiCom?

Ubiquitous computing: smart devices, environments and

interaction

10

Introduction.
Some More Scenarios(Transport)

Ubiquitous computing: smart devices, environments and

interaction

11

Introduction.

How can we enhance the transport service

using UbiCom?

Ubiquitous computing: smart devices, environments and

interaction

12

Introduction.
Select & Buy
food at physical
or virtual market

Transport
food to
home store

Ubiquitous computing: smart devices, environments and

interaction

Put in home
store

13

Select food from

store, get &
transform food
into a meal

Consume
food

Introduction..
How can UbiCom enhance the foodstuff management scenario?

Ubiquitous computing: smart devices, environments and

interaction

14

Introduction.

Ubiquitous computing: smart devices, environments and

interaction

Introduction..
Utility regulation concerns energy, water, waste regulation by
end-users.
How can UbiCom enhance the Utility Regulation scenario?

Ubiquitous computing: smart devices, environments and

interaction

16

Overview

Introduction
Modelling the Key Ubiquitous Computing
Properties
Ubiquitous System Environment
Interaction
HCI in pervasive computing

Architecture Elements of pervasive

computing
Ubiquitous computing: smart devices, environments and
interaction

17

UbiCom: Different Combinations of Core

Properties versus a Single Definition
No single, absolute definition for ubiquitous computing.
Instead propose many different kinds of definition based
upon combining different sets of core properties
What core system properties would you propose to define
ubiquitous computing?
.

Ubiquitous computing: smart devices, environments and

interaction

18

UbiCom: Weisers 3 Internal

System Properties
3 main properties for UbiCom Systems were proposed
by Weiser (1991)
1. Computers need to be networked, distributed and
transparently accessible

In1991, little wireless computing, Internet far less

pervasive

2. Computer Interaction with Humans needs to be more

hidden

Because much HCI is overly intrusive

3. Computers need to be aware of environment context

In order to optimise their operation in their physical

& human environment.

Ubiquitous computing: smart devices, environments and

interaction

19

Devices: Extended set of Internal

System Properties
To which two additional properties are added:
4.Computers can operate autonomously, without
human intervention, be self-governed
5.Computers can handle a multiplicity of
dynamic actions and interactions, governed by
intelligent decision-making and intelligent
organisational interaction. This entails
some form of artificial intelligence.

Ubiquitous computing: smart devices, environments and

interaction

20

UbiCom: Different Combinations of Core

Properties versus a Single Definition
No single, absolute definition for ubiquitous computing.
Instead propose many different kinds of UbiCom based upon combining different
sets of core properties
Here are some examples proposed by others
Weiser (1991): distributed, iHCI, physical environment context aware
Ambient Intelligence (AmI), similar to UbiCom - intelligence everywhere?

Arts and Marzano (2003) define 5 key features for AmI to be embedded, context-aware, personalised,
adaptive and anticipatory.

Buxton (1995): ubiquity and transparency

Endres et al. (2005): distributed mobile, intelligence, augmented reality
Millner (2006): autonomy, IHCI
Etc.
Exercise: Do your own survey of UbiCom definitions and highlight the properties
they define.
.
Ubiquitous computing: smart devices, environments and
interaction

21

Five main properties for UbiCom

Ubiquitous computing: smart devices, environments and

interaction

22

UbiCom System Properties:

Distributed
Networked ICT Devices
Transparency
Openness

Ubiquitous computing: smart devices, environments and

interaction

23

Distributed System: sub-properties

Often designed middleware, set of generic services

Universal, Seamless, Heterogeneous
Networked
Synchronised, Coordinated
Open
Transparent, Virtual
Mobile, Nomadic

Ubiquitous computing: smart devices, environments and

interaction

24

Internal System Properties: iHCI

Concept of calm / disappearing computer has several
dimensions
Implicit (iHCI) versus Explicit HCI
Embodied Virtuality as opposite of VR (people in virtual world)

Ubiquitous computing: smart devices, environments and

interaction

25

iHCI: Sub-Properties

Implicit Human Device Interaction (iHCI)

Non-intrusive, Hidden, Invisible, Calm, computing
Tangible, Natural
Anticipatory, Speculative, Proactive
Affective, Emotive
User-aware
PostHuman
Sense of Presence
Immersed, Virtual , Mediated reality

Ubiquitous computing: smart devices, environments and

interaction

26

Internal System Properties:

context-aware
Often Context-based ubiquity rather than global
ubiquity
Why?

3 Main Types of Context

Physical Environment Context
Human Context (or User context or person context)
ICT Context or Virtual Environment Context:
Ubiquitous computing: smart devices, environments and
interaction

27

Context-aware: sub-properties

Also referred to as Sentient, Unique, Localised, Situated

Presupposes sensing of environment to be aware of its
context
Adaptive: active versus passive context-aware
Types of environment aware
Person-aware, User-aware, Personalised, Tailored,
Environment-aware, Context-aware, Physical contextaware
ICT infrastructure aware
(See Chapter 7)
Ubiquitous computing: smart devices, environments and
interaction

28

Internal System Properties:

Autonomy
Challenge 1: increasing computer systems can
overload humans humans become a bottleneck
What can be done?

Ubiquitous computing: smart devices, environments and

interaction

29

Internal System Properties:

Autonomy
Challenge 2: automated system can be become to
complex to maintain
- must reduce maintenance of complex systems

What can be done?

Ubiquitous computing: smart devices, environments and

interaction

30

Autonomy: Sub-Properties
Automatic
Embedded, Encapsulated
Embodied
Resource-constrained
Untethered, Amorphous
Autonomic, Self-managing, self-star
properties
Emergent, self-organising
Ubiquitous computing: smart devices, environments and
interaction

31

Internal System Properties:

Intelligence
Intelligent UbiCom systems (IS) can:
Act more proactively, dynamically & humanely through:
Model how their environment changes when deciding how it
acts.
Goal-based / planning
Reasoning for re-planning
Handle uncertainty.
semantic based interaction
etc
Ubiquitous computing: smart devices, environments and
interaction

32

Individual Intelligence: SubProperties

Referred to as Intelligent Systems, AI, agent-based system
etc.
Sub-Properties (sub-types of individual intelligence)
Reactive, Reflex
Model-based,
Rule/Policy-based
Logic/Reasoning
Goal-oriented, Planned, Proactive
Utility-based, Game theoretic
Learning, Adaptive
(See Chapter 8)
Ubiquitous computing: smart devices, environments and
interaction

33

Multiple Intelligence: SubProperties

Referred to as Distributed AI, Multi-Agent Systems, Collective
or Social Intelligence
Sub-Properties
Cooperative , Collaborative, Benevolent
Competitive, self-interested, antagonistic, adversarial
Orchestrated, Choreographed, Mediated
Task-sharing
Communal, shared meaning
Shared knowledge
Speech-act based , Intentional, Mentalistic
(See Chapter 9)
Ubiquitous computing: smart devices, environments and
interaction

34

Overview

Introduction
Modelling the Key Ubiquitous Computing
Properties
Ubiquitous System Environment
Interaction
HCI in pervasive computing

Architecture Elements of pervasive

computing
Ubiquitous computing: smart devices, environments and
interaction

35

UbiCom System versus ICT

System focus
Conventional focus on ICT systems
Conventional use a restrict view of physical environment
interaction:
Conventional ICT systems often require humans in the loop
UbiCom represents a powerful shift in computation:

Ubiquitous computing: smart devices, environments and

interaction

36

Different Degrees of HCI

Ubiquitous computing: smart devices, environments and interaction

37

Different Degrees of HCI

From less to more C Interaction with H

H2H: human interaction
H2C or explicit (e)HCI:
C2H or implicit (i)HCI:
C2C:
Ubiquitous computing: smart devices, environments and
interaction

38

Different Degrees of CPI

Ubiquitous computing: smart devices, environments and

interaction

39

Different Degrees of CPI

From less to more C Interaction with P

P2P
Physical interaction (No ICT mediation)

C2P / CA (Physical Env. Context-aware)

C Senses P. C Aware of Ps Context

P2C/AR/MR

C augments or mediates Ps reality.

C actively adapts to Ps context

C2C /VR

Virtualisation of reality facilitated by C

Ubiquitous computing: smart devices, environments and

interaction

40

Overview

Introduction
Modelling the Key Ubiquitous Computing
Properties
Ubiquitous System Environment
Interaction
HCI in pervasive computing

Architecture Elements of pervasive

computing
Ubiquitous computing: smart devices, environments and
interaction

41

UbiCom System Model: Smart DEI

No single type of UbiCom system

Different UbiCom systems support:
3 basic architectural design patterns for UbiCom:
smart Devices, smart Environments, smart Interaction

Smart means systems are:

active, digital, networked, autonomous, reconfigurable,
local control of its own resources, e.g., energy, data
storage etc.
Ubiquitous computing: smart devices, environments and
interaction

42

UbiCom System Model: Smart DEI

Device Trends
Increasing capability to
embed devices in the
physical environment

Increasing capability to
manufacture low power,
micro, more complex devices
Use more complex, multifunctional, mobile, personalised
(& private) smart devices to
ease access to & embody
services rather than just to
virtualise them
e.g., phone is also a
camera, music player, is
also a printer??

Use smarter environments

to sense and react to events
such as people, with mobile
devices , entering & leaving
controlled spaces
e.g., walls can sense camera
is recording and modify
lighting to improve
recording

Ubiquitous computing: smart devices, environments and

interaction

Ubiquitous
43 Computing

Increasing capability for

more interoperable
distributed mobile devices

Use more service access

devices with simpler
functions and allow them to
interoperate smarter
interaction between devices
e.g., camera can interconnect
to phone to share recordings,
direct to printer to print

Smart DEI Model: types of smart

device
3 main types of UbiCom system may themselves
contain smart sub-systems at a lower level of
granularity
e.g., a smart environment device may consist of smart
sensors and smart controllers

Ubiquitous computing: smart devices, environments and

interaction

44

UbiCom System: Smart Sub-Systems or

Components

Ubiquitous computing: smart devices, environments and

interaction

45

Smart Device Form Factors

Devices tend to become smaller and

lighter in weight, cheaper to produce.
Devices can become prevalent, made more
portable and can appear less obtrusive.
Weiser proposed a range of device sizes
1.Smart Tabs
2.Smart Pads:
3.Smart Boards:
Ubiquitous computing: smart devices, environments and
interaction

46

Smart Device Form Factors (2)

Form Factors can be extended to support

4.Smart Dust
5.Smart Skins
6.Smart Clay

Ubiquitous computing: smart devices, environments and

interaction

47

Design challenges in Using Varying

Form Factors
What are the design Challenges?

Ubiquitous computing: smart devices, environments and

interaction

48

Smart Device: Variations

Many variations or sub-types of smart devices,

e.g. smart mobile device, smart environment device

Use different combinations of sub-components
Support different combinations of UbiCom properties
Interact in different types of environment
Take different form factors

Ubiquitous computing: smart devices, environments and

interaction

49

Smart Device Variations: Smart

Mobile Device
Many flavours or variants of these basic designs
One of the most important variants of the smart
device is the smart mobile device
Combines several main types of UbiCom property:
Note mobile devices are sometimes taken to be
synonymous with wireless devices but these are
different

Ubiquitous computing: smart devices, environments and

interaction

50

Smart Mobile Devices

Multi-purpose ICT devices, operating as a single portal to

multiple services
eases access & interoperability versus decreased openness
Usually personalised devices, specified owner.
Locus of control and user interface resides in the smart
device.
Main characteristics: mobility, open service discovery,
intermittent resource access.

Ubiquitous computing: smart devices, environments and

interaction

51

Smart Devices: Mobility

Many dimensions for mobility

Static:
Accompanied:
Wearable:
Embedded (into objects):
Implanted (into humans):
Untethered or unanchored:
Division between statics and mobiles can be more finely
grained and multi-dimensional

Ubiquitous computing: smart devices, environments and

interaction

52

Volatile Service Access

Devices access software services and hardware

intermittently . Why?
Devices can dynamically discover services
available.
Context-aware discovery can improve basic
discovery.
Asymmetric remote service access, more
downloads than uploads, tends to occur.
Ubiquitous computing: smart devices, environments and
interaction

53

Smart Mobile Devices: ContextAware

Can use contexts to filter information & service
access.
Often designed to work with a reference location in
physical environment called a home location,
Mobile devices are ICT resource constrained.
Mobile devices tend to use a finite internal energy
cache.
Devices configuration, operation tends to be
personalised
Ubiquitous computing: smart devices, environments and
interaction

54

Smart Environment (Devices)

Definitions
Consists of a set of smart devices specialised to interact with their
(virtual, phys., human) environments.
Typically, embedded single task devices; not MTOS devices
Can automatically respond to or anticipate users, using iHCI
Smart environments support bounded, local user context
Smart environment devices may also be:

fixed versus anchored mobile versus unanchored mobile devices

macro to micro to nano

See Chapter 6, 7

See Chapter 12

Smart environment device design issues what?

Smart environment device management issues what?

Ubiquitous computing: smart devices, environments and

interaction

55

Types of Smart Environment

Device Interaction

Tagging and Annotating

Sensing & monitoring
Filtering
Adapting
Controlling
Assembling
Regulating

Ubiquitous computing: smart devices, environments and

interaction

56

Smart Interaction

Additional type of design is needed to knit together many

individual system components and interactions.
Smart interaction promotes unified & continuous interaction
model between UbiCom applications & their UbiCom
infrastructure, physical world & human environments.
Internal self-organising system vs. externally driven system
Components can interact cooperatively versus competitively
Several benefits to designs based upon set of interacting
components:
A range of levels of interaction between UbiCom System
components
Distinguish between (basic) interaction and (smart) interaction
Ubiquitous computing: smart devices, environments and
interaction

57

Basic Interaction

Typically involves two interlinked parties, a

sender and a receiver.
Sender knows things in advance:
Two main types of basic interaction
synchronous versus asynchronous

Ubiquitous computing: smart devices, environments and

interaction

58

Smart Interaction

Smart Interaction extends basic interactions

as follows.
Coordinated interactions
Cooperative versus competitive interaction
Policy and Convention based Interaction
Dynamic Organisational Interaction
Semantic and Linguistic Interaction
Ubiquitous computing: smart devices, environments and
interaction

59

Smart DEI Model Summary

Basic Smart Device has many variations

6 different physical form factors

5 different groups of internal properties & over 70 subproperties

Multiple flavours of smart device,

e.g., Smart Mobile type of Smart device

e.g., Smart Environment type of Smart Device

UbiCom System interact across 3 main types of

environment: physical, virtual & human
System of systems models in terms of multiple device
combinations and interactions
Ubiquitous computing: smart devices, environments and
interaction

60

Common Myths of Ubiquitous

Computing
1.
2.
3.
4.
5.
6.
7.

There is a single definition which will accurately

characterises Ubiquitous Computing
The ideal type of Ubiquitous computing is where all the
properties of ubiquitous must be fully supported
Ubiquitous computing means making computing
services accessible everywhere.
Ubiquitous computing is boundless computing
Ubiquitous computing is just about HCI
Calm Computing should be used as a model for all HCI.
Ubiquitous computing is just about augmenting reality

Ubiquitous computing: smart devices, environments and

interaction

61

Common Myths of Ubiquitous

Computing 2
8. Ubiquitous computing is just distributed or virtual
computing
9. Ubiquitous computing is just mobile wireless
computing
10.Ubiquitous computing is just about smart
environments
11.Ubiquitous computing need to be highly autonomous
systems
12.Ubiquitous computing is just about physical world
context-awareness
13.Ubiquitous computing is just distributed
intelligence
14.Ubiquitous computing systems can operate
effectively in all kinds of environments:
Ubiquitous computing: smart devices, environments and
interaction

62

Exercise: Applying New Concepts

What are the main properties of UbiCom?
What are the latest trend in UbiCom?
New Application area of UbiCom?

Ubiquitous computing: smart devices, environments and

interaction

63