Multimedia Applications
Multimedia Applications
2
Big picture: from creation to consumption
3
A short introduction
4
2020 – 50 bilion devices in Internet
Internet of Things
Machine2machine
Future Internet
NGN
5
OTT g
growth and forecasts
Smart TV (source: GII – Global Information company)
T i l play
Triple l isi on the
th rise
i (source : Digital TV research)
400 million new subscribers worldwide (China almost 50%)
Mostly cable (66%), IPTV also on the rise
income : USA 59 billion $, France and Germany 4 billion $,…)
pay TV services
i (2011-2015)
(2011 2015) (source : Digital TV research)
STB growth 17%
second screen device growth 400%
(smart) TV
Trends and services
7
Current situation-
situation- ((smart)) TV
Many devices , standards, manufacturers
Content recommendations
By other users through FB
FB, twitter
System recommendations (personalisation)
Most wanted services ((3))
Additional content
Catch-upp TV
“Behind the scenes”
Recipes, tourist destinations
I t
Interactive
ti adds
dd related
l t d to
t the
th events
t on the
th screen
Linking of related sites (wiki)
…
Example: Forecasting of the dead in the Walking Dead series
20% of viewers have installed the 2nd screen app Walking Dead
Most wanted services ((4))
Extinction of linear TV?
Linearna TV is beingg watched more than
ever
VoD is also on the rise
Social networks
Usage depends on age and culture
USA 60% , GB 86% of viewers use
tablets while watching TV
Not in a second screen fashion
Most are using Facebook, Twitter,
wikipedia,
ped a, games
ga es
Viri:
•GFK media efficiency panel 2011
(20000 uporabnikov)
• RedBee media (2000 uporabnikov),
•Nielsen mobile study 2012
2nd screen: a g
good example
Cinergy
Trend
2nd screen advertising
Personalised addvertising, game based advertising
Much easier interaction as on a TV screen
M i from
Moving f “push”
“ h” to
t “pull”
“ ll” model
d l
Requires a good connection between TV content and 2nd screen
content
From technological perspective
From consumption perspective
Advertising
g – new approaches ((2))
Example:
Magic
g Ruby
y for “Sons of Anarchy”:
y
Advertising and sales of items from the show (clothes, eye glasses,
motorbikes,…)
Information about actors in the series
Cooking recipes for food
Synchronises with the current show on the airm(audio finger-printing)
Mobile services
19
Mobile internet
Full web browsers
Rich HTML5 web apps
Flash
Demanding for low performance mobile devices
Existing flash apps not adapted for touch interaction
Not supported on iOS
Only partially supported on Android
No support on 4.1 and higher
Android iPhone,
phones Facebook iPod Touch
>100M (>100M)
845mio
users
Netbooks
(39 mio)
Readers
e- books(3G)
iPad (3G) ((Kindle))
YouTube ( 2011))
More than 3 bilion hours watched
per month
1000 bilion watched in 2011
25% from mobile devices
C t t fformats
Content t
HD 480p, 720p, 1080p (1920x1080), 4k
Do 24MB ((200 Mbit)) for each minute of
video
NFC services
2011 first Android phones with NFC
A long list...
list
Most of the apps are developed by
3rd p
party
y developers
p
Also increases value of the platform
Other apps - LTFE ((2))
RTV SLO Live
40
What are the big-ones up to?
(
(smart
t TV&mobile
TV& bil )
Microsoft
XBOX as central home platform
Games
OTT content!!!
U
Users can access over 200
200.000
000 movies,
i shows…
h
Netflix, BBC iPlayer, HBO, Hulu, sport1, sky, Canal+,…
OTT content
t t repository
it
Netflix, Hulu, YouTube, ESPN, MLB, Blockbuster,…
Hot topics:
Gesture and voice control
User recognition
http://www.samsung.com/us/2012-smart-tv/#navigation
http://www.samsung.com/us/2012 smart tv/#navigation
Google
g
Google TV
First version a failure
New version on the market (July 2012 GB, August
Germany, September France…)
OTT content
Netflix, HBO, CartoonNetwork, YouTube, Nascar,
Vimeo, DailyMOtion, TuneIn, FoxNews…
Apps, games
Apps games,…
Linear content through Google TV platform
Indexing
g and recording
g
Hot topics:
Search?
Google
g glass
g
Eyeglasses with a camera and a display
Voice controlled
Internet access, video-chat, virtual reality…
Android OS
1500$ price
i ffor d
developers
l
http://www.google.com/glass/start/how-it-feels/
Apple
Apple TV
Apple
pp style:
y well designed
g and simple
p to use
3rd generation
OTT content:
t t
Netflix, Hulu, YouTubre, NBA, MLB, NHL, Wall street journall, iTunes
Hot topics:
Simple to use remote
Also with iPhone, iPad,…
Apple ((2))
Connectivity with other apple devices:
AirPlay:
y content or screen transfer from other devices,, dual screen
(at home)
iCloud: consumption of bought content on all Apple devices
(anywhere)
BBC
“The” broadcaster
Top
p content
User experience?
Sony
y
PlayStation
Basically
yggames
Movies and TV series
Most Netflix content is sold on PS3, more than
on PCs
Cross-platform
Start a game on PS3 continue on PS Vita
PS Move: motion based control
Same story in a sligtly different way
Senzorske storitve
(vse platforme)
Internet of Things
g (IoT)
( )
An increasing number of data sources are available over
interent
All we need are devices and sensors and automatisation of
data processing
Numerous fileds of usage
Traffic, environment, medicine, energy…. anything
Data accessible on an
any de
device,
ice eeven
en on TV
Sensors
Sensor is a device transforming physical quantity into a
electrical signal
Advancement in technology
Technological solutions are getting smaller and cheaper
They are also getting more accurate
Nowadays almost any senssors are available:
Light, temperature, humidity, air pressure
Wind speed and direction, precipitation, sun and UV lights
Gas concentration(CO, CH4, C4H10 CO2, NO)
Radioactivity, vibrations, IR movement
GPS, magnetometer, camera, microphone, proximity sensors
H
Heart
t rate,
t sugar bl
bloodd levels,
l l pH,
H accelerometer
l t
Sensor platforms
More than a sensor
“ l ” connecting
“glue” ti sensor in
i a bigger
bi system
t
Embedded platforms on the rise
CPU, RAM and communication interfaces
RaspberryPi (PC for 30 EUR)
Popular platform Arduino (cheap and simply programable)
Possible integration into a modem, STB,…
+ =
Usage scenarios
Cooling and heating systems
IInner/outer
/ t temperature
t t detction
d t ti
Open windows detection
User presence detection
Lights
IR presence sensors
Light sensors
Medicine
Until recently not suported
Doctors
D t prescribe
ib medicine
di i off which
hi h they
th know
k little,
littl tot cure
diseases of which they know less, in human beings of which they
know nothing. —François-Marie Arouet Voltaire, about 250 years ago
T di i
Tradicionalno
l nepersonalizirana
li i
A lot of interest by people
50% of Slovene population was looing for health related data (Q1
2012)
“only” 27% for watching Web TV (Vir: RIS)
P t bl sensors are generating
Portable ti huge
h amounts
t off data
d t
before: 1 sample every n years
today:
y 10 samples
p a day
y
B i
Basis
Fitness watch + data mining (5 sensors)
Air temperature,
p , skin temperature
p
and conduction, accelerometer, heart rate
Many other solutions
iPod Nano
Touch screen
iOS-like operating system
Some time left?
63
Mobile platforms
Windows Mobile
System based on platform Windows CE
Closed environment
Costly development tools (Microsoft Visual
Studio)
From 2009 includes services ffor app
distribution
Mobile GUI similar to PC based GUI
Microsoft Office Mobile, Outlook Mobile,
Internet Explorer
VPN for business users
advantage: close integration with MS
environment ( AD, Exchange)
L t version:
Last i 6.5
6 5 (2010)
Replaced by Windows Phone 7
Windows Phone 7
Upgrade of Windows Mobile
Completely
p y new GUI
Adapted for a mobile device
koncept ploščic
i t
interakcija
k ij s prstom
t (nič
( ič več
č stylusa)
t l )
Rich media funkcionality
Zune player
Speech recognition
Minimal HW requirements
Capacitive touch screen 480x800
1GHz CPU, 256MB RAM
accelerometer, compass, light sensor, GPS
Separate buttons for search, start, back, sleep, camera
Windows Phone 7
Development platform
Silverlight
Sil li ht (WP7 version)
i ) and
d
XNA platform (based on Xbox new architecture)
Development environment
Visual Studio 2010 Express and
Expression Blend (free)
Wi d
Windows Phone
Ph Marketplace
M k t l
60.000 apps (2012)
(12.000 in 2011)
Also “trial”
Need approval
Microsoft takes
30%
Windows Phone 8 ((RT))
Based on Windows 8 platform
Unfortunately quite different from previous platform (development)
Support fo screens 1280x768
Similar GUI as WP7
XBOX connectivity
music, video, podcasts
games: standalone or 2nd screen
Cooperation: Nokia-
Nokia-Microsoft
Mutual benefit
Why not Nokia-Android?
Nokia would be “just another Android terminal”
Microsoft would have to target niche high
high-end
end devices and couldn’t
couldn t
compete with Googand in Apple
Togehther they are 3rd major player
F i
Furious reaction
ti off Nokia
N ki hardcore
h d fans
f
Apple Inc.
Ecosistem
iPad ••Music
glasba
g
••Movies
filmi
••TV series
nadaljevanke
••Apps
aplikacije
••Newspapers
časopisi
••Books
knjige iPhone
Apple TV
iPhone
Presented in 2007
Sceptical acceptance
“Apple has no experince”
Partially justified critics (poor radio, partially useful as
phone
h
Innovative GUI
At first limited functionality (2G)
No MMS, vCard, no 3G and second camera
Current model (iPhone 5)
GPS, accelerometer,
GPS accelerometer compass
compass, gyroscop
gyroscop, autofocus camera,
camera
bluetooth, voice recognition, text-to-speech, ActiveSync, VPN client
960x640 screen, 512MB RAM, 800MHz CPU, 64GB Flash
Very clear product line
Only 1 current model (little fragmentation)
Apple iPad
Technical specs (iPad4)
1.4GHz
1 4GH CPU,
CPU 1GB RAM
WiFi, 3G, up to 128 GB flash memory
2048x1536, 10” diagonal
GPS, accelerometer, bluetooth, compass
Same ecosystem as iPhone
Can run same apps -2x
C 2 magnified
ifi d
Today >100.000 apps specially developped
for iPad
Final result
15 MIO sold in first 9 month (end of 2010)
Best sold consumer device of all times
Apple SDK
Development tool for iPhone apps
One year after iPhone (2008)
Runs only only Macs
Phone simulator
SDK presents
Programing environment for applications - Xcode (IDE)
A set of interfaces for access to device data and events
ex multitouch
ex. multitouch, camera,
camera location
location, accelerometer,
accelerometer recording and
playback of audio and video, Core Animation, Core Graphics
(Quartz)
App development
Membership in development programme required (Apple developer):
$100 per year
Apple terms of service (TOS)
Every app is checked before release (problem!)
App Store
Free and paid apps
Paid:
Apple takes 30% (most po $0.99)
Install over PC or web
AppStore today
800.000+ apps
sum >40.000.000.000
>40 000 000 000 downloads
Apple gets an estimated $1.8B a year
(profit wasn’t expected at first )
60% of developers don’t even cover
development costs
A model to look up to!
Increases the usability of the device!
Android Marketplace, OVI store
Android
Mobile OS based on Linux kernel
Google
g and Open
p Handset Alliance
Fastest growing mobile platform
Mobile terminals
HW requirements are specified
Connectivity: GSM/EDGE, CDMA, EV-DO, UMTS, Bluetooth, Wi-Fi
Touch screen, GPS, accelerometer,
compass, gyroscope, NFC, camera
Android Mobile terminals
First terminal available in 2008
HTC D
Dream,
A year later
20 models available
Today over 200 models of different manufacturers
HTC, Samsung, Huawei, Motorola, Acer, Dell,
Lenovo, LG,
G Sony
S Ericsson, Philips
Google-branded Nexus One (HTC), Nexus S (Samsung)
Samsung
g Galaxy
y Tab
Technical specs (v2)
1GHz CPU,
1GH CPU 512 MB RAM
WiFi, 3G, 32GB flash
1200x800, 10” diagonal
GPS, accelerometer, 2 cameras, BT
Platform
Android
A d id
Lack of real tablet designed apps
Business results
6 months after iPad
2M pieces sold in 2010, >10M in 2011
App development and distribution
Development in Java
W ll known
Well k ttools
l (E
(Eclipse
li IDE,
IDE iipd.)
d)
Distribution through Android Marketplace
App verification similar to Apple
Device settings relevant for app versions
Problems
Fragmentation: Android is open thousands of existing device
verions
Everyy device p
provider adds it’s own specifics
p to OS to make some
diferenciation
Apps no longer work on all devices
Similar p
problem to J2ME
VIR: http://opensignal.com/reports/fragmentation.php
Android – development and apps
App distribution supported by Android Market
Anyone can participate, individuals or companies
Licencing is not necessary
Development tools and libraries
Free (Eclipse, Android SDK)
Available on all platforms (Windows, Linux, Mac)
BlackBerry
y ((RIM))
Proprietary HW in SW
BlackBerry
Bl kB OS
SDK - Java, coming HTML5, ActionScript
Strong market share by business users
Push e-mail from 2002
BlackBerry Messenger
U i own server-side
Using id
BlackBerry Enterprise Server (BES)
Intermediary between mail servers and mobiles
Checks many mailboxes
Changes immediately communicated to the mobile
Coming: BlackBerry 10
Touch screen (Z10), also with a keyboard (Q10)
Uncertain future
Market share comparison
All phones!
I l di
Including "dumbphones"
"d b h "
War of platforms: Android vs. iOs
Market share: 2011:
Q1 2011: sold
alltogether428 M mobiles
100 M smartphones
36 mio Android,
28 mio Symbian,
17 mio iOS
Comparison of profits
Apple accounts for 50% of total profit [2011]
Vpliv na računalniško industrijo
POSLOVNA SKRIVNOST
smartTV
tTV platforms
l tf
Middleware
• Middleware – proprietary
• OpenTV,
O TV MediaHighway,
M di Hi h Microsoft
Mi ft TV,
TV Liberate,
Lib t PowerTV,
P TV NDS
Core
• Middleware –open
p standard
• MHEG, MHP, OCAP, ACAP, ARIB B23, Java TV, HbbTV
HBBTV
87
Specification HbbTV
• HBBTV v1.5 (March 2012)
• http://www hbbtv org/
http://www.hbbtv.org/
88
HbbTV supported devices
89
Architecture
• Broadcast:
• Transmittion of live TV,
TV radio and data
• Signaling of “broadcast-related” applications
• Transmition of “broadcast-related” applications and data
• Synchronistaion of application and broadcast TV/radio/data
services
• Broadband:
• Content on demand
• Transmition of “broadcast-related” and “broadcast-independent”
applications and coresponding data
• Information exchange between servers and applications
• Access to “broadcast-independent” applications
90
Architecture
91
Hybrid
y device (STB,
( IDTV))
Application Data:
HTML 4, JavaScript, CSS, XML, multimedia files
92
Some time left?
93
W b technologies
Web t h l i
Web evolution
At first intended for scientific exchange of information
CERN
Later on, growing with availability of bandwidth
Multimedia content
A growing number of web pages…
…and users
Ever more performant HW
(decoding of video, audio, 3D rendering)
Today
Web page in a browser can replace
a native app
On both mobiles and desktop
Tommorow
Google
G l Chrome
Ch OS instead
i t d off Windows?
Wi d ?
Web based operating systems?
95 /
66
Tehnologije
gj
Some web technologies are almost 20 years old
Most obvious one: HTTP
Some changes ahead: SPDY draft (Google)
Nekatere so se razvijale počasi, v koraku s številom spletnih
strani
t i in
i uporabnikov
b ik
HTML 1.0 [1991] … first web page
HTML 2.0 [1995] … web has 10.000 web pages
HTML 3.2 [1997] … web has 500.000 web pages
HTML 4.0 [1997] … web has 1M web pages
HTML 5 [2008]
Today [2013] … web has bilions of web pages
96 /
66
Web is open
World wide web was always open
An example
p to other systems
y
Anyone can add their own HW
And by that physically extend the web
Anyone can add content
And extend the web with new web pages
Openess
All standards and protocols are known
(published in ASCII/txt on the Web)
One can implement its own server without patent infringement
Or setup their web site
98
Web2.0 application technologies
WEB 2.0 API
“Write Applications not Code”
Basic building blocks are available
Open interfaces over HTTP/REST/SOAP
A set of supported libraires
100
WEB 2.0 API ((2))
Google API (http://code.google.com/more/)
A number of APIs
Search
You TUBE
Open Social
Maps
Earth
Calendar
Chart
...
Basics
User registration
Basic and advanced search
Uploadig and transcoding of videos
Serving of videos
102
WEB 2.0 API ((4))
YouTube API (2)
Y T b player
YouTube l
Basic player with all options
Simple use (HTML Embed)
Advanced player with reduced functionality (just video)
Player control imeplemented using Javascript
Customisable controls (graphics)
(g p )
At what price?
Standard YouTube player (embeded video)
Advanced player with embedded YouTube logo
Not really annoying
Dependent on Google
103
Examples-- Google
Examples g Charts
Primer
104
Examples-- Google
Examples g Maps
Primer
105
Application development -
E
Ecosystem
t
SW development: past
Traditional environment
Closed
Cl d OS
Tedious app development
Limited HW capabilities
Expensive data links
No sensors on devices
Consequently:
Low motivation for app development
Developers are mostly operators and device manufacturers
Poor documentation
Specialized knowledge needed
(details, details,...)
Ecosystem
y in the past
Operator
• price
• Quality of service
Developers
•Manufacturers adapts • Specialized companies
applications to the •Rarely individuals
Users infrastructure
Manufacturers
• Openess
• performance
p
• price
SW development: today
y
Trend and reality: open platforms
Platform
f owner ((Nokia, Microsoft,
f GGoogle, Apple))
opens the platform and offers development tools
Usually free of charge
Developers over the world are developing the applications
And by that enrich the platform
Contribute some % from revenue to the platform owner
Platform owner
makes% from app sales
Gets a more appealing and better platform
The latter attracts more users
Ecosystem
y today
y
More users result in
more interest for app
Operater and service
development
• Price
• Quality of service
Developers
• Specialized companies
• Individuals
Users
Manufacturers invest in
development and increase interest
i d
in devices
i
112