VI- SEM/III B.E. CSE Prepared By: R.

Reshma/AP/CSE

CCS352: MULTIMEDIA AND ANIMATION

LECTURE NOTES

UNIT V MULTIMEDIA APPLICATIONS 6

Multimedia Big data computing, social networks, smartphones, surveillance, Analytics, Multimedia Cloud Computing,
Multimedia streaming cloud, media on demand, security and forensics, Online social networking, multimedia ontology, and
Content-based retrieval from digital libraries.

Introduction
Multimedia applications and services provide more opportunities to compute multimedia big data. Multimedia big data
computing refers to the processing and analysis of large volumes of multimedia data using computational techniques.
Multimedia data typically includes a combination of text, images, audio, video, and other types of rich media. This field
leverages advanced computing techniques, such as machine learning, deep learning, and natural language processing, to
extract valuable insights, patterns, and knowledge from these diverse data sources. The sheer volume, variety, and velocity
of multimedia big data present unique challenges in terms of storage, retrieval, and processing. Consequently, multimedia
big data computing often employs distributed and parallel computing frameworks, cloud computing infrastructure, and
scalable algorithms to handle massive datasets efficiently. Applications of multimedia big data computing span various
domains, including social media analytics, healthcare, entertainment, surveillance, and e-commerce, offering opportunities
for innovation and advancement in data-driven decision-making and understanding complex phenomena in our digital
world.

Multimedia Big Data Computing

Multimedia Big Data Computing Framework

1. Data Acquisition: This phase involves collecting multimedia data from various sources such as social media platforms,
sensors, cameras, and databases. Data may include images, videos, audio recordings, text, and metadata associated with
each type of media.

Department of Computer Science and Engineering Dhanalakshmi Srinivasan College of Engineering

 VI- SEM/III B.E. CSE Prepared By: R. Reshma/AP/CSE

2. Data Preprocessing: Once collected, the raw multimedia data often undergoes preprocessing to clean, filter, and format
it for further analysis. This may include tasks such as noise removal, normalization, feature extraction, and data
transformation to make the data suitable for subsequent processing steps.
3. Data Storage and Management: In this phase, the preprocessed multimedia data is stored in appropriate data storage
systems, which may include traditional databases, distributed file systems, or cloud-based storage solutions. Efficient data
management strategies are essential to handle the large volumes of multimedia data effectively and ensure fast retrieval
when needed.
4. Data Analysis and Mining: This phase involves applying various computational techniques, such as machine learning,
deep learning, image processing, and natural language processing, to analyze and extract insights from multimedia data.
Tasks may include object detection and recognition, sentiment analysis, content-based retrieval, clustering, classification,
and anomaly detection.
5. Interpretation and Visualization: Once the analysis is complete, the results need to be interpreted and visualized to
make them understandable and actionable for decision-makers. Visualization techniques such as charts, graphs, heatmaps,
and interactive dashboards can help in conveying complex findings and patterns discovered in the multimedia data.
6. Knowledge Discovery and Application: In the final phase, the insights gained from multimedia big data analysis are
used to drive decision-making, solve problems, and create value in various domains such as healthcare, entertainment,
marketing, security, and finance. This may involve developing new products, optimizing processes, improving customer
experiences, or making policy recommendations based on the knowledge derived from the multimedia big data.
Multimedia in Social Networks/ Multimedia Social Networks
Multimedia in social networks refers to the integration of diverse digital media formats, including images, videos, audio
clips, and animations, within the fabric of online social platforms. It has transformed the landscape of communication and
interaction, offering users dynamic ways to express themselves, share experiences, and engage with others. Through
multimedia, social network users can convey nuanced emotions, tell compelling stories, and create visually captivating
content, fostering deeper connections and enhancing engagement within their networks.
From sharing photos and videos to live streaming events and posting Stories, multimedia has become instrumental in
shaping the culture, trends, and dynamics of social networking platforms. Businesses and brands also leverage multimedia
content to capture attention, drive engagement, and build relationships with their audience. As social networks continue to
evolve, multimedia remains a cornerstone, enriching the user experience and enabling new forms of expression and
connection in the digital age.

Department of Computer Science and Engineering Dhanalakshmi Srinivasan College of Engineering

 VI- SEM/III B.E. CSE Prepared By: R. Reshma/AP/CSE

In social networks, multimedia content is widely used for various purposes:

1. Visual Storytelling: Users leverage multimedia content to tell stories, share experiences, and express themselves
creatively. Images and videos can convey rich narratives and evoke emotions more effectively than plain text,
enabling users to connect with their audience on a deeper level.
2. Content Sharing: Social media users frequently share multimedia content, such as photos, videos, memes, and
GIFs, with their networks. This sharing behavior facilitates the dissemination of information, trends, and viral
content across social platforms, contributing to the interconnectedness and dynamics of social networks.
3. Engagement and Interaction: Multimedia content tends to generate higher levels of engagement and interaction
compared to text-only posts. Users are more likely to like, comment, share, and interact with multimedia posts,
leading to increased visibility and reach within social networks.
4. Branding and Marketing: Businesses and brands utilize multimedia content to promote their products and
services, build brand awareness, and engage with their target audience. Visual content, such as product images,
promotional videos, and influencer collaborations, can effectively capture the attention of users and drive customer
engagement and conversion.
5. User-generated Content: Social networks facilitate the creation and sharing of user-generated multimedia content,
allowing individuals to contribute to the platform's content ecosystem. User-generated content adds authenticity,
diversity, and richness to social networks, fostering community engagement and participation.
6. Live Streaming and Stories: With the rise of live streaming and ephemeral content features like Stories, social
media users can share real-time experiences and updates with their followers through multimedia formats. Live
streaming platforms enable interactive and immersive experiences, while Stories offer a casual and ephemeral way
to share moments and updates with a sense of urgency.
Multimedia in Smartphones
Multimedia in smartphones refers to the integration of various forms of media content, such as text, images, audio, and
video, within the device's functionality. Smartphones have evolved to become powerful multimedia hubs, offering users the
ability to consume and create diverse types of media on a single device. From streaming videos and music to capturing high-
resolution photos and videos, smartphones provide an immersive multimedia experience. Additionally, users can access
social media platforms, gaming apps, and multimedia messaging services, further enriching their multimedia interactions.
The advancements in smartphone technology, including high-resolution displays, advanced camera systems, and robust
audio capabilities, have significantly enhanced the multimedia capabilities of these devices, making them indispensable
tools for entertainment, communication, and productivity. Multimedia in smartphones refers to the integration of various
forms of media content, including text, images, audio, and video, into the device's functionality. Smartphones serve as
versatile multimedia devices, offering users the capability to consume, create, and share diverse types of media content
seamlessly. Users can access a wide array of multimedia content through various applications and services available on
smartphones. They can stream high-definition videos, listen to music, view photos, and engage with interactive multimedia
content on social media platforms, news apps, and entertainment services.
Moreover, smartphones are equipped with advanced camera systems that enable users to capture high-quality photos and
videos. With features such as image stabilization, HDR (High Dynamic Range), and advanced editing tools, users can create
professional-looking multimedia content directly from their smartphones. In addition to consumption and creation,
smartphones facilitate the sharing of multimedia content through various communication channels, including messaging
apps, social media platforms, and email. Users can instantly share photos, videos, and other multimedia files with friends,
family, and colleagues, enhancing connectivity and collaboration. The continuous advancements in smartphone technology,
such as improved display quality, enhanced audio capabilities, and augmented reality (AR) features, further enrich the
multimedia experience on these devices. As a result, smartphones have become indispensable tools for entertainment,
communication, and creative expression in the digital age.

Department of Computer Science and Engineering Dhanalakshmi Srinivasan College of Engineering

 VI- SEM/III B.E. CSE Prepared By: R. Reshma/AP/CSE

Multimedia in Surveillance
Multimedia in surveillance refers to the utilization of various forms of media, including video, audio, and sometimes text,
in the process of monitoring and observing activities, events, or people for security or investigative purposes.
Video surveillance, one of the most common forms of multimedia surveillance, involves the use of cameras to capture visual
information in real-time. These cameras are strategically placed in locations of interest, such as public spaces, buildings, or
critical infrastructure, to monitor for potential security threats, criminal activities, or safety breaches. The captured video
footage can be stored, analyzed, and accessed for review by security personnel or law enforcement agencies.
Audio surveillance involves the recording and analysis of sound or spoken conversations using microphones or specialized
audio recording devices. This form of surveillance is often employed in conjunction with video surveillance to provide
additional context or evidence in security investigations.
Text-based surveillance involves the monitoring and analysis of written or digital communications, such as emails, text
messages, social media posts, or other forms of electronic communication. Text-based surveillance may utilize automated
systems or algorithms to scan and analyze large volumes of text data for keywords, patterns, or suspicious activities.
Multimedia surveillance systems often incorporate advanced technologies such as facial recognition, object detection,
motion tracking, and data analytics to enhance the effectiveness and efficiency of surveillance operations. These
technologies enable automated monitoring, alerting, and decision-making capabilities, allowing security personnel to
identify and respond to potential security threats in real-time.
While multimedia surveillance can play a crucial role in enhancing security and public safety, it also raises important ethical
and privacy concerns related to the collection, storage, and use of personal information and data. Therefore, it is essential
to implement appropriate safeguards, regulations, and oversight mechanisms to ensure that surveillance practices are
conducted responsibly, transparently, and under legal and ethical standards.
Multimedia Analytics
Multimedia analytics is a new and exciting research area that combines techniques from multimedia analysis, visual
analytics, and data management, with a focus on creating systems for analysing large-scale multimedia collections.
Multimedia analytics is a process that involves analyzing and extracting meaningful insights from diverse forms of media
data, including images, videos, audio recordings, and text. This analytical approach encompasses a range of techniques from
fields like machine learning, computer vision, natural language processing, and signal processing to uncover patterns, trends,
and valuable information within large volumes of multimedia content. By leveraging advanced algorithms and tools,
multimedia analytics enables organizations to gain deeper understanding and actionable intelligence from their multimedia
data. This can be applied across various domains, such as marketing, security, healthcare, social media analysis, and
entertainment, to inform decision-making, enhance operational efficiency, and drive innovation. Ultimately, multimedia
analytics serves as a powerful tool for transforming multimedia data into actionable insights that can drive business growth,
improve services, and address complex challenges in today's data-driven world.
The size and complexity of media collections is ever increasing, as is the desire to harvest useful information from these
collections, with expected impacts ranging from the advancement of science to increased company profits. Indeed,
multimedia analytics sees potential applications in diverse fields, including data journalism, urban computing, lifelogging,
digital heritage, healthcare, digital forensics, natural sciences, and social media.
The process of multimedia analytics typically involves several stages:
1. Data Collection: Gathering multimedia data from diverse sources such as social media platforms, surveillance
systems, digital cameras, satellites, and sensors. This step involves acquiring raw data in different formats and
structures.
2. Preprocessing: Cleaning, filtering, and preparing the raw multimedia data for analysis. This may involve tasks such
as noise reduction, normalization, data integration, and feature extraction. Preprocessing aims to enhance the quality
and usability of the data for subsequent analysis.

Department of Computer Science and Engineering Dhanalakshmi Srinivasan College of Engineering

 VI- SEM/III B.E. CSE Prepared By: R. Reshma/AP/CSE

3. Analysis: Applying various analytical techniques and algorithms to extract meaningful insights from the
multimedia data. This may involve methods from fields such as machine learning, computer vision, natural language
processing, signal processing, and data mining. Different types of analysis can be performed depending on the
nature of the data and the objectives of the analysis, such as image classification, sentiment analysis, object
detection, event detection, pattern recognition, and anomaly detection.
4. Visualization: Presenting the analytical results and insights in a visual format, such as charts, graphs, maps,
heatmaps, timelines, or interactive dashboards. Visualization helps in understanding complex relationships, trends,
and patterns within the multimedia data and facilitates communication of findings to stakeholders.
5. Interpretation and Decision-Making: Interpreting the analytical results and using them to make informed
decisions, develop strategies, and take actions. The insights derived from multimedia analytics can be used across
various domains such as business, healthcare, finance, marketing, security, and research to drive innovation,
optimize processes, improve decision-making, and solve real-world problems.
Multimedia analytics finds applications in a wide range of domains, including:
• Marketing and Advertising: Analyzing multimedia content to understand consumer behavior, preferences, and
sentiment for targeted advertising and marketing campaigns.
• Healthcare: Analyzing medical images, patient records, and sensor data to assist in diagnosis, treatment planning,
and healthcare management.
• Security and Surveillance: Analyzing surveillance videos, images, and audio recordings to detect anomalies,
identify threats, and enhance public safety.
• Social Media Analysis: Analyzing multimedia content shared on social media platforms to track trends, monitor
public opinion, and understand social dynamics.
• Entertainment and Media: Analyzing user-generated content, such as videos, images, and music, to personalize
content recommendations, detect copyright infringement, and improve user engagement.
Multimedia (images, videos, text, and speech or combinations of more than one modality) forensics is the science of
computer algorithms (especially machine learning and artificial intelligence) to automatically analyze multimedia, construct
provenance trees and identify and provide evidence of multimedia forgeries. Multimedia forgeries range from manually
manipulating multimedia assets, to multimedia repurposing and deep neural network-generated content.
IMAGE MANIPULATION AND IMAGE REPURPOSING IDENTIFICATION
Manipulating visual information (images and videos) have perhaps existed since visual information itself existed. Take, for
example, the iconic portrait of President Lincoln of 1860. As it turns out, the portrait was actually a fake composite of
Lincoln’s head and the body of Vice President John Calhoun, who died in 1850.

Department of Computer Science and Engineering Dhanalakshmi Srinivasan College of Engineering

 VI- SEM/III B.E. CSE Prepared By: R. Reshma/AP/CSE

Image manipulation forgeries include any methods that attempt to change the original digital content of the image (i.e.
pixels) including, for example, image splicing and copy-move forgeries, where part(s) of the image are altered by copying
and pasting parts of the same or other images (e.g. to introduce or hide evidence). Meanwhile, another infamous multimedia
forgery technique depends on repurposing an unaltered image to convey misleading information by changing one or more
of the accompanying metadata, such as changing the geolocation where the image was captured, events or names of people
in the image. At VIMAL, we have been developing deep learning methods for detecting image splicing and copy move
image forgeries, as well as deep learning methods for identifying repurposed multimedia packages (i.e. images accompanied
by other types of metadata).
Multimedia Cloud Computing
Internet is having a significant impact on the media-related industries which are using it as a medium to enable delivery of
their content to end-users. Rich web pages, software downloads, interactive communications, and ever-expanding universe
of digital media require a new approach to content delivery. Size and volume of multimedia content is growing
exponentially. For example, more than 30 billion pieces of content such as web links, news stories, blog posts, notes, and
photo albums are shared each month on Facebook. On the other hand, Twitter users are tweeting an average 55 million
tweets a day that includes web links and photo albums. Web pages and other multimedia content are being delivered through
content delivery networks (CDN) technologies. These technologies optimize network usage through dedicated network
links, caching servers and by increasingly using peer-to-peer technologies. The concept of a CDN was conceived in the
early days of Internet but it took until the end of 1990’s before CDNs from Akamai and other commercial providers managed
to deliver Web content (i.e., web pages, text, graphics, URLs and scripts) anywhere in the world and at the same time meet
the high availability and quality expected by their end users. For example, Akamai delivers between fifteen to thirty percent
of all Web traffic, reaching more than 4 Terabits per second. Commercial CDNs achieved this by deploying a private
collection of servers and by using distributed CDN software system in multiple data centres around the world.
A different variant of CDN technology appeared in the mid 2000’s to support the streaming of hundreds of high definition
channels to paid customers. These CDNs had to deal with more stringent Quality of Service (QoS) requirements to support
users’ experience pertaining to high definition video. This required active management of the underlying network resources
and the use of specialized set-top boxes that included video recorders (providing stop/resume and record/playback
functionality) and hardware decoders (e.g., providing MPEG 4 video compression/decompression). Major video CDNs
where developed by telecommunications companies that owned the required network and had Operation Support Systems
(OSSs) to manage the network QoS as required by the CDN to preserve the integrity of high definition video content. Just
like the original CDNs, video CDN also utilize a private collection of servers distributed around the network of video service
provider. The first notable CDNs in this category include Verizon’s FiOS and AT&T’s U-verse. Some CDN providers such
as Limelight Networks invested billions of dollars in building dedicated network links (media-grade fiber-optic backbone)
for delivering and moving content from servers to end-users.

Dissemination and presentation in multimedia big data computing

Dissemination and presentation in multimedia big data computing play critical roles in effectively communicating insights,
findings, and results derived from the analysis of large volumes of multimedia data to relevant stakeholders. Dissemination
involves the strategic sharing and distribution of information through various channels to ensure that it reaches its intended
audience. This process ensures that the valuable knowledge obtained from multimedia big data analysis is accessible and
actionable. Dissemination strategies may include publishing research papers, presenting at conferences and workshops,

Department of Computer Science and Engineering Dhanalakshmi Srinivasan College of Engineering

 VI- SEM/III B.E. CSE Prepared By: R. Reshma/AP/CSE

sharing data and code repositories, engaging in collaborative networks, and communicating findings through media outreach
and public engagement efforts.
Presentation, on the other hand, focuses on conveying complex insights and results in a clear, concise, and visually
compelling manner. Multimedia big data often entails diverse types of data, including images, videos, text, and audio, which
require appropriate visualization techniques to effectively communicate findings. Presentations may involve creating
interactive dashboards, data visualizations, infographics, and multimedia-rich reports to facilitate understanding and
interpretation by diverse audiences.
Effective dissemination and presentation in multimedia big data computing are essential for maximizing the impact of the
analysis and driving decision-making, innovation, and action in various domains. By making insights accessible,
understandable, and actionable, dissemination and presentation efforts ensure that the potential of multimedia big data is
realized in addressing real-world challenges and driving positive change.
A more recent variant of video CDNs involves the caching video content in cloud storage and the distribution of such content
using third-party network services that are designed to meet QoS requirements of caching and streaming high definition
video. For example, Netflix’s video CDN has been developed on top of Amazon AWS. CloudFront is Amazon’s own CDN
that uses Amazon AWS and provides streaming video services using Microsoft Xboxes. While Cloud-based CDNs have
made a remarkable progress in the past five years, they are still limited in the following aspects:
• CDN service providers either own all the services they use to run their CDN services or they outsource this to a
single cloud provider. A specialized legal and technical relationship is required to make the CDN work in the latter
case.
• Video CDNs are not designed to manage content (e.g., find and play high definition movies). This is typically done
by CDN applications. For example, CDNs do not provide services that allow an individual to create a streaming
music video service combining music videos from an existing content source on the Internet (e.g., YouTube),
his/her personal collection, and from live performances he/she attends using his/her smart phone to capture such
content. This can only be done by an application managing where and when the CDN will deliver the video
component of his/her music program.
• CDNs are designed for streaming staged content but do not perform well in situations where content is produced
dynamically. This is typically the case when content is produced, managed and consumed in collaborative activities.
For example, an art teacher may find and discuss movies from different film archives, the selected movies may then
be edited by students. Parts of them may be used in producing new movies that can be sent to the students’ friends
for comments and suggestions. Current CDNs do not support such collaborative activities that involve dynamic
content creation.
Major Research Issues in Multimedia Cloud Computing
1. Collaborative content management workflows
2. Ubiquitous content delivery
3. Flexible content storage, compression, & indexing
4. Content personalisation and contextualisation
5. Community building
6. Quality of service optimization
Multimedia Streaming Cloud
A Multimedia Streaming Cloud is a cloud-based infrastructure designed to efficiently deliver multimedia content, such as
videos, audio streams, and live broadcasts, over the internet to end-users. This specialized cloud environment leverages
various components and technologies to ensure seamless and high-quality streaming experiences.

Department of Computer Science and Engineering Dhanalakshmi Srinivasan College of Engineering

 VI- SEM/III B.E. CSE Prepared By: R. Reshma/AP/CSE

At its core, the multimedia streaming cloud includes storage solutions optimized for hosting large volumes of multimedia
files. These files are stored in distributed data centers around the world, ensuring low-latency access for users regardless of
their geographical location. Additionally, the streaming cloud employs content delivery networks (CDNs) to cache and
deliver multimedia content closer to end-users, reducing buffering and improving playback performance.
Furthermore, the multimedia streaming cloud incorporates robust transcoding and encoding capabilities to adapt multimedia
content to different devices, network conditions, and bandwidth capacities. This ensures compatibility and optimal viewing
experiences across a wide range of devices, from smartphones and tablets to smart TVs and desktop computers. Moreover,
the multimedia streaming cloud utilizes advanced streaming protocols and technologies, such as HTTP Live Streaming
(HLS), Dynamic Adaptive Streaming over HTTP (DASH), and WebRTC (Real-Time Communication), to deliver adaptive
bitrate streaming, real-time communication, and interactive multimedia experiences. Security is also a key consideration in
a multimedia streaming cloud, with encryption, access controls, and digital rights management (DRM) mechanisms
implemented to protect multimedia content from unauthorized access, piracy, and content theft.
Streaming files is a way to deal with large content of files in pieces so that playback can begin more quickly. Cloud
computing and its ever-growing technology has a heavy impact on streaming. Streaming is performed where there is
broadcasting of current events, viewing TV series or movies on user's computer or cell-phones, listening to radio jockey
(RJ) or Internet radio and various other applications.
Defining Media Streaming
It is a method of delivering multimedia elements usually video or audio from a data-streaming service provider to the users.
The protocols it uses includes:
• HTTP (Hypertext Transfer Protocol)
• TCP/IP (Transmission Control Protocol / Internet Protocol)
• HTML Protocols
Streaming media is either video or audio content sent in compressed form over internet, rather than saved to hard-disk.
Because of media streaming, users does not have to download a file to play and view it. Users can fast-forward, rewind,
pause, stop that streaming video what they can with a downloaded file.
Advantages of Streaming in Cloud
Cloud technology has a number of advantages that aid content providers to deliver systems. A large storage facility is
available for cloud computing providers for maintaining libraries along with a high computation power for streaming
servers. It also provides an engine for encoding, decoding and transcoding content. Audio streaming was the first to gain
widespread fame as a media application on Internet. The streamed audio gets expanded its field using cloud computing to
make radio. VoIP is a form of audio streaming based on specialized protocol.
Some of the advantages of streaming media are:
• It makes possible for cloud users to view interactive applications like video, searching them and personalized
playlist.
• It provides the content makers with more control over his intellectual property as the video file isn't stored on the
viewer's computer.
• It provides an efficient use of bandwidth. This is because the transferred file is the only part that is being watched.
• It allows content delivers for monitoring what visitors are watching.
Encoding.com offers a great example of how the cloud can be leveraged for providing service on demand. This site
advertises itself as the world's most popular encoding/transcoding service. Encoding.com also provides static picture file
conversion, audio, and video file conversion. Most of the cloud business focuses on streaming file formats used for audio
and video work. There is the most popular conversion provided by that site. These are:

Department of Computer Science and Engineering Dhanalakshmi Srinivasan College of Engineering

 VI- SEM/III B.E. CSE Prepared By: R. Reshma/AP/CSE

• AVI to WMV
• Any video to 3GP
• MP4 to WMV
• FLV to MPG
• 3GP to WMV
• WMV to MPEG
• AVI to MPEG-4
• VP6 to MPEG
• 264 to MPEG
• VP6 to AVI
• 264 to AVI
• FLV to MOV
• MP4 to 3GP etc….
Working with VoIP Applications
Voice over Internet Protocol (VoIP) is a set of protocols for communicating and delivering voice over the internet. Some of
these services migrated to the cloud that requires a large number of servers. Other than audio streaming, VoIP uses additional
protocols and standards, which are most commonly used and are listed below:
• 323
• Session Description Protocol (SDP)
• Media Gateway Control Protocol (MGCP)
• Real-time Transport Protocol (RTP)
• IP Multimedia Subsystem (IMS)
• Session Initiation Protocol (SIP)
Skype is an example of the VoIP application, and its protocol is a proprietary protocol which uses the Global Index P2P
protocol. Skype communication is encrypted using standard encryption methods: RSA key & Advanced Encryption
Standard (AES) algorithms. Skype accounts do not require that the name of the caller be identified during a call, only his
username.
Media on Demand
Media-on-Demand (MOD) refers to a type of service that provides users with access to media content such as videos,
movies, music, and TV shows at their convenience. With Media-on-Demand, users can choose the content they want to
watch or listen to and when they want to access it. It is a type of service that is becoming increasingly popular as more
people switch to streaming services for their entertainment needs.
How Does Media-on-Demand Work?
Media-on-Demand services work by using a server or network of servers that host digital media content. Users can access
this content through a variety of devices such as computers, smartphones, tablets, and smart TVs. The content is typically
stored on servers and can be streamed or downloaded by users depending on the service they choose.
The process of accessing media content on Media-on-Demand services usually involves the following steps:

Department of Computer Science and Engineering Dhanalakshmi Srinivasan College of Engineering

 VI- SEM/III B.E. CSE Prepared By: R. Reshma/AP/CSE

1. User registration: Users sign up for the service by creating an account and providing their personal information.
2. Content selection: Users browse the available content and select the media they want to access.
3. Payment: Users pay for access to the content either on a per-use basis or through a subscription model.
4. Content delivery: Once payment is confirmed, the media is delivered to the user's device via streaming or download.
Advantages of Media-on-Demand Services
1. Convenience: Media-on-Demand services offer users the ability to access media content at their convenience. Users can
choose the content they want to watch or listen to and when they want to access it.
2. Access to a wide range of content: Media-on-Demand services provide users with access to a vast array of media content,
including movies, TV shows, music, and more.
3. Cost-effective: Compared to traditional cable or satellite TV services, Media-on-Demand services can be more cost-
effective, especially for users who do not want to pay for channels they do not use.
4. Flexibility: Media-on-Demand services offer users flexibility in terms of the devices they can use to access the content.
Users can access the content on a variety of devices, including smartphones, tablets, and smart TVs.
Disadvantages of Media-on-Demand Services
1. Dependence on the internet: Media-on-Demand services rely heavily on internet connectivity. Poor internet
connectivity can result in buffering and interruptions while streaming media content.
2. Limited content availability: Some Media-on-Demand services may not have the same range of media content available
as traditional cable or satellite TV services.
3. Subscription costs: While Media-on-Demand services can be cost-effective, users may end up paying more if they
subscribe to multiple services to access a wide range of content.
Types of Media-on-Demand Services
1. Video-on-Demand (VOD): VOD services allow users to access movies and TV shows on-demand.
2. Music-on-Demand (MOD): MOD services allow users to access a vast library of music and create custom playlists.
3. Game-on-Demand (GOD): GOD services allow users to access and play video games on-demand.
4. Book-on-Demand (BOD): BOD services allow users to access e-books on-demand.
Examples of Media-on-Demand Services
1. Netflix: A popular VOD service that offers a vast library of movies and TV shows.
2. Spotify: A popular MOD service that allows users to access a vast library of music and create custom playlists.
3. PlayStation Now: A GOD service that allows users to access and play video games on-demand.
4. Kindle Unlimited: A BOD service that allows users to access a vast library of e-books on-demand.
Multimedia In Security And Forensics
When applied to the field of multimedia, digital forensics started to face challenges, as multimedia is content that uses a
mix of audio, video, images, text. Thanks to the wide adoption of mobile devices, cheaper storage, high bandwidth, online
users are generating a humongous amount of data. This growth has pushed digital multimedia in the forefront. The amount
of data is so massive that it has surpassed the capabilities of the forensic experts to effectively analyze and process the data.
Multimedia forensics has now become an integral part of the Cyber Forensics. Multimedia forensics involves the set of
techniques used for the analysis of multimedia signals like audio, video, images. It aims to:
• Reveal the history of digital content.

Department of Computer Science and Engineering Dhanalakshmi Srinivasan College of Engineering

 VI- SEM/III B.E. CSE Prepared By: R. Reshma/AP/CSE

• Identifying the acquisition device that produced the data.

• Validating the integrity of the contents.
• Retrieving information from multimedia signals.
What are the approaches to Multimedia Authentication?
Internet content is not only limited to text form, it comes in a lot of different varieties, so the forensic approaches developed
to analyze them must also vary in scope. The goal here is to analyze images, text, audio, video, in order to generate a piece
of logical Forensic evidence. Multimedia Forensics divides its efforts between 2 main approaches – Active Image
Authentication and Passive Image Authentication.

As you can see from the diagram, Active image authentication, and Passive image authentication are further divided into
categories. Let’s get started and discuss each category in detail.
Active Image Authentication:
In this technique, a known authentication code is embedded in the image at the time of image generation or sent with the
image for accessing its integrity at the receiving end. Verifying this code authenticates the originality of the image. Active
Authentication is further classified into 2 categories: Digital Watermarking and Digital Signatures. Drawbacks of Active
image authentication:
• The authentication code needs to be embedded in the image at the time of recording using special equipment thus
prior information about the image becomes indispensable.
• This approach requires a digital watermark or a digital signature to be created precisely when the image is recorded,
which limits its ability to handle specially equipped digital devices.
• As the majority of the images on the Internet, today don’t have a watermark or a digital signature, which has forced
this image authentication method to consider additional techniques – Digital Watermarking: In this technique a
digital watermark is embedded into the image at the time of either image acquisition or in the processing
stage. Digital Signatures: Digital signatures embed some secondary information that is usually obtained from the
image, at the acquisition end into the image.
Passive Image Authentication:
Passive authentication also known as image forensics uses the only image with no prior information for accessing the
integrity of the image. Passive authentication works on the assumption that even though tampering with the image may not
leave any visual trace but they are likely to alter the underlying statistics. This means that digital forgeries may disturb the

Department of Computer Science and Engineering Dhanalakshmi Srinivasan College of Engineering

 VI- SEM/III B.E. CSE Prepared By: R. Reshma/AP/CSE

underlying properties of the image, quality of the image, even though no physical clue has been left behind. Passive
techniques are further classified into Forgery-type dependent and Forgery-type independent techniques.
Forgery-type dependent –
These are designed to detect only certain types of forgeries like copy-move and image-splicing which are dependent on the
type of forgery carried out on the image. It is further classified into 2 categories: Copy-move detection and Image-splicing
detection. 1. Copy-move detection: Copy-move is the most popular photo tampering technique because of the ease with
which it can be carried out. It involves copying some regions in the image and moving the same to some other region in the
image. Since the copied region belongs to the same image so the dynamic range and color remain compatible with the rest
of the image. In copy-move detection post-processing operation like blurring is used to decrease the effect of border
irregularities between the two images.

2. Image-splicing detection: The Image-splicing method involves merging 2 or more images changing the original image
significantly to create a forged image. Please note when merging images with differing backgrounds, it becomes difficult to
make the border and boundaries indiscernible. Image-splicing detection is a challenging task involving the following
techniques:
• Composite regions are investigated by a variety of methods.
• The presence of abrupt changes between different regions that are combined to create a composite image and their
backgrounds, provide valuable traces to detect splicing in the image under consideration.

Forgery-type independent –
These methods detect forgeries independent of forgery type but based on artifact traces left during the process of re-sampling
and due to lighting inconsistencies. It is further classified into 2 categories: 1. Retouching detection: This method is most
commonly used for commercial and aesthetic applications. Retouching is mostly carried out to enhance or reduce the image

Department of Computer Science and Engineering Dhanalakshmi Srinivasan College of Engineering

 VI- SEM/III B.E. CSE Prepared By: R. Reshma/AP/CSE

features or to create a convincing composition of 2 images that requires rotation, resizing, or stretching of one of the images.
Image retouching detection is done using the following techniques:
• Find the blurring, enhancements, or color changes and illumination changes in the forged image.
• Retouching Detection is easy if the original image is available however blind detection is a challenging task.

2. Lighting Conditions: Images that are combined during tampering are taken in different lighting conditions. It becomes
very difficult to match the lighting condition from combining photographs. This lighting inconsistency in the composite
image can be used for the detection of image tampering.
What are Digital Fingerprints?
Although Cryptographic tools and access control mechanisms ensure the safe delivery of multimedia content across the
Internet. But this protection ends as soon as the content is delivered to the end-user and safely decrypted. Digital
Fingerprinting has emerged to cater to this post-delivery by identifying the end-users who have authorized access to plaintext
but use it for unauthorized purposes. Digital Fingerprinting process involves investigators to trace the illegal usage of
multimedia content through a unique identifying information known as “Fingerprint” that is embedded in the content before
distribution. Youtube is using this technology to scan files and match the digital fingerprints they find against a database of
copyrighted material to see if any intellectual property is being violated. Digital Fingerprints are technically coded strings
of binary digits generated by mathematical algorithms, they are as unique as the analog fingerprints of a person. The more
images and videos continue to flood the Internet, the more difficult it becomes to protect the information through forensic
investigations. As online multimedia content grows, it becomes important for the users and creators to understand the legal
boundary of the virtual world.
Multimedia in Online Social Networking
Multimedia plays a central role in online social networking, enriching user experiences, facilitating communication, and
fostering engagement among users. Here are several ways multimedia is integrated into online social networking platforms:
1. Photo Sharing: One of the most common multimedia features in social networking platforms is photo sharing.
Users can upload, share, and view photos with their friends, family, and followers. Platforms like Instagram,
Facebook, and Snapchat have made photo sharing a cornerstone of their user experience, enabling users to express
themselves, document their lives, and connect with others through visual storytelling.
2. Video Sharing: Video sharing has become increasingly popular on social networking platforms, allowing users to
share short clips, longer videos, and live streams with their followers. Platforms like YouTube, TikTok, and
Facebook Live enable users to create and share video content, engage with their audience in real-time, and discover
new content creators and influencers.

Department of Computer Science and Engineering Dhanalakshmi Srinivasan College of Engineering

 VI- SEM/III B.E. CSE Prepared By: R. Reshma/AP/CSE

3. Live Streaming: Live streaming enables users to broadcast live video content to their followers or to a wider
audience in real-time. Platforms like Twitch, Instagram Live, and Twitter's Periscope allow users to stream live
events, gameplay, tutorials, Q&A sessions, and more, fostering interactive and engaging experiences.
4. Stories: Stories have become a popular multimedia format on social networking platforms, allowing users to share
ephemeral content, such as photos, videos, and text, that disappears after a certain period of time. Platforms like
Snapchat, Instagram, and Facebook offer Stories features, enabling users to share casual and spontaneous moments
with their followers.
5. Audio Content: Audio content has seen a resurgence in online social networking, with the rise of podcasts, audio
messages, and audio-based social networking platforms. Platforms like Clubhouse, Twitter Spaces, and Facebook's
Audio Rooms enable users to engage in real-time audio conversations, participate in discussions, and connect with
others through voice interactions.
6. Augmented Reality (AR) Filters and Lenses: AR filters and lenses have become popular features on social
networking platforms, allowing users to enhance their photos and videos with interactive and immersive effects.
Platforms like Snapchat, Instagram, and Facebook offer a wide range of AR filters and lenses, enabling users to add
fun and creative elements to their multimedia content.
Multimedia Ontology
In a multimedia ontology concept might be represented by multimedia entities (images, graphics, video, audio, segments,
etc.) or terms. A multimedia ontology is a model of multimedia data, especially of sounds, still images, and videos, in terms
of low-level features and media structure. Multimedia ontologies enable the inclusion and exchange of multimedia content
through a common understanding of the multimedia content description and semantic information. Multimedia ontology
refers to the formal representation of knowledge about multimedia content and its relationships in a structured and machine-
readable format. It serves as a framework for organizing and describing multimedia data, enabling computers to understand
and reason about the semantics of multimedia content.
At its core, multimedia ontology defines concepts, entities, properties, and relationships relevant to multimedia content,
such as images, videos, audio recordings, and text. These elements are typically organized in a hierarchical manner, with
higher-level concepts representing broader categories and more specific concepts representing finer-grained details. For
example, a multimedia ontology may include concepts such as "image," "video," "audio," "person," "object," "location,"
and so on, along with their respective properties and relationships. Multimedia ontology facilitates interoperability and
integration across different multimedia applications and domains by providing a common vocabulary and semantics for
describing multimedia content. It enables multimedia data to be annotated with metadata that describes its content, context,
and structure in a standardized way, making it easier for computers to search, retrieve, analyze, and process multimedia
content effectively. Furthermore, multimedia ontology enables advanced capabilities such as multimedia retrieval,
recommendation systems, content-based indexing and searching, semantic annotation, and knowledge discovery from
multimedia data. By leveraging the rich semantic information encoded in multimedia ontology, computers can better
understand the meaning and context of multimedia content, leading to more intelligent and personalized multimedia
applications and services.
Multimedia ontology plays a crucial role in advancing multimedia technologies and applications by providing a formal
framework for representing and reasoning about multimedia content in a semantically meaningful way. It enables computers
to bridge the semantic gap between low-level multimedia data and high-level semantic concepts, paving the way for
innovative multimedia applications and services in various domains such as education, entertainment, healthcare, and
security.
Multimedia ontology is a structured representation of knowledge about multimedia content, capturing the semantics,
relationships, and properties of various multimedia elements such as images, videos, audio clips, and textual information.
Ontology, in general, refers to a formal representation of concepts within a domain and the relationships between those
concepts.
In the context of multimedia, ontology serves several purposes:

Department of Computer Science and Engineering Dhanalakshmi Srinivasan College of Engineering

 VI- SEM/III B.E. CSE Prepared By: R. Reshma/AP/CSE

1. Semantic Understanding: By defining concepts, relationships, and properties related to multimedia content,
ontology helps in understanding the meaning and context of multimedia data. For example, ontology can define
concepts such as "image," "video," "audio," "tag," "annotation," "creator," "format," "resolution," etc., and specify
relationships such as "isPartOf," "hasFormat," "hasCreator," etc.
2. Interoperability: Multimedia ontology facilitates interoperability between different systems and applications by
providing a common framework for describing and exchanging multimedia data. It allows different systems to
understand and interpret multimedia content in a consistent manner, enabling seamless integration and
communication between heterogeneous environments.
3. Multimedia Retrieval and Management: Ontology assists in multimedia retrieval and management tasks by
providing structured metadata that can be used to index, search, filter, and organize multimedia content. For
instance, ontology can be used to annotate multimedia files with descriptive metadata, keywords, semantic tags,
and other relevant information, making it easier to retrieve specific content based on user queries and preferences.
4. Semantic Multimedia Analysis: Ontology supports semantic multimedia analysis by enabling reasoning and
inference over multimedia data. By formalizing relationships between multimedia concepts, ontology allows for
advanced reasoning capabilities, such as inferencing relationships between multimedia entities, detecting semantic
similarities, resolving ambiguities, and deriving new knowledge from existing data.
5. Multimedia Applications and Services: Ontology serves as the foundation for developing intelligent multimedia
applications and services. It provides a conceptual framework for designing and implementing various
functionalities, such as content recommendation, personalized multimedia delivery, context-aware multimedia
processing, and multimedia content generation.
Content-Based Retrieval from Digital Libraries
Content-based information retrieval has been shown to be an effective way to search for the type of multimedia documents
that are increasingly stored in digital libraries. Content-based retrieval from digital libraries is a method used to search for
and retrieve multimedia content based on its inherent characteristics, such as visual features, audio attributes, or textual
information. Unlike traditional keyword-based search methods, which rely on metadata or user-provided tags, content-based
retrieval analyzes the actual content of multimedia items to find relevant matches. This approach is particularly useful for
digital libraries containing large collections of images, videos, audio recordings, and documents, where manual annotation
or tagging of every item may be impractical or incomplete. Content-based retrieval systems use various techniques such as
image analysis, audio fingerprinting, text mining, and machine learning algorithms to extract and analyze features from
multimedia content. These features are then used to index the digital library, allowing users to search for similar or related
items based on their content characteristics. Content-based retrieval enables users to find multimedia content that matches
their specific preferences, interests, or criteria, enhancing the accessibility and usability of digital libraries for researchers,
scholars, educators, and other users.
Content-based retrieval from digital libraries refers to the process of searching and retrieving digital resources based on
their content characteristics rather than relying solely on metadata or keywords. This approach allows users to find relevant
information by analyzing the actual content of digital resources, such as text, images, audio, or video. In content-based
retrieval, digital resources are typically indexed and represented using features extracted from their content, such as visual
features for images, acoustic features for audio, or textual features for text documents. These features capture various aspects
of the content, such as color, texture, shape, audio spectrum, or textual semantics, enabling efficient and effective retrieval
of similar or relevant resources.
When a user initiates a search query, the system compares the features of the query with those of the indexed digital resources
to identify potential matches. Similarity metrics or algorithms are used to measure the similarity between the query and the
indexed resources, ranking the results based on their relevance to the query. Content-based retrieval is particularly useful in
scenarios where metadata is incomplete, inaccurate, or inconsistent, as it enables users to find relevant resources based on
their visual, auditory, or textual content. It is widely used in digital libraries, multimedia databases, image repositories, and
audio/video archives to facilitate access to large collections of digital resources and support various applications such as
image retrieval, music recommendation, video summarization, and document search.

Department of Computer Science and Engineering Dhanalakshmi Srinivasan College of Engineering

 VI- SEM/III B.E. CSE Prepared By: R. Reshma/AP/CSE

Content-based retrieval from digital libraries refers to the process of searching, filtering, and retrieving digital content based
on its intrinsic characteristics and features rather than relying solely on metadata or textual descriptions. This approach
allows users to find relevant content within vast collections of digital materials such as documents, images, videos, audio
recordings, and other multimedia assets.
Here's how content-based retrieval from digital libraries typically works:
1. Feature Extraction: The first step involves extracting relevant features from the digital content. For text-based
materials, features may include word frequencies, semantic representations, or document embeddings. For
multimedia content like images, videos, or audio recordings, features may include color histograms, texture
descriptors, shape features, motion vectors, or audio spectrograms. These features capture the intrinsic
characteristics of the content that can be used for comparison and similarity assessment.
2. Indexing and Storage: The extracted features are then indexed and stored in a searchable database or index. This
index allows for efficient retrieval of content based on similarity measures calculated from the extracted features.
Indexing techniques vary depending on the type and volume of content, with methods such as inverted indices, hash
tables, or tree-based structures commonly used for large-scale digital libraries.
3. Querying and Retrieval: When a user submits a query to the digital library system, the system compares the
features of the query with those of the indexed digital content to identify items that are similar or relevant to the
query. The similarity measure used for comparison depends on the type of features extracted and the specific
retrieval algorithm employed. For example, cosine similarity may be used for text-based retrieval, while Euclidean
distance or correlation coefficients may be used for multimedia content.
4. Ranking and Presentation: The retrieved items are ranked according to their similarity scores or relevance to the
query. The top-ranked items are then presented to the user in the search results interface, along with relevant
metadata and contextual information. Users can further refine their search results, explore related content, or
navigate through the digital library based on their preferences and interests.
Content-based retrieval from digital libraries offers several advantages, including:
• Enhanced Precision: By analyzing the content itself, rather than relying solely on metadata or textual descriptions,
content-based retrieval can often achieve higher precision in search results.
• Multimedia Support: Content-based retrieval can handle various types of digital content, including text, images,
videos, and audio recordings, making it suitable for multimedia digital libraries.
• Semantic Similarity: By considering the intrinsic characteristics of digital content, content-based retrieval can
capture semantic similarities that may not be apparent from metadata alone, leading to more accurate and relevant
search results.
*****************

Department of Computer Science and Engineering Dhanalakshmi Srinivasan College of Engineering