Unit 2 Multimedia and Computer Vision
Unit 2 Multimedia and Computer Vision
What is Sound?
• Sound is a form of energy that travels as waves through a medium (like air, water, or solids).
• These waves are created by vibrations. When an object vibrates, it causes the particles in the
surrounding medium to vibrate as well, creating a chain reaction that spreads outwards.
1. Frequency:
o The number of vibrations that occur per second.
2. Amplitude:
Digital Audio
• Multimedia uses digital audio, which is a representation of sound waves in a digital format
that computers can understand and manipulate.
• Sampling: The process of measuring the amplitude of a sound wave at regular intervals to
convert it into a series of numbers.
o The sampling rate is the number of samples taken per second (e.g., 44.1 kHz for CD-
quality audio).
• Quantization: The process of assigning a digital value to each sample, representing its
amplitude.
o Bit depth determines the precision of the amplitude measurement (e.g., 16-bit
audio has more detail than 8-bit audio).
Sound in Multimedia
o Add interactivity
• WAV: Uncompressed audio format, high quality but large file size.
• MP3: Compressed audio format, smaller file size but some loss of quality.
• AAC: Compressed audio format, similar to MP3 but often with better quality.
Important Considerations
• File size: Uncompressed audio files can be very large, so compression is often used for
storage and transmission.
• Quality: The quality of digital audio depends on the sampling rate and bit depth.
• Copyright: Be aware of copyright restrictions when using sound in your multimedia projects.
• It's a protocol: MIDI defines how these messages are formatted and transmitted
between devices.
• It's versatile: MIDI can be used to control synthesizers, samplers, drum machines, and
even software instruments on a computer.
Applications of MIDI in Multimedia
1. Music Production:
o Recording and editing: MIDI allows musicians to record performances as data,
which can then be edited and manipulated in a digital audio workstation
(DAW).
o Controlling virtual instruments: MIDI keyboards and controllers can be used
to play software instruments within a DAW, giving access to a vast library of
sounds.
o Automating music: MIDI data can be used to automate changes in volume,
panning, effects, and other parameters, creating dynamic and expressive
music.
2. Interactive Audio:
o Game development: MIDI can be used to trigger sound effects and music in
video games, creating a more immersive experience.
o Interactive installations: MIDI can be used to create interactive sound
installations where users can trigger sounds and manipulate music in real-
time.
3. Live Performance:
o Controlling instruments: Musicians can use MIDI to control multiple
instruments from a single controller, simplifying their setup and expanding
their sonic possibilities.
o Synchronization: MIDI can be used to synchronize different devices, such as
lighting and video, to music performances, creating a cohesive and dynamic
show.
4. Education:
o Music learning: MIDI can be used to create interactive music lessons and
tutorials, helping students learn music theory and play instruments.
Advantages of MIDI
• Small file size: MIDI files are much smaller than audio files, making them easier to
store and transmit.
• Flexibility: MIDI data can be easily edited and manipulated, allowing for changes in
instrumentation, tempo, and other musical elements.
• Versatility: MIDI can be used to control a wide range of devices and software, making
it a powerful tool for music creation and performance.
In Summary
MIDI is a powerful tool for working with music and sound in multimedia. It provides a
flexible and efficient way to create, manipulate, and control musical performances, making it
an essential part of the modern multimedia landscape.
Speech in multimedia
Speech is a fundamental aspect of human communication, and it plays a crucial role in multimedia
experiences. Here's a breakdown of how speech is used in multimedia:
Images are a fundamental part of multimedia, adding visual appeal, conveying information,
and enhancing user experience. Here are the basic concepts of images in multimedia:
1. What is a Digital Image?
• A digital image is a representation of a visual scene as a collection of tiny dots called
pixels.
• Each pixel has a specific color and brightness value.
• These pixels are arranged in a grid (rows and columns) to form the image.
2. Image Resolution
• Resolution refers to the number of pixels in an image.
• It's often expressed as width x height (e.g., 1920x1080).
• Higher resolution means more pixels, resulting in a sharper and more detailed image.
3. Color Models
• RGB (Red, Green, Blue): A color model where colors are created by combining
different intensities of red, green, and blue light. Used for displaying images on
screens.
• CMYK (Cyan, Magenta, Yellow, Key/Black): A color model used for printing, where
colors are created by mixing inks.
• Grayscale: An image that contains only shades of gray, from black to white.
4. Image Formats
• JPEG (.jpg, .jpeg): A widely used format that employs lossy compression to reduce file
size, suitable for photographs and images with gradual color changes.
• PNG (.png): A format that supports lossless compression, preserving image quality,
and is suitable for images with sharp lines, text, and graphics.
• GIF (.gif): A format that supports animation and is commonly used for simple
animated images.
5. Image Editing
• Image editing software (e.g., Photoshop, GIMP) allows you to manipulate images:
o Adjust brightness, contrast, and color balance
o Crop, resize, and rotate images
o Add text, shapes, and effects
o Remove blemishes and imperfections
6. Image Optimization
• Image optimization is the process of reducing file size without significantly
compromising image quality.
• This is important for web use, where smaller file sizes lead to faster loading times.
7. Image Copyright
• Images are often protected by copyright, which grants the creator exclusive rights to
their use.
• It's important to respect copyright and obtain permission before using images in your
multimedia projects.
Key Considerations for Images in Multimedia
• Purpose: What is the image intended to convey?
• Composition: How are the elements arranged in the image?
• Quality: Is the image clear, sharp, and properly exposed?
• File size: Is the image size appropriate for its intended use?
• Accessibility: Is the image accessible to people with visual impairments (e.g., with
alternative text descriptions)?
By understanding these basic concepts, you can effectively use images to enhance your
multimedia projects and create engaging visual experiences.
Graphics format in Multimedia
Why So Many Formats?
Imagine trying to store a detailed photograph and a simple logo using the same method.
You'd run into problems! Some images need to be small files, others need to be perfectly
sharp, and some might even need to animate. That's why we have different graphics
formats, each designed with specific strengths.
Two Main Types
1. Raster (Bitmap) Graphics:
o These are made of a grid of pixels, like a digital mosaic.
o Good for: Photos, images with lots of colors, and anything that looks realistic.
o Examples:
▪ JPEG (.jpg, .jpeg): The workhorse of the web. Great for photos
because it compresses well (smaller file size) but can lose some detail
in the process (lossy compression).
▪ PNG (.png): Excellent for images with sharp lines, like logos or graphics
with text. Uses lossless compression, so quality is preserved. Also
handles transparency well.
▪ GIF (.gif): Famous for animation! Uses lossless compression and
supports transparency, but limited to a 256-color palette.
2. Vector Graphics:
o These are based on mathematical equations that define lines, curves, and
shapes.
o Good for: Logos, illustrations, anything that needs to be scaled without losing
quality.
o Examples:
▪ SVG (.svg): A web standard. Can be scaled to any size without
becoming pixelated, and files can be quite small.
▪ AI (.ai): Adobe Illustrator's native format. Powerful for creating and
editing vector graphics.
▪ EPS (.eps): A versatile format that can contain both vector and raster
data. Often used in printing.
Choosing the Right Format
Here's a quick guide:
• Photographs: JPEG is usually your best bet.
• Logos, illustrations, web graphics: PNG or SVG are ideal.
• Simple animations: GIF is still commonly used.
• High-quality printing: TIFF is often preferred.
Important Note:
• Compression: Some formats use compression to reduce file size. This can be lossless
(preserving all image data) or lossy (discarding some data to achieve a smaller file).
• Transparency: Some formats support transparency, allowing images to have see-
through areas.
By understanding the strengths of different graphics formats, you can choose the best one
for your multimedia projects, ensuring the right balance of quality, file size, and features.
Video
• What it is: Video is a sequence of images displayed in rapid succession to create the
illusion of motion. Think of it like a flipbook, but instead of drawings, you have
photographs or digitally created images.
• Key Concepts:
o Frames: Individual images that make up the video.
o Frame Rate (FPS): The number of frames displayed per second. Higher frame
rates result in smoother motion. Common frame rates include 24fps (used in
film), 30fps, and 60fps.
o Resolution: The size and clarity of the video image, typically expressed in
pixels (e.g., 1920x1080 for Full HD).
o Aspect Ratio: The ratio of the width to the height of the video frame (e.g.,
16:9 is a common widescreen aspect ratio).
o Compression: Techniques used to reduce the file size of video data, making it
easier to store and transmit.
• Types of Video:
o Live Action: Video captured in real-time using a camera.
o Animated Video: Video created using animation techniques (see below).
o Computer-Generated Imagery (CGI): Video created using computer graphics.
Animation
• What it is: Animation is the process of creating the illusion of motion by displaying a
series of still images in rapid sequence. It's like bringing drawings, puppets, or digital
models to life.
• Key Concepts:
o Frames: Individual images that make up the animation.
o Keyframes: Important frames in the animation that define the main poses or
actions.
o In-betweening: The process of creating the frames between the keyframes to
make the motion appear smooth.
o Principles of Animation: A set of guidelines developed by Disney animators to
make animations more believable and engaging. These principles include
squash and stretch, anticipation, staging, and more.
• Types of Animation:
o 2D Animation: Animation created using flat images or drawings, often drawn
by hand or created using computer software.
o 3D Animation: Animation created using digital models in a 3D environment.
o Stop-Motion Animation: Animation created by physically manipulating
objects (like puppets or clay figures) and photographing them frame by
frame.
Similarities and Differences
• Both video and animation rely on the principle of persistence of vision: Our eyes
perceive a series of still images displayed rapidly as continuous motion.
• Video captures reality, while animation creates an illusion of reality.
• Video can be edited and manipulated, but animation is created from scratch.
Applications in Multimedia
• Video and animation are used in a wide range of multimedia applications, including:
o Entertainment: Movies, TV shows, video games
o Education: Explainer videos, tutorials
o Marketing: Commercials, promotional videos
o Communication: Video calls, video messages
By understanding the basic concepts of video and animation, you can appreciate the
creativity and technical skill that goes into creating these engaging forms of multimedia.
Conventional Multimedia:
Before computers became powerful enough to handle audio and video, multimedia relied
heavily on analog technologies. Think of it as a world of physical media and specialized
equipment.
Key Components of Conventional Multimedia Systems
• Text: Typewriters, printing presses, and even handwriting were used to create and
display text.
• Images: Photography, paintings, drawings, and printed materials were the primary
ways to capture and share images.
• Audio: Records, tapes, and phonographs were used to store and play back sound.
• Video: Film projectors, videotape recorders, and television broadcasts were the
dominant forms of video.
Challenges of Conventional Systems
• Integration: Combining different media types was often cumbersome. For example,
syncing audio with a film required careful manual alignment.
• Editing: Making changes to audio or video was a time-consuming and labor-intensive
process, often requiring physical cutting and splicing of film or tape.
• Storage: Analog media like film reels and tapes were bulky and prone to degradation
over time.
• Accessibility: Sharing and distributing multimedia content was limited by the physical
nature of the media.
Examples of Conventional Multimedia Applications
• Slide shows: Using a projector to display a sequence of images while a presenter
provided narration.
• Film screenings: Showing movies in theaters or using projectors at home.
• Audio-visual presentations: Combining slides, audio recordings, and sometimes even
video clips to create presentations.
• Educational films: Using film or video to teach students in classrooms.
Transition to Digital
The advent of computers and digital technologies revolutionized multimedia. Digital formats
made it easier to:
• Integrate different media types: Combining text, images, audio, and video became
seamless.
• Edit and manipulate content: Digital editing tools provided much greater flexibility
and control.
• Store and distribute content: Digital files could be easily stored and shared
electronically.
Legacy and Influence
While digital multimedia has largely replaced conventional systems, the principles and
techniques developed in the analog era still influence how we create and experience
multimedia today. For example, the concepts of storytelling, visual composition, and sound
design remain essential, regardless of the technology used.
In Summary
Conventional multimedia systems relied on analog technologies to combine different media
types. While they had limitations, they laid the foundation for the digital multimedia
revolution that followed.
The shift to digital has transformed multimedia, bringing incredible enhancements in how
we create, interact with, and experience content.
Key Features of Enhanced Multimedia Systems
• Digital Integration: Seamlessly combining text, images, audio, video, and interactive
elements into cohesive experiences.
• Interactive Content: Enabling users to engage with multimedia through menus,
buttons, hyperlinks, and other interactive features.
• Rich Media: Supporting high-quality audio and video formats, including HD, 4K, and
even virtual reality experiences.
• Accessibility: Providing features like captions, subtitles, and alternative text
descriptions to make multimedia accessible to a wider audience.
• Connectivity: Leveraging the internet and networking technologies to deliver
multimedia content to a wide range of devices and locations.
• Personalization: Tailoring multimedia experiences to individual user preferences and
needs.
Components of Enhanced Systems
• Hardware: Powerful computers, smartphones, tablets, and specialized devices like
VR headsets and interactive displays.
• Software: Sophisticated applications for creating, editing, and delivering multimedia
content, including:
o Authoring tools: Adobe Creative Suite, Final Cut Pro
o Media players: VLC, Windows Media Player
o Web browsers: Chrome, Firefox
• Networks: High-speed internet connections, Wi-Fi, and mobile networks for
distributing multimedia content.
Enhancements and Capabilities
• Increased Interactivity: Users can actively participate in multimedia experiences,
making choices, exploring content, and even contributing their own creations.
• Immersive Experiences: Virtual reality (VR) and augmented reality (AR) technologies
create highly immersive and engaging multimedia experiences.
• On-Demand Content: Streaming services and online platforms provide access to a
vast library of multimedia content whenever and wherever users want it.
• Collaboration: Online tools enable collaborative creation and sharing of multimedia
projects.
• Artificial Intelligence: AI is being used to enhance multimedia in various ways, such
as:
o Automated content creation: AI can generate images, music, and even video
content.
o Content analysis: AI can analyze multimedia to understand user preferences
and provide personalized recommendations.
o Improved accessibility: AI can generate captions and subtitles automatically.
Examples of Enhanced Multimedia Applications
• Interactive websites and web applications: Featuring rich media, animations, and
interactive elements.
• Video games: Immersive and interactive experiences with high-quality graphics and
sound.
• E-learning platforms: Providing engaging and interactive educational content.
• Social media: Platforms for sharing and interacting with multimedia content.
• Virtual and augmented reality applications: Creating immersive and interactive
experiences for entertainment, education, and training.
The Future of Enhanced Multimedia Systems
As technology continues to evolve, we can expect even more enhancements in multimedia
systems, including:
• More realistic and immersive experiences: Advancements in VR and AR will blur the
lines between the real and virtual worlds.
• AI-powered personalization: Multimedia experiences will become even more
tailored to individual user preferences and needs.
• Seamless integration with other technologies: Multimedia will be seamlessly
integrated with other technologies, such as the Internet of Things (IoT) and artificial
intelligence.
Enhanced multimedia systems have revolutionized how we create, share, and experience
content. They have become an integral part of our lives, transforming the way we learn,
communicate, and entertain ourselves.
Multimedia products encompass a vast range of digital content and interactive experiences.
Here's a breakdown of the main types:
1. Entertainment Products:
• Video Games: Interactive games for consoles, computers, mobile devices, and VR/AR
platforms. They often combine stunning visuals, immersive sound, and engaging
gameplay.
• Films and Videos: Movies, documentaries, short films, music videos, and online
video content. They can be live-action, animated, or a combination of both.
• Interactive Storytelling: Experiences that allow users to influence the narrative
through choices and interactions, blurring the lines between stories and games.
• Virtual Reality (VR) and Augmented Reality (AR) Experiences: Immersive and
interactive environments that transport users to other worlds or overlay digital
information onto the real world.
2. Educational and Training Products:
• E-learning Courses: Online courses and tutorials that use multimedia to teach a
variety of subjects. They often include videos, interactive exercises, quizzes, and
progress tracking.
• Simulations: Interactive programs that simulate real-world situations for training
purposes, such as flight simulators or medical simulations.
• Interactive Textbooks and Educational Apps: Digital versions of textbooks and
educational materials that incorporate multimedia elements like videos, animations,
and interactive exercises.
3. Informational and Reference Products:
• Encyclopedias and Databases: Digital resources that provide access to a wealth of
information through text, images, audio, and video.
• Interactive Maps and Atlases: Digital maps that allow users to explore geographic
information through zooming, panning, and interactive features.
• Museum Exhibits and Virtual Tours: Multimedia installations and online experiences
that showcase museum collections and provide virtual tours of historical sites.
4. Business and Communication Products:
• Presentations: Multimedia presentations used for business meetings, conferences,
and training.
• Marketing and Advertising Materials: Videos, animations, and interactive content
used to promote products and services.
• Interactive Websites and Web Applications: Websites that incorporate multimedia
elements to engage users and provide information.
• Video Conferencing and Collaboration Tools: Applications that allow people to
communicate and collaborate remotely using video, audio, and screen sharing.
5. Creative and Artistic Products:
• Digital Art and Design: Images, illustrations, and animations created using digital
tools.
• Interactive Art Installations: Multimedia installations that respond to user
interaction, creating dynamic and engaging art experiences.
• Digital Music and Sound: Music and sound effects created and manipulated using
digital tools.
6. Social Media Content:
• Short-Form Videos: Engaging video content shared on platforms like TikTok,
Instagram Reels, and YouTube Shorts.
• Interactive Stories: Content formats that allow users to interact with stories through
polls, quizzes, and other interactive elements.
• Live Streams: Real-time video broadcasts that allow creators to interact with their
audience.
Key Considerations:
• Target Audience: Who is the intended audience for the multimedia product?
• Purpose: What is the goal of the multimedia product (e.g., entertainment, education,
information)?
• Platform: What platform will the multimedia product be delivered on (e.g., web,
mobile, VR)?
By understanding the different types of multimedia products, you can better appreciate the
diverse range of digital content and interactive experiences that are available today.