Multimedia computing unit 1: Introduction to multimedia The word multimedia is the combined form of two different words “multi and “media” in which Mul multiple/many or several whereas “Media” means source that refers to different kind of information that including text, graphics, audio, video, animation etc. Thus, multimedia is the field concerned with the computer-controlled integration of text, graphics, drawings, still and moving images (video), animation, and any other media where every type of information can be represented, stored, transmitted, and processed digitally. Using multimedia, the computer information can be represented in audio, video and animated format in addition to traditional format. History of multimedia Assimple example of media communications would be the telegraph, developed in the U.S. by Samuel Morse in the mid 1800's. After that the timeline of history of development of multimedia is listed below. vvy y The concept of multimedia was sketched by a German citizen Paul Nipkow, who developed the first video disc in 1884. In 1895, Gugliemo Marconi sent his first wireless radio transmission at Pontecchio, Italy. in 1901, Gugliemo Marconi detected radio waves beamed across the Atlantic. In 1927,Electronic television was designed and first successfully demonstrated in San Francisco by Philo Taylor Farnsworth. Thomas Alva Edison ‘commissioned the invention of a motion picture camera in 1887 and Silent feature films appeared from 1910 to 1927. In 1945,Vannevar Bush wrote about Memex a device in which an individual stores all his books, records, and communications, and which is mechanized so that it may be consulted with exceeding speed and flexibility In 1960s, Ted Nelson started Xanadu project a kind of deep Hypertext. In 1967, Nicholas Negroponte formed the Architecture Machine Group( A combination lab and think tank responsible for many radically new approaches to the human-computer interface) at Massachusetts Institute of Technology.> In 1968, Douglas Engelbart demonstrated Shared-screen collaboration involving two persons at different sites communicating over a network with audio and video. > In 1969 The birth of Internet by DARPA > In 1974 ~ Email was introduced > In 1976 - Architecture Machine Group proposal to DARPA: Multiple Media > In 1980 - Lippman & Mohl: Aspen Movie Map > In 1985 — Negroponte, Wiesner: opened MIT Media Lab Research at the Media Lab comprises interconnected developments in an unusual range of disciplines. > In 1989 - Tim Berners-Lee proposed the World Wide Web to CERN (European Council for Nuclear Research) > In 1990-K. Hooper Woolsey, Apple Multimedia Lab gave education to 100 people > In 1992 - The first M-Bone audio multicast on the net (MBONE- Multicast Backbone) > In 1993 - U. lilinois National Center for Supercomputing Applications introduced NCSA Mosaic (a web browser) > In 1994 — Jim Clark and Marc Andersen introduced Netscape Navigator (web browser) > In 1995 — Java for platform-independent application development. > In 1996 ~ DVD video was introduced; high-quality, full-length movies were distributed on a single disk. The DVD format promised to transform the music, gaming and computer industries. 1998 — XML 1.0 was announced as a W3C Recommendation. > In 1998 - Handheld MP3 devices first made inroads into consumer tastes in the fall, with the introduction of devices holding 32 MB of flash memory. > In 2000 - World Wide Web (WWW) size was estimated at over 1 billion pages. Advantages and disadvantages of multimedia Advantages of multimedia ‘© Multimedia makes teaching learning easier, attractive and effective. ‘* Multimedia has the potential to be one of the most powerful forms of communicating idea, searching for information, and experiencing new concepts of any communication media ever developed. Multimedia helps to share, views ideas and information in more effective and attractive ways. Using multimedia, the things can be presented to cover a wide range of people using projector. Movie making, cartoon making etc is easy and fast through multimedia. Multimedia is possible to get virtual classes at home as a part of distance learning because of multimedia. Multimedia is also used in Internet to make up the interactive web-page contents. Multimedia helps to make advertisement more attractive and realistic. Multimedia makes the presentation more interesting and audience concentered. ‘+ Simulation of air battle, space shuttle flight are the use of multimedia. Disadvantages of multimedia: ‘+ Multimedia is highly expensive to produce multimedia systems and contents, © Multimedia needs well trained manpower to create and use it. ‘© Multimedia files are too large so, itis time consuming to transfer across the internet and intranet. * New networking protocols are needed to handle bandwidth requirements of video and sound for multimedia. * Copying and altering digital images may unknowingly lead to copyright infringement in multimedia. Types of Multimedia This is very important to understand about the various types of multimedia with purposes. The following are the most common and purposeful types of multimedia.Non-linear (Interactive) Multimedia The nor-linear or interactive multimedia is a form of multimedia that the user can have some control what and when the multimedia is displayed or run. The easiest examples of this multimedia are games and virtual reality. This type of multimedia is arguably the most interesting and enjoyable because we can interact more with it to better feel the content. Hyperactive Multimedia This type of multimedia has a structure of various related elements that the user can control. Although it is not like interactive multimedia, this type of hyperactive multimedia also has certain functions. For example, websites and online games. Linear/Sequential Multimedia This type of multimedia also known as sequential multimedia is a type of multimedia that runs linearly or in a straight line. This type of multimedia only moves in the same direction as we often encounter in all types of video tutorial films. In contrast to interactive multimedia, there is an interaction between the user and the multimedia by using computers, mice, and keyboards. At the same time, linear multimedia in its use must be scheduled and sequentially from start to finish. Examples such as music and television station broadcasts. so J Multimedia computer system A Multimedia computer system is a system capable of processing multimedia data and applications. It is characterized by the processing, storage, generation, manipulation and rendition of Multimedia information. A multimedia system is responsible for developing a multimedia application. A multimedia application is a bundle of different kinds of data. So, multimedia computer system is one that can create, integrate, store, retrieve delete two or more types of media materials in digital form, such as audio, image, video, and text information.Following are some major concerns of a Multimedia System: Hardware Processor (CPU): Intel Core iS (sixth generation or newer) or equivalent Operating System: Microsoft Windows 10 Professional x64 (free via Azure Dev Tools for Teaching. Restrictions apply.) Memory: 16 GB RAM or more Storage: 512 GB intemal Solid-State Drive (SSD) or 1 TB internal HDD Sustainability EPEAT Silver rating (preferably EPEAT Gold) Monitor/Display: 24" LCD monitor, Accelerated Graphics Card Network Adaptor: 802.1 1ac 2.4/5 GHz wireless adapter Input Devices : (Keyboards, Mice, Digital Cameras, MIDI Keyboards, Touch Screens, Trackballs, Scanner , Voice Recognition Systems, Magnetic Card Encoders and Readers, Tablets etc.) Output Devices: (High resolution monitors, Speakers, Printer, Projector, Video Devices etc.) Storage Devices: (CD-ROM Drives, Pen drive, Magneto-optical drives, Communication devices, Modems etc.) Other devices: — Webcam, lock, external drive for backups Multimedia File System The multimedia file system must be efficient to meet the requirements of continuous media. These media files require very high-disk bandwidth rates, Disks usually have low transfer rates and high latency rates. To satisfy the requirements for multimedia data, disk schedulers must reduce the latency time to ensure high bandwidth. File formats that support multimediMultimedia data consists of a variety of media formats or file representation including, JPEG, MPEG, AVI, MID, WAV, DOC, GIF, PNG, etc. AVI files can contain both audio and video data in a file container that allows synchronous audio-with-video playback. Like the DVD video format, AVI files support multiple streaming audio and video. Because of restrictions on the conversion from one format to the other, the use of the data in a specific format has been limited as well. Network Support: It includes internet, intranet, LAN, WAN, ATM, Mobile telephony and others. In recent years, there has been a tremendous growth of multimedia applications on the internet like streaming video, IP telephony, interactive games, teleconferencing, virtual world, distance learning and so on. These multimedia networking applications are referred as continuous-media applications and require high communication latency. Communication Latency is the time it takes for a data packet to be received by the remote computer. Software Tools: For the development of multimedia applications, the various software tools like programming languages, graphics software's, multimedia editing software's scripting languages: authoring tools, design software's etc are required. In addition to these the device drivers are required for interfacing the multimedia peripherals. Some of them are listed below > Music sequencing and notation took (Cakewalk, Cubase, Macromedia etc) > Digital audio tools (Cool Edit, Sound Forge ,Pro Tools etc;) > Graphics and image editing too! (Adobe Illustrator, Adobe Photoshop, Macromedia Freehand) > Video editing tools: (Adobe Premiere, Adobe After Effects, Final Cut Pro) > Multimedia API (Java3D, DirectX, OpenGL ) Rendering Tools: (3D Studio Max, Maya animation, GIF Animation ) Multimedia authoring tools: (Author ware, Quest) » Presentation tools: (PowerPoint, keynotes, Google slides) COMPUTER-BASED MULTIMEDIA Technological advances have changed the hardware and software used for developing multimedia from the traditional analog equipment to computer-based or digital multimedia systems. Computers use Os and 1s to store and process sounds, still graphics, motion video etc. Text scanning, digital imaging, sound cards etc converts analog media into digital signals for processing by the computer. Once analog media are converted to a digital format, a computer can be used to manipulate the various media. Multimedia systems have different levels of components that handle tasks ranging in difficulty from simple text processing to complex digital-motion video. e yComponents/elements of multimedia ‘The elements/components of multimedia work together to represent information in an effective and easy manner. There are various elements/components of multimedia: The major and most important one are discussed below. DYNAMIC ELEMENTS. Text Text is the most popular and common method of presenting information in multimedia. Text is commonly used in multimedia for names, headlines, menus, and so on. The keyboard is the most common text input devices, Microsoft Word, Notepad, and Word Pad are the most widely used applications for displaying text files. The text files may have extensions such as DOC, TXT, etc. Text can be further simplified with the following points. > Abroad term for something that contains words to express. » Text is the most basic element of multimedia. > Characters that are used to create words, sentences, and paragraphs, is a fundamental element used in many multimedia applications. » Different text sizes, fonts, color or styles may be used to emphasis the text. Graphics Graphics are at the heart of any multimedia presentation. They refer to images, digital art, drawings, outlines, sketch, etc. The use of graphics in multimedia increases the usefulness and presentation of thedefinition. The extension of Graphics file may be jpeg, .gif, tif, etc. Adobe is a popular graphics editing application that allows you to easily edit graphics and make them more powerful and appealing. Some other examples of graphics based software’s are Ms. Paint, Corel Draw, etc, Graphies can be further simplified with the following points. > A graphic or graphic image or still graphical image, such as drawing, chart, or photograph. > Itis used in multimedia to show more clearly what a particular information is all about (diagrams, picture). > Produced manually (by drawing, painting etc.) or by computer graphics technology. Audio Audio in multimedia refers to recording, playing, and other similar activities. Audio is a most important component of multimedia because it increases the concept's understanding and clarity. For ‘example speech, music, and other forms of audio. Its extensions might be .mp3, .wav, etc. Audio can be further simplified with the following points. > Audio is produced by vibration, as perceived by the sense of hearing. > In multimedia, audio could come in the form of speech, sound effects and also music score. The following programs are widely used to play audio files: > Windows Media Player > Real Player > VLC Media Player Video Moving pictures with sound are referred to as video, The video part of a multimedia application conveys a great deal of information in a short period of time. Hence, It is the most effective means of communication In multimedia applications, digital video is useful for displaying real-life things. The extensions used by video files may be mp4, mkv, etc. Video can be further simplified with the following points. > Video is the technology of capturing, recording, processing, transmitting and reconstructing moving pictures. > Video is more towards photo realistic image sequence / live recording as in comparison to animation. > Video also takes a lot of storage space. So plan carefully before you are going to use it. Following programs are widely used to display videos: > VLC media player > Windows Media Player > Real Player > MX player, etc Animation Computer animation is the process of modifying images in such a way that the sequence of images appears to be moving. Animation plays a number of frames per second to give the user the feeling of motion. The extensions of animation and the programs used in displaying animations are the same as in Video. Animation can be further simplified with the following points. > Animation is the illusion of motion created by the back-to-back display of images of static elements. > Itdisplays a series of still graphics in sequence to give the look of movement. > Animation is used to further enhance the experience of the user to further understand the information conveyed to them.Two broad classes: @ Static, time-independent discrete media: Text, graphics, images. Information in these media consist exclusively of a sequence of individual elements without a time component. & Dynamic, time-dependent continuous media: Sound, video. Information is expressed as not only of its individual value, but also by the time of its occurrence. A note: These notions of time-dependent, discrete and continuous media do not have any connection to the internal representation. They only relate to the impression of the viewer of listener. Properties/ features of multimedia > Combination of media: According to the definition of multimedia system, a multimedia system must be composed with the help of different mediums and devices and all together when works or comes in function then it forms the multimedia system. > Communication systems: Communication capable multimedia system must be approached. Multimedia information not only be created, proceed and stored but also be distributed above the single computer boundary which makes the multimedia application much popular and useful in distributed environment. > Computer supported integration: The different independent media are combined in arbitrary forms to work together as a system with the ‘support of computers. Computer supported integration also called control through the computer in multimedia systems. > Independence: In the multimedia system different media should be independent from each other whereas there should be integrally close-fitting connection between different media to work together also. limitations of Multimedia > Developing a successful multimedia system is non-trivial. Continuous media types such as video need a lot of space to store and very high bandwidth to transmit > They also have tight timing constraints. > Automatically analyzing, indexing and organizing information in audio, image and video is much harder than from text. > Multimedia involves many different research areas and needs more complex and more efficient algorithms and hardware platforms. Multimedia Technology The multimedia technology techniques which includes interactive, computer-based applications that allow people to communicate ideas and information with digital and print elements. Multimedia technologies enable the users to integrate and manipulate data from diverse sources such as video, images, graphics, animation, audio, text etc on a single hardware platform and store them for further reference. Now multimedia has become the latest cultural phenomena in the information sector. The rapid rise of multimedia technology over the last decade has brought about fundamental changes to computing, entertainment and education. The exponential growth of multimedia technologies and applications has > >presented the computerized society with opportunities and challenges that in many cases are exceptional. It is becoming more and more popular because of the effectiveness of its applications in all aspects of human life. Multimedia applications have progressed to the point of sparking a fundamental paradigm shift in the very concept of information transfer and presentation. Global structure of multimedia The global structure of multimedia provides the basic configuration and illustration about the interrelated domains. The global structure contains major four sections including compressions, computer technology, tools and technology also synchronizations with their related domains. The four domains of global structure of multimedia are described below. » Device domain Device domain deals with the interaction between multimedia and multimedia devices such as AGP cards, sound card etc. Itis the basic concepts and skill for processing of images, animation, audio and video data Which are based on digital signal processing. The audio techniques section includes music (MIDI) and speech processing. The understanding of video techniques is built mainly on TV development, including digital representation and HDTV. The current quality requirements and available technology, corresponding compression methods. > System Domain The interface between the device domain and the system domain is specified by the computer technology. To utilize the device domain, several system services are needed. System domain including all supports for using the functions of the device domain that is operating systems, communication systems (networking) and database systems. The operating system serves as an interface between computer hardware with system software and all other software components. (Network, memory, processor, input/output devices etc). Database System allows a structured access to data a management of large database. Communication System data transmission according to the timing and reliability requirements of the networked multimedia application. > Application Domain The services of the system domain are offered to the application domain through proper programming abstractions. Application domain provides functions to the user to develop and present multimedia projects. This includes Software tools and multimedia projects development methodology. A document consists of a set of structural information, represented in different media and generated or recorded at the time of presentation. Many functions of document handling and other applications are accessible and presented to the user through a user interface. > Cross Domain Compositions must allow any type of logical structure besides those ones for synchronization (presentation) purposes. Multimedia involves multiple elements of text, audio, images, drawings, animation, and video. It provides the ability to manually or automatically access or transfer information between two or more different domains. Synchronization are difficult to locate in one or two components ‘or domains. Synchronization the temporal relationship among various media, relates to many components across all domainsApplication domain — provides functions the user to develop and present multimedia Une Prosammue fg Projects. This includes Software tools, and Atsacties Mf multimedia projects development methodology System domain — including all supports for ce ore “"] using the functions of the device domain, 8. operating systems, communication eat systems (networking) and database systems. ao “Bi processing Various multimedia elements and for Medium in multimedia Medium in multimedia means for distribution and presentation of information. A medium is a third-party or ‘element through which a message is communicated. This seems to apply to information technology as ‘well as to multimedia. Some of the common medium related to multimedia are discussed below. >» Perception Medium Perception medium refers to the nature of information perceived by humans, which is not strictly identical to the sense that is stimulated. For example, a still image and a movie convey information of a different nature, though stimulating the same sense. The question to ask here is how do humans perceive information in a computer? What we see and what we hearing information. The perception of information through hearing, auditory media include music, sound, noise and voice. Perception of information through seeing the visual media include text, graphics, still image and moving pictures consist of a series of scenes. Sycheniatin > Representation Medium The term representation medium refers to how information is represented internally to the computer. The encoding used is of essential importance. The question to ask here is: How is information encoded in the computer? There are several option * Each character of a piece of text is encoded in ASCII. * A picture is encoded by the CEPT or CAPTAIN standard, or the GKS graphics standard can serve as a basis. * An audio data stream is available in simple PCM encoding and a linear quantization of 16 bits per sampling value. * Asingle image is encoded as group-3 facsimile or in JPEG format. * A combined audio-video sequence is stored in the computer in various TV standards in MPEG format. » Presentation Medium The term presentation medium refers to the physical means used by systems to reproduce information for humans. Presentation media refer to the tools and devices for /O of information. For example, a TV set, monitor, printer, paper speaker etc. The question to ask here is which medium is used to output information from the computer or input in the computer? Media such as TV, monitor, printer and speakers are output media, while keyboard, mouse, camera and microphone are input media> Storage Medium The storage medium is used in computing to refer to various physical device means for storing computer data, such as hard disk, magnetic disk/tapes, pen drive, compact disk, or digital optical disk. Data ‘storage is not limited to the components available in a computer, and paper is also storage medium. The ‘question to ask here is where is information stored? > Transmission Medium The transmission media different data storage and continuous information transmitted over networks. for ‘example electronic mailing systems, The transmission medium refers to the physical means cables of various types, radio tower, satellite, or ether (the medium that transmit radio waves) that allow the transmission of telecommunication signals. The question to ask here is which medium is used to transmit data? The information is transmitted over networks, which use wired cable transmission such as coaxial cable, fiber optical, wireless as well as free air space transmission. > Information Exchange Medium Information exchange media include all data media used to transport information, e.g., all storage and transmission media. The question to ask here is: Which data medium is used to exchange information between different locations? For example, information can be exchanged by storing it on a removable medium and transporting the medium from one location to another. These storage media include microfilms, paper, and floppy disks. Information can also be exchanged directly, if transmission media such as coaxial cables, optical fibers, or radio waves are used. Representation values and Representation space: Representation values: Representation values determines the information representation of different media. A sentence can be represented by text, which is a sequence of characters or by speech, which is in a waveform. Representation values can be considered as a range or a sequence of discrete values. Examples of representation values are given below. > Continuous representation values: Electro-magnetic waves. > Discrete representation valuies: Text characters in digital form. Representation space Representation space determines the surrounding where the media are presented. Each representation ‘space consists of one or more representation dimensions. Examples of visual representation spaces are Paper or screen. During a computer-controlled slide show with simultaneous projection of the computer screen content, the whole movie counts as a representation space. The most common representation space are as follows. » Visual representation space: paper or screen. > Acoustic (audio) representation space: stereo. Representation dimensions: - Each representation space consists of one or more representation dimensions. A computer screen has two spatial dimensions; holography and stereophony require an additional spatial dimension. Time can ‘occur in each representation space as an additional dimension as it has central meaning to multimedia system. The common examples of representation dimension of representation space are: 1. Spatial dimensions > Two dimensional (2D graphics)> Three dimensional (holography) 2. Temporal dimensions > Time independent (document)- Discrete media > _ (Information consists of a sequence of individual elements without a time component) > Time dependent (movie)- continuous media > (Information is expressed not only by its individual value but also by its time of occurrence) Challenges for Multimedia Systems There are several challenges occur while building multimedia system. One of the principal challenges in building a multi-media system lies in balancing the platform's resources against the demands of the presentation. One of the major issue and challenge is a temporal relationship between many forms of media such as Video and Audio. There are also other forms of problems known as challenges which exists in multimedia, They are discussed below. Sequencing Sequencing the components of multimedia in multimedia one of the key challenges while developing multimedia, Sequencing is the process of translating timed data into correctly timed presentation. Though sequencing logic is a central part of all multimedia applications, it tends to be tightly integrated with specific media formats, authoring models, timing/control primitives and/or predefined Ul elements. Within the media which occurs by playing frames in correct order/time frame in a video. Synchronization Synchronization is another challenge while producing multimedia content. Multimedia synchronization refers to the coordination of multimedia information along three axes: content, space, and time. The media streams must be highly synchronized, so that speech remains lip synchronized, and the Whiteboard updates are synchronized with audio references to them. Inter-media scheduling The next challenge of multimedia system is an inter-media scheduling which is concerned with maintaining the requirements of the temporal relationships between two or more media. Lip scheduling between video and audio is an example of inter media scheduling where the display of video must synchronize with audio. Distributed network Multimedia systems in distributed network consist of multimedia databases, proxy, information servers, clients and several hardware and software application which are intended to for the distribution of multimedia content over the networks. This is also an important challenge in multimedia system since the multimedia systems should support multiuser system and run smoothly. Components of multimedia Systems: Capture devices: Video Camera, Video Recorder, Audio Microphone, Keyboards, mice, graphics tablets, 3D input devices, tactile sensors, Video Recording devices. Digitizing/Sampling Hardware etc. Storage devices: — Hard disks, CD-ROMs, Zip drives, DVD, etc Communication Networks: Ethernet, Token Ring, FDDI, ATM, Intranets, Internets, extranets etc. Computer System: Multimedia Desktop machines, Workstations, MPEG/VIDEO/Digital Signal Processing (DSP) Hardware etc. Display Devices: CD-quality speakers, HDTV(High Definition Television), SVGA, Hi-resolution monitors, Color printers etc. Assignment 1. What is multimedia? Write the advantages and disadvantages of multimedia. 2. Differentiate between linear and non-linear multimedia.Explain the elements/components of multimedia. Mention the features and challenges of multimedia. Explain with illustration about global structure of multimedia. Describe any five applications (fields) of multimedia. Explain the types of medium related to multimedia NO Unit 2: Text Text is one of the most imperative elements of multimedia and an essential source of presenting information to a wide range of people which consists of characters, punctuation symbols, etc. to convey a message. Text can be found in the form of newspapers, magazines, pamphlets, billboards, books, websites, etc. It is also an integral part of songs, movies, TV, radio, advertisements, video games, presentations, etc. Text is concemed with the following issues and sub elements. Fonts and Faces A font is a collection of character or glyphs of a single size and style belonging to a particular typeface. A font typeface is a family of many characters often with many type sizes and styles. A glyph is a graphic representation of a character's shape where a character may be represented by many glyphs. Most computer users are familiar with word processing and know the processes of entering and editing text and working with fonts and font sizes. Many fonts are also available online and people can download them from a server. They are classified on the basis of spacing between characters, words, presence or absence of serifs, their shape, stretch and weight such as bold or italics. Font size is measured in points and it does not describe the height or width of its character. Some of the common types of fonts are: Arial, Times NewRoman, Arial black, Bodoni, Bookman, Georgia, Impact, verdana, sans serif etc. Factors affecting legibility of text (font) are as follows: © Size and style ‘+ Background and foreground colors * Leading Font styles(types) Arial Arial Black Comic Sans MS Courier New Georgia Impact Times New Roman Different sizes of fonts There are three airy RAGEs OF OO MUN CAL Project. “There are three main stages OF a muttimedia project. THERE ARE THIREE MAIN sTACES oF A MULTIMEDIA PROJECT. Use of background and foreground colors Effects of different leading beneUsing Text Elements in a Multimedia Presenta The text elements used in multimedia are given below: Menus for Navigation ‘+ Auser navigates through content using a menu. ‘+ Assimple menu consists of a text list of topics Interactive Buttons ‘+ Abbutton is a clickable object that executes a command when activated + Users can create their own buttons from bitmaps and graphics. * The design and labelling of the buttons should be treated as an industrial art project. Symbols and Icons * Symbols are concentrated text in the form of stand-alone graphic constructs and are used to convey meaningful messages and human emotions are called emoticons. ‘+ loons are symbolic representations of objects and processes Use of Text in Webs Using text in websites attract a visitor's attention as well as help him in understanding the webpage better. Itis far better than the use of meaningless graphics and images which do not contribute in understanding of the page. Website Loading Speed Website loading speed is one of the important factors that influences conversion as visitor's starts to leave the page if it takes more than eight seconds to load. A website which contains a lot of text loads faster than the websites that contains the following: Internal code (not placed in external CSS, JS, ete. files and linked to) A lot of images and graphics JavaScript (for menus, including various stat tracking scripts, such as Google Analytics). Audio and video clips on the page (especially without transcripts, which hurts accessibility if you do use audio/video, do not auto-launch it and have a button to tum it on/off). ‘+ Table-based layouts that are twice larger in file size, than the ones built in CSS. Text in Films Such as Titles and Credits Notes Most films start with titles and end with credits. The text is shown over either plain background or colored background. Typography look different in different formats such as a in film subtitles, on websites, poster, essay, etc. To include a text in multimedia, a designer has to keep in mind the points given below: The theme or look of the multimedia product. The amount of text needed. The placement of the text (heading, body text or logo). The format of the project (video, website, blog, video, slideshow, etc.) The content of the information Use of text in a film posterText in Advertisements The explosive growth of multimedia data on the web creates huge opportunities for further monetizing them with multimedia advertisements. Multimedia content becomes a natural information carrier for advertising in a way similar to radio wave to carry bits in digital communications. More and more business models are rolled out to freely distribute multimedia contents and recoup the revenue from the multimedia advertisements it carries. With the increasing importance of online multimedia advertising, researchers from multimedia community have made significant progresses along this direction. Local business, such as a restaurant or bookstore, coffee shop, boutique or convenience store owners use text advertising. Similarly, event planners and party organizers publicize their events by treating in text advertising. Importance of Text Based Advertising Since the text ads are more of keyword oriented, they draw more attention than banner advertising. The text ads are inexpensive, thus making it affordable and effective for your business. There are a few websites which offers a flat free rental services to place your text based advertisements. A few websites request for a onetime payment to place your text ads The foremost benefit of having text based advertisements is that it helps in improving your search engine ranking, Since it creates more visibility and draws more traffic to your site, your page rank will be improved ‘Font Editing and Design Tools A font editor is a class of application software specifically designed to create or modify font files. Font editors differ greatly depending on if they are designed to edit bitmap fonts or outline fonts. Most modem font editors deal with the outline fonts. Special font editing tools can be used to make your own type, so we can communicate an idea or graphic feeling exactly. With these tools, professional typographers create distinct text and displays faces. Font Monger Font Monger from Ares Software offers a proprietary hinting technology to ensure that your fonts will look good regardless of size. To create new fonts or to manipulate existing ones, Font Monger includes a freehand drawing outline. Type-Designer Type-Designer for windows from DS Design is a font editor that lets you create, convert, and manipulate PostScript Type1 and TrueType fonts as well as EPS file format illustrations. An extensive palette of editingtools allows you to make changes to a font’s outline. With Type-Designer you can open up to eight typefaces simultaneously and cut and paste characters between them. ResEdit ResEdit is a source editor available from apple that is useful for creating and changing graphic resource such as cursors, icons, dialog boxes, patterns, keyboard maps, and bitmapped fonts on the Macintosh. It can be used to edit or create new font resources for storing the bitmaps of screen fonts. Font Chameleon Font Chameleon from Ares software for both Macintosh and Windows platforms builds millions of different fonts from a single master font outline. The program provides a number of pre-set font descriptors, which you build into a PostScript Type 1, or True Type Font. With slide bars you can manipulate various aspects of the font, including its weight, width, x-height, ascenders, descenders, and the blend of the serifs. The fonts we do build from the master outline can be used on the Macintosh, Windows, or OS/2 platforms Cool 3D Text Cool 3D Production Studio is a program for creating and animating 3D text and graphics, for videos and other multimedia products. This software runs on Windows 98SE/ ME/2000/XP. With this program, a user ‘can create 3D graphics, animations for videos. It includes new modelling tools, animations plugs-in, and new features for animation and video. Hypertext Hypertext is different from normal text in that it is nonlinear. The reader need not read a document from beginning to end, but can jump around within the document by clicking on hot spots (or hyperlinks) in the text. Example: Hypertext systems include web pages and Windows help pages. The function of hypertext is to build links and generate an index of words. The index helps to find and group words as per user's search criteria. Hypertext systems are very useful in multimedia interactive education courseware. Hypertext systems provide both unidirectional and bi-directional navigation. Navigations can be through buttons or through simple, plain text. The simple and easy navigation is through linear hypertext where information is organized in linear fashion. Nonlinear hypertext, however, is the ultimate goal of effective navigation. ETF free gromenimbernet ; Liesl We Sil snl ft i ‘* Using Hypertext Systems Information management and hypertext programs present electronic text, images, and other elements in a database fashion. Software robots visit Web pages and index entire Web sites. Hypertext databases make use of proprietary indexing systems. Server-based hypertext and database engines are widely available. Hypertext databases make use of proprietary indexing systems. Hypermedia Hypermedia is an extension of the term hypertext, is a nonlinear medium of information that includes audio, video, graphics, plain text, and hyperlinks. Hypermedia is communicational, cultural, and social is the blending of the media, telecommunications, and computing. This convergence is the adoption of digital technologies across three sectors: Audio, Video, and hypertext. The development of increasinglypowerful personal computers enables one-to-many transmission of information to be combined with one- {o-one interaction and in a combination of various media. Difference between Hypertext and Hypermedia Hypertext Hypermedia 1. Hypertext refers to the text which links to 1. Hypermedia refers to connecting the hypertext with other pieces of text of same or another — media such as graphics, sounds, and animations. document. 2. Hypertext is a part of Hypermedia 2. Hypermedia comes in the superior level entity 3. Hypertext involves only text 3. Hypermedia involves graphics, image, video, and audio. 4, Hypertext is a non-linear way. 4. Hypermedia is a linear way. 5. It doesn't provide a better user experience. _5. It provides a better user experience. Text filed Formats in Multimedia The following is an outline of current file formats used for the production and delivery of multimedia data RTF Rich Text Format is the primary file format introduced in 1987 by Microsoft with the specification of their published products and for cross-platform documents interchange. Plain text Plain text files can be opened, read, and edited with most text editors. commonly used are Notepad (Windows), Gedit or nano (Unix, Linux), TextEdit (Mac OS X) and so on. Plain text is the original and popular way of conveying an e-mail. Some other text file Formats are: + .docand docx Microsoft Word file * odt :OpenOffice Writer document file «pdf : PDF file tex :ALaTeX document file © txt : Plain text file Assignment1. What is text? Elaborate the comparisons between hypertext and hypermedia. 2. Define the term font also discuss about some font editing and design tools. 3. Explain about the text elements used in multimedia, 4. Write the use and purpose of text in web and advertisement. 5. What is file format? Discuss the types of text file format. unit 3: Sound and audio system Introduction to audio/sound Audio: A multimedia application may require the use of speech, music and sound effects. These are called audio or sound element of multimedia. Speech is also a perfect way for teaching. Audio are of analog and digital types. Analog audio or sound refers to the original sound signal. Computer stores the sound in digital form. Therefore, the sound used in multimedia application is digital audio.What is “Audio?” c Audio means "of sound" or "of the reproduction of sound.” Sound: Sound is perhaps the most affective element of multimedia. It is meaningful "Speech" in any language from a soft voice to a scream. It can provide the listening pleasure of music, the startling accent of special effects or the ambience of a mood-setting background. It is a complex relationship involving: a vibrating object. Sound is a pressure wave which travels in air at 330ms — with a frequency between 20 and 20,000 Hz (variations/second). Sound is a perceptual effect caused by a pressure wave of between 20 and 20KHz being detected at the ear. = Sound comprises the spoken word, voices. music and even = itis a complex relationship involving > a vibrating object (sound source) 2 transmission medium (usually ai) > a receiver (ean ana: Sate a preceptor iran) ap om Difference between sound and audio ‘Sound ‘Audio ‘Sound is mechanical wave energ ‘Audio is an electrical energy ‘Sound is circulated through a medium causing __| Audio is transmitted through variation in pressure within the medium such as | audio communications medium such as an audio gas, liquid or solid channel or audio track ete. The properties of sound include amplitude, The properties of audio include volume, frequency, wavelength etc. normalize, reverse, speed etc. Sound intensity/power/pressure is measured in | Audio Precision APx525 analyzer is used for decibels, making audio system measurements. Conclusion Sound is vibrations through materials or medium. It’s the energy that travels through a medium or a substance as sound frequency waves. Audio is a description of what we hear from digital or human-made sources which give us information, history, and memories that we share with others. Audio can also describe the end result of sound. The power of soundy >>> | Something vibrates waves of pressure Ear drums will translate in the air these changes in wave Forms as sound e Sound is measured in > dB (decibel) e Sound waves are known as waveforms. Key note about sound Sound is a continuous wave which is made up of pressure differences. The sound waves have normal wave properties such as reflection (echo), refraction (alteration), diffraction etc. The sound is detected by measuring the pressure level at a location. « Sound transmission Sound is transmitted by molecules bumping into each other and sound waves are transmitted through the air. « Sound receiving Microphone in sound field moves according to the pressure exerted on it. The transducer converts energy in to voltage level. «¢ Sending sound Speaker transforms electrical energy into sound waves. Computerization/digitization of sound The computerization/digitization process of sound technique of converting the analog signals to a digital signal. There are three steps of digitization of sound. Sampling Ore Analog Signal PCM Encoder Digital DataComputer Representation of Sound « Sound waves are continuous while computers are good at handling discrete numbers «In order to store a sound wave in a computer, samples of the wave are taken. Each sample is represented by a number, the ‘code” ‘© This process is known as digitisation. © This method of digitising sound is know as pulse code modulation (PCM), Refer to Unit 1 for more information on digitisation. # According to Nyquist sampling theorem, in order to capture all audible frequency components of a sound, ie., up to 20k //=, we need to set the sampling to at least twice of this. This is why one of the most popular sampling fate for high quality sound is 4410/ = * Another aspect we need to consider is the resolution, ie., the number of bits used to represent a sample. Often, 16 bits are used for each sample in high quality sound. This gives the SNR of 96d 2. Sound wave Awave is a vibratory disturbance in a medium which carries energy from one point to another without there being a direct contact between the two points. We can say that a wave is produced by the vibrations of the particles of the medium through which it passes. There are two types of waves: * Longitudinal Waves A wave in which the particles of the medium vibrate back and forth in the ‘same direction’ in which the wave is moving. Medium can be solid, liquid or gases. Therefore, sound waves are longitudinal waves. « Transverse Waves: Awave in which the particles of the medium vibrate up and down ‘at right angles’ to the direction in which the wave is moving. These waves are produced only in a solids and liquids but not in gases. Fundamental characteristics of sound A sound wave is associated with the following physical characteristics: amplitude, frequency, waveform and speed of propagation. Amplitude Amplitude of a wave is the maximum displacement of a particle in the path of a wave and is a measure of the peak-to-peak height of the wave. The physical manifestation of amplitude is the intensity of energy of the wave. For sound waves this corresponds to the loudness of sound. Loudness is measured in a unit called decibel (dB). Wavelength | Cycle start point Cycle end Amplitude pointFrequency The number of complete waves or cycles produced in one second is called frequency of the wave. Since one complete wave is produced by one full vibration of the vibrating body, so we can say that the number of vibrations per second is called frequency. For example: if 10 complete waves or vibrations are produced in one second then the frequency of the waves will be 10 hertz or 10 cycles per second ae = >> Relation between time-period and frequency of a wave The time required to produce one complete wave is called time-period of the wave. Suppose the time- period of a wave is T seconds. In T seconds number of waves produced = 1 So, in 1 second, number of waves produced will be = 1/T But the number of waves produced in 1 second is called its frequency. Therefore, F = 4/Time-period f=4T where f = frequency of the wave T = time-period of the wave Sampling rate Sampling is a process of measuring air pressure amplitude at equally spaced moments in time, Where each measurement constitutes a sample. A sampling rate is the number of times the analog sound is taken per second. A higher sampling rate implies that more samples are taken during the given time interval and ultimately, the quality of reconstruction is better. The sampling rate is measured in terms of Hertz, Hz in short, which is the term for Cycle per second. Each between each line/sample represents the sampling ‘ period, which equals 1/44,100 of a second, for a CD. with a sampling rate of 44/1 kHz. Quantization ine represents a new sample. The timeQuantization is a process of representing the amplitude of each sample as integers or numbers. Commonly used sample sizes are either 8 bits or 16 bits. The larger the sample size, the more accurately the data will describe the recorded sound. An 8-bit sample size provides 256 equal measurement units to describe the level and frequency of the sound in that slice of time. A 16-bit sample size provides 65,536 equal units to describe the sound in that sample slice of time. The value of each sample is rounded off to the nearest integer (quantization) and if the amplitude is greater than the intervals available, clipping of the top and bottom of the wave occurs. Quantization and quantization errors Nn Loudness and amplitude The other important perceptual quality is This is known as the threshold of feeling. If the loudness or volume intensity is 10“ watt/m?, we may just be able to hear it. This is know as the threshold of Amplitude is the measure of sound levels, For a | hearing digital sound, amplitude is the sample value The relative intensity of two different sounds is The reason that sounds have different loudness | seasured using the unit Bel or more commonly is that they carry different amount of power. deciBel (dB) Itis defined by The unit of power is watt. The intensity of sound is the amount of power transmitted relative intensity in d3 = 101og 2 through an area of |’ oriented perpendicular h to the propagation direction of the sound. ‘Very often, we will compare a sound with the ; threshold of hearing If the intensity of a sound is | watt/m®, we may start feel the sound. The ear may be damaged.Iypical sound levels generated by various sources Taten: Cwatt far ‘Typical sound levels in music Dynamic and bandwidth 160 a5 13048 100 aR. 70 ae soan 30 aR 20 aR ‘* Dynamic range means the change in sound levels. For example, a large orchestra can reach 1304B at its climax and drop to as low as 304B at its softest, giving a range of 10048. Jet engine Large orchestra at fortissimo Car on highway Voice conversation Quiet residential areas Very soft whisper Sound studio a6 120 Threshold af fealing 20 a 80 a 70 7 60 mt 50 P 40 PP 30 Pep 0 Threshold of hearing + Bandwidth 1s the range of frequencies a device can produce or a human can hear. FM radio ‘SOHz— 15kHz AM radio ‘80Hz—SkHz ©D player 20Hz— 20ktiz Sound Blaster 16 sound card 30Hz— 20kHz Inexpensive microphone ‘80Hz — 12kHz ‘Telephone 300Hz— 3kHz Children’s ears 20H2 ~ 20kt4z Older ears Sorte — 10kH1z Male voice 12082 — 7kHz Female voice ‘200Hz —9kHz Quality vs File size “The size of a digital recording depends on the sampling rate, resolution and mumber of IF Mile size bytes channels R samplingrate samples per second ; : 2 resolution bits See eo eee © channels 1 - mono, 2 - stereo Higher sampling rate, higher resolution gives D fecording duration scconde higher quality but bigger file size For example, if we record 10 seconds of stereo music 44.1kHz, 16 bits, the size will be: S = 44100 x (16/8) x 2x 10 1, 764, 000bytes = 1722.7Kbytes LGSMbytes at Note: IXbytes = 1024bytes ete: iMibytes — 1024K bytes High quality sound files are very big, however, the file size can be reduced by compression.File size for some common sampling rates and resolutions Sampling Stereo Size for Rate Resolution /Mono for 1 Min. Comments 4a1KHz 16-bit Stereo 105MB CD-quality recording 44.1KHz 16-bit Mono 5.25MB A good trade-off for high-quality recordings of ‘mono sources such as voice-overs 44.1KH2 8-bit Stereo 5.2SMB Achieves highest playback quality on low-end devices such as most of the sound cards 44.1KHz 8-bit Mono 26MB Aa appropriate trade-off for recording a mono 22.0SKHz 16-bit Stereo 5.2SMB Darker sounding than CD-quality recording because of the lower sampling rate 22.0SKHz 16-bit Mono 25MB Not a bad choice for speech, but better to trade some fidelity for a lot of disk space by dropping down to 8-bit 22.0SKHz 8-bit Stereo 2.6MB_A-very. popular choice for reasonable stereo recording where full bandwidth playback is not possible 22.0SKHz 8-bit Mono 13MB A thinner sound than the choice just above, but very usable UIKHz — 8-bit Stereo 13MB At this low a sampling rate, there are few advantages to using stereo KHz —8-bit Mono 650K ——_Inpractice, probably as low as you can go and still get usable results S.SKHz 8-bit Stereo 650K Stereonot effective SSKHz 8-bit Mono 325K About as good as.a bad telephone connection Music Music is any organized collection of sounds. Computer music is any music constructed, reproduced or performed with the aid of computer technology. More specifically, the term usually refers to real-time music a computer software program generates based on a predetermined set of constraints. Computer music: - MIDI The relationship between music and computers has been very important for the development specially MIDI. The MIDI interface between electronic musical instruments computer is a small piece of ‘equipment that plugs directly into the computer serial port and allows the transmission of musical signal. Common MIDI devices include electronic music synthesisers, modules, and MIDI devices in common sound cards. General MIDI is a standard specified by MIDI Manufacturers Association. To be GM compatible, a sound generating device must meet the General MIDI system level 1 performance requirement. «minimum of 24 filly voices + 6chamels, percussion on chanel 10 ENERA + immu 16 smatapeous and diferent tinbe insmuents This sign indicated that | manu 128 preset insruments the device is a general «Suppor certain conrolers MIDI device. Concept of MIDI MIDI stands for Musical Instrument Digital Interface is a music and sound technology developed in 1983 by a panel of music industry representatives and is maintained by the MIDI Manufacturers Association (MMA). MIDI is a protocol which allows electronic instruments and other digital musical tools tocommunicate with each other. MIDI itself does not make sound, it is just a series of messages like "note on, ‘note off." “note/pitch," "pitch bend,” and many more. These messages are interpreted by a MIDI instrument to produce sound. A MIDI instrument can be a piece of hardware (electronic keyboard, synthesizer) or part of a software environment (ablation, garage band, digital performer, logic...). Advantages of MIDI Since they are small, MIDI files embedded in web pages load and play promptly. Length of a MIDI file can be changed without affecting the pitch of the music or degrading audio quality MIDI files will be 200 to 1000 times smaller than CD-quality digital audio files. Therefore, MIDI files are much smaller than digitized audio. MIDI files do not take up as much as RAM, disk space and CPU resources. A single MIDI link can carry up to sixteen channels of information, each of which can be routed to a separate device. MIDI hardware MIDI hardware is an electronic musical instrument or a computer which has MIDI interface with one ‘or more MIDI ports. The MIDI ports on musical instrument are usually labelled with. IN: For receiving MIDI data OUT: For outputting MIDI data that are generated by the instrument. THRU : For passing MIDI data to the next instrument.Components of a MIDI System Synthesizer: It is a sound generator (various pitch, loudness, tone color). Sequencer: It can be a stand-alone unit or a software program for a personal computer. (It used to be a storage server for MIDI data. Nowadays it is more a software music editor on the computer. It has one or more MIDI INs and MIDI OUTs. ‘Track: Track in sequencer is used to organize the recordings. Tracks can be tured on or off on recording or playing back Channel: MIDI channels are used to separate information in a MIDI system. There are 16 MIDI channels in one cable. Channel numbers are coded into each MIDI message. Pitch: Musical note that the instrument plays. Voice: Voice is the portion of the synthesizer that produces sound. Synthesizers can have many (12, 20, 24, 36, etc.) voices. Patch: The control settings that define a particular quality. MIDI Messages The sound hardware such as MIDI transmits the information between devices and determine what kinds ‘of musical events can be passed from device to device. Such messages are divided into two different types 1. Channel Messages: Channel messages go only to specified devices. There are two types of channel messages: a. Channel Voice Messages: Send actual performance data between MIDI devices. Example: Note On, Note Off, Channel Pressure, Control Change etc. b. Channel Mode Messages: Determine the way that a receiving MIDI device responds to channel voice messages. Example: Local Control, All Notes Off, Omni Mode Off ete. 2. System Messages: ‘System messages go to all devices in a MIDI system because no channel numbers are specified There are three types of system messages: a. System Real-time Messages: System real time messages are very short and simple, consisting of only one byte. They carry extra data with them. Example: System Reset, Timing Clock etc. b. System Common Messages: System common messages are commands that prepare sequencers and synthesizers to play a song. Example: Song Select, Tune Request etc. (iii) System Exclusive Messages: System exclusive messages allow MIDI manufacturers to create customized MIDI messages to send between their MIDI devices. MIDI Software‘MIDI player for playing MIDI music. This includes: ‘* Windows media player ean play MIDI files * Player come with sound card — Creative Midi player Freeware and shareware players and plug-ins— Midigate, Yamaha Midplug, etc MIDI sequencer for recording, editing aul playin MIDI ‘© Cakewalk Express, Home Studio, Professional © Cubasis © Encore ‘© Voyetra MIDI Orchestrator Plus ‘Configuration — Like audio devices, MIDI devices require a driver. Select and configure MIDI devices from the control panel. Basic concept of SMPTE ‘SMPTE stands for Society of Motion Picture & Television Engineers developed by NASA is the interface standard for electronic musical instruments and the time code standard in the television and film world. This standard specifies the content, format, and recording method of the data blocks containing video, audio and associated data which form the spiral records on 12.65-mm (0.5-in) tape in cassettes as specified in ANSI/SMPTE 263M. —In addition, this standard specifies, the content, format, and recording method of the longitudinal record containing tracking information for the scanning head associated with the helical records, and also the longitudinal cue audio, and time and control code. — One video channel and four independent audio channels are recorded in the digital format. Each of these channels is designed to be capable of independent editing, eed Digital Audio Digital audio is a technology that uses digital signals for sound reproduction which includes analog-to-digital conversion, digital-to-analog conversion, storage, and transmission. It is alsoreferred to as the way that music and sound is stored on digital devices such as hard drives, iPods, websites (servers) and CDs. by > Difference between digital audio and MIDI Digital Audio ‘* Digital representation of physical sound ‘* File size is large if without compression ‘© Quanity 1s in proportion to nie size + More software available ‘* Play back quality less dependent on the sound sources ‘* Can record and play back any sound including speech Audio hardware ‘* Recording and Disitising sound + An analog-to-digital converter(ADC) ‘converts the analog sound signal into digital samples, + A digital signal processor(@DSP) processes the sample, e.g. filtening, modulation, compression, and so on. * Play back sound: + A digital signal processor processes the sample, e.g. decompression, demodulation, and so on + An digital-to-analog converter(DAC) converts the digital samples into sound signal + All these hardware devices are integrated into a few chips on a sound card Audio software MIDI ‘© Abstract representation of musical sounds and sound effects ‘* MIDI files are much more compact ‘+ File size is independent to the quality ‘* Much better sound if the sound source is of ‘hugh quality ‘* Need some music theory ‘+ Camot generate speech ‘¢ Different sound card have different capability of processing digital sounds ‘When buying a sound card, you should ook at: + maximum sampling rate = stere or mone = duplex or simplex eos fey =© Windows device driver — controls the hardware device Many popular sound cards are Plus and Play Windows has drivers for them and can recognise them automatically. For cards that Windows does not have deivers, you need to get the diver foes the manufacturer and install it with the card # Ifyou do not hear sound, you should check the settings, such as interrupt, DMA channels, and so on. * Device manager — the user interface to the hardware for configuring the devices, + You can choose which audio device you want te use + You can set the audio volume Preparing Digital Audio Files We can follow the steps given below to prepare digital audio files: * Balancing the need for sound quality against available RAM and hard disk resource. * Setting appropriate recording levels to get a high quality and clean recording * To digitize the analogue material recording it into a computer readable digital media. The sampling rate determines the frequency at which samples will be drawn for the recording. Digital Audio File Formats Digital audio formats are file formats for storing digital audio data on computer systems. These formats can be uncompressed (all the original data) or compressed in one of two ways: lossy (unnecessary info is irreversibly removed to save space) or lossless (redundant data is removed to save space). Digital sound files can be created in a number of different formats. Broadly, the main categories are Compressed or Uncompressed. Compressed formats (such as MP3) have much smaller file size than Uncompressed formats, but there are sacrifices in terms of the quality of the sound WAV (Waveform Audio File Format) Itis the most popular audio file format in windows for storing uncompressed sound files. In order to attain the reduced file size it can also be converted to other file formats like MP3. MP3 (MPEG Layer-3 Format) MPEG Layer-3 format is the most popular format for storing and downloading music. The MP3 files are roughly compressed to one-tenth the size of an equivalent WAV file OGG A free, open-source container format that is designed for obtaining better streaming and evolving at high end quality digital multimedia. It can be compared to MP3 files in terms of quality. AIFF (Audio Interchange File Format) It is an audio file format developed by Apple Inc. used for storing sound data for personal computers and other electronic audio devices. WMA (Windows Media Audio) Itis a popular windows media audio format owned by Microsoft. WMA is a file extension used with windows media player. RA (Real Audio Format) Real Audio format is designed for streaming audio over the Internet. The digital audio resources are usually stored as a computer file in computer's hard drive or CD/DVD. Besides the variety of audio file formats available, the most common formats are wave files (.WAV) and MPEG Layer-3 files (MP3), WMA and RA.SND (SouND) file format SND is the Sound file format developed by Apple. It is used mainly within the operating system and has a limited sampling rate of eight bits. For a multimedia application to work on both PCs and Mags, save it using either the Musical Instrument Digital Interface (MIDI) or the Audio Interchange File Format (AIFF) file format. It is recommended to use AIFF format if sound is a part of the application. AIFF is a cross platform format and it can also reside outside the multimedia application. Now the file occupies less space and play faster. Moreover, if a user wants to burn the multimedia application onto a CD, AIFF format can be used. Digital audio file formats Audio compression When songs are recorded, mixed, and stored, it’s standard practice to work with equipment, programs and files that are capable of preserving the full quality of the music. Before digital audio, songs were stored in analog format: first vinyl records, then tape cassettes, then eventually moving on to CDs at the beginning of the digital age of music. Sure, these folks take up some space, but the other thing they have in common is the ability to store all (or at least most) of the data required to play your music in high fidelity. There are lossy or lossless audio file compressions. Lossy audio compression Lossy audio compression refers to a process where audio signals are first stripped down and then compressed. The compressed data can be decompressed to something which can sound fine but isn't a 1:1 copy of the original. Compressed lossy audio files are made by removing certain types of audio data to shrink the file size. Lossy compression can be adjusted to compress audio a lot, or to compress audio very little. As a result, most audio file formats strive for a balance between audio quality and file size. The most common compressed lossy audio formats are AAC (Advanced Audio Coding) AAC and MP3. Lossy compression is about quantity. We can take the typical example of excellent from a CD to MP3. CD-quality connects to a sample rate of 44.1 kHz and a bit-depth of 16-bit. The above numbers give CD audio a bit rate (the number of bits/data stored per second of audio) of 1,411 Kbps. While the bitrate never changes for CDs, an MP3 file's bitrate can range from 96 — 320 Kbps. So, you can see, even at the highest bitrate, MP3 has a significant drop-off when compared to CD. Lossless audio compression Compressed lossless audio files are designed to squeeze audio data into a smaller file size. They stil take up a lot of space, but the benefit is that you still don’t lose any audio data. Theoretically, this file type should sound identical to the aforementioned lossless audio files. Two examples of compressed lossless audio file formats are FLAC (Free Lossless Audio Codec) and Apple Lossless. Lossless formats can be referred to as high-resolution audio. Lossless compression is about quality. Speech Speech is the expression of or the ability to express thoughts and feelings by articulate sounds which can be perceived, understood and generated by humans and machines. Human own-self can adjust speech to the different speakers and speech habits. The human speech signal comprises a subjective lowest spectral component known as tone which is not proportional to frequency. Speech generationSpeech generation is the mean for producing spoken messages in response to signals from a data processing or control system. Speech generation is used to communicate between humans and machines. Rather than using our hands and eyes, we use our mouth and ears. This is very convenient when our hands and eyes should be doing something else, such as: driving a car, performing surgeries etc. Two approaches are used for computer generated speech: digital recording and vocal tract simulation. In digital recording, the voice of a human speaker is digitized and stored, usually in a compressed form. During playback, the stored data are uncompressed and converted back into an analog signal. Note: Generated speech must be understandable and should sound natural. The requirement of understandable speech is a fundamental assumption and the natural sound of speech increases user acceptance. Speech-generating devices Speech-generating devices are hand-held electronic devices that play words or phrases when the user touches a switch or presses buttons or keys, Some devices ‘speak’ words as the words are typed on a keyboard. Speech-generating devices can be specialized devices used only for communication. They can also be tablets, laptops or other common devices with speech-generating apps or functions. These devices are also known as communication devices, electronic augmentative and alternative communication (AAC) devices or systems and voice output communication aids. Some of them are : AAC speech recognition device, GOTALK, Light Writer, Allora, Grid Pad Eye etc. People who have difficulty in developing language or communicating using speech, use speech-generating devices. These devices are most commonly used by people who have difficulty pronouncing words because of a physical disability like cerebral palsy or acquired brain injury. Autistic children sometimes use speech-generating devices, particularly when they're developing speech and language. Basic notions about speech % The lowest periodic spectral component of the speech signal is called the fundamental frequency which is presented in voiced sound. Aphone is the smallest speech unit, such as the m of mat and b of bat in English that distinguish one utterance or word from another in a given language. Allophones mark the variants of a phone. For example, the aspirated p of pit and the aspirated p of spit are allophones of the English phoneme p. “ The morph marks the smallest speech unit which carries a meaning itself. Therefore, consider is a morph, but reconsideration is not. % Avoiced sound is generated through the vocal cords. m,v and | are examples of voiced sounds. The pronunciation of a voiced sound depends strongly on each speaker.+ During the generation of an unvoiced sound, the vocal cords are opened. f and s are unvoiced sounds. Unvoiced sounds are relatively independent from the speaker. Reproduced Speech Output The easiest method of speech generation output is to use pre-recorded speech and play in timely fashion. Speech can be stored as PCM (Pulse Code Modulation) samples. There are two way of speech generation/output performed by time-dependent sound concatenation and a frequency- dependent sound concatenation. Time-dependent Sound Concatenation Individual speech units are composed like building blocks, where the composition can occur at different levels. The individual phones are understood as speech units. The individual phones of the word crumb. It is possible with just a few phones to create an unlimited vocabulary. Two phones can construct a dl-phone. To make transition problem easier, syllables can be created. Speech is generated through the set of syllables. Transition between individual sound units create and essential problem, called co-articulation, which is the mutual sound influence throughout several sounds. Prosody should be considered during speech generation or output. Prosody means the stress and melody course, Frequency Dependent Sound Concatenation Speech generation/output can also be based on a frequency dependent sound concatenation. This can be done through a formant synthesis. Formants are frequency maxima in the spectrum of the speech signal. Formants synthesis simulates the voce! tract through a filter. A pulse signal ith @ frequency, corresponding to the fundamental speech frequency, is chosen as a simulation ed sounds. Unvoiced sounds are created through a noise generator. for vc Human speech can be generated using a multi-pole lattice filter. The 1:four or five formants, occurring in human speech are modeled correctly with this filter type. Unvoiced sounds, created by vocal chords are simulated through a noise and tone generator. The method used for the sound synthesis in order to simulate human speech is called the linear predictive coding (LPC) method. Using speech synthesis, an existent text can be transformed into an acoustic signal.Letter to phone rules and dictionary of exceptions Sound Speech tk Transcription synthesis Figure: Components of a speech synthesis system with time dependent sound concatenation Step 1: Y Performs transcription Y Texts translated into sound script This process is done using letter-to-phone rules and dictionary of exceptions Y User recognizes the formula deficiency in the transcription and improves the pronunciation manual Y Sound scriptis translated into a speech signal. ¥ Time or frequency dependent concatenation can follow Speech analysis Speech Synthesis and Analysis Speech analysis are important aspects of multimedia systems. As multi-modal user interfaces become more common, it will become increasingly important for humans to communicate with computers using spoken language approaching natural language, and for computer systems to communicate with humans using artificially generated speech. The human acceptance of computer-generated speech is dependent ‘on the speech sounding natural and easy to understand. However, speech synthesis and analysis have ‘a multitude of other applications. Voice recognition systems are an important class of security systems; speech synthesis can give those who are vocally impaired a means for spoken communication. Speech synthesis and analysis are also an important aspect for computer systems which can be used by illiterate and visually impaired users.Speech analysis can serve to analyze who is speaking i.e. to recognize a speaker for his identification and verification. The computer identifies and verifies the speaker using an acoustic fingerprint. An acoustic fingerprint is a digitally stored speech probe of a person Speech analysis what has been said i.e. to recognize and understand the speech signal itself. Based on speech sequence, the corresponding text is generated (e.g. speech-controlled typewriter) Speech analysis also tries to research speech patterns with respect to how a certain statement was said. E.g. a spoken sentence sounds differently if a person is angry or calm which can be used for lie detector. Speech Analysis Figure: Research Areas of speech analysis Speech transmission Speech Transmission is a process of speech conduction quality. The absolute measurement of speech intelligibility is a complex science. The Speech Transmission Index measures some physical characteristics of a transmission channel (a room, electro-acoustic equipment, telephone line, etc.), and expresses the ability of the channel to carry across the characteristics of a speech signal. STI is a well- established objective measurement predictor of how the characteristics of the transmission channel affect speech intelligibility. The effect that a transmission channel has on speech intelligibility is dependent on: the speech levels frequency response of the channel non-linear alterations background noise level quality of the sound reproduction equipment Echo (reflections with delay) covering effectsSource , Pee Source Chane! speck Sampler DAD converter FP er Py cocoder ‘Channel Output cr Source Chane! speech ¢ Filter }¢ D/A converter. decoder decoder L__} i Assignment 1, What is sound? Differentiate between sound and audio. Define audio file and calculate the file size in bytes for a 30 second recording at 44.1 khz, 8 bits resolution stereo sound. What is MIDI? Discuss the components of MIDI system Explain in brief about the fundamental characteristics of sound. Illustrate and discuss about speech transmission process. What is audio compression? Explain lossy audio compression with example. What is speech generation? Illustrate about the speech transmission process. Discuss the types of digital audio file formats. Identify speech analysis also draw the figure of research analysis of speech analysis. Nn ©C@PNOALO Unit 4: Image and Graphics Image ‘An image is a visual representation of something such as picture or other representation of a person or thing or it can be someone's public perception. An image is a longitudinal representation of an object, a two dimensional or three-dimensional scenes of another image. Conceptually, an image is a continuous function defining a rectangular region of a plane.Digital image An image must be converted to numerical form before processing and this conversion process is called digitization. Digital image or still mage is a binary representation of visual information, such as drawings, pictures, graphs, logos, individual video frames etc. Digital images can be saved electronically on any device. An image is represented by its dimensions (height and width) based on the number of pixels. Basic concept(components) of digital image Pixel: The pixel also known as picture element is the smallest element of digital image. A digital image usually consists of a rectangular matrix of images. ff LL EE EEF Ky H] i hy N I BIST ILE LY "aa he hs" Se Ne heh SS Depth: The depth of an image is the number of bits used to represent each pixel. The concept of depth ‘on image is described below.L-bit black-and-white image, also called bitmap image +bit can represent 16 colours, used in low resolution sereens(EGA/VGA) 8-bit can have 256 colours. The 256 colour images are often known as indexed eet colour images. The values are actually foe +] omen indexes to a table of many more rani different colours. For example, Colour 25] cueio 3 is mapped to (200, 10, 10) 3] eo S-bit grey 256 grey-levels. The image contains - Tepe oper only brightness/ntensity data without Ss] OL Bs OTe colour information, fate [5 [3s] [t] rope a ooo BF 16-bit can have 65536 colours, also known as hi-colour in Windows systems. The 16 bits are divided into 5 bits for RED, 6 bits for GREEN and 5 bits for BLUE. 24-bit 2% = 16, 77, 216 colours, true colour. Each byte is used to represent the intensity of a primary 25 "0 0 Ret colour, RED, GREEN and BLUE. Each colour 9 559, Gnen can have 256 different levels. 2552550 Yellow 2 0) Magenta 7 7 OT Lighter White 0 0 0 Black 32-bit 2° = 4, 294,967,296 (4G). Usually, 3 bytes are used to represent the three primary colours and the fourth byte is used as the alpha channel Resolution: The resolution on digital image measures how much detail an image can have. There are several resolutions relating on multimedia digital image.Color system! Color is one of the vital and very important component of multimedia images. The color management in multimedia creation and development is both subjective and technical. Image resolution is the number of pixels in an image 820 x 240 = 76800 pixels, 700 x 400 = 280000 pixels Display (Monitor) resolution — refers to number of dots per inch (dpi) on a monitor. Windows systems usually have 96dpi resolution. Some high resolution video adapters/monitors support 120dpi. For example, a 288 x 216 image displayed on a monitor with 96dpi will be 3” x 24” Output resolution — refers to number of dots per inch (dpi) on a (hard copy) output device Many printers have 300dpi or 600 dpi resolution. High-quality imagesetters can print at a range between 1200dpi and 2400dpi, or higher. The above image printed on a 300dpi printer will be 0.96 x 0.72 inch: © Colour is a physical property of light, but # Colour perception is a human physiological activity. © Choosing a right colour or colour combination involves many trials and aesthetic judgement + Colour is the frequency/wave-length of alight wave within the narrow band of the electromagnetic spectrum (380 ~ 760am) to which the human eye responds. Wavelength Intensity Spectral Purity Hue Brightness Saturation RGB color model This model is based on the light absorbing quality of inks printed on paper. Combining three primary colour = © — Cyan pigments, Cyan, Magenta and Yellow, should absorb all M— Magenta light, thus resulting in black. Y— Yellow Itis a subtractive model. The value of each primary colour is assigned a percentage from the lightest (0%) to the darkest (100%). Because all inks contain some impurities, three inks actually produce a muddy brown, a black colour is added in printing process, thus CMYK model Note: the primary colours in RGB and CMY models are complementary colours