25/4/2018 The Basic Architecture of Avid Media Composer

How Avid Media Composer Uses a

Computer
By Chris Bové (http://www.alwaysediting.com) in Tech Talk
(http://www.avidblogs.com/category/topics/tech-talk/), Video Editing
(http://www.avidblogs.com/category/topics/video-editing/)
October 27, 2017

In the past, acquiring and running so ware for desktop and laptop computers was a slow, thoughtful process. We would stand in stores (back
when they existed) and stare at so ware boxes (back when they existed). Turning a box around, we’d comb through the specs, making
informed decisions based on our intimate knowledge of the computers we owned.
That was for simple “home stuff” like Quicken or Doom but what about “work stuff”? As professional cra editors responsible for large
projects it was even more critical to understand the specs and the tech behind the whole process.
At the time Avid Media Composer (http://www.avid.com/media-composer) wasn’t just so ware in a box. It was acquired as part of an
expensive “turnkey” system – a machine designed from the ground up with the sole purpose of running it as best as possible. It’s the
etymology behind referring to an edit system as “an Avid”.
Today we download and install everything we see to our phones, tablets and even desktops. We’re app-happy. It’s how we test and consume
new workflows, and it’s all thanks to this massive D.I.Y. culture. The result is editors and assistants building and supporting their own
machines. Unless we’re responsible for outfitting a massive facility, turnkey systems are largely gone. Today all we need to do is click,
download and pay a monthly subscription fee to get a working Media Composer system. But what kind of computer are we downloading it
to? Will it handle Media Composer? Or perhaps the more appropriate question: How will Media Composer handle the computer?
We’ve all asked these questions over the years, especially when we were students learning the trade. In fact today, places like the Assistant
Editors Bootcamp (http://www.assistbootcamp.com/) are great examples of how we bring new women and men into the industry. But are
they learning these basics? When Noah at the AE Bootcamp reached out to me with these questions, they were at the request of his Lead AE
and Independent Editor class. I was eager to help. But to truly get answers, we need to get them from Avid’s engineers directly.

A group of students at the Assistant Editors Bootcamp, lead by Noah Chamow in 2017. (assistbootcamp.com
(http://www.assistbootcamp.com))

Conveniently, Avid is in a wonderful mindset of transparency right now.

As a Vice Chair of Avid’s ACA, as well as a volunteer Moderator of Avid’s Pro Video Community, I was able to have conversations with a number
of senior level peeps at Avid. Reponses came in from many of them – from Avid offices in Massachusetts, Québec and California.
Video Audio Live Sound Notation Broadcast EDU (http://www.avidblogs.com/education/) Events ACA 
http://www.avidblogs.com/how-avid-media-composer-uses-a-computer/ 1/11
25/4/2018 The Basic Architecture of Avid Media Composer

Avid Technology Inc., 75 Network Drive, Burlington, MA 01803

It was the response from Shailendra Mathur, VP and Chief Architect that kicked everything into gear. Here’s what I asked:
For the purpose of assisting editors and AE’s, I’m hoping to create a 1-page guide that explains what parts of a computer are used by Media
Composer – broken down by processes like rendering, AVX Plug-ins, real time playback, processor-intensive codecs and so on. Would you or one of
your sta be willing to assist me in its creation?

The response from Shailendra Mathur:

Hi Chris, sure we can help. We have quite a lot of info on this since it has been a popular question through the ages, and it will be a wonder to fit it
into a single page :-).

Shortly therea er I got an email from a number of Avid’s engineers and away we went. The first question: How simple would this one-pager
be?
I opted for a simple concept we could start with – listing only the “intentions” of the app, meaning answers to questions like, “For video
e ects, does MC use the CPU or the GPU?” and, “How many cores are used and for what tasks?” From there we started diving into the details
behind those simple explanations.
It took a while, which wasn’t their fault but rather mine. In addition to my cra -editing schedule, there were a lot of emails and phone
conversations with Avid’s engineers to help me understand things. I’m not an engineer, so I’ve been very thankful for their patience. I still
can’t say I comprehend it all, yet the goal from the beginning was to have this written by an editor for editors.
Here is the result:

(http://www.avidblogs.com/wp-content/uploads/2017/10/Image-4-one-pager.jpg)

Video Audio Live Sound Notation Broadcast EDU (http://www.avidblogs.com/education/) Events ACA 
http://www.avidblogs.com/how-avid-media-composer-uses-a-computer/ 2/11
25/4/2018 The Basic Architecture of Avid Media Composer

So… Are you begging for more details? Of course! We’re cra editors and that’s what we do.
First… What is this whole thing and how long has it been here?
The Avid Intelligent Compute Architecture as it’s called was initially developed when Avid Media Composer v3.0. It evaluates the whole system
– the OS, hardware, GPU capabilities, availability of processors and number of cores. It dynamically distributes the processing to the device
best suited to the specific task for different segments of the timeline. Rather than targeting just the GPU, just the CPU, or just the FPGA based
cards, the philosophy changed to use them all in a holistic fashion. Thus the whole system is turned into an accelerator. The Intelligent media
player in the application acts as an orchestra conductor, keeping as many of the resources playing to provide the performance required.
Keeping a holistic view of the whole system in mind, particular attention is paid to the cost of transferring heavy video data across the system
bus when deciding which compute hardware should be used for a particular process.
OK let’s get into it. Below are cutouts from the above 1-pager, followed by the notes I jotted-down during my conversations with Avid’s
Architecture team.

RAM and Cache

(http://www.avidblogs.com/wp-content/uploads/2017/10/Image-5-Ram-and-Cache.jpg)

Video Audio Live Sound Notation Broadcast EDU (http://www.avidblogs.com/education/) Events ACA 
http://www.avidblogs.com/how-avid-media-composer-uses-a-computer/ 3/11
25/4/2018 The Basic Architecture of Avid Media Composer
1. Everything works better with more RAM. Filling a computer with the maximum it can handle is now a standard recommendation. Having less
RAM constricts Media Composer’s abilities.

Media Composer works best when it encounters the least restrictions possible, especially when it comes to RAM. For example, take a 2014
MacPro with 16GB of RAM. Hit play and watch the RAM usage in the macOS Activity Monitor. (This is how we monitor apps and their
efficiency.) Media Composer may hover generally around the 8GB area. But if you take that same system and change the RAM to 32GB, Media
Composer may hover generally around the 14-16GB area. This isn’t Media Composer “hogging” more resources, but rather the smaller
RAM is constricting the system’s ability to use any available resources. This is a prime argument for increasing a system’s RAM to max
capacity.
Maxing-out the RAM “li s the computers ceiling” as high as possible. Thus all functions have the possibility to operate at their own
maximums on that computer, without constraints placed on them by lesser amounts of RAM. If a new iMac can physically hold a maximum of
64GB of RAM, then that’s what is recommended by Avid.
Since RAM can be the easiest and least expensive way a user can upgrade a computer, Avid has been architecting the 64-bit playback engine
to take advantage of RAM first.
Minimum requirements will, of course, still exist. For example, a laptop with only 8GB of RAM and a 5400RPM external drive holding a
project’s DNxHD media will work as a minimally qualified machine.

2. Larger raster sizes (UHD, 4K etc.) use more RAM than smaller ones.

Larger raster sizes means more pixels, which require more processing power to play in real time.

(http://www.avidblogs.com/wp-content/uploads/2017/10/Image-6-Aspects.jpg)

h ps://commons.wikimedia.org/wiki/File:Aspect_Ra os_and_Resolu ons.svg

3. The Interactive Frame Cache within Composer o en assists playback. Since cache from Composer is stored in RAM, increasing the Media
Cache -> Video Memory can improve stream counts. This allows more streams to be played in real time without rendering. Users should learn to
use this setting as needed.

Video Audio Live Sound Notation Broadcast EDU (http://www.avidblogs.com/education/) Events ACA 
http://www.avidblogs.com/how-avid-media-composer-uses-a-computer/ 4/11
25/4/2018 The Basic Architecture of Avid Media Composer

Avid Media Composer: Se ngs ‐> Media Cache

Leaving the Desired Video Memory cranked-up all the time may negatively affect other processes and apps.
In Media Composer’s processing algorithms for playback, video is actually looked at as individual frames. During processing, Composer
determines how those frames get played in streams. For example, a single video layer of DNxHD media qualifies as one stream. More layers
and effects add more streams.
The term processing refers to the heavy li ing of playback. There may be pre-processing involved at some stage (where things are
transformed – partial renders would be one form), but getting everything from the drives put forth into the output, all of that is referred to as
processing.

The term cache in a timeline refers to a way that processing distributes the handling of playback. When processing gets really dense and
complicated, here is where CPU cache can assist.
The cache might be thought of as a sort of invisible Video Mixdown that the system uses to reduce strain and help playback. That’s a pretty
narrow and somewhat incorrect definition, but it hopefully gets the basic point across. A better way of explaining it would be: Instead of
doing a complex evaluation over and over again (processing), Composer keeps the result in RAM, saving the pain of fully reprocessing over
and over. Cache is downstream of processing, and will remember the results of processing.
Cache does eventually fill up. At a certain point, when there is no longer room in the cache, the oldest used frame is thrown away in favor of
the new frame. Want more frames saved in cache? Get more RAM.

Video Audio Live Sound Notation Broadcast EDU (http://www.avidblogs.com/education/) Events ACA 
http://www.avidblogs.com/how-avid-media-composer-uses-a-computer/ 5/11
25/4/2018 The Basic Architecture of Avid Media Composer

Avid Media Composer: Se ngs ‐> Media Cache (showing the default se ng)

Avid Media Composer: Se ngs ‐> Media Cache (showing the Set High se ng)

Note: On this particular system, which is loaded with 32GB of RAM, Media Composer is operating at around 8 GB. Setting the Media Cache’s
Video Memory to 22 GB adds to that memory pressure, reaching a grand total of around 30GB. This means only 2 GB of RAM is le , which can
cause memory issues if any other apps are launched or if background processes like Dropbox or iTunes begin to sync.

When Media Composer is closed, the Memory Cache setting is not saved, thus allowing for fast, easy re-launching later.
Note: Increasing this Video Memory setting by large amounts may negatively a ect other processes that require RAM, so do not to leave this
cranked-up all of the time. Adjust it up/down as needed.

4. The Playback Video Frame Cache improves single frame play responsiveness.

Video Audio Live Sound Notation Broadcast EDU (http://www.avidblogs.com/education/) Events ACA 
http://www.avidblogs.com/how-avid-media-composer-uses-a-computer/ 6/11
25/4/2018 The Basic Architecture of Avid Media Composer

Avid Media Composer v8.9.2: Se ngs ‐> Media Cache ‐> Video Memory tab

While the Media Cache -> Video Memory increases the number of frames saved, this setting increases the responsiveness of those frames
during playback. It gets better results with Media Cache -> Video Memory set higher.

Codecs

(http://www.avidblogs.com/wp-content/uploads/2017/10/Image-12-Codecs.jpg)

5. Currently the CPU handles encoding/decoding of codecs.

RED Camera files (R3D) are the exception, which encode/decode with help from the GPU, and certainly more so when a Red Rocket card is
assigned the workload.
Most codecs are structured in a way that one might call “GPU averse”. But this could (and likely will) change in the future. If codecs become
more GPU-friendly as bitstream formats, then Avid will no doubt chase that philosophy.

6. Playing codecs smoothly in a timeline requires processing, which benefits from more CPU cores. Codecs with large raster sizes also benefit from
more cores.

Playing a timeline that contains Linked (AMA) clips plus many effects results in a higher stream count, especially with more complex codecs.
With more cores there are more opportunities to distribute the processing of those streams effectively.

7.All codecs benefit from more RAM, but some codecs (LongGOP, AVC/H.264) need much more to work e ectively.

Some codecs are easy for computers to play and edit (like DNxHD). Other codecs are more complex and require a great deal more processing
power to get minimally acceptable results. There are certainly situations where computers that are considered minimally qualified to run MC
are not powerful enough to run MC plus those codecs. More RAM will have to be added.
Files accessed through the Source Browser can also be transcoded in to Avid codecs (Clip > Consolidate/Transcode) which use less streams
and less system resources to play. Workflows using Avid HD-sized codecs do not require a high number of cores to work effectively.
Note: The H.264 codecs (.mov and .mp4) are no longer being handled by the legacy 32-bit QuickTime engine. As of Media Composer 8.9.1, the
64-bit playback engine handles them. This applies to XAVC-S (.mp4) as well.

Video Quality Menu

(http://www.avidblogs.com/wp-content/uploads/2017/10/Image-13-Video-Quality-Menu.jpg)
Video Audio Live Sound Notation Broadcast EDU (http://www.avidblogs.com/education/) Events ACA 
http://www.avidblogs.com/how-avid-media-composer-uses-a-computer/ 7/11
25/4/2018 The Basic Architecture of Avid Media Composer

8. The Video Quality Menu changes the raster size of the viewed output to allow weaker computers to play complex codecs smoother.
Green/Green mode plays the codec in full raster. Yellow/Green mode reduces that raster to 25%. Full Yellow mode reduces that raster to 6.25%
(1/16th size). Currently the CPU, not the GPU, handles this raster resizing.

The CPU is doing this resizing from its original raster size. Operating in these lower modes on HD-sized projects usually allows for a weaker
computer to play back a timeline smoother.
For larger-than-HD raster sizes however, this can add a CPU-based bottleneck. The computer is using more resources to do a real time
conversion of the raster.

9. If a codec is greater than 8-bit, switching the Video Quality Menu to Green/Green/10-bit mode playback can sometimes be a more e ective
use of overall processing. This depends on the amount of e ects on a clip.

Video Effects, Timeline & Playback

(http://www.avidblogs.com/wp-content/uploads/2017/10/Image-14-VFX.jpg)

10. GPU is front loaded/preferred by Composer when playing from a timeline. Playback looks at the topmost layer in a timeline first, which is seen
as one stream.

The term “front loading” means that the top-most-layer in a Media Composer timeline will target the GPU first. When playing a sequence,
Media Composer looks at a timeline’s play head (blue vertical bar) from above, and not from the side like we users do. (Imagine the play head
as the light bar in a photocopier or a scanner, hovering over a sequence.)
Before playback of a sequence with effects begins, Composer’s algorithms allocate as much of the timeline as possible to the GPU. It also
allocates some CPU power, but only a er it identifies specifics within the timeline that need CPU-only processing and/or multithreading over
a number of cores.
The primary reason for all of this is we want to read back only one stream from the GPU, and that starts with the topmost layer. So loading a
computer with the hottest qualified GPU and highest amount of GPU RAM possible helps with this. The better and more RAM-heavy the
graphics card, the less data needs to be sent to the CPU and the cores.
Render and Expert Render can relieve system stress by collapsing multiple streams.

11. Some e ects are processed only using the CPU. They were engineered to do so. More recent e ects have been engineered towards GPU usage.

Video Audio Live Sound Notation Broadcast EDU (http://www.avidblogs.com/education/) Events ACA 
http://www.avidblogs.com/how-avid-media-composer-uses-a-computer/ 8/11
25/4/2018 The Basic Architecture of Avid Media Composer

Avid Media Composer v8.9.2: Effect Pale e

Video Audio Live Sound Notation Broadcast EDU (http://www.avidblogs.com/education/) Events ACA 
http://www.avidblogs.com/how-avid-media-composer-uses-a-computer/ 9/11
25/4/2018 The Basic Architecture of Avid Media Composer
12. Some Color Adapters (source e ects for example) are processing-intensive, so a more powerful GPU will handle them more e ectively.

Random Notes
That’s it on my notes in-context with the basic document, but here are a few other tidbits of information I picked up in conversation.
– As of Media Composer v8.8.3, the QuickTime AMA plugin relies a lot less on the QuickTime engine (operating in the background outside of
Media Composer), and more on Composer’s own playback engine.
– Higher frame rate clips benefit from more cores as well as bandwidth. This is because higher frame rates can be handled by something
called stream-based parallelism (the processing of sub-sequent frames in parallel). This eats up a lot of buffer memory. Thus another
argument for more RAM. Note: this is only for I-frame codecs. Hence why GOP codecs are so much more difficult. Their processing of frame
rates cannot be handled in any sort of method of divide-and-conquer. By design, GOP codecs have a lot of interdependencies between the
video’s frames. If processing of a later frame can only proceed once the dependent earlier frame has been decoded, then you have a result
that needs to be addressed much more practically and linearly.

Further Reading
If you’re really into learning the architecture behind all of this, Avid has public access granted to many documents created over the last few
years by Avid’s architecture team.
Here are the links:
1) A blog discussing the architecture itself – the Avid Intelligent Compute Architecture:
http://community.avid.com/blogs/mediacomposer/archive/2013/03/12/how-intelligent-computing-powers-our-editorial-architecture.aspx
(http://community.avid.com/blogs/mediacomposer/archive/2013/03/12/how-intelligent-computing-powers-our-editorial-architecture.aspx)
2) And here is my personal favorite – the public file ion Google of the patent filed which includes the performance architecture:
https://www.google.com/patents/US8358313 (https://www.google.com/patents/US8358313)

Ques ons?
Any questions? I’m sure there are. If this info doesn’t prompt more questions, I’d be surprised. For the sake of being heard as best as possible,
let’s please post all questions and comments at one place, on the Avid Community.

Here is the link: http://community.avid.com/forums/p/182676/849425.aspx#849425

(http://community.avid.com/forums/p/182676/849425.aspx#849425)

Ask as many questions as you’d like. If it’s something I don’t know (which is plenty), then I’ll pass it along to others. Or perhaps other users
here can step up? The goal is a universal understanding, in order to make us better at our cra . Hopefully this is one good step in the right
direction.
Thank you for you time!
Chris Bové
(AKA “Pixel Monkey” on the Avid Community)

Discover Media Composer

Accelerate storytelling with the tools embraced by top movie, television, and broadcast editors. And power
through HD and high‐res edi ng faster and easier than ever.

AVID MEDIA COMPOSER (http://www.avid.com/media-composer/?utm_source=avidblogs)

Video Audio Live Sound Notation Broadcast EDU (http://www.avidblogs.com/education/) Events ACA 
http://www.avidblogs.com/how-avid-media-composer-uses-a-computer/ 10/11
25/4/2018 The Basic Architecture of Avid Media Composer

GET IT NOW (http://shop.avid.com/ccrz__Products?

categoryId=a3yi0000000011qAAA&isCSRFlow=true&portalUser=&store=shop/?utm_source=avidblogs)

 SHARE THIS

Chris Bové October 27, 2017

Categories: Tech Talk
(http://www.avidblogs.com/author/cbove/) (http://www.avidblogs.com/category/topics/tech-talk/), Video
(h p://www.avidblogs.com/author/cbove/) Editing (http://www.avidblogs.com/category/topics/video-
editing/)
I have spent 20 years as an editor and a story editor
on films, docs and TV shows. Working at the highest-
Tags: Media Composer (http://www.avidblogs.com/tag/media-
composer/), Video Editing (http://www.avidblogs.com/tag/video-
rated PBS station in the USA, I have created more
editing/), Video Post Production
than 200 hours of broadcast programming—all on
(http://www.avidblogs.com/tag/video-post-production/)
Avid Media Composer.
♥ 6 Likes

(http://www.avid.com)
©2018 Avid Technology, Inc. Terms & Legal (http://www.avid.com/about-
avid/legal-notices) | Privacy (http://www.avid.com/about-avid/legal-
notices/privacy-policy)


http://www.avidblogs.com/how-avid-media-composer-uses-a-computer/ 11/11