By Derek Wan, HKSC

November 17, 2002

SMPTE told me that the general theme of today’s meeting would be post-production. But I made my topic a little vague on purpose. I’d like to talk about issues in a wider scope because I came from a cinematography background and began involved in producing, directing and post-production in recent years. As a result, I’d like to address issues of broader concern.

Digital Mastering is the core of my presentation today. Make no mistakes. I am not introducing a new technology. What I mean “new” in the topic is the trend of how this technology is being used in acquisition and particularly, in post-production workflow of the whole film production chain. The technology has been around for at least ten years. Digital mastering has been widely used in finishing high-end TV commercials and creating visual effects for feature films.

Post facilities in Hong Kong and some members of the audience should be quite familiar with the process. Producers and directors around the world have adopted this technology.
I am sure Hong Kong movies with large amount of visual effects such as etc, have applied some form of digital mastering extensively.

The difference is that digital mastering process used to be cost prohibitive. Producers could afford it exclusively for special effect scenes, as it was a significantly better process than the traditional optical method. Recently, more and more filmmakers are aware of the advantages of using digital mastering. The cost of using this process has come down rapidly as the computer processing power increases and hard disk prices drop. Digital Mastering is beginning to be applied to the entire film. And it is becoming more and more common among bigger budget productions.

In the meantime, the advent of High Definition digital system, especially the 24p Universal Mastering Format, has offered an alternative in high quality imaging for film and video content creation. The second half of my presentation will examine the inexpensive workflow of using HD 24p equipment as a tool for acquisition and finishing.

The central theme of my presentation today is high end imaging workflows for motion picture productions, particularly in the post-production process. The technique is recently referred to as Digital Mastering. And the specific process is called Digital Intermediate.

Let’s first take a look at the typical workflow we have been using in the last 10 years. The flowchart should represent the common practice of local motion picture post-production work.

As far as I understand, in the case of Hong Kong, we don't shoot 16mm, we shoot 35mm film only. We process the negative, take it to a telecine session and create video dailies, usually in a standard definition format, Betacam SP for example. With this video dailies we do a nonlinear offline edit, convert the offline EDL to a film cut list with key code numbers on the film strip for negative conforming. Conforming the negative means cutting the negative and splicing them together based on the offline edit decisions. Traditionally we add fade-ins, fade-outs, dissolves, and other visual effects optically and assemble them to the negative reels before striking a positive print.

In a situation where many copies of release prints are needed the labs would recommend making intermediates to avoid damage to the original negative. We first create an inter-positive from the original negative and than generate multiple inter-negatives from the inter-positive if huge numbers of release prints are required. And than we use the inter-negative to make release prints.

Usually the producer picks a release print or, in some better scenarios, uses the inter-positive for another Telecine session to create different video versions, PAL, NTSC, DVD, VCD, VHS...and whatever format is needed for distribution.

One of the problems with the traditional workflow is that the image quality deteriorates as we go through each step, especially when you are dealing with intermediate positive and intermediate negative film. In most cases, optical sound track and subtitle stock exacerbate the situation. We lose so much sharpness and color details in the release print as a result. In a word, we are not getting the best possible image out of the original negative. Another limitation is the color timing process. With the dated analog system, color timing is limited to overall picture changes in red, green, and blue dimensions plus brightness adjustments. We don’t have the tools to really rectify problems from the shoot or use color correction as a creative process to generate stylized look.

Since film is still the highest quality image recording medium available, digital mastering process for motion picture usually means we shoot film, scan the developed negative into the computer as data files, manipulate the data files with software before recording back to film. As technology evolves, inspired by still image and video footage manipulation hardware and software, film post-production began to adopt some form of digital processing to create visual effects.

The basic idea is very simple. We use a film scanner to scan filmed images into data form, i.e. a computer file, pretty much like how we bring still photo film into the computer. Once the images are in a digital format, we can apply the software tools that have been available to still pictures and video footage for all kinds of manipulations and reconstructions, and the degree of manipulation and refinement are virtually unlimited. Once we create our “finished” images we can record the digital files back to film emulsion, similar to how we turn a digital image back to slide film for presentation.

Last 10 years, we saw significant progress in the field of digital still imaging, desktop publishing, in digital video’s nonlinear editing and visual effect creation. We saw a lot of tools that were available for video finishing business. We wanted to take that into our hands to create images for theatrical display. We wanted to be as flexible and as refined as we could with video manipulation. We used to envy the video industry for a long time because film did not have that flexibility.

Digital photography and video technologies have opened up new possibilities for motion picture image manipulation. It is just a matter of getting enough resolution and image details into the computer; processing and storing the huge amount of data; and finally outputting the data files back to the film emulsion while maintaining enough sharpness, details and color information for large screen projections. The digital process works somewhat like the intermediate film in a traditional workflow. That’s why we call it “digital intermediate”.

In the early stage, extremely high costs due to expensive equipment and media storage; and due to slow capture and output speeds deterred producers from using the technology extensively. As a result, digital intermediate process was first introduced to the film industry to do visual effects only. When people began to realize how much more, how much better and how much easier digital process could do to create complex visual effect shots, the demand started to heat up.

As more equipment manufacturers seize the opportunity and pump out faster film scanners and recorders and as the hard drives prices drop faster than what Moore’s Law dictates, the cost of using this process has come down significantly to a point that even finishing the entire movie with a digital mastering workflow is affordable to some producers. Recently digital process and computer assisted finishing are no longer limited to a 3 second dissolve or a 1 minute multi-layer composite sequence. That’s when the trend to use digital intermediate on the whole movie begins.

Let’s look at the applications of digital mastering. First, you could do color grading within the digital domain and that’s the biggest reason why people want to use the process in the first place. If you are familiar with a Telecine session, i.e. how we convert film into video, you’ll know the unbelievable possibilities today’s technology offers. We probably should not even call it a “color correction” session since we don’t necessarily go through the process to “correct” any mistakes. It has become an image re-creation process. The degree of control we have in the process is just amazing.

With digital mastering workflow, we can apply the same kind of controls into feature film, the entire two hours of footage. There are other applications such as digital blow up. We can shoot 16mm, for example, go through this process and than output to 35mm with much higher quality than an optical blow up. We could do format conversions such as shoot Super 35 format and digitally squash the image to create an anamorphic inter-negative and release prints. Digital mastering not only allows us to create visual effects such as speed changes, reframing, compositing, rotoscoping, morphing, chroma keying, etc., it is also an excellent tool for restoring damaged film. With digital intermediates, we can make last minute changes in edit decision and visual effects while retaining pristine picture quality before recording back to film. Moreover, if the director, producer or the cinematographer understands how much digital mastering can do in post-production, they could save time and energy during the shoot because some optical filter effects can be re-created and other visual imperfections can be easily repaired through the process.

Please refer to the flowchart for a digital intermediate workflow. Typically, film frames are scanned into the computer in two different resolutions, 2K and 4K. These numbers refer to number of vertical lines the scanned file can resolve. Generally, 4K file is perceived as preserving most details recorded on negative film. However, time, storage space and processing power required to handle 4K files for the entire movie at this point of technology development is still cost prohibitive. It is not viable to process a two-hour film all in 4K. On the other hand, general consensus of the industry is that we could live with the quality of 2K files and realistically, the additional quality preserved by a 4K file is usually lost due to the reproduction limitations of film output, release print film stock characteristics and projection weaving.

The first attempt to use digital mastering was O’Brother, Where Art Thou? for it’s washed out look. Recent films utilizing digital mastering include The Lord of the Rings and its sequel, We Were Soldiers, Panic Room, Hart’s War and HBO TV mini series Band of Brothers, among others.

If shooting film and mastering the show digitally is the best route to go in terms of image manipulation and quality preservation, the alternative route with significantly lower cost would be HD acquisition and finishing. We have just seen the footage from Star Wars Episode II in the beginning of today’s meeting. That entire movie was shot with a 24p HD camera.

I must admit, shooting with current HD formats doesn’t capture as much information as 35mm film does, but the cost is significantly lower. In some cases HD capture could look somewhat better than the traditional workflow of finishing film because we don’t need to go through the inter-positive and inter-negative film again. I will explain this with another flowchart later on.

The most common format in HD has 1920 X 1080 pixels, i.e. there are 1920 pixels across and 1080 vertical lines in a frame. It sounds like a much lower resolution than a 2K scan, but in reality, based on subjective evaluation, the image is extremely sharp and full of details both in a large screen digital projection or output to film for cinema projection.

Besides, with the limitations of current recording and projection technology, we are not utilizing all 1920 X 1080 pixels in its full color depth on tape formats. We are actually just recording a highly compressed signal and projecting substantially down-sampled images. Imagine what it would look like when the technology and equipment in the future allow us to enjoy a true 1920 X 1080 pixel picture with 4:4:4 uncompressed color depth.

The beauty of 24p HD is that it records 24 frames per second in progressive scan mode, which is very similar to how film camera captures images. This format eliminates a lot of artifacts and motion issues inherent in traditional interlaced video recording for TV broadcast while enjoying all the benefits video has to offer. While 1080p HD has sufficient resolution and image information for a large screen projection, 24fps running speed makes it compatible with output to film (one frame to one frame) for theatrical release. Therefore, 24p HD has similar motion characteristics as film capture. It’s like having the best of both worlds.

Furthermore, while 1080 line 24p HD has the highest spatial resolution among all accepted HD formats; it is also an ITU-709 international acquisition and program exchange standard. This means an edit master of this format is highly compatible and flexible for worldwide distributions.

Please look at the simple workflow in the flowchart. The shooting, offline editing and online process of HD is very similar to that of SD video program. We enjoy all the convenience, flexibilities, and tools available to video post-production work. In the non-linear world, all you need is just a more powerful computer, a dedicated capture card, and a huge hard disk.

If you are working with a minimal budget, the offline editing can be done on a standard definition system by down converting the HD footage to a SD format and use the offline EDL to conform the HD camera master later in an HD online system. In fact, even a lot of visual effects and color correction work can be done with a lesser system such as a person computer as long as you have an HD NLE system to capture the original footage and assemble the finished effects sequence back to your HD online program for playback. As the HD format gets more popular, HD NLE systems are improving constantly. Even some inexpensive hardware and software packages are capable of capturing and rendering in uncompressed 10 bit YUV format, offering same image quality as the tape to tape editing workflow and are comparable with most other solutions available.

For producers, getting a film output from an HD master tape is pretty much the same as the digital mastering. We can output to camera negative or intermediate stock. From there we make answer print and release print. The general rule of thumb is that if the source is HD capture, you want to use camera negative. That gives you the right latitude and more film look. If the source material is originated on film, you might want to output it to an intermediate stock for finer grain.

Between the high-end shoot film/digital mastering approach and the low cost HD shoot/finish approach, there is a wide spectrum of high quality imaging workflows for different creative, technical, and financial reasons. Producers can choose the combinations that best fit their budget and needs. Any of the listed combinations should be able to deliver very pleasing results.

For example, if you shoot film but can’t afford scanning the film into the computer, you can use a “Datacine” machine to turn the film into 10 bit .sgi files for digital processing. If the producer just wants to shoot film for the look and doesn’t need to go back to film for theater projection he can telecine the film original to a less compressed HD format, such as D5 for maximum video quality. We can also shoot with an HD camera, record directly to a digital disk recorder (DDR), capture the signal in uncompressed 4:4:4 color depth for manipulation. This approach is particularly useful in green screen, blue screen type of applications.

New technologies have changed the whole working relationship. I’d say post-production actually starts from the preproduction stage. Cinematographers, directors, and producers have to think about what the possibilities are and what the cost would be in post. They need to acquire new knowledge to take full advantage of technology; decide early on how to most effectively achieve certain visual effects and realize their creative goals; and technically what needs to be done to facilitate post-production work.

On the other hand, post people may want to be involved in the production stage to help out the filmmakers. Let them know what the possibilities are; what the most cost effective ways of creating certain effects are; and what the crew needs to watch out for during the shoot to facilitate best possible post results.

Producers who are more familiar with the possibilities and costs involved with digital processing will be able to make the film look bigger than what the budget actually can afford or save some money along the way. Directors with more post-production knowledge will open themselves up for more creative possibilities.

Cinematographers will be able to control their images much better by means of digital processing. They may want to extend their responsibilities and creative controls to the color correction sessions in post-production. Their photography, lighting, and vision don’t stop when the camera stops. Directors of Photography can continue their creative process in post. All these take some learning. It requires some understanding in digital photography, video engineering, and knowledge of new post-production protocols.

If people are still not convinced with what current HD equipment can do for them. Let me show you briefly some new technologies and equipment that will be at your disposal in a not too distant future.

Recent advancements in imager technology have put CMOS in a leading position over CCD. CMOS eliminates or minimizes a number of artifacts that are inherent in CCD image sensors. Some have more film characteristics while others have four times higher resolution than the current imagers. The recording side has advanced to hard disk recording that preserves the true uncompressed 4:4:4 color signals. Although not very practical for location shoot at the moment, it certainly opens up new horizons in recording technology for even higher image quality. Most of these new innovations are in prototype stage. But imagine what would happen if all of these technologies were incorporated into the design of a future HD product. If that’s not exciting enough, please note that consumer HD formats are catching up. HD capture becomes more affordable and we can actually enjoy HD quality at home with pre-recorded D-VHS movies now in a region without HDTV broadcast.

To conclude my presentation, let me quickly share some thoughts with you. HDTV is going to stay, now in Japan and the USA, later in China, Korea and Australia. Film will remain a high-end capture medium for a long time, at least for the next ten years. It will be a coexistent and hybrid situation. It’s our job to take advantage of both technologies and to use both tools for creative and logistic purposes. The emergence of HD actually allows people with less resource to express their vision in high visual quality.

Secondly, we all agree film records more visual information than standard definition video. Unfortunately economy dictates that we could no longer afford to record our history on film on a daily basis. Since we started using ENG video cameras to shoot news footage some 20 years ago, we have been recording news, national, or family events with fuzzy SD video. Allow me to dramatically characterize the last 25 years or so as the “dark age” of archiving our civilization because we could only afford film on specific science projects or documentary films with a certain budget. Now high definition digital system is available and very soon all countries with HD broadcast will capture news footage and produce documentaries in HD format. With approximately 6 times more spatial resolution, we are preserving the visual history in motion of the coming years in a better form for our future generations. That’s another reason why I advocate HD technology and application.

Lastly, over my years of working abroad, I have been deeply impressed by the professional attitude of most members of the production community. Their relentless pursuit for improvement in their work resulted in everlasting advances in quality and innovation. I hope my colleagues at home would share the same spirit and continue to produce high quality work as our contribution to help revive the industry.