Netflix says its magenta approach beats green screen

Green screen technology has been around nearly a century. Some 45 million moviegoers were mesmerized by the special effects of "The Wizard of Oz" when it hit theaters in 1939. The magical scenes in Emerald City were the first to film actors in front of a green screen that later would be replaced with footage of fantastical scenes recorded at a different time and place.

The technique was refined over the years in such fantastical, breathtaking and action-packed films such as "Avatar," "The Matrix" and "Iron Man" and "Titanic."

But the technique is not perfect. Proper, carefully-balanced lighting is required for the most realistic results. Green color spillage from the backdrop onto other objects affects color consistency. Shadows on the green screen also affect color accuracy and must be closely monitored. And motion of fine objects, such as strands of hair, can produce distracting visual artifacts.

And the shade of green matters. A search of YouTube for red-faced weather forecasters who unintentionally wore clothing with the wrong shade of green—one that turned some of their body invisible—will turn up plenty of examples of the perils of mismatched clothing hues.

Manual intervention can address these issues, but it can be time-consuming and the results are not always perfect.

Netflix now says it has come up with a technique that improves upon the longstanding green chroma key technique.

In an article published on the preprint server arXiv, Netflix researchers said they have developed a new approach called Magenta Green Screen (MGS) that achieves greater accuracy and quick results.

The name stems from the lighting approach they use: Actors are lit from the front with red and blue LEDs against a backdrop lit by bright green LEDS. The red and blue lighting creates a magenta glow on the actors.

Since digital cameras capture color values along red, green and blue tracks, one track captures the background (the green screen channel) with the foreground all black, while the red and blue (magenta) channels record the foreground, with the background displaying all black.

Netflix utilizes AI tools to eliminate the magenta hues by comparing photographs of the actors in natural lighting to action scenes with hues.

"Our technique yields high-quality compositing results when implemented on a modern LED virtual production stage," Netflix senior researcher Dmitriy Smirnov said in the report, "Magenta Green Screen: Spectrally Multiplexed Alpha Matting with Deep Colorization."

The process can capture action with filled-in virtual backgrounds in real-time, Smirnov says. No special camera equipment or manual color adjustments are required.

One issue they addressed is the problem of pixels along the periphery of objects that can be part of the object or background.

"Partial coverage at edges, wispy and transparent structures, defocused and motion-blurred areas all exhibit partial transparency," Smirnov said. "Determining the RGB color of the foreground element at a pixel, as well as the pixel's transparency, is … tricky."

The researchers improved upon a method called triangulation in which stationary objects are filmed in front of differing backgrounds in order obtain highly accurate definitions of the objects.

"Our technique appears to outperform the matting that which can be obtained with either traditional chromakey green screen or time-multiplexed techniques," Smirnov said.

"Computers already have provided such powerful tools to make a lot of stuff easier," said Paul Debevec, another member of the research team. As a result, "the talented artists that we have can focus on the artistry, actually making things look better."