Netflix built an army of servers to stream TV—now it’s powering games

A little over a decade ago, Netflix decided to take streaming into its own hands: Instead of relying on commercial content delivery services, the streamer built its own servers from scratch, and gave them away to internet service providers. Since then, Netflix has distributed over 18,000 of these servers, now installed in 6,000 locations spread across 175 countries, forming the company’s Open Connect content delivery network.

Now, Netflix is ready to take this tech beyond movies and TV shows: The company has begun to develop its own cloud gaming infrastructure, with servers that could eventually allow any Netflix member to play complex games on their smart TVs without an expensive game console.

This kind of cloud gaming requires a whole new generation of Open Connect servers. “The appliances that are going to stream games will need to look different,” says Netflix CTO Elizabeth Stone. “That’s something that we’re building now.”

Netflix’s servers have improved by 40x

Netflix began working on Open Connect in 2011, and placed a first set of servers in the data centers of select U.S.-based internet providers in 2012. At the time, Netflix engineers taped a different movie quote on each custom-built, bright red Open Connect server.

The concept behind Open Connect has stayed pretty much the same ever since: Once a server gets installed in an internet provider’s local data center, it gets loaded up with copies of Netflix’s movies and shows, which are then streamed to the provider’s customers whenever they’re ready to watch something. 

“When you click play, the content is coming from around the corner,” Stone explains. That allows Netflix to deliver movies faster and more reliably, while also reducing congestion for upstream pipes that connect a service provider to the internet at large—a win-win for everyone involved.

These days, the movie quotes are long gone, as are the red chassis housings. The technology itself has changed as well: Netflix server racks now incorporate both machines with fast flash drives that stream Netflix’s most popular fare, as well as massive hard drive clusters to store thousands of additional movies and shows for customers with more eclectic taste.

“One half-rack of our servers serves all of Netflix’s catalog,” Stone says. “It can serve about 500,000 simultaneous streams. When we first got started, this half-rack could only serve about 13,000 simultaneous streams.”

Video games are a different beast

Open Connect’s next challenge is Netflix’s budding video game business. The company has launched more than 120 mobile games since it began exploring gaming in 2021, but its ambitions reach far beyond mobile: Netflix ultimately wants to allow its subscribers to access games comparable to what’s available on Microsoft’s Xbox and Sony’s PlayStation directly from the Netflix app running on a regular smart TV.

To do this, Netflix has begun to build its own cloud gaming service, which it is currently testing with a subset of its subscribers in eight countries, including the U.S. and Canada. Subscribers in the beta test get to play around a dozen titles, including the indie game hit Oxenfree and the rebooted Atari classic Centipede: Recharged. This trial run helps Netflix test its cloud gaming technology, and optimize the design of its new Open Connect game servers.

These appliances are very different from Netflix’s existing streaming hardware. “If you are streaming games to the TV, you execute the game on a server,” Stone explains. The server then captures a live video feed of the game and beams it to the consumer’s TV. That requires Netflix’s game servers to be optimized for real-time graphics rendering. “More GPU-heavy, different types of chips, and a different type of design,” Stone says.

Cloud games also need to be delivered with as little latency as possible so that players don’t miss a critical jump or turn due to a delayed stream. For some games, synchronization across locations is also needed to allow Netflix subscribers to play together remotely. “Cloud games tend to be pretty social, in many cases,” Stone says.

Stone readily admits Netflix’s current cloud gaming test is small. “As we feel more confident in our approach, we [will] start to scale to other countries and other game types,” she says. “That will come in the coming years.”

The Tyson-Paul disaster helped future-proof Open Connect

Getting a new technology like cloud gaming right can be challenging, especially for a company the size of Netflix; opening up the service too early to all of its 300 million subscribers could easily overwhelm the company’s infrastructure. Netflix did get a lesson in humility last year when it aggressively expanded into livestreaming—only to have the broadcast of a match between boxing legend Mike Tyson and YouTube star Logan Paul turn into a train wreck.

“Everything about Tyson-Paul was extraordinary, including how large that event was,”  Stone says.

An estimated 65 million viewers tuned in simultaneously, overwhelming even Netflix’s content delivery network. The result: widespread buffering, and some users being kicked from the stream entirely. Stone describes the experience as a painful but necessary lesson. “For that fight, there was no way to simulate that in a lab,” she says.

The insights her team gained from that knockout not only helped Netflix successfully stream two NFL games over the holidays, but are helping make Open Connect more resilient for whatever comes next, from live sports to cloud gaming.

“Our aspirations are much larger than 300 million members,” Stone says. “In order to stream film and TV to 400 million members, 500 million members all around the world, we need Open Connect to continue to evolve.”

View the full article