I Tested Nvidia’s AI Tool for Making Your Webcam Better, and Oof

When Nvidia introduced the free Nvidia Broadcast app in 2020, it promised to use the AI capabilities of RTX GPUs to boost the video and sound quality of users' recordings, presuming they had the right hardware. This could be useful when a laptop’s built-in webcam and microphones may not capture the best looking footage or the richest audio, and could potentially help streamers get by with a cheaper setup. With version 2.0 of the Nvidia Broadcast app, released at the tail end of January, the capabilities are stepping up even higher...perhaps a little too close to the sun.

In Nvidia Broadcast 2.0, microphones still have options for noise removal and room echo removal, but there’s now also a studio voice effect in beta that “enhances the quality of your mic to simulate a high end recording studio.” For video, the app can still make tweaks to your background (replacing, blurring, or essentially green-screening it), remove noise from grainy footage, zoom in and automatically keep you in frame, and strangely enough, make it look like your eyes are looking at the camera. But new with the update is virtual lighting, to better highlight you in your video. 

Some of these new features call for powerful GPUs. Studio voice, video noise removal, and virtual key light all call for a “high-end GPU” and aren’t recommended for use while gaming or using other GPU-intensive applications. All features require RTX hardware, meaning you’ll need at least an RTX 2060 or above to try them, but for the features that require a “high-end GPU,” Nvidia says an RTX 4080, 5080, or higher is “required.” That said, I was able to run both features on an RTX 4060 mobile GPU.

Nvidia Broadcast in action

The idea behind these AI features is cool, but how well they actually work is still in question. For one thing, they may really be as demanding as Nvidia says. Running either the virtual key light or studio voice feature on my RTX 4060-powered laptop showed the GPU was absolutely slammed by the process. Nvidia’s built-in GPU Utilization monitor was showing red, with the RTX 4060 all but maxed out and the performance overlay showing it drawing 60 watts. My laptop’s fans even ramped up as if I were gaming at full throttle. So just from an economics standpoint, these features are costly no matter how you look at them. You’ll need to have powerful hardware to run them, and then run that hardware hard. Plan on using these features on a desktop computer or with your laptop plugged in. 

Then there’s the even more crucial matter of how they really look and sound. Let’s start with video.

Credit: Mark Knapp

The eye contact tool, despite being available before Broadcast 2.0, has now come out of beta. But I’m not convinced it should have. Sure, enabling it makes it look like I’m staring into the camera in video footage. But in my testing, it invariably gave me big blue eyes that made me look like a White Walker right out of Game of Thrones. For reference, I do not have blue eyes. Even when I was making eye contact with the camera, Nvidia Broadcast still insisted on editing my eyes and making them blue. 

Credit: Mark Knapp

The virtual key light did what it said. It created artificial lighting to boost brightness on me without bumping up the brightness on the whole video. The results failed to impress me, though. With it enabled, I simply look like I’ve gone radioactive. The lighting is very unnatural.

As for the audio, at first blush, it sounds fairly impressive. The mics on my laptop are not very good. Even in a quiet room, they put out audio that has me sounding far away and slightly muffled. With studio voice enabled, my voice ends up much fuller and clearer sounding. But listening closely, there’s an odd digitization going on. It’s hard to characterize, but it doesn’t sound like it’s really my voice. It sounds more like a recording of my voice was used in a voice cloner, and then that repeated everything I said. It’s all just a little stilted and quavering. Listen below:

The studio voice feature also can’t save the mic from a bad recording environment. Testing in a small room with a box fan running at full blast, the audio was a dramatic improvement in clarity compared to the raw recording from the microphones, but it was still audibly processed, and the efforts to subdue the background noise made my voice sound especially odd.  

If you have a half-decent microphone, studio voice might even make it worse. I made additional test recordings using the built-in boom mic on the Audeze Maxwell headset with it directly wired into my laptop. In both a quiet and loud room, it provided a loud, clear, and full recording of my voice without studio voice enabled. In both cases, turning on studio voice then introduced hard-to-miss digitization that not only made the audio sound worse but also made it harder to comprehend.

Can Nvidia AI replace a proper streaming setup?

Given the hardware requirements, performance demands, and quality of the results, the stars really need to align for these newer Nvidia Broadcast features to feel truly worthwhile. If you have an Nvidia-powered system, by all means, play with the tool. Some of the features can come in handy, like the auto-framing one. But I wouldn’t recommend shelling out for a new Nvidia GPU just so you can save money on audio and video recording gear, especially if you want to get anywhere close to professional quality. And don’t forget that the power draw of the GPU trying to run these features will add up over time.

The audio quality I got from studio voice — perhaps limited by the RTX 4060 in my system — wasn’t something I’d want to share with any kind of audience on a regular basis, and it paled in comparison to the quality I could get just from having a headset with a boom mic. I’ve tested a lot of gaming headsets, and even very cheap wired headsets with a boom mic are leagues better than what I heard from studio voice. 

The eye contact feature failed to be anything other than unsettling, and I don’t think it’s going to fool anyone into believing you’re actually making eye contact with them. And the virtual key light, much like studio voice, doesn’t appear to be a quality substitute for a real key light, especially when affordable LED lights are a dime a dozen.

View the full article