Specialist in artificial intelligence, Nvidia has just unveiled a collection of tools supposed to make our videoconferencing meetings less painful. By using two neural networks, the company is able to change the direction of your gaze, and replicate your head movements on a virtual avatar.
Nvidia wants to change the way we do video conferencing. The graphics card maker and AI specialist unveiled new advancements in the field on June 24, with a bunch of tools meant to improve our video meetings.
The company has demoed an artificial intelligence capable of analyzing your face in real time so that it appears flawless on all your calls, even if you’ve just got out of bed and you’re still in your pajamas.
Vid2Vid Cameo, of its small name, is in fact able to replicate your head movements and to animate your avatar in real time, to make your interlocutors believe that you are perfectly attentive. So, if you watch TV from home during a video meeting, all you need to do is nod your head occasionally for your avatar to appear involved and serious, looking straight into the eyes of your interlocutors.
Test the tool on one of your photos
In its research paper, Nvidia explains using a technique called “facial redirect” that will sort of straighten the angle of your face so that you appear to be looking straight into the camera, even though your face is actually at an angle. The neural network behind the tool will somehow animate a photo of you with your eye, jaw and head movements to make you appear as natural as possible.
A tool has been made available by Nvidia to better understand how it all works. By sending a simple portrait of yourself, you can edit the image to roll your eyes, turn your head right, or look down, all in seconds without touching Photoshop. The result is far from flawless at the moment. The edited photo lacks detail and sports some notable artifacts, but beyond the raw quality of the shot, the work on the face and eye rotation is quite stunning. The AI easily manages to get you up and down in a relatively realistic fashion. The real-time demonstration is even more striking.
Another notable advantage of this technique: it considerably reduces the bandwidth required to make a video call. Rather than sending the raw video feed from your camera, it is possible to send only facial movement information and therefore reduce the bandwidth required for a smooth phone call.
10 times less bandwidth
” By compressing and sending only the head position and key points back and forth, instead of the entire video stream, this technique can reduce bandwidth requirements for video conferencing by 10 times, resulting in a smoother experience. », Says Nvidia.
All these advances are still at the experimental stage, but the demonstration of Nvidia shows a future where video will be less painful than today. All that remains is to convince users to let a neural network analyze their facial expressions, and as with anything related to facial recognition, that doesn’t look easy.