Nvidia has showcased an AI model that converts a single 2D image of a person into a “talking head” video. Well, that would make it really easy for people who want to make video calls sitting in their pajamas.
Dubbed Vid2Vid Cameo, the deep learning model aims to amplify the video conferencing experience. For instance, if a person is running late for a video call, they can upload an image of themselves dressed well, and the AI will map their facial movements to the reference image.
Cutting bandwidth by 10 times
The AI will do all this without any other attendee knowing about it. It could be a really useful tool for the unkempt, but before you show up in your birthday suit relying on the AI, more testing should be done.
The system is capable of adjusting a person’s talking head viewpoint to make them appear to be looking straight into the screen. These features are very useful for people who aren’t very fond of video calls, but the handiest feature of the system is bandwidth reduction. Nvidia claims the system is capable of cutting the bandwidth required for video conferencing by up to 10 times.
Neural networks play a huge role
Vid2Vid Cameo is powered by generative adversarial networks (GANs), which put two neural networks against one another to create the videos. One of the two tries to create realistic samples, while the other attempts to figure out whether they are real or fake.
The allows the two networks to create a realistic video from a single image, which could be a real photo or an animated avatar. The model will capture their real-time motion during the call and apply it to the reference image.
The AI model was fed around 180,000 talking-head videos, teaching the network to identify 20 important points such as eyes, nose, and mouth. These points are extracted from the uploaded image, allowing the system to mimic their appearance.
