Nvidia has developed a generative art system that leverages AI to turn words into visually-pleasing artwork. This isn’t the first time we’re seeing such a concept; however, it is the first time we’ve seen such a system work with such accuracy and speed.
To see a project named DALL·E, you can check out OpenAI. That’s an image-generating project based on GPT-3, which you can learn about at Cornell University. Using the Deep Dream Generator, you can start making wild imaginations of style.
Turning words into images
The NVIDIA Project GauGAN2 dwells on what researchers developed with Nvidia Canvas. That application is in the Beta phase currently and works with the first GauGAN model. With AI at your disposal, anyone can produce a relatively realistic appearing artwork with input that’s not anything more than what’s needed to create a finger painting.
With GauGAN2, Nvidia researchers expanded what can be done with simple input and AI interpretation of the said output. This model uses around 10 million high-quality landscape images, as its visual knowledge bank, and draws upon said bank to determine what inputs could mean in a work of art.
One single GAN framework in GauGAN2 features numerous modalities. Nvidia points to text, semantic segmentation, sketch, and style. The demonstration shows the interface that is an extension of Nvidia Canvas.
Other ambitious projects from Nvidia
Previously, Nvidia showed off its new voice AI that is something far more realistic and levels above anything seen before. Merging AI and a human reference recording, the fake voice sounds almost identical to a human’s.
Nvidia’s in-house creative team describes the process of achieving accurate voice synthesis. The team equates speech to music that has complex rhythms, pitches, and timbres that aren’t easy to replicate. Nvidia is creating tools that are capable of producing these intricacies with AI. The company showcased its latest advancements at the Interspeech technical conference that focuses on the research of speech processing technologies.
