Finger On The PULSE: Using AI To Turn Blurry Images Into Hi-Res CGI Faces
For all the photographers out there who haven’t mastered the art of the steady hand: this one’s for you. Researchers at Duke University in North Carolina have applied an AI-based solution to touching up blurry photographs, creating a program capable of touching up blurry faces into an image sixty times sharper. It’s not going to turn you into an artist, but it could work wonder on your holiday snaps!
The Duke team’s system is called PULSE, standing for Photo Upsampling via Latent Space Exploration. The system creates entirely new high-resolution computer-generated images based on the blurry sample photograph, and then scours then scales the new images down, working backwards to finding the closest matches to the original low-res sample.
PULSE is actually made up of two neural networks, a machine learning tool called a GAN, or a generative adversarial network. When combined, these two networks produce lifelike high-resolution output images. The first AI generates new human faces, initially using its own programming but learning from feedback from the second network to improve its processes. The second network analyses the output images to “decide” if they’re convincing enough, feeding its analysis back into the first network to improve the process.
Traditional upscaling tools have assumed that there’s one true upscaled image based on any low-resolution (LR) input and have then worked linearly to add detail to an input image, slowly improving its resolution. But this assumption about a “true image” requires the program to smooth out any details that it can’t guess at. Typically, the image produced is around eight times sharper than its original, but details such as wrinkles, hairs and pores are absent, as the computer doesn’t want to guess at them. Output images suffer from a lack of texture, and look uncanny, like a too thoroughly airbrushed model – somewhere between human and CGI.
PULSE throws this assumption out the window, as any LR image could actually have hundreds of corresponding high-resolution outputs. By recognizing this, says Larry Gomez, a business writer at Revieweal and Assignment Help, “the team that worked on PULSE knew they could produce a vastly sharper image, where no details are smoothed out. The generative adversarial network works backwards, producing images and then shrinking them to see if they closely match the input image, all the while reappraising its process through machine learning to get better at both the image production process and the matching process.”
(PULSE Authors Example)
When the results from PULSE were tested against the images generated from other models, the PULSE images scored the best, approaching the scores given to real hi-res photographs. The algorithm can take an image of just 16×16 pixels and produce a realistic output image of 1024×1024 pixels. Even images where features are barely recognizable, eyes or mouths reduced to single pixels, can be used to produce life-like outputs.
The team at Duke focused on images of faces, but this was just a proof of concept for PULSE. “In theory, the process can be applied to blurry images of anything. This could have practical applications beyond your family photo album, in disciplines such as medicine and astronomy where blurry images are the norm. Sharp, realistic images could be generated to help us guess at what we’re looking at, whether that’s in the human body or deep space,” says Julia McAdams, a marketer at Coursework help service and State Of Writing.
One thing the team are keen to emphasize that PULSE can’t do is upgrade existing blurred images – say, blurred out photographs to protect individuals’ identities – and produce the original image. The output images produced by PULSE are computer generated – they look plausibly real, but aren’t actually images of anyone who exists. This dystopian model for unmasking protected identities doesn’t exist: indeed, the PULSE team say it’s impossible.
The power of AI and machine learning continues to be harnessed to create self-learning algorithms with impressive power to produce lifelike results. PULSE’s upscale tool is only one possible application of generative adversarial networks. It has been used in other areas to have algorithms produce their own video games. Smart algorithms will be infiltrating all walks of life in a few years, with impressive results.