Currently, only residents from certain countries and US states can opt out of certain Tracking Technologies through our Consent Management Platform. Additional options regarding these technologies may be available on your device, browser, or through industry options like AdChoices. Please see our Privacy Policy for more information.
Social Media
On
These cookies are set by a range of social media services that we have added to the site to enable you to share our content with your friends and networks. They are capable of tracking your browser across other sites and building up a profile of your interests. This may impact the content and messages you see on other websites you visit. If you do not allow these cookies you may not be able to use or see these sharing tools.
Essential
On
This website uses essential cookies and services to enable core website features and provide a seamless user experience. These cookies and services are used to facilitate features such as navigation, remember user preferences, and ensure the security of the website.
Targeted
On
These cookies may be set through our site by our advertising partners. They may be used by those companies to build a profile of your interests and show you relevant adverts on other sites. They do not store directly personal information, but are based on uniquely identifying your browser and internet device. If you do not allow these cookies, you will experience less targeted advertising.
Performance
On
These cookies allow us to count visits and traffic sources so we can measure and improve the performance of our site. They help us to know which pages are the most and least popular and see how visitors move around the site. All information these cookies collect is aggregated and therefore anonymous. If you do not allow these cookies we will not know when you have visited our site, and will not be able to monitor its performance.
Functional
On
This website uses functional cookies and services to remember your preferences and choices, such as language preferences, font sizes, region selections, and customized layouts. They enable this website to offer enhanced and personalized functionalities.
Audience Measurement
On
We use audience measurement cookies in order to carry out aggregated traffic measurement and generate performance statistics essential for the proper functioning of the site and the provision of its content (for example to measure performance, to detect navigation problems, to optimization technical performance or ergonomics, to estimate server power needed and to analyse content performance). The use of these cookies is strictly limited to measuring the site's audience. These cookies do not allow the tracking of navigation on other websites and the data collected is not combined or shared with third parties. You can refuse the use of this cookie by switching off the slider to the right.
Google Brain has devised some new software that can create detailed images from tiny, pixelated source images. Google's software, in short, basically means the "zoom in... now enhance!" TV trope is actually possible.
Credit:
Google Brain
First, take a look at the image on the right. The left column contains the pixelated 8×8 source images, and the centre column shows the images that Google Brain's software was able to create from those source images. For comparison, the real images are shown in the right column. As you can see, the software seemingly extracts an amazing amount of detail from just 64 source pixels.
Of course, as we all know, it's impossible to create more detail than there is in the source image—so how does Google Brain do it? With a clever combination of two neural networks.
The first part, the conditioning network, tries to map the the 8×8 source image against other high resolution images. It downsizes other high-res images to 8×8 and tries to make a match.
Left column: source image. Other columns: various outputs produced by the neural networks. There's a bit of variation. Fourth celebrity from the bottom is particularly scary.
Google Brain
Left column: source image. Other columns: various outputs produced by the neural networks. There's a bit of variation. Fourth celebrity from the bottom is particularly scary.
Google Brain
Input: left column. Fourth column: the original image. Other columns: various super-resolution techniques. NN = nearest neighbour (looking for a high-res image in the dataset that closely matches the 8x8 image)
Google Brain
Input: left column. Fourth column: the original image. Other columns: various super-resolution techniques. NN = nearest neighbour (looking for a high-res image in the dataset that closely matches the 8x8 image)
Google Brain
Left column: source image. Other columns: various outputs produced by the neural networks. There's a bit of variation. Fourth celebrity from the bottom is particularly scary.
Google Brain
Input: left column. Fourth column: the original image. Other columns: various super-resolution techniques. NN = nearest neighbour (looking for a high-res image in the dataset that closely matches the 8x8 image)
Google Brain
Google Brain
Google Brain
Various different super-resolution techniques. The three right-most columns are the Google Brain method.
Google Brain
Various different super-resolution techniques. The three right-most columns are the Google Brain method.
Google Brain
Google Brain
Various different super-resolution techniques. The three right-most columns are the Google Brain method.
Google Brain
The second part, the prior network, uses an implementation of PixelCNN to try and add realistic high-resolution details to the 8×8 source image. Basically, the prior network ingests a large number of high-res real images—of celebrities and bedrooms in this case. Then, when the source image is upscaled, it tries to add new pixels that match what it "knows" about that class of image. For example, if there's a brown pixel towards the top of the image, the prior network might identify that as an eyebrow: so, when the image is scaled up, it might fill in the gaps with an eyebrow-shaped collection of brown pixels.
To create the final super-resolution image, the outputs from the two neural networks are mashed together. The end result usually contains the plausible addition of new details.
Google Brain's super-resolution technique was reasonably successful in real-world testing. When human observers were shown a real high-resolution celebrity face vs. the upscaled computed image, they were fooled 10 percent of the time (50 percent would be a perfect score). For the bedroom images, 28 percent of humans were fooled by the computed image. Both scores are much more impressive than normal bicubic scaling, which fooled no human observers.
One of the best videos of all time.
It's important to note that the computed super-resolution image is not real. The added details—known as "hallucinations" in image processing jargon—are a best guess and nothing more. This raises some intriguing issues, especially in the realms of surveillance and forensics. This technique could take a blurry image of a suspect and add more detail—zoom! enhance!—but it wouldn't actually be a real photo of the suspect. It might very well help the police find the suspect, though.
Sebastian is the editor of Ars Technica UK. He usually writes about low-level hardware, software, and transport, but it is emerging science and the future of technology that really get him excited.
Loading comments...
