Deep Learning Turns Night Into Day by Converting Monochrome Infrared Imagery Into Glorious Color
Trained on multi-spectral imagery, this neural network can figure out what black-and-white infrared captures would look like in the day.
A team of computer vision and machine learning researchers has developed a method for turning black-and-white night-vision imagery into a full-color display, by predicting what the scene should look like based on a deep learning system.
"Some night vision systems use infrared light that is not perceptible to humans," Andrew W. Browne and colleagues explain in the abstract to their paper, "and the images rendered are transposed to a digital display presenting a monochromatic image in the visible spectrum."
While being able to see in the dark at all is a great breakthrough, being unable to see color — and, worse, materials sometimes appearing dramatically different under infrared light than visible light — is a drawback, which is where the team's machine learning system comes in.
"We sought to develop an imaging algorithm powered by optimized deep learning architectures whereby infrared spectral illumination of a scene could be used to predict a visible spectrum rendering of the scene as if it were perceived by a human with visible spectrum light," the team explains. "This would make it possible to digitally render a visible spectrum scene to humans when they are otherwise in complete 'darkness' and only illuminated with infrared light."
To achieve that, the team used a standard infrared-sensitive camera combined with a suitable illuminator to capture the usual monochromatic imagery in full darkness — but then fed the imagery through a convolutional neural network (CNN), which had been trained on print-outs of faces that were then illuminated by red, green, and blue lighting as well as infrared. The result: a system, which could take the infrared imagery and guess what it would look like when viewed in color.
"This study suggests that CNNs are capable of producing color reconstructions starting from infrared-illuminated images," the team concludes, "taken at different infrared wavelengths invisible to humans. Thus, it supports the impetus to develop infrared visualization systems to aid in a variety of applications where visible light is absent or not suitable."
The team's work has been published under open-access terms in the journal PLOS ONE.