Evaluation of deep neural networks for single image super-resolution in a maritime context

Evaluation of deep neural networks for single image super-resolution in a maritime context

Nieuwenhuizen, R.P.J.
Kruithof, M.
Schutte, K.

Huckridge, D.A. (editor)
Ebert, R. (editor)
Bursing, H. (editor)

2017

High resolution imagery is of crucial importance for the performance on visual recognition tasks. Super-resolution (SR) reconstruction algorithms aim to enhance the image resolution beyond the capability of the image sensor being used. Traditional SR algorithms approach this inverse problem using physical models for the image formation combined with a regularization function to prevent instabilities in the solution. Recently deep neural networks have been put forward as an alternative approach to the SR reconstruction problem. They learn a mapping from low resolution images to their high resolution counterparts from pairs of training images, which allows them to capture more specific information about the space of possible solutions than traditional regularization functions. These networks have achieved state-of-the-art performance on single image SR for sets of generic test images. Here we investigate whether the same performance can be realized when these neural networks for single image SR are applied specifically in the maritime domain. In particular we investigate their ability to reconstruct undersampled images of ships at sea, and demonstrate that the performance is similar to what is achieved on generic test images. In addition we quantify the gain in performance that is achieved when the networks are trained specifically on images of ships, which allows the networks to capture more prior knowledge about the space of possible solutions. Finally we show that the performance deteriorates when the resolution of test images is limited by image blur, for example due to diffraction, rather than undersampling. This highlights the importance of using representative training data that account for the part of the image formation process that limits the resolution in the sensor data. © 2017 SPIE. The Society of Photo-Optical Instrumentation Engineers (SPIE)

2015 Observation, Weapon & Protection Systems
II - Intelligent Imaging
TS - Technical Sciences
Algorithms
Deep learning
Neural networks
Software
Super-resolution
Algorithms
Computer software
Deep learning
Deep neural networks
Image processing
Image reconstruction
Image resolution
Infrared devices
Inverse problems
Optical resolving power
Ships
Vehicle performance
High resolution imagery
Image formation process
Low resolution images
Reconstruction problems
Regularization function
State-of-the-art performance
Super resolution reconstruction
Image enhancement

http://resolver.tudelft.nl/uuid:8533f756-5c9f-4e58-8755-278561906d3c

782875

SPIE

9781510613300

0277-786X

Electro-Optical and Infrared Systems: Technology and Applications XIV 2017. 13 September 2017 through 14 September 2017, 10433

Proceedings of SPIE - The International Society for Optical Engineering

104330T

conference paper