A Deep Recurrent Neural Network with Iterative Optimization for Inverse Image Processing Applications

Abstract

Many algorithms and methods have been proposed for inverse image processing applications, such as super-resolution, image de-noising, and image reconstruction, particularly with the recent surge of interest in machine learning and deep learning methods. As for Computed Tomography (CT) image reconstruction, the most recently proposed methods are limited to image domain processing, where deep learning is used to learn the mapping between a true image data set and a noisy image data set in the image domain. While deep learning-based methods can produce higher quality images than conventional model-based algorithms, these methods have a limitation. Deep learning-based methods used in the image domain are insufficient to compensate for lost information during a forward and backward projection in CT image reconstruction, especially with high noise. This dissertation proposes new iterative reconstruction algorithms implemented by the Recurrent Neural Network (RNN). The RNN is usually used to process sequential data, such as a stock price prediction or natural language processing. In this dissertation, we use the RNN to implement the iterative reconstruction (IR), where the RNN performs an iterative optimization for CT image reconstruction. Besides, we propose new RNN memory cells called Gated Momentum Unit (GMU) and Recurrent FISTA Unit (RFU) to keep the RNN cell preserve a long-term memory. The GMU and GFU are similar to the Long-Short Term Memory (LSTM) and the Gated Recurrent Unit (GMU), in which the RNN cells alleviate a banishing and an exploding gradient problem. The GMU and GFU have simpler network structures than the LSTM and the GRU, and they are particularly designed to accelerate the convergence of the training optimization process. We conducted a simulation study and a real CT image study to demonstrate that these proposed methods achieved the highest Peak Signal to Noise Ratio (PSNR) and Structure Similarity (SSIM). The GMU was evaluated in CT image reconstruction, and the GFU was evaluated in CT Metal Artifact Reduction (CT MAR). Also, we showed these algorithms converged faster than other well-known methods. Furthermore, in the fourth chapter of this dissertation, we discuss how vital image texture is in inverse image processing problems. Many methods have been proposed for these problems; however, the most popular methods, the convolutional neural network (CNN) based methods with a Mean Squared Error (MSE) are known to over-smooth images due to the nature of the MSE. The MSE-based methods minimize Euclidean distance for all pixels between a baseline image and a CNN-generated image and ignore the pixels' spatial information, such as image texture. The chapter of this dissertation proposes a new method based on Wasserstein GAN (WGAN) for inverse problems. We showed that the WGAN-based method was effective in preserving image texture. It also used a maximum likelihood estimation (MLE) regularizer to preserve pixel fidelity. Maintaining image texture and pixel fidelity is an essential requirement in medical imaging. We used PSNR and SSIM to evaluate the proposed method quantitatively. We also conducted first-order and second-order statistical image texture analysis to assess image texture.