Deep Learning for Computational Photography
Overview
In addition to dealing with computer vision problems, deep learning has also shown great promise for computational photography problems.
We have applied deep learning to two different categories of computational photography problems: improving image quality and learning to see the unseen.
-
Improving image quality.
Cameras have a number of limitations. As a result, the photographs they capture are imperfect and do not always appear satisfactory to users. We approach this issue from two different directions: physically and aesthetically. Our CVPR 2020 paper takes a physical approach to recover the information lost due to the camera pipeline. In other words, our method reconstructs a high-dynamic-range image from a single image by learning to inverse the camera pipeline.
Our CVPR 2018 paper takes an aesthetic approach. Instead of generating physically accurate images, the method aims to generate photographs that are pleasing to the user. The method is unsupervised. The user only needs to choose a set of images that appeals to him or her. Our GAN model learns the mapping between a set of captured photographs and a set of user-preferred images. The two sets of images are not paired. The learned model can then convert given images into images in the style that the user prefers.
-
Learning to see the unseen.
Due to limitations of the camera or capture setting, certain information may be missed or unwanted objects may appear in the picture. We propose a set of deep methods for dealing with these situations, including full-frame video stabilization by predicting missing pixels around the border (ICCV 2021), video frame interpolation by predicting what a high-speed camera would see (AAAI 2019), removing unwanted obstructions such as reflections and occlusions (CVPR 2020), and removing shadows from document images (CVPR 2020).
Publications
- Learning to See Through Obstructions with Layered Decomposition
-
Yu-Lun Liu,
Wei-Sheng Lai,
Ming-Hsuan Yang,
Yung-Yu Chuang,
Jia-Bin Huang
- IEEE PAMI 2022
- Hybrid Neural Fusion for Full-frame Video Stabilization
-
Yu-Lun Liu,
Wei-Sheng Lai,
Ming-Hsuan Yang,
Yung-Yu Chuang,
Jia-Bin Huang
- ICCV 2021
- BEDSR-Net: A Deep Shadow Removal Network from a Single Document Image
-
Yun-Hsuan Lin,
Wen-Chin Chen,
Yung-Yu Chuang
- CVPR 2020
- Single-Image HDR Reconstruction by Learning to Reverse the Camera Pipeline
-
Yu-Lun Liu,
Wei-Sheng Lai,
Yu-Sheng Chen, Yi-Lung Kao,
Ming-Hsuan Yang,
Yung-Yu Chuang,
Jia-Bin Huang
- CVPR 2020
- Learning to See Through Obstructions
-
Yu-Lun Liu,
Wei-Sheng Lai,
Ming-Hsuan Yang,
Yung-Yu Chuang,
Jia-Bin Huang
- CVPR 2020
- Deep Video Frame Interpolation using Cyclic Frame Generation
-
Yu-Lun Liu,
Yi-Tung Liao,
Yen-Yu Lin,
Yung-Yu Chuang
- AAAI 2019
- Deep Photo Enhancer: Unpaired Learning for Image Enhancement from Photographs with GANs
-
Yu-Sheng Chen, Yu-Ching Wang, Man-Hsin Kao,
Yung-Yu Chuang
- CVPR 2018
cyy -a-t- csie.ntu.edu.tw