Stereoscopic Media Editing

Overview

• Display adaptation. Due to the diversity among display resolutions and aspect ratios, similar to 2D media, stereoscopic images require adaptation to be displayed properly on different devices. In addition, for stereoscopic displays, we often must adapt images to their comfort zones, especially for binocular images with excessive depth ranges. We proposed a content-aware stereoscopic image display adaptation and editing method which simultaneously resizes a binocular image to the target resolution and adapts its depth to the comfort zone of the display while preserving the perceived 3D shapes of prominent objects. Our method also allows users to interactively adjust the sizes, locations, and depths of the selected objects, giving users aesthetic control for depth perception. The paper was published by TMM in 2011.
  
  
• Image resizing. Previous stereoscopic image resizing methods suffer from the problems of noticeable discontinuity on structural objects or degraded stereoscopic quality. We proposed a layer-based stereoscopic image resizing method which decomposes an image into layers and applies image warping on each layer to synthesize scene-consistent resizing results. It provides the following nice properties: (1) it avoids distortions and holes as much as possible; (2) it maintains good stereoscopic properties and (3) it contains as many important pixels as possible in the reduced image space. This work has been published by CVPR 2012.
  
  
• Image cloning. We have developed a technique for seamless stereoscopic image cloning, which performs both shape adjustment and color blending such that the stereoscopic composite is seamless in both the perceived depth and color appearance. Our method guarantees not only depth continuity across the boundary but also models local perspective projection in accordance with the disparities, leading to more natural stereoscopic composites. It allows for easy cloning of objects with intricate silhouettes and vague boundaries because it does not require precise segmentation of the objects. The paper was published in SIGGRAPH Asia 2012.
  
  
• Patch-based 3D image editing We extended the patch-based synthesis framework for image editing to stereoscopic images. Our method offers patch-based solutions to a wide variety of stereoscopic image editing problems, including depth-guided texture synthesis, stereoscopic NPR, paint by depth, content adaptation, and 2D to 3D conversion. The work was published at IEEE TVCG 2015.
  
  
• 3D Cinemagraphy Principles and Their Applications Our ACM MM 2011 paper introduced a set of 3D cinematography principles such as maintaining coordination among views, having a continuous depth chart and using a shallow depth of field for shots with excessive depth brackets. It is important for 3D media processing algorithms to pay attention to these principles so that their 3D results yield more comfortable viewing experiences. We demonstrated these principles by incorporating them into two popular media authoring applications, video stabilization and photo slideshow. User studies showed the proposed methods yield more comfortable and enjoyable 3D viewing experiences than those delivered by methods without taking these principles into account.
  
  
• A tool for stereoscopic parameter setting. It is a necessary but challenging task for creative producers to have an idea how the target audience might perceive when watching a stereoscopic film in a cinema during production. We propose a novel metric, geometric perceived depth percentage (GPDP), to numerate and depict the depth perception of a scene before rendering. In addition to the geometric relationship between the object depth and focal distance, GPDP takes the screen width and viewing distance into account. As a result, it provides a more intuitive mean for predicting stereoscopy and is universal across different viewing conditions. Based on GPDP, we design a practical tool to visualize the stereoscopic perception without the need of any 3D device or special environment. The work was published by IEEE TCSVT in 2016.
  
  

Publications

A Tool for Stereoscopic Parameter Setting Based on Geometric Perceived Depth Percentage

Shuen-Huei Guan, Yu-Chi Lai, Kuo-Wei Chen, Hsuang-Ting Chou, Yung-Yu Chuang

IEEE TCSVT 2016

Geometrically Consistent Stereoscopic Image Editing using Patch-based Synthesis

Sheng-Jie Luo, Ying-Tse Sun, I-Chao Shen, Bing-Yu Chen, Yung-Yu Chuang

IEEE TVCG 2015

Perspective-Aware Warping for Seamless Stereoscopic Image Cloning

Sheng-Jie Luo, I-Chao Shen, Bing-Yu Chen, Wen-Huang Cheng, Yung-Yu Chuang

ACM SIGGRAPH Asia 2012

Scene Warping: Layer-based Stereoscopic Image Resizing

Ken-Yi Lee, Cheng-Da Chung, Yung-Yu Chuang

IEEE CVPR 2012

3D Cinematography Principles and Their Applications to Stereoscopic Media Processing

Chun-Wei Liu, Tz-Huan Huang, Ming-Hsu Chang, Ken-Yi Lee, Chia-Kai Liang, Yung-Yu Chuang

ACM MM 2011

Content-Aware Display Adaptation and Interactive Editing for Stereoscopic Images

Che-Han Chang, Chia-Kai Liang, Yung-Yu Chuang

IEEE TMM 2011

Support

• NSC101-2628-E-002-031-MY3
• NSC100-2628-E-002-009
• NSC100-2622-E-002-016-CC2
• NSC99-2628-E-002-015
• NSC99-2622-E-002-026-CC2