We have been working on semantic content analysis for media such as photos and videos. Such an analysis will enable more efficient and effective utiliziation of massive media, for example, search, organization, summarization and retrieval. In particualr, we are invloved in TRECVID benchmark, in which we have focused more on high-level concept detection and rushes video summarization tasks.
We investigate better ways for users to utilize multimedia data, including better methods for presenting photos in a spatial order, making slideshows with transitions of a 3D navigation style, and displaying photo and music with synchronized emotions.
Photorealism and interactivity are historically two contrary goals of computer graphics. However, due to the tremendous advance of graphics processing units, real-time photorealisitc rendering becomes probable. In this project, we explore possibilities to improve visual quality of rendering under the constraint of rendering in real time by utilizing the computation power of modern GPUs.
We developed new models and methods for digital matting and compositing, crucial for making visual effects. Conventional matting methods either require a carefully controlled studio setup or demand intensive user interactions. Our Bayesian matting algorithm is capable of pulling an alpha matte of a complex silhouette from a natural image with limited user interactions. We also extended this method to handle videos by interpolating user-drawn ' keyframes using optical flow. Traditional compositing approach only models color blending effects like anti-aliasing, motion blur and transparency, but not reflections, refractions and shadows. Environment matting enables capture of reflections and refractions with a limited accuracy at a modest cost. We further improved this process by developing a more sophisticated sampling scheme to capture environment mattes with higher accuracy, and by developing a technique that requires fewer images to allow for real-time capture. Shadows are yet another effects that the traditional method fails to model correctly. We introduced a novel process called shadow matting and compositing to acquire the photometric and geometric properties of the background for making realistic shadow composites.
Linux Snoop
A zoomin tool for visualizing a portion of your window on Linux and Windows.