Digital Visual Effects

Digital Visual Effects, Spring 2008

Forum

Jump to...

project #1
project #2
scribe
final project
assignments

project #3: MatchMove

Assigned: 2008/04/22
Due: 2008/05/19 11:59pm
artifacts, winning artifacts

Project description

Matchmove is a technique of estimating camera parameters from an input video sequence so that computer generated imagery (CGI) could be seamlessly inserted into the sequence. In this project, you will use tools to create visual effects using matchmove. You can either use camera calibration techniques or structure from motion methods. Essentially, for this project, you have to do the following things:

Shoot a video sequence and digitalize it. (Of course, you should plan ahead on how camera moves and what to insert afterwards.)
Apply a calibration or structure-from-motion tool to estimate camera parameters
Import these camera parameters into some 3D packages to add CGI into your scene
Optionally, add sound effects to your video and edit it.

Taking video

We will provide a Sony TRV900 camera for the students with no access to video camera. Please sign up here to borrow the camera. For better tracking results, we suggest you to set the camera in the progressive mode. Hence, the video could be of only 15fps. You can check with this when you convert your video into a sequence of images. If it is 15fps, two neighboring frames are exactly the same. When taking your video, for easier extracting, press the record button earlier to leave a redudant video segment before and after the segment you want. That is, it is easier to extract a video segment if it is in the middle part of a longer video segment. It is a problem with the time code written into the tape. We had problems with this before, but are not sure if the problem is still there. In addition, please note that this camera could have problems with recording sound.

After taking videos, you need to convert it into a sequence of images. We will have a laptop with a 1394 card and VirtualDub installed so that you can borrow from us to digitalize your video.

MatchMove and rendering

If you prefer to use calibration approaches, you can either Bouget's matlab calibration toolbox or Zhengyoun Zhang's calibration program. However, to use this approach, you have to figure out how to insert CGI by yourself.

For structure from motion, there are many excellent commercial software. However, most of them are very expensive (around $5,000) which we can't afford to use in this class. Hence, we will use icarus, the predecessor of PFTrack, for this project. You can download icarus here. Please note that icarus is not distributed online any more. Hence, don't distribute it and only use it for academic usage. The other viable option is Voodoo. This one is free and could be used along with blender as weel.

For inserting CGIs, we recommend that you use Blender 3D to add CGI. Here is a wondeful tutorial on using icarus and Blender for the complete MatchMove process. You will need this Python script to import Icarus output to Blender.

For your reference, this is the slides from last year's TA's help session.

Submission

You have to turn in two videos, the raw footage before inserting CGI and the final composite video as your artifacts. Send me the links to these two videos. It is not required but preferred to Write a simple report in html describing what tools you have used and what lessones you have learned.