DigiVFX
Digital Visual Effects, Spring 2018

Jump to...

project #2
project #3
final project
assignments

project #1: High Dynamic Range Imaging

Assigned: 2018/03/21
Due: 2018/04/11 2:00pm
submission, voting

Project description

High dynamic range (HDR) images have much larger dynamic ranges than traditional images' 256 levels. In addition, they correspond linearly to physical irradiance values of the scene. Hence, they are useful for many graphics and vision applications. In this assignment, you are asked to finish the following tasks to assemble an HDR image in a group of two.

1. Taking photographs. Take a series of photographs for a scene under different exposures. (As discussed in the class, changing shutter speeds is probably the best way to change exposures for this application.) For that, you need a digital camera that allows you to set exposures. You can use your own camera or borrow one from us. It is suggested that you use a tripod when you take pictures. If you need to borrow the camera, please sign up here.

One thing to note is that you should avoid touching your camera during this process so that all pictures are well registered. Some cameras have their own accompanying software for remotely controlling the cameras. For Nikon cameras, there are programs allowing you to set exposures and release shutter remotely, for example, Breeze system's NKRemote. Another solution would be to use AHDRIA for some models from Canon, but it is obsolete and not maintained since 2007. Cameras of other brands might have a similar solution for remote capturing, but you have to discover on your own. You are welcome to write down your findings for that matter in your report.

If you decide to manually change exposures, it would help if you align your images before proceeding to the next step. You can probably try to use Photomatix. Otherwise, you are also welcome to write a program for alignment as bonuses. A good candidate is the median threshold bitmap we discussed in the class.

2. Write a program to assemble an HDR image. Write a program to take these captured images as inputs and output an HDR image and, optionally, the response curve of the camera. You can use OpenCV, a popular vision library with image I/O functions, for reading/writing images. You are free to use any languages (including Matlab) for this assignment. However, you should not use the library or built-in routines whose function is to assemble HDR. We recommend that you output your radiance map as a Radiance RGBE image (.hdr) or OpenEXR format (.exr). As discussed in the class, there are several ways for assembling a radiance map.
  • Paul Debevec's method. Please refer to Debevec's SIGGRAPH 1997 paper. The details of this algorithm has been discussed in class. The most difficult part probably is to solve the resulted over-determined linear system. You can use these packages, GSL, Boost, LAPACK, LAPACK++, ATLAS, or write your own solver. You can refer to Chapter 2 of Numerical Recipes if you want to write your own linear solver. If you want to test your program before taking pictures of your own, you can download test images from here or here (hdr here).

  • On the other hand, to avoid the trouble with camera's nonlinear mapping, you can recover a radiance map directly from CCD snapshots. To use this method, remember to have your camera save images in raw format during capturing. You can then use dcraw or libraw to convert raw images into 16-bit ppm images. Dcraw can decode raw formats for many cameras including most of Canon, Nikon and Kodak's. Since the recorded pixels are linear to irradiance, you can just recover the slope of the recorded pixel values under different exposures and use it as the irradiance value. Notice that, to recover colors, you have to perform demosaicking on the recovered radiance map.
3. Develop your radiance map using tone mapping. Finally, tone map your radiance map with some programs. Here are some options: Photomatix (algorithm unknown), logview and tmo (24 algorithms implemented). The following two tone mapping programs are for Linux only: Fast Bilateral Filtering and PFStmo. Note that implementing tone mapping is a bonus and not required.

Bells and whistles

Students will get extra points if they implement any of the following extensions. Please note that this assignment is considered easy. It is strongly recommended that you implement at least one of these extensions, especially if you implemented Debevec's method using Matlab (which only gives you a very baseline grade).
  • Image alignment methods for HDR imaging. A good suggestion is Ward's MTB algorithm.
  • Other HDR creation methods such as Mitsunaga and Nayar's algorithm. Please refer to their CVPR 1999 paper, Radiometric Self Calibration. The main difference with Debevec's method is that this method assumes that the response curves are high-order polynomials. RASCAL uses this method. Another popular method is Robertson's method. It is more robust.
  • Tone mapping. Refer to the reading list for more tone mapping algorithms.
You are welcome to do any other extensions or develop algorithm related to high dynamic range images. The bonus depends on how useful and difficult these extensions are.

Submission

You have to turn in your complete source, the executable, the original pictures you have taken, a recovered HDR image, a tone-mapped image, instruction to run your program and a report in an html/pdf format (including all resources). The report could contain a description of this project, what you have learned from this project, description of the algorithm you implemented, implementation details, results (either good or bad), and what extensions you have implemented.

You also have to submit your favorite artifact generated by the program you have implemented (not the reference software). For the artifacts for this project, submit the tone-mapped image. All class participates will vote for the top three artifacts. The creators for these artifacts will get extra bonus points. Please submit your homework through http://ntuvfx.csie.org/vfx/2018/ (if you are within the cmlab domain, use 192.168.1.110/vfx/2018/ instead).

Reference software

  • Photomatix: a high-quality commercial program to create HDR images and tone mapping. It has a free version with basic operations.
  • RASCAL: a HDR creation program with image alignment capability.

References