Practical SVBRDF Capture In The Frequency Domain

Code and data

Return to main project page


This is a source code and data release for the ACM Siggraph 2013 paper Practical SVBRDF Capture In The Frequency Domain, by Aittala, Weyrich and Lehtinen.

Most of the code is written in Matlab and requires some toolboxes to run (image processing, preferably also parallel toolbox for performance). The capture tool is C++ code and requires the Canon EDSDK. The code is research code, and hence unfortunately is not particularly readable, flexible or efficient. We hope that you will find it useful nevertheless.


Copyright (c) 2013-2015 Miika Aittala, Jaakko Lehtinen, Tim Weyrich, Aalto University, University College London. This code and data is released under the Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International license (


Source code (Matlab/C++)
Dataset: Mix (~700 MB)
Dataset: Pynchon (~700 MB)
Dataset: Crumpled (~700 MB)
Dataset: Eco (~700 MB)
Dataset: Tile (~700 MB)
Dataset: Bluebook (~700 MB)
Raw TIFF images for Lindt dataset (3.5 GB)


The source code archive contains the following directories:

Quick start

To run the optimizer on all the datasets:

Running the optimizer

The actual optimization is performed by optimizer.m. It shows the current iterate (starting at a low resolution) and gradually improves it. Finally it outputs a Matlab data file sols.mat and the resulting images as 16-bit TIFFs in the specified directory. The displayed images are: diffuse albedo, specular albedo, kurtosis, normals, glossiness, unused (shows upsampling progress), and normals in another visualization. Notice that the glossiness is "inverse" (dark is shiny), as it corresponds to our sigma parameter.

For performance, you should launch a few parallel jobs using matlabpool (if you have it.) The optimizer outputs quite a bit of messy progress data into the console. Thanks to the low-resolution initial stage, you should be able to see rather quickly whether the solution is going to be something reasonable.

    matlabpool 6        
    % Always include the final \ and make sure that the directory exists!
    sols = optimizer(Data, 'Z:\whatever_your_path\mix\'); 

The solution will be in sols, which is a 1024 * 1024 * 10 array. The 10 channels are (for historical reasons): diffuse R, specular R, diffuse GB, specular GB, glossiness, normal x, normal y and kurtosis.

Capture tool

The tool that displays the patterns on the monitor and drives the camera is in the capture_tool/ directory. It is a Visual Studio 2010 project. It also requires the Canon EDSDK for camera control.

Again, this is a crude tool used for research and experimentation. A proper product-like implementation could be significantly more friendly and automated.

We have used a Canon EOS 5D Mk II camera. The temporal syncing between shutter and patterns has been chosen experimentally and may not match the times needed for some other camera model. The important thing is that the shutter must be open by the time the pattern starts showing, and must close after it has finished; preferably with some safety buffer at each end.

We have included a dataset (Lindt) that contains an entire output of a capture session as TIFF files converted from Canon RAW files by dcraw with maximally linear settings. Have a look at the photos (especially the first ones) to get an idea of what they should look like.

The overall steps for capturing are:

Calibration tool

Run calibration_ui.m. This is a calibration and solver launch tool.

This, too, is a crude research tool. Note in particular that it is currently hardcoded assuming a 16:9 capture monitor aspect ratio and a given monitor emission distribution. Change the values from the code if needed. (You can probably recycle the monitor calibration values if they happen to work for you, but the aspect ratio should correspond to that used in the capture.)

The steps are:

  • Click Pre-process and choose the first image of the actual frequency set, namely the black frame (IMG_5924.tiff.) The program will now rectify the images and combine the partial waves into the actual complex-valued Fourier basis images. It takes some time. You will see the current image on the upper left, and the magnitude and phase on the lower row. After it is done, you will find a large file in the directory, called data.mat. It is precisely the thing we saw in a previous section.
  • Finally, you can compute the solution by pressing Solve. It simply launches the solver as discussed before, and outputs the solution images when it's done. You should say something like matlabpool 4 in the console before running this, if you have parallel toolbox; otherwise it will be much slower.
  • We've also included a button called Solve (heuristic). It simply outputs the magnitude and phase images, and a heuristic guess for the normal map. They could be potentially useful for artists or other purposes.

    Exploring the data

    The data packages each contain a single .mat file, which contains a Matlab struct. Let us use the Mix dataset as an example. Load it up:


    The data will be placed into a struct called Data. To see its contents, simply type Data. The struct contains the image data itself, and all relevant calibration information. For historical reasons, names and conventions do not always match to those in the paper.

    Let's first familiarize ourselves with the near-raw data. The instructions for running the actual optimizer are in the end of this section.

    To see the DC component (zeroth frequency image, i.e. simply illuminated by the plain window function), type


    (the factor 4 is just to make it a bit brighter; imagec is our simple color image viewing function which does gamma correction and is a bit less picky than Matlab's image)

    The actual frequency pattern images are stored as complex numbers in Data.Z, which is a 1024 px * 1024 px * 3 colors * 8 frequencies * 4 orientations array. The actual frequencies themselves are listed in Data.freqs. Let's see all the data images at once.

    Here are the magnitudes, in a 8*4 array corresponding to orientation and frequency. Notice the effect of increasing frequency: the diffuse component fades away rather quickly.

        % Magnitudes of all data
        for o = 1:4
            for f = 1:8

    Here's a representative example in higher resolution:


    Let's have a similar look at the phase. Clearly the phase seems to contain strong clues about the normals, but it is still distorted especially at low frequencies:

        % Phases of all data
        subplot(8, 4,1);
        for o = 1:4
            for f = 1:8

    One could extract quite a bit of useful heuristic information out of this sort of near-raw data itself, if accuracy is not critical and the surfaces are glossy (it is the mixing of diffuse and specular that makes it difficult.) In fact, we base our initial guess on similar reasoning.

    Questions, comments