Scene radiance

Code overview

We have collected and posted online tutorials and scripts that illustrate how to create scenes

---

Creating simple scenes

scene = sceneCreate;
scene = sceneCreate('slanted bar');

The ISETCam scene structure is a description of the scene spectral radiance. The scene structure includes the parameters necessary to describe the spectral radiance at sample points in the world. Each point is at a different distance. The model is simplified because each point is assumed to emit light uniformly in all of the directions that reach the camera aperture. (For a discussion of the full scene description, see this page about the light field.)

We store the spectral radiance of each point in the scene structure using units of quanta/sec/sr/nm/m^2 (scene.data.photons). In almost all cases we further store the spectral radiance distribution of the light source. In some cases we store a space-varying representation of the illuminant spectral radiance distribution. In general cases we store the depth map as well - the distance from the observer to each point in the scene (scene.depthMap), or the distance from the observer to the central point in a planar scene (scene.distance).

This level of scene representation is useful for many calculations. For example, a scene whose points are all far away, or a planar scene of a test chart can be well-represented this way. These approximations suffice for many simple but important scenes used in image quality metrics and camera evaluation. Many users never need more than this basic structure.

Creating 3D scenes

A more complete and accurate representation of a 3D scene describes all the rays arriving at the camera aperture. At every point in the aperture we represent the intensity, wavelength and direction of the incident rays (light field). For simulations that account for the light field, we use graphics rendering methods ISET3d toolbox. This toolbox traces rays from the scene through a model of the camera optics. The ISET3d calculations account for multi-element optics and diffraction. The rendering accounts for occlusions and depth of field depth (bokeh) effects.

The ISET3d methods are constantly being expanded. At present they account for various types of bidirectional reflectance distribution functions, transparency, fluorescence, and retroreflective materials. The ISET3d package also simulates light field cameras with microlens arrays. When using ISET3d with a lens model, we return an optical image that can be used by ISETCam.

A special case arises when we use ISET3d with a pinhole model of the camera and diffraction turned off. In that case the iset3d code returns an ISETCam scene structure.

Basic scene functions

Like other ISETCam structures, the scene has these basic methods

• sceneCreate - Method that produces many different types of scenes for testing
• sceneSet - Set scene parameters (e.g., field of view, or mean luminance)
• sceneGet - Get scene parameters
• scenePlot - Plot scene functions and parameters
• sceneWindow - Open up the scene graphical user interface (GUI), which can hold a collection of scenes

More information about the scene parameters and programming with scenes can be found in the scene programming wiki pages, and in the scene Scripts and tutorials.

sceneFromFile

In addition to creating many simple images using sceneCreate, the ISETCam function sceneFromFile can read an RGB image and return a scene radiance structure. This enables you to test camera features using almost any image as an input to the simulation.

There are significant limits because the scene returned from an RGB image is not a physically-precise description of the scene; it is just a useful way to produce an approximation. Scenes created this way have no depth or illuminant information.

The algorithm implemented in sceneFromFile creates the scene spectral radiance that one would measure if (a) the RGB data are displayed on a monitor, and (b) the scene illumination is the monitor white point. It is possible to choose a monitor so that the implicit surface reflectance functions are within the first three principal components of natural surface reflectances. Spatial patterns, light levels, and related effects will be represented.

Here is the sceneFromFile code in action.

d = displayCreate('LCD-Apple');
meanLuminance = 100;
scene = sceneFromFile('eagle.jpg','rgb',meanLuminance,d);
ieAddObject(scene); sceneWindow;

Notice that the scene depends on the display model, and these can differ substantially. Here is a comparison between two different displays.

d = displayCreate('LCD-Apple'); meanLuminance = 100;
scene = sceneFromFile('eagle.jpg','rgb',meanLuminance,d);
rgb1 = sceneGet(scene,'rgb'); imshow(rgb1);

d = displayCreate('CRT-Dell');
scene = sceneFromFile('eagle.jpg','rgb',meanLuminance,d);
rgb2 = sceneGet(scene,'rgb'); imshow(rgb2);

We have created a special theoretical display (reflectance_display.mat) that is created so that the RGB data are rendered assuming a D65 illuminant and the radiances are all products of that illuminant and weighted sums of the first three principla components of the naturally occurring spectral reflectance functions.