Scene radiance

Code overview

We have collected and posted online tutorials and scripts that illustrate how to create scenes

---

Creating simple scenes

scene = sceneCreate;
scene = sceneCreate('slanted bar');

The ISETCam scene structure is a description of the scene spectral radiance. The scene structure includes the parameters necessary to describe the spectral radiance at sample points in the world. The scene model is simplified because each point is assumed to emit light uniformly in all of the directions that reach the camera aperture. (For a discussion of the full scene description, see light field.) We store the spectral radiance of each point in the scene structure using units of quanta/sec/sr/nm/m^2 (scene.data.photons).

In general the scene structure includes a depth map - the distance from the observer to each point in the scene (scene.depthMap), or the distance from the observer to the central point in a planar scene (scene.distance).

We also store the spectral radiance distribution of the light source. The light can be represented either as a uniform light by a single spectral vector, or as a space-varying representation of the illuminant spectral radiance distribution.

The scene representation is useful for many calculations. For example, many simple but important scenes are used in image quality metrics and camera evaluation. The sceneCreate function can create a large variety of scenes. The sceneFromFile function enables you to create a scene from JPEG, PNG, EXR and related image files. These are described below.

Creating 3D scenes

• N.B. We are starting to rely on ISET3d-tiny. More to come here in the near future (May, 2024).

A more complete representation of a 3D scene describes all the rays arriving at the camera aperture. At every point in the aperture we represent the intensity, wavelength and direction of the incident rays (light field). For calculations that account for the light field, we use graphics rendering methods ISET3d toolbox. This toolbox traces rays from the scene through a model of the camera optics. The ISET3d calculations account for multi-element optics and diffraction. The rendering accounts for occlusions and depth of field depth (bokeh) effects.

The ISET3d methods are constantly being expanded. At present they account for various types of bidirectional reflectance distribution functions, transparency, fluorescence, and retroreflective materials. The ISET3d package also simulates light field cameras with microlens arrays. When using ISET3d with a lens model, we return an optical image that can be used by ISETCam.

A special case arises when we use ISET3d with a pinhole model of the camera and diffraction turned off. In that case the iset3d code returns an ISETCam scene structure.

Basic scene functions

Like other ISETCam structures, the scene has these basic methods

• sceneCreate - Method that produces many different types of scenes for testing
• sceneSet - Set scene parameters (e.g., field of view, or mean luminance)
• sceneGet - Get scene parameters
• scenePlot - Plot scene functions and parameters
• sceneWindow - Open up the scene graphical user interface (GUI), which can hold a collection of scenes

More information about the scene parameters and programming with scenes can be found in the scene programming wiki pages, and in the scene Scripts and tutorials.

sceneFromFile

In addition to creating many simple images using sceneCreate, the ISETCam function sceneFromFile can read an RGB image and return a scene radiance structure. This enables you to test camera features using almost any image as an input to the simulation.

There are significant limits because the scene returned from an RGB image is not a physically-precise description of the scene; it is just a useful way to produce an approximation. Scenes created this way have no depth or illuminant information.

The algorithm implemented in sceneFromFile creates the scene spectral radiance that one would measure if (a) the RGB data are displayed on a monitor, and (b) the scene illumination is the monitor white point. By default we assume a special theoretical display (reflectance_display.mat) such that RGB data are rendered assuming a D65 illuminant and the radiances are all products of that illuminant and weighted sums of the first three principla components of the naturally occurring spectral reflectance functions. The script s_displaySurfaceReflectance.m explains how we did this.

Here is the sceneFromFile code in action using other display models.

d = displayCreate('LCD-Apple');
meanLuminance = 100;
scene = sceneFromFile('eagle.jpg','rgb',meanLuminance,d);
ieAddObject(scene); sceneWindow;

Notice that the scene depends on the display model, and these can differ substantially. Here is a comparison between two different displays.

d = displayCreate('LCD-Apple'); meanLuminance = 100;
scene = sceneFromFile('eagle.jpg','rgb',meanLuminance,d);
rgb1 = sceneGet(scene,'rgb'); imshow(rgb1);

d = displayCreate('CRT-Dell');
scene = sceneFromFile('eagle.jpg','rgb',meanLuminance,d);
rgb2 = sceneGet(scene,'rgb'); imshow(rgb2);