Home
This repository is the next generation of ISET3d. This version is smaller, simpler, and uses a different, distributed, model for storing and managing assets.
** January 10, 2025: We are nearly ready to replace ISET3d with this repository. It appears to be fully functional. **
-
When we originally designed ISET3d most work was done on local machines. With PBRT's GPU implementation, we have been increasingly using remote rendering and data management. This is because we do not have Nvidia GPUs on our local machines, but we do have them on our shared servers. With this distributed environment, we needed new ways to control rendering computations.
-
Further, the size of our data (assets, textures, materials) has grown. We have come to think most data should be stored remotely but made accessible through a data management system. It was hard to layer the distributed data environment on the older environment, and in particular to do so in a way that other labs could easily copy and use.
-
We hope this re-write will make the software and data simpler to share.
ISET3d manages computer graphics software to render physics-based complex scenes. The outputs are intended to accurately represent natural scene properties (illumination, surface reflectance, multiple bounces). ISET3d is integrated with ISETCam and ISETBio.
A priority of the whole ISET software collection is to produce code that makes the ideas clear. Computational efficiency is valuable, but a secondary concern. Please let us know if you have a clearer way to implement an idea, or please propose a clarifying comment in the code or wiki.
We expect ISET3d and ISETCam will be toolboxes (libraries) on a user's Matlab path. Projects for specific papers will be in a separate directory or GitHub repository.
NOTE We are working actively on the 'dev' branch. If you have an issue getting something to run using the master branch, it is worth doing a checkout of the dev branch to try it.
Computer graphics software requires a description of the scene, including the objects, lights, materials, and textures. There is an effort to create a universal scene description (USD) format that can be shared by many software developers. This is an excellent idea. Alas, PBRT and other open-source rendering tools that we had available for our work do not yet use USD. Thus, ISET3d developed an equivalent idea that we called a rendering 'recipe'. This recipe includes all the information that PBRT needs to render a scene.
In ISET3d, the typical workflow is to create and edit a recipe. We then invoke a series of functions - write, render, show.
There are several ways to create a recipe. The most common is to use the ISET3d function piRead reads a PBRT scene file, and parses the contents into a Matlab class we call a recipe. The user edits the recipe using recipe.set() to specify how the scene will be rendered. This includes specifying the spatial resolution, number of rays per pixel, the multi-element optics, material properties, and the positions of objects.
When the user is finished editing the recipe, we use piWrite to create an updated version of the PBRT files. By default, these are written out into the iset3d/local/sceneName folder.
The piRender function calls a docker image to render the PBRT files. Rendering options include generating (a) the radiance or irradiance, (b) the depth map, and (c) pixel-by-pixel labels for the material or object. Typically, we show the rendered data in an ISETCam (or ISETBio) window invoking sceneWindow() or a lens is specified, oiWindow(). It is so common to write/render/show in a group that we implemented a method, piWRS() for that purpose.
ISET3d works with both ISETCam and ISETBio. Specifically, this means that it manages to render with pinhole, a realistic lens model (we call omni) from ISETCam, and a human eye model based on sceneEye in ISETBio.
When the recipe specifies pinhole optics, piRender returns the spectral radiance of a scene. If the recipe specifies a lens, the pRender output is the spectral irradiance at the surface of the sensor (aka film or retina). The radiance and irradiance formats are those used by ISETBio and ISETCam (scene and optical image).
ISET3d ray traces using code based on Matt Pharr's PBRT V4 (Physical based ray tracing). Our modifications to PBRT are encapsulated in a Docker image that can be launched as a container. The Docker image can run on some modern GPUs. To do so requires an image generated for a specific Nvidia GPU architecture (e.g. Pascal, Turing, Volta, or Ampere). We will add instructions on how to run the system on a CPU or GPU here, but for now please consult the Remote Rendering section of the ISET3d wiki.
We use the prefix pi for functions in this repository. The two letters refer to PBRT and ISET. (The original name of this repository was pbrt2iset). This prefix acts as an informal version of a namespace.