Merge pull request #245 from princeton-vl/rc_1.3.3

Merge v1.3.1 - v1.3.3

.gitignore

@@ -42,7 +42,8 @@ Blender-FLIP-Fluids
 *.blend1
 *.out
 profiles_*.npz
-*outputs
+outputs
+outputs_scratch
 snippets
 resources
 times.txt

README.md

@@ -45,8 +45,9 @@ Next, see our ["Hello World" example](docs/HelloWorld.md) to generate an image &
 - ["Hello World": Generate your first Infinigen scene](docs/HelloWorld.md)
 - [Configuring Infinigen](docs/ConfiguringInfinigen.md)
 - [Downloading pre-generated data](docs/PreGeneratedData.md)
-- [Extended ground-truth](docs/GroundTruthAnnotations.md)
 - [Generating individual assets](docs/GeneratingIndividualAssets.md)
+- [Exporting to external fileformats (OBJ, OpenUSD, etc)](docs/ExportingToExternalFileFormats.md)
+- [Extended ground-truth](docs/GroundTruthAnnotations.md)
 - [Implementing new materials & assets](docs/ImplementingAssets.md)
 - [Generating fluid simulations](docs/GeneratingFluidSimulations.md)
 

docs/CHANGELOG.md

@@ -36,3 +36,16 @@ v1.2.6
 - Fix bug where manage_jobs.py would ignore CUDA_VISIBLE_DEVICES that didnt start at 0
 - Add NotImplementedError for dynamic hair.
 
+v1.3.1
+- Fix configuration bug causing massive render slowdown 
+- Create noisier video trajectories optimized for training
+
+v1.3.2
+- Bugfix USD/OBJ exporter, add export options to generate_individual_assets
+
+v1.3.3
+- Bugfix camera code to allow multiple cameras, prevent all-water frames
+- Tweak rendering settings
+- Tweak test lists & add timeouts, pass all tests
+
+

docs/ExportingToExternalFileFormats.md

@@ -0,0 +1,56 @@
+
+# Asset Exporter
+
+This documentation details how to create an OBJ, FBX, STL, PLY or OpenUSD file from a `.blend` file, such as those produced by [Hello World](HelloWorld.md) or [Generating Individual Assets](./GeneratingIndividualAssets.md). 
+
+Blender does provide a built-in exporter, but it wont work out of the box for Infinigen since our files contain procedural materials and assets defined using shader programs. This tool's job is to "bake" all these procedural elements into more standard graphics formats (i.e, simple meshes with materials made of texture maps), before invoking the standard blender exporter. This process can be slow, since it uses a rendering engine, and lossy, since the resulting textures have a finite resolution.
+
+To convert a folder of blender files into USD files (our recommmended format), use the command below:
+```bash
+python -m infinigen.tools.export --input_folder {PATH_TO_FOLDER_OF_BLENDFILES} --output_folder outputs/my_export -f usdc -r 1024
+```
+
+If you want a different output format, please use the "--help" flag or use one of the options below:
+- `-f obj` will export in .obj format, 
+- `-f fbx` will export in .fbx format
+- `-f stl` will export in .stl format
+- `-f ply` will export in .ply format. 
+- `-f usdc` will export in .usdc format. 
+- `-v` enables per-vertex colors (only compatible with .fbx and .ply formats). 
+- `-r {INT}` controls the resolution of the baked texture maps. For instance, `-r 1024` will export 1024 x 1024 texture maps.
+- `--individual` will export each object in a scene in its own individual file.
+
+## :warning: Exporting full Infinigen scenes is only supported for USDC files. 
+
+:bulb: Note: exporting OBJ/FBX files of **single objects** *generally works fine; this discussion only refers to large-scale scenes.
+
+Infinigen uses of *instancing* to represent densely scattered objects. That is, rather than storing millions of unique high-detail pebbles or leaves to scatter on the floor, we use a smaller set of unique objects which are stored in memory only once, but are repeated all over the scene with many different transforms. 
+
+To our knowledge, no file formats except '.blend' and '.usdc' support saving 3D files that contain instanced geometry. For all file formats besides these two, instanced will be *realized*: instead of storing just a few unique meshes, the meshes will be copied, pasted and transformed thousands of times (once for each unique scatter location). This creates a simple mesh that, but the cost is so high that we do not recommend attempting it for full Infinigen scenes. 
+
+If you require OBJ/FBX/PLY files for your research, you have a few options:
+- You can use individual objects, rather than full scenes. These *generally dont contain instancing so can be exported to simple mesh formats.
+- You can use advice in [Configuring Infinigen](./ConfiguringInfinigen.md) to create a scene that has very small extent or low detail, such that the final realized mesh will still be small enough to fit in memory.
+- You can use the advice in [Configuring Infinigen](./ConfiguringInfinigen.md) to create a scene which simply doesnt contain any instanced objects. Specifically, you should turn off trees and all scattered objects.
+    - The simplest way to do this is to turn off everything except terrain, by including the config `no_assets.gin`. 
+
+*caveat for the above: Infinigen's implementation for trees uses instances to represent leaves and branches. Trees are also generally large and high detail enough to cause issues if you realize them before exporting. Therefore, exporting whole trees as OBJs also generally isnt supported, unless you do so at very low resolution, or you turn off the tree's branches / leaves first. 
+
+## Other Known Issues and Limitations
+
+* Some material features used in Infinigen are not yet supported by this exporter. Specifically, this script only handles Albedo, Roughness and Metallicity maps. Any other procedural parameters of the material will be ignored, so you should not expect complex materials (e.g skin, translucent leaves, glowing lava) to be perfectly reproduced outside of Blender. Depending on file format, there is limited support for materials with non-procedural, constant values of transmission, clearcoat, and sheen.
+
+* Exporting *animated* 3D files is generally untested and not officially supported. This includes exporting particles, articulated creatures, deforming plants, etc. These features are *in principle* supported by OpenUSD, but are untested by us and not officially supported by this export script.
+
+* Assets with transparent materials (water, glass-like materials, etc.) may have incorrect textures for all material parameters after export.
+
+* Large scenes and assets may take a long time to export and will crash Blender if you do not have a sufficiently large amount of memory. The export results may also be unusably large.
+
+* When exporting in .fbx format, the embedded roughness texture maps in the file may sometimes be too bright or too dark. The .png roughness map in the folder is accurate, however.
+
+* .fbx exports ocassionally fail due to invalid UV values on complicated geometry. Adjusting the 'island_margin' value in bpy.ops.uv.smart_project() sometimes remedies this
+
+
+
+
+

docs/GeneratingIndividualAssets.md

@@ -1,20 +1,16 @@
 # Generating Individual Assets
 
-This tutorial will help you generate images or .blend files of specific assets of your choice.
-
-Limitations (to be addressed soon): 
-- This tool only exports .blend files. [See here](../infinigen/datagen/tools/export/README.md) for a prototype tool to convert these to standard mesh file formats, but it itself has some limitations. 
-- This tool cannot currently generate or export terrain meshes.
+This tutorial will help you generate images, .blend files, or baked OBJ/USD/etc files for any asset of your choosing.
 
 ### Example Commands
 
-Shown are three examples of using our `generate_individual_assets.py` script to create images and .blend files.
+Shown are three examples of using our `generate_individual_assets.py` script to create images and .blend files. If you want to create asset files in another format (e.g. OBJ, FBX, USD) you should first generate blend files then use our [](./ExportingToExternalFileFormats.md) docs to bake them to the format of your choosing. 
 
 ```bash
 mkdir outputs
-python -m infinigen_examples.generate_individual_assets -f CoralFactory -n 8 --save_blend
-python -m infinigen_examples.generate_individual_assets -f seashells -n 1 --save_blend
-python -m infinigen_examples.generate_individual_assets -f chunkyrock -n 1 --save_blend
+python -m infinigen_examples.generate_individual_assets --output_folder outputs/corals -f CoralFactory -n 8 --save_blend
+python -m infinigen_examples.generate_individual_assets --output_folder outputs/seashells -f seashells -n 1 --save_blend
+python -m infinigen_examples.generate_individual_assets --output_folder outputs/chunkyrock -f chunkyrock -n 1 --save_blend
 ```
 
 <p align="center">
@@ -23,15 +19,26 @@ python -m infinigen_examples.generate_individual_assets -f chunkyrock -n 1 --sav
   <img src="images/individual_assets/chunkyrock.png" width="150" />
 </p>
 
-Running the above commands will save images and .blend files into your `outputs` folder.
+Running the above commands will save images and .blend files into your `outputs` folder. You can customize what object is generated by changing the `-f` argument to the name of a different AssetFactory defined in the codebase (see the file `tests/test_meshes_basic.txt` for a partial list).
 
 Please run `python -m infinigen.tools.generate_individual_assets --help` for a full list of commandline arguments.
 
-The most commonly used arguments are:
-- `-f` to specify the name(s) of assets or materials to generate. `-f NAME` can specify to generate three different types of objects:
-   - If `NAME` is the name of a class defined in `infinigen/assets`, then it will be treated as an AssetFactory and used to generate objects from scratch. For example, you can say `-f CactusFactory` or `-f CarnivoreFactory`, or use the name of any similar Factory class in the codebase.
-   - If `NAME` is the name of a file in `infinigen/assets/materials`, that material will be applied onto a sphere
-   - If `NAME` is the name of a file in `infinigen/assets/scatters`, that scatter generator will be applied nto a plane
-- `-n` adjusts the number of images / blend files to be generated.
+### Creating OBJ, USD, FBX and other file formats
+
+You can use the `--export` flag to export each generated asset to a format of your choosing. Please see [ExportingToExternalFileFormats](./ExportingToExternalFileFormats.md) for details and limitations regarding exporting. 
+
+Examples:
+
+```bash
+
+# Save a coral as an OBJ with texture maps
+python -m infinigen_examples.generate_individual_assets --output_folder outputs/corals -f CoralFactory -n 1 --render none --export obj
+
+# Save a bush as OpenUSD 
+python -m infinigen_examples.generate_individual_assets --output_folder outputs/bush -f BushFactory -n 1 --render none --export usdc
+
+# See the full list of supported formats
+python -m infinigen_examples.generate_individual_assets --help
+```
 
 

infinigen/__init__.py

@@ -1,3 +1,3 @@
 import logging
 
-__version__ = "1.2.6"
+__version__ = "1.3.3"

infinigen/assets/creatures/reptile.py

@@ -392,14 +392,6 @@ def chameleon_postprocessing(body_parts, extras, params):
 
     #chameleon_eye.apply(get_extras('Eye'))
 
-def purge_empty_materials(obj):
-    with butil.SelectObjects(obj):
-        for i, m in enumerate(obj.material_slots):
-            if m.name != '':
-                continue
-            bpy.context.object.active_material_index = i
-            bpy.ops.object.material_slot_remove()
-
 @gin.configurable
 class LizardFactory(AssetFactory):
 
@@ -420,7 +412,7 @@ def create_asset(self, i, animate=False, rigging=False, cloth=False, **kwargs):
         else:
             joined = butil.join_objects([joined] + extras)
 
-        purge_empty_materials(joined)
+        butil.purge_empty_materials(joined)
 
         return root
 
@@ -518,6 +510,8 @@ def create_asset(self, i, placeholder, **kwargs):
                 root.parent = butil.spawn_empty('snake_parent_temp') # so AssetFactory.spawn_asset doesnt attempt to parent
                 butil.parent_to(joined, root, keep_transform=True)
 
+        butil.purge_empty_materials(joined)
+
         return joined
 
 @gin.configurable

infinigen/assets/fruits/general_fruit.py

@@ -169,6 +169,9 @@ def create_asset(self, **params):
         obj.scale *= normal(1, 0.1) * self.scale * scale_multiplier
         butil.apply_transform(obj)
 
+        # TODO remove when empty materials from geonodes is debugged
+        butil.purge_empty_materials(obj) 
+
         tag_object(obj, 'fruit_'+self.name)
         return obj
 

infinigen/assets/leaves/leaf_pine.py

@@ -366,6 +366,7 @@ def create_asset(self, **params):
         obj = bpy.context.object
         obj.scale *= normal(1, 0.05) * self.scale
         butil.apply_transform(obj)
+        butil.purge_empty_materials(obj) # TODO remove when geonodes emptymats solved
         tag_object(obj, 'leaf_pine')
 
         return obj

infinigen/assets/lighting/__init__.py

@@ -1,3 +1,2 @@
 from . import sky_lighting
-from .caustics_lamp import CausticsLampFactory
-from .glowing_rocks import GlowingRocksFactory
+from .caustics_lamp import CausticsLampFactory

infinigen/assets/materials/ice.py

@@ -45,7 +45,7 @@ def shader_ice(nw: NodeWrangler):
         Nodes.PrincipledBSDF,
         input_kwargs={
             'Subsurface': 1.0000, 
-            'Subsurface Radius': (0.1000, 0.1000, 0.2000), 
+            'Subsurface Radius': (0.0010, 0.0010, 0.0020), 
             'Subsurface Color': tuple(col_ice),
             'Roughness': color_ramp.outputs["Color"], 
             'IOR': 1.3100

infinigen/assets/rocks/__init__.py

@@ -1,2 +1,3 @@
 from .blender_rock import BlenderRockFactory
-from .boulder import BoulderFactory
+from .boulder import BoulderFactory
+from .glowing_rocks import GlowingRocksFactory

infinigen/assets/small_plants/snake_plant.py

@@ -269,6 +269,8 @@ def create_asset(self, **params):
         obj.scale = (0.2, 0.2, 0.2)
         butil.apply_transform(obj, scale=True)        
 
+        butil.purge_empty_materials(obj)
+
         tag_object(obj, 'snake_plant')
         return obj
 

infinigen/core/execute_tasks.py

@@ -96,7 +96,7 @@ def populate_scene(
     **params
 ):
     p = RandomStageExecutor(scene_seed, output_folder, params)
-    camera = bpy.context.scene.camera
+    camera = [cam_util.get_camera(i, j) for i, j in cam_util.get_cameras_ids()]
 
     season = p.run_stage('choose_season', trees.random_season, use_chance=False, default=[])
 
@@ -118,7 +118,7 @@ def populate_scene(
     p.run_stage('populate_clouds', use_chance=False,
         fn=lambda: placement.populate_all(weather.CloudFactory, camera, dist_cull=None, vis_cull=None))
     p.run_stage('populate_glowing_rocks', use_chance=False,
-        fn=lambda: placement.populate_all(lighting.GlowingRocksFactory, camera, dist_cull=None, vis_cull=None))
+        fn=lambda: placement.populate_all(rocks.GlowingRocksFactory, camera, dist_cull=None, vis_cull=None))
 
     populated['cached_fire_trees'] = p.run_stage('populate_cached_fire_trees', use_chance=False, default=[],
         fn=lambda: placement.populate_all(fluid.CachedTreeFactory, camera, season=season, vis_cull=4, dist_cull=70, cache_system=fire_cache_system))
@@ -283,6 +283,7 @@ def execute_tasks(
     resample_idx=None,
     output_blend_name="scene.blend",
     generate_resolution=(1280,720),
+    fps=24,
     reset_assets=True,
     focal_length=None,
     dryrun=False,
@@ -315,6 +316,7 @@ def execute_tasks(
     bpy.context.scene.frame_start = int(frame_range[0])
     bpy.context.scene.frame_end = int(frame_range[1])
     bpy.context.scene.frame_set(int(frame_range[0]))
+    bpy.context.scene.render.fps = fps
     bpy.context.scene.render.resolution_x = generate_resolution[0]
     bpy.context.scene.render.resolution_y = generate_resolution[1]
     bpy.context.view_layer.update()
@@ -338,7 +340,8 @@ def execute_tasks(
 
     if Task.FineTerrain in task:
         terrain = Terrain(scene_seed, surface.registry, task=task, on_the_fly_asset_folder=output_folder/"assets")
-        terrain.fine_terrain(output_folder, optimize_terrain_diskusage=optimize_terrain_diskusage)
+        cameras = [cam_util.get_camera(i, j) for i, j in cam_util.get_cameras_ids()]
+        terrain.fine_terrain(output_folder, cameras=cameras, optimize_terrain_diskusage=optimize_terrain_diskusage)
 
     group_collections()