Scene Selection in HITL App

[ambervg]

System information & package versions

• OS: Ubuntu 22.04.1
• Architecture: x86_64
• habitat-sim version: 0.3.0
• habitat-lab version: 0.3.0
• habitat-hitl version: 0.3.0

❓ Questions and Help

Problem description:

For a simple habitat HITL app I want to use a scene that differs from the default one. I have tried to add a scene .glb file to the .yaml configuration file like this:

habitat:
  # https://github.com/facebookresearch/habitat-lab/blob/main/habitat-lab/habitat/config/default_structured_configs.py
  simulator:
    scene: "data/scene_datasets/habitat-test-scenes/skokloster-castle.glb"

However, when running the script, the scene is still the default one instead of the just defined skokloster-castle. It seems like the scene is being overwritten somewhere. Let’s take the minimal app as example. I am wondering which files have to be linked at what location(s) so that the scene being used is changed.

I have checked the passed configurations with print(omegaconf.OmegaConf.to_yaml(config)) in the main function and will link the results of that here: used_config.txt

I feel like this question should have a very straightforward answer, but I am stuck on it and would greatly appreciate it if anyone could point me in the right direction. So far, I have not been able to find the answer to this in the documentation, but maybe I am not looking in the right places.

My questions:

1. For the minimal.py & minimal.yaml example, which files would I have to link at what location(s) so that the actual scene being used is changed?
2. Where can the scene still be overwritten?
3. Do I need more files than just the .glb and .navmesh for a scene to be loaded successfully?

[ojss]

I have the same issue! perhaps, @0mdc @aclegg3 any insights you could share?

[0mdc]

The HITL app sits on top of habitat-lab rather than just the sim, which works with episode sets instead of scenes. I'm also interested in adding support for loading specific scenes (via config or at any time via a command).

As a work-around, you can create an episode set (json.gz file) with only 1 episode which loads skokloster-castle.glb.

You can use an existing file as a template, e.g. data/versioned_data/hab3-episodes/train/social_rearrange.json.gz.
This one is downloaded using the python -m habitat_sim.utils.datasets_download --uids hab3-episodes command.

[ambervg]

Tagging @rutadesai based on our discussion this morning.

[ambervg]

I have been trying to get this work-around to work by using run_episode_generator.py to create an episode set, such as data/versioned_data/hab3-episodes/train/social_rearrange.json.gz. Let me start by describing the steps that I have taken.

Steps I have taken:

1. Creating a .yaml file with configs for run_episode_generator.py. I will link one of my attempts here: config_ep_v10.zip. It is a .zip format because GitHub did not accept .yaml.
2. I run the below command to create rearrange_ep_dataset_v10.json.gz. I will then copy this to my desired location.

python habitat-lab/habitat/datasets/rearrange/run_episode_generator.py --config habitat-lab/habitat/datasets/rearrange/configs/config_ep_v10.yaml --out habitat-lab/habitat/datasets/rearrange/configs/rearrange_ep_dataset_v10.json.gz --run --num-episodes 1

3. Now, I go to hitl_rearrange.yaml and at habitat.dataset.data_path I link path/to/rearrange_ep_dataset_v10.json.gz.
4. Ensure that hitl_rearrange.yaml is in fact being used by rearrange.py in this line.
5. Finally running python path/to/rearrange.py. However, this results in an error for me: KeyError: 'any_targets|0'. I will attach a file with the full error message here: error_msg_keyerror.txt.

Flow:
If I understand it correctly, the flow is as follows:

My question:

• Any tips on how to handle the KeyError: 'any_targets|0'? The error does not only happen for the episode sets that I create myself, but also when I use the config_ep.yaml files that are provided in the repo.

@rutadesai @0mdc

[0mdc]

Hmm. I believe that the problem is due to the config activating incompatible pddl code.
hitl_rearrange.yaml is indeed referencing pop_play.yaml, which is probably breaking with this setup.

Let me see if I can come up with a bare-bone configuration that doesn't do all this unnecessary work.

[ambervg]

@0mdc That would be super helpful. Thank you so much.

Based on recommendations from @rutadesai + @aclegg3 I tried changing something in the config_ep.yaml because the PDDL implementation is hardcoded to look for "any_targets" with an "s".

So from:

object_target_samplers:
  - name: 'any_target'

To:

object_target_samplers:
  - name: 'any_targets'

This changed the error from KeyError: 'any_targets|0' to KeyError: 'any_targets|1'.