Model submission for BOSDYN_SPOT and two sensor configs of CTU-CRAS-Norlab

[peci1]

This PR adds Boston Dynamics Spot robot. It adds robot BOSDYN_SPOT which is the bare platform with no additional sensors. On top of this common base, two sensor configs CTU_CRAS_NORLAB_SPOT_SENSOR_CONFIG_X are submitted.

It is expected that the bare robot model will also be utilized by other teams when submitting their own sensor configs of Spot. We're coordinating the effort together in ctu-vras#1 . It is not expected that the base robot would be used alone without any sensor config - that's why it doesn't have the _sensor_config_x suffix. Sensor config 1 and 2 differ just in the (non-)presence of breadcrumbs.

So far we were unsuccessful collecting the validation data. CTU-CRAS-Norlab's Spots are still fighting with administrative steps for buying them. Teams CoSTAR and MARBLE who have the real robots agreed to help collecting the validation data, but it is unclear whether they will succeed in doing so anytime soon, probably because of COVID-19 restrictions. In the worst case, we have arranged a rental of Spot for end of April/beginning of May, so this is the latest time when the validation data will be provided for sure. This approach was allowed in an email communication with Angela Maio on 18 Feb 2021:

model validation data can be submitted up to 3 months after submission of the model (in accordance with the "Model Validation Sessions" contract deliverable)

The design of flippers and robot body have been already finished, while the top deck and sensor box might still undergo minor changes. I'll try to keep the model up-to-date with latest iterations with the manufacturer (until the PR is merged).

We have already uploaded the validation data ZIP with datasheets of our sensor config's sensors. As said earlier, validation data are missing at the moment. It is impossible to find any detailed specs of the Spot's sensors, so their simulation and specification is just an approximation of what could be devised looking at the robot and data. We used the following information sources:

• https://github.com/clearpathrobotics/spot_ros/blob/0cfb0c1c43460238bc2ca113b1d567d6dd1c1985/spot_driver/scripts/spot_wrapper.py#L28 for the list of cameras
• https://github.com/boston-dynamics/spot-sdk/blame/master/python/bosdyn-client/README.md#L58 for a list of resolutions of the onboard cameras
• the Clearpath URDF model
• https://github.com/tu-darmstadt-ros-pkg/image_projection/tree/master/image_projection_demo/doc for sample camera images

The robot can be controlled by any force- or postional- control gait generation algorithm. For ease of use, we prepared https://github.com/ctu-vras/subt_spot package which uses CHAMP to generate a simple gait that works on flat ground (it is open-loop, so terrain will be problematic). As this package depends on unreleased ROS packages (CHAMP), we did not include it directly in the SubT repo. Instead, the users should install this package and its dependencies manually in the solution containers.

The model.sdf files should not be edited manually. The model configuration process of Spot starts with Xacro files in urdf/ folder and config files in config/model. A combination of these files yields an intermediate URDF, which is converted to SDF via ign sdf -p and then polished and postprocessed using scripts/clean_sdf. This whole conversion process is done in script scripts/update_robot_sdf. If a change in BOSDYN_SPOT should affect all sensor configs of Spot, the update_robot_sdf script has to be called in each of the packages separately.

There is no URDF file as it is generated on-the-fly by calling scripts/print_robot_urdf script. This script is utilized by launch/description.launch.

The additional sensor configs reuse most of the code and configuration of the base robot and sensor config 1 instead of copy-pasting it.

---

I was looking into porting gazebo_ros_control to work with Ignition gazebo, but I got quickly completely lost. The main problem is the missing API plugin which would wait for roscore and such things and, most importantly, read and parse the remapping arguments. It seems Ignition Gazebo doesn't yet support plugins that could read the CLI args. That is mostly a show-stopper for any use of ROS in IG. So I ended up writing double-layered plugins - one reads the components of simulator and publishes them on an Ign topic, and the other reads the Ign topic and passes it to ROS. This is not optimal, but as said, there's no way so far to run a ROS publisher inside IG (or I haven't found it).

These plugins might be of more general interest, but due to time constraints, I published them as a part of the bosdyn_spot package:

• configurable_joint_position_controller
• logical_contact_system
• mono_camera_system

The mono_camera_system works around the non-support of mono cameras in IG. It reads the RGB camera and exposes a new topic mono/image which contains the RGB image converted to 8-bit mono. I used the wikipedia-style conversion

L = 0.3*R + 0.6*G + 0.1*B

But the result is weird in places lit by a spot light. I'm not sure if some gamma magic doesn't need to be done before/after the conversion.

[peci1]

This is a preview of all data produced by the base platform, together with a preview of the walking capabilities:

spot.mp4

[peci1]

Mobility of the virtual robot together with an example odometry (in this case only base_link->base_footprint).

state_estimation_better.mp4

[peci1]

A few Gazebo Classic screenshots that might come handy for model assessment.

Please note that the light visuals are affected by the not yet fixed bug gazebosim/gazebo-classic#2947 .

[peci1]

Proof of working breadcrumb deploy via ROS topic for sensor config 2 :)

[peci1]

Preview of sensors available on CTU_CRAS_NORLAB_SPOT_SENSOR_CONFIG_1:

Please note that "Omni Front" and "Omni Rear" cameras are upside-down on purpose. This is how the real cameras will be mounted.

[peci1]

This is the glitch in mono camera rendering I talked about in the first post:

It seems to me that the black parts are mostly in places with strong spotlight illumination. However, it should not be a problem as the mono cameras are just a postprocessing of RGB images...

[peci1]

I downgraded model.sdf to SDF version 1.6 as requested for all our models.

[acschang]

The initial assessment of the Boston Dynamics' Spot model and associated CTU-CRAS-NORLAB payload is complete however there are some issues to resolve before this model can be merged and utilized in a competitive setting:

Base Spot:

• Please update BOSDYN_SPOT's IMU parameters to match the new standard SubT IMU parameters.
• Please add intrinsic values for all camera sensors.
• Please confirm the robot mass and make sure the mass in specifications.md and model.sdf agree.
• Please update the endurance to meet the value specified in the specifications.md. This can be done by adjusting the power_load parameter in common.rb to 9.9 as opposed to 4.95.
• Please provide validation data as it is necessary to complete the evaluation. If not provided, the model's maximum velocity will be limited to 1m/s and battery life set to 60min. When the model validation data can be provided later, please submit a PR updating the model.

CTU-CRAS-NORLAB Payload:

• Please add intrinsic values for all cameras sensors in the OmniCamera.
• Please update the Ouster range resolution and stddev values to match the standard in the SubT repository (0.01 for both).
• Please update IMU parameters on the CTU-CRAS-NORLAB sensor payload to match the new standard SubT IMU parameters.

Please address the issues noted above in this pull request. Once the above issues are fixed, your submission will move onto the next review stage!

[peci1]

I addressed all of the issues except validation data, which should however be already being collected by CoSTAR.

Base Spot:

• Please update BOSDYN_SPOT's IMU parameters to match the new standard SubT IMU parameters.
• Please add intrinsic values for all camera sensors.
• Please confirm the robot mass and make sure the mass in specifications.md and model.sdf agree.
  • the model I based the work on apparently did not model the 4 kg battery. I added it. That required tuning of the PIDs for CTU_CRAS_NORLAB_SPOT as with the 4 kg more it became very unstable.
• Please update the endurance to meet the value specified in the specifications.md. This can be done by adjusting the power_load parameter in common.rb to 6.6 as opposed to 4.95.
• Please provide validation data as it is necessary to complete the evaluation. If not provided, the model's maximum velocity will be limited to 1m/s and battery life set to 60min. When the model validation data can be provided later, please submit a PR updating the model.

CTU-CRAS-NORLAB Payload:

• Please add intrinsic values for all cameras sensors in the OmniCamera.
• Please update the Ouster range resolution and stddev values to match the standard in the SubT repository (0.01 for both).
• Please update IMU parameters on the CTU-CRAS-NORLAB sensor payload to match the new standard SubT IMU parameters.

The body mono camera intrinsics are based on this camera info from the real Spot provided by CoSTAR:

header: 
  seq: 214
  stamp: 
    secs: 1603401456
    nsecs: 366378422
  frame_id: "back_fisheye"
height: 480
width: 640
distortion_model: "plumb_bob"
D: [0.0, 0.0, 0.0, 0.0, 0.0]
K: [257.48651123046875, 0.0, 316.9465637207031, 0.0, 256.9732971191406, 245.06680297851562, 0.0, 0.0, 1.0]
R: [1.0, 0.0, 0.0, 0.0, 1.0, 0.0, 0.0, 0.0, 1.0]
P: [257.48651123046875, 0.0, 316.9465637207031, 0.0, 0.0, 256.9732971191406, 245.06680297851562, 0.0, 0.0, 0.0, 1.0, 0.0]
binning_x: 0
binning_y: 0
roi: 
  x_offset: 0
  y_offset: 0
  height: 0
  width: 0
  do_rectify: False

I chose the use the ideal values for cx and cy though, as it doesn't really make sense to use other values in the simulation.

The depth camera intrinsics are so far based on the mono camera, just different resolution. When I get my hands on a camera info from these, I'll update the models to be as close to the real ones as possible.

I also added a link to experimental validation data for Basler cameras on CTU_CRAS_NORLAB_SPOT.

[peci1]

Sensor config 2 also works if I manually add the SC1 files in .ignition/fuel cache. What is surprising (or not at all?) is that the previous relative references worked too, in this case (i.e. Fuel had to add the /1 version directory to the mesh URL itself). So maybe the relative references would work, too, but now when the models will already be uploaded to Fuel, having the absolute URLs is better.

[peci1]

When uploading Sensor Config 1 to Fuel, don't forget that most DAE files in the meshes directory are actually symlinks to bosdyn_spot meshes. So be sure to follow symlinks when uploading.

[mjcarroll]

@acschang anything else from you?

[mjcarroll]

@peci1 as a follow up, please verify: https://app.ignitionrobotics.org/OpenRobotics/fuel/models/CTU_CRAS_NORLAB_SPOT_SENSOR_CONFIG_1

and

https://app.ignitionrobotics.org/OpenRobotics/fuel/models/CTU_CRAS_NORLAB_SPOT_SENSOR_CONFIG_2

[nkoenig]

Bounding box size: [0.87244, 0.448977, 1.07188]

[peci1]

@mjcarroll I tested both sensor configs and they work well. I tried clearing them from local Fuel cache and starting with sensor config 2, and it correctly downloaded both sensor config 1 and bosdyn_spot.

I've discovered a problem with the use of nodelets, though. See #898 where I disabled running nodelets in nodelet managers to get a more reliable behavior.

[peci1]

I've seen you assigned CTU-CRAS-Norlab Spot a credit price of 300 credits (on https://app.ignitionrobotics.org/OpenRobotics/fuel/models/CTU_CRAS_NORLAB_SPOT_SENSOR_CONFIG_1 ). I think it is a lot compared to other robots and taking into account how difficult it is to control this robot and how easy it is to tip it over (I haven't found a way to stand it once it falls). If possible, please, reconsider the price, otherwise Spot is practically useless...

[peci1]

Thanks, I see Angela decreased the cost to 255/280!