skin emulation

[matejhof]

Hi everyone,
as I'm working on the iCub "self-touch" project, I was not so happy to see the icubSim going through his body without colliding. I decided to change that, created a branch of the simulator and redefined the collision spaces. At the moment, I have a version that collides, I read out the coordinates, map them into a local reference frame of a respective skin part and emulate a skinEvent / skinContact port (/skinManager/skin_events:o), not the raw /icub/skin/... My thing is still not working with the covers though, as they have no bodies and mass in icubSim.

Anyway, I just discussed with Silvio the state of affairs in the gazebo - it seems that much of this functionality could come quite easily there - adding a contact sensor to individual body parts and reading the coordinates. For now, I just wanted to make you aware that there would be interest to have a skin emulation there and that some of the code I developed for icubSim could be ported to gazebo too.
Please do keep me updated on developments in this direction.

Thanks!
Matej Hoffmann

[traversaro]

Just adding pointers to some materials needed if someone want to tackle this issue:

• a tutorial on how to adding a contact sensor plugin to Gazebo: http://gazebosim.org/wiki/Tutorials/1.9/sensors/contact .

[traversaro]

The SkinSim [1] framework can be of interest for proper skin simulation, even if it simulates skin at physical level, while I guess we are more interested in simulating it an higher level.

[1] : http://spring.mae.cuhk.edu.hk/~cmdl/activity/201406_ICRA%20Soft%20robot%20workshop/SkinSim_uta.pdf

[matejhof]

We should keep an eye on this one: http://gazebosim.org/haptix
Nice hands with tactile and proprioceptive sensors.
So far only proprietary, for the DARPA Haptix challenge, but maybe they will make it available one day.

[traversaro]

The models of the hands are proprietary , but the software is opensource.
In particular the implementation of the tactile sensors is available in the HaptixControlPlugin, available in the handsim package:

• https://bitbucket.org/osrf/handsim/src/b641e22241cc78745b90049df3420a93d5f0aed5/include/handsim/HaptixControlPlugin.hh?at=default
• https://bitbucket.org/osrf/handsim/src/b641e22241cc78745b90049df3420a93d5f0aed5/src/HaptixControlPlugin.cc?at=default

[traversaro]

Just checked, and apparently for the Haptix project they just merged an interesting feature in Gazebo: https://bitbucket.org/osrf/gazebo/pull-request/1413/allow-multiple-contact-sensors-per-link/diff https://bitbucket.org/osrf/gazebo/issue/960 , namely the support for multiple contact sensor for link .

[traversaro]

This would definitely simplify the implementation of the gazebo_yarp_skin plugin a lot, even for simulation at taxel level.

I personally don't have the time at the moment for looking into it. Anyway, if anyone is interested in this feature and willing to work on it I will be happy to share knowledge and know how.
cc @robotology/codyco-developers @MoritzN89 @robertocalandra

[robertocalandra]

We currently have some ongoing effort (together with @MoritzN89) to create a plugin for gazebo/yarp that simulate the skin (see robotology/icub-main#93).
So far the results are positive and we can simulate contacts on the whole upper-body skin (with a simple linear activation function).

I would generally be interested in collaborating about this topic as I think that it is an important feature also within the CoDyCo project. Anyway, we aim to publish a first release of our code for the end of March.

[iron76]

👍 @robertocalandra I think that also @jeljaik could be interested in collaborating but definitively not before the IROS deadline.

[matejhof]

Hi @robertocalandra,
as is probably apparent from this thread, I'm also keen on this enhancement.
Recently, I added the first version of such enhancement to the old icubSim:
http://eris.liralab.it/wiki/Simulator_README#TOUCH_SENSORS:
see Whole-body skin emulation section.
I also provided emulation of the standard skin ports, such that we can use the existing Guis. Illustration videos are here:
https://www.dropbox.com/sh/vw3y6jum6wfbmhk/AAA2PM4YmSAfC_fIRt4cPRMba?dl=0
But the resolution is minimal - so far whole skin parts get activated.

Anyway, we are looking forward to your progress and please let us know if we can help!

[traversaro]

@robertocalandra Really cool. Even if it is in primitive form, the easy way to avoid integration problems in the future is to directly develop your plugin in the gazebo-yarp-plugins repo (perhaps in a new branch). If there are integration problems it is much easier to catch them at early development phases, while fixing them in a finished software can be much more cumbersome.

If you agree, I can add you and @MoritzN89 to this repo (i.e. to get writing access to the repo). When the plugin will be ready, we can merge it in the master branch.

[matejhof]

Following up on the HAPTIX and Silvio's post, someone apparently well informed about the project, Nate Koenig, replied to my post in ROS Robot Skin Special Interest Group (https://groups.google.com/forum/?utm_medium=email&utm_source=footer#!forum/ros-sig-robot-skin) with the following info:
" There are plans to make this available. You can install and run everything right now, with the caveat that you'll have to install from source.

Here are the dependencies:
a. https://bitbucket.org/osrf/gazebo (default branch)
b. https://bitbucket.org/osrf/handsim (default branch, plus the open pull requests)
c. https://bitbucket.org/osrf/haptix-comm (default branch)
d. https://bitbucket.org/ignitionrobotics/ign-transport (default branch)

Tutorials:
http://gazebosim.org/haptix/tutorials
Debians should be generated mid to late next week.
Enjoy, and feedback is most welcome."

[robertocalandra]

Hi all, a brief follow up.
The student project is now terminated and at http://www.ausy.tu-darmstadt.de/Research/SkinGazeboYarpPlugin you can find the code developed and the final report.
We currently do not plan further work in this direction (mostly due to lack of manpower) but I believe that in the long term it would be a very important development for the iCub community to have a fully functional and realistic skin simulation.
Cheers

[lornat75]

Nice work! Do you think you can ask the student the extra bit of work required to upload it to github/robotology?

[robertocalandra]

I will ask, but since the project is over it is up to them :)

edit: I asked them.

[matejhof]

In the meantime, I made some progress in skin emulation in the old iCub Simulator, best illustrated by this video: link
(Employing doubleTouch code by @alecive)
Note that this is not more than emulation - collisions are physically (in ODE physics) detected with the skin covers as as whole (e.g. 1 for whole forearm). Then using the coordinates of the collision and doing the coordinate transformations to link FoR, they can then be compared with taxel positions in the real robot. Because this is subject to different inaccuracies, we did a rough "tesselation" only, giving a resolution of roughly min. 2 triangles activated.
See the Simulator readme for instructions how to use it.

The "emulation infrastructure" that opens the skin and skin_event ports can be partly reused if the taxels were implemented physically like in the Darmstadt solution.
Btw., the method of @MoritzN89 / @robertocalandra , was performing fast enough? Are there any visualizations available?

[randaz81]

Uhm, this issues seems a bit starving... 🍝
What is the current status of this issue?
Having the skin emulation on gazebo would be awesome!

[robertocalandra]

Hi @randaz81,

Sorry, but on my side things are completely blocked: I currently do not have any time to push this topic any further.
The code is available online (http://www.ausy.tu-darmstadt.de/Research/SkinGazeboYarpPlugin) so if you find any volunteer he/she might continue from there.

Best

[matejhof]

With the help of a student, the skin emulation on the old icubSim is completed (part of icub-main already, see video) for the hands and forearms with a roughly 2-triangle resolution.

It would be nice to have this in gazebo too - either with "physical" sensors or emulated from the collision coordinates. Maybe we can sit down on this once at the IFI day or so with @traversaro and @randaz81?

[emilio-cartoni]

Dear all,
our lab in Rome would be very much interested in having the skin functionality in Gazebo for our GOAL-Robots project.
Are there any updates or plans to implement the skin in the near future?

[traversaro]

Hi @emilio-cartoni ,
currently there is no one at IIT actively working on this feature.
If you are interested in working on this, we would be happy to give you indications about possible way on how to implement this feature.

[emilio-cartoni]

@robertocalandra
Dear Roberto, we would like to continue your work so that we can add the hand skin as well.
I see that in the code downloaded from http://www.ausy.tu-darmstadt.de/Research/SkinGazeboYarpPlugin there is no licence file.
How would you like your work to be acknowledged?
If you upload it to GitHub we could officially fork it from there; then, when it's ready for the gazebo-yarp-plugins repository I'll ask IIT for a pull request.

[robertocalandra]

Uploaded here: https://github.com/robertocalandra/icub-gazebo-skin
I think that a normal citation to the github repo is sufficient, alternatively you could cite the final report of the project linked in the repository.

Anyway, best of luck with the project :)

[emilio-cartoni]

Thanks Roberto! 👍

[matejhof]

Hello @robertocalandra ,
we're also trying along these lines with @martinj8649 (even though now primarily for Nao in Gazebo), but are experiencing quite a slowdown when many sensors are used (e.g., 10 and more). Do you recall if your students could run reasonably fast on a normal laptop or what was it like? Is there a video of the system in operation?

[robertocalandra]

I vaguely remember that it was slower then real-time, but it was long time ago so I don't remember any other detail.

[tfietzek]

Hello,
what is the current state of the development of a skin emulation? Is there a working plugin for the full right arm? This would be very interesting for us in Chemnitz.

[traversaro]

Hi @torstenfol ! As far as I know there is no plugin to simulate the iCub skin out of the box in Gazebo.

[alecive]

man, this was a really old issue!!

[DanielePucci]

CC @martinj8649 @matejhof

[traversaro]

man, this was a really old issue!!

Recently during a meeting a lab member that will not be mentioned commented an issue in this repo I was explaining by simply saying: "Interesting. When you opened that issue, I was still in high school." :D

[matejhof]

Haha, we're not that old! ;)
@torstenfol I'm curious, can you share some info about what you're doing and why?
In Prague, we have the "iCub skin" on a Nao robot (https://youtu.be/3Z28oXHRaXo) and we developed a simulation of this one in Gazebo (using the ContactSensor plugin) - will try to make the code accessible and show a video. Would you be interested in that?

[tfietzek]

Thanks for the information.
@matejhof We want to use the skin in student projects and the main objective is a research project with the goal to implement the rubber hand illusion on the iCub. Since this would be easier to implement in the Gazebo than in the iCub-Simulator, it is necessary to have the skin available there.
Therefore, I would be very interested in your code, since then I could try to adapt it to the iCub.

[matejhof]

Interesting! Which group are you part of at TU Chemnitz @torstenfol?
Here's two illustrations how it works on our Nao in Gazebo:

• low-res with skin contact events: https://youtu.be/7f9Mfl7sO18

• high-res - iCub-like, from the same skin triangles: https://youtu.be/XvfdKautB-Q

@maxymczech will make our code available here: https://github.com/maxymczech/nao-gazebo-skin as soon as he gets around it.

[xEnVrE]

cc @2103simon

[tfietzek]

I am working in the group of prof. Hamker, who holds the professorship for artificial intelligence. The focus of the research group are neuro-computational models, which could help to understand the human brain.

Thank you @matejhof @maxymczech for the illustrations and the code!

[traversaro]

Related PR (courtesy of @prajval10): openai/gym#1299 .

[traversaro]

Related PR in ignition-gazebo: gazebosim/gz-sim#229 . cc @xEnVrE

[matejhof]

thanks, @traversaro for keeping an eye on all this. Very useful!

[matejhof]

@maxymczech still did not get around making our iCub skin on Nao in Gazebo public. Sorry!
How is the progress in Chemnitz, @torstenfol ?

[tfietzek]

@matejhof No problem! Since I had a lot other stuff to do in the last weeks, I did not had the time to work a lot on the Plugin. I only take also a lokk at @xEnVrE 's implementation of the skin plugin for the fingertips.

[diegoferigo]

Related PR in ignition-gazebo: gazebosim/gz-sim#229 . cc @xEnVrE

Maybe related to this PR, there's a GSoC project that was accepted this year to add nice contact features to Ignition Gazebo.

[maxymczech]

@matejhof @torsten93
Hello, guys. I apologize for the delay, I was really devastated after the exams this year, finally started pulling myself together. I have published my NAO skin emulation code as a stand-alone ROS package: https://github.com/maxymczech/nao-gazebo-skin

There are basic instructions in readme. I have tested this on clean ROS Melodic installation, seems to work and generate contact data properly. If you have trouble installing / using it, feel free to ask.

[matejhof]

Hi there,
in the meantime, we have published our work on the Nao in Gazebo with skin - video, arxiv, but we continue to develop it.
We would be interested in iCub as the next target. @torsten93, please keep us posted!

One more general question, what is the plan regarding iCub in (good old) Gazebo vs. Ignition Gazebo? From what I know, Ignition Gazebo may be the future, but currently couple of years behind Gazebo as we know it.... @traversaro?

[maxymczech]

@matejhof They have a neat feature comparison table for Ignition Gazebo vs Gazebo 11: https://ignitionrobotics.org/docs/citadel/comparison

[traversaro]

One more general question, what is the plan regarding iCub in (good old) Gazebo vs. Ignition Gazebo? From what I know, Ignition Gazebo may be the future, but currently couple of years behind Gazebo as we know it.... @traversaro?

A bit of discussion is in #395 . At the moment the work on this is not scheduled, but the hope is to get someone to work on this sooner or later. Without the YARP interfaces that are available on the real robot, we are already using iCub + Ignition Gazebo for RL research, see https://github.com/robotology/gym-ignition and https://github.com/robotology/gym-ignition-models .

[matejhof]

Hi guys,
this is where we currently are thanks to @maxymczech . It is in Gazebo 11. https://youtu.be/xccVWibz_KQ
The repo is here: https://gitlab.fel.cvut.cz/body-schema/code-icub-gazebo-skin
Please let is know any plans if this could be incroporated into the iCub gazebo master or something....

[pattacini]

Nice progress @matejhof @maxymczech 👍🏻
I think the first step would be to fork this repo and port your changes into the fork in order to get prepared for a PR.
A description of the architecture you implemented posted in the body of the PR could be helpful as well.

cc @traversaro

[maxymczech]

@pattacini Hello. I have finished working on my thesis and will start porting the changes. I have a few quick questions before I start:

• Apart from gazebo plugin, changes will also have to be made to the iCub URDF models. Can you please tell me what is the correct way to do that?
• In case I have further small questions, what is the best way to contact you? Is it here in github issues, or maybe you have discord/slack group?

Thank you very much.

[pattacini]

Hi @maxymczech

Apart from gazebo plugin, changes will also have to be made to the iCub URDF models. Can you please tell me what is the correct way to do that?

I think that the best option would be to wait for @traversaro and coordinate this activity with him as he's the maintainer of this repo as well as one of the main contributors of https://github.com/robotology/icub-models.

In particular, as written in the documentation, icub-models hosts models that are in turn generated from https://github.com/robotology/icub-models-generator.

Which model to upgrade or judging whether we should create a brand new one for the skin is right a matter to be discussed with @traversaro.

In case I have further small questions, what is the best way to contact you? Is it here in github issues, or maybe you have discord/slack group?

We don't have discord/slack groups (maybe in the future we should evaluate the opportunity to launch a discord public server for online interaction), so GitHub issues are the way to go.