Skip to content

Latest commit

 

History

History
182 lines (157 loc) · 8.42 KB

README.md

File metadata and controls

182 lines (157 loc) · 8.42 KB

Reval

Reval is an open-source framework to evaluate the performance of Robotics platforms. Currently it supports Husky platform, Turtblebot3. The useres can evalute the performance of a mission for a given gazebo envirnoment (or on their own gazebo envirnment) for different configurations in an automated fashion and log the results. In addition, Reval supports the following metrics to evaluate the quality of a mission:

Evaluation metrics

Metrics Description
DWA F # of failed produced path by DWA planner
DWA NP # of re-planning by DWA planner
DWA IT # of DWA invalid trajectory
RR # of rotate recovery excuted
RCU # of ClearCostMaps recovery executed for unstuck robot
RCL # of ClearCostMaps layer recovery executed
IRC # of invalid rotation cmd
ERG # of error rotating on the goal
ED Euclidean distance between actual goal location and the location the robot reached.
RNS robustness in narrow spaces, ; where Ns is the total narrow spaces in the gazebo environment, and passed_Ns is the narrow spaces that the robot successfully crossed.
DT total distance traveled during a mission
BP battery percentage. For more details: Gazebo-ROS battery plugin
MT time taken to complete a mission
MS mission success. Example: if the robot successfully reached point A to B

Reval supports both the Husky simulator and Hysky physical robot. The instructions provided below are for Husky simulator. To run Reval on the physical Husky, first setup your husky using Husky UGV Tutorial then follow the below instructions.

         Probe      Probe
         +            +
         |            |
+--------|------------|-------+     +-----------------------------+
|        |            |       |     |                             |
|     +--|------------|-+     |     |                             |
|     |  v            v |     |     |          - TD      <--------------+ Probe
|     |                 |     |     |          - DWA F   <--------------+ Probe
|     |        MS       |     |     |          - MT      <--------------+ Probe
|     |                 |     |     |          - RNS     <--------------+ Probe
|     +-----------------+     |     |                             |
|      System under test      |     |       System under test     |
+-----------------------------+     +-----------------------------+
          Functional                        Non-functional

Build status

Build Type Status
ROS melodic ROS melodic
ROS noetic ROS noetic
Platform Status
Husky UGV Husky UGV
TurtleBot3 TurtleBot3
OceanWATERS OW

Requirements

  • Ubuntu 18 or Ubuntu 20
  • ROS Melodic or ROS Noetic
  • Python 2.7 (for ROS Melodic), Python 3.6+ (for ROS Noetic)

Installations

Installing the husky simulator

For ROS Melodic

sudo apt-get install ros-melodic-husky-simulator
sudo apt-get install ros-melodic-husky-navigation
sudo apt-get install ros-melodic-husky-desktop

For ROS Noetic

sudo apt-get install ros-noetic-husky-simulator
sudo apt-get install ros-noetic-husky-navigation
sudo apt-get install ros-noetic-husky-desktop

Installing rosbag for Python

Download and install ros_readbag.py using these commands:

Download the file

wget https://raw.githubusercontent.com/ElectricRCAircraftGuy/eRCaGuy_dotfiles/master/useful_scripts/ros_readbagfile.py

Make it executable

chmod +x ros_readbagfile.py

Ensure you have the ~/bin directory for personal binaries

mkdir -p ~/bin

Move this executable script into that directory as ros_readbagfile, so that it will be available as that command

mv ros_readbagfile.py ~/bin/ros_readbagfile

Create a symlink in ~/bin to this script so you can run it from anywhere:

ln -si "${PWD}/ros_readbagfile.py" ~/bin/ros_readbagfile

If this is the first time ever creating the "~/bin" dir, then log out and log back in to your Ubuntu user account to cause Ubuntu to automatically add your ~/bin dir to your executable PATH.

Re-source your ~/.bahsrc file

source ~/.bashrc

Install dependencies

Clone the repo

git clone https://github.com/softsys4ai/Reval.git

Installing the dependencies

sudo apt install ripgrep
pip install pandas
pip install tqdm
pip install tabulate

Or cd Reval/ run ./requirements.sh. If you face permission denied, first run chmod +x requirements.sh

Running Reval

cd cure/

source your new setup.sh file. You need source this setup.sh file everytime you open a new Terminal

source devel/setup.bash

To evaluate the mission run

cd cure/src/Reval
python reval_husky_sim.py
optional arguments:
  -h, --help    show this help message and exit
  -v , -viz     turn on/off visualization of gazebo and rviz (default: On)
  -e , -epoch   number of data-points to be recorded (default: 1)

examaple: python reval_husky_sim.py -v off -e 10

Demo

Visualization off:

Visualization on:

Turtlebot 3 physical platform

  1. Run roscore on the Remote PC
  2. Launch Bringup on the Turtlebot 3 SBC
ssh pi@{IP_ADDRESS_OF_RASPBERRY_PI}
roslaunch turtlebot3_bringup turtlebot3_robot.launch
  1. Launch the Navigation on the Remote PC
export TURTLEBOT3_MODEL=burger
roslaunch turtlebot3_navigation turtlebot3_navigation.launch map_file:=$HOME/map.yaml
  1. Finally, run Reval python reval_turtlebot_phy.py

Demo

reval_turtlebot3_demo.mp4
Reval_Turetlebot3_demo.mov

Contacts

Please feel free to contact via email if you have any feedbacks. Thank you for using Reval!

Name Email
Md Abir Hossen [email protected]
Pooyan Jamshidi [email protected]