Autonomous car simulator

This project is a big one. I have already adapted the sensor fusion and localization modules into the project and I plan on adding path planning, controls and behavior in.

I am using the Carla simulator bult on Unreal Engine to run simulations, and using ROS to send data across modules. The aim to make the project portible and to have each module abstracted such that I can run each module independent of the others and independent of Carla. My plans for the future are two fold. First, I want to run a Carla simulation on computer A, and send ros topics over a local network to computer B which then runs the modules. With this framework in place, it will be easy to impliment in a real controlled environment. For instance, raspberry pi A collects camera and lidar data from real sensors attached to a vehcile then sends ros topics to computer B also in the vehicle running the modules.

Requirements

OS: Ubuntu 18.04
Unreal engine: 4.26
carla-simulator/bionic (apt installation): 0.9.13
Ros (apt installation): Melodic
carla-ros-bridge/bionic (apt installation): 0.9.10-1

Follow this guide to enable python3 in ros melodic

You also will need simple-pid for manual control.

pip install --user simple-pid

Initially, the installed directory is not in the path, so I needed to add it to the path.

export PATH=$PATH:/home/mike/.local/lib/python3.7/site-packages

I have an anaconda enviornment called waymo with all of the libraries needed for the sensor fusion part of the udacity course.

You will need to compile tf2 for python3, so follow these steps

Installation

Source the carla ros bridge enviornment

source /opt/carla-ros-bridge/melodic/setup.bash

Then make the catkin workspace

catkin_make

How to Run

Source the current workspace:

cd .../catkin_ws/
source devel/setup.bash

Then the current workspace will be a child of the parent carla_ros_bridge catkin workspace.

Start carla on another terminal

cd /opt/carla-simulator
./CarlaUE4.sh

By default, running the project mike_av_stack

roslaunch mike_av_stack mike_av_stack.launch

will do the following:

• Starts basic carla_ros_bridge with an ego vehicle. You can find more info about this in the docs.
• Opens rviz
• Runs helpful tools
  • point cloud stacker
• Runs the sensor_fusion node
• Runs the localization node

Docs

Sensor Fusion

Localization

Todo

Next steps, I want to:

• Clean up config files

• Move the rviz config file locally and clean it up a bit

• Sensor Fusion

  1. Seperate topic and topic_combined into configs and put that back in
  2. Fix tracking
  3. Train a cnn on my parameters
  4. Fix intensity scaling
  5. Add image detection
  6. Make filter generic and not dependant on trackmanagement. This way I can use for Localization

• Localization

  1. Move subscriber initialization into the scan matching constructor
  2. Implement configs
  3. Finish scan matching, and add thread safe transformation matrix
  4. Add gnss
  5. Add covariances
  6. Maybe use EKF to compare gnss and scan matching?
  7. Far off: Train a CNN to detect features like trees, buildings signs and such and create a feature map from that

• Tools

  1. Fix traffic gen

Problems to fix

For localization, seperating the point clouds by each sensor makes no sense. So I would like to combine all sensor's point clouds to produce one global pointcloud. To do this, I would need to transform each sensor's point cloud by its extrinsic transformation matrix.

For sensor fusion, each measurement from the detection algorithm/model is in sensor space and needs to be transformed to vehicle space.

Installation Issues

carla-ros-bridge colcon build failed. Turns out it would somehow find my other ros2 humble installation even though there was no reference to it in PATH or PYTHONPATH. Deleting it worked

The carla-simulator installed in apt only had a python3.7.egg and building carla-ros-bridge required python3.6. So I started from scratch, apt remove carla-sim, deleted ~/carla Re install following very closely https://carla.readthedocs.io/en/latest/build_linux/ On make PythonAPI i get this gem: clang 8 is required, but it's not installed.

Retrying with clang set to 10 but that didn't work.

Turns out I need to edit a few files such as Setup.sh and manually set clang to 10. That works, back to carla-ros-bridge

Okay I just restarted from scratch and got another error while building carla from source. I can either use master branch and change the Setup.sh script and others from clang8 to clang10 or use dev env Got this error and fixed it this way

sudo ln -s /usr/lib/x86_64-linux-gnu/libstdc++.so.6 /usr/lib/libstdc++.so

sudo ln -s /usr/lib/x86_64-linux-gnu/libstdc++.so.6 /usr/lib/libstdc++.so.6

On master new error: CMake Error at cmake/modules/CheckCompilerVersion.cmake:72 (message): libstdc++ version must be at least 4.8.

Trying dev

Build: Carla -> CMake Ros2 -> through apt carla-ros-bridge -> colcon

Ok this is super confusing but here goes

• colcon
  • catkin
  • ament_package
    • cmake
    • setuptools (python)