Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Tests in Webots for NUbots #58

Open
ysims opened this issue May 26, 2021 · 0 comments
Open

Tests in Webots for NUbots #58

ysims opened this issue May 26, 2021 · 0 comments
Labels
enhancement New feature or request

Comments

@ysims
Copy link
Member

ysims commented May 26, 2021

Isolated tests allow us to observe the performance of a single system without the influence of any other systems.
Many systems in the NUbots main codebase have suffered from a lack of testing due to systems they depend on not performing well. This prevents us from being able to develop these areas.
Now that we have support for Webots as a result of the RoboCup2021 Humanoid League turning to simulation, we can create some environments for observing the performance of, developing and testing these individual systems.
Note that it's still important to test in conjunction with the other real systems as well, since our data is not going to be perfect like the webots ground truth and we need to account for that.

  • Odometry test: This involves sending raw sensor data from webots to NUbots, which NUbots filters. This filtered data is sent back to Webots, which compares it against its own ground truth.
    Odometry Test
  • Vision test: The Visual Mesh is easy enough to test since images can be fed to it and the performance determined all within its own code, completely separate from the NUbots code. This focuses more on testing the post processing algorithms. The idea is that we use ground truth filtered sensor data from webots in the vision system. That is, webots sends NUbots ground truth filtered sensor data. NUbots uses this in the vision system, where the Visual Mesh runs and then the post processing is performed. The post processing will output vision objects, such as the position of balls, goals, etc. This will be sent to webots, which compared it with its own ground truth about where the objects are.
    Vision Test
  • Localisation test: This involves testing where the robot thinks it is on the field vs where it actually is. Vision data and odometry data is provided by Webots. NUbots uses this to run localisation, and sends the result to Webots. Webots picks it up and compares it against the true position of the robot.
    Localisation Test
  • Motion test: While we currently use open loop systems for motions, we would like to use close loop systems that use sensor data. This test involves getting Webots to send ground truth filtered sensor data, which is used by any motion systems, and then can be viewed on Webots.
    Motion Test
  • Behaviour Test: Like the motion test, there is no real comparison for this test and is more something that needs to be observed. Webots will need to send NUbots everything it needs. This includes vision objects, sensors and localisation ground truth data. NUbots will determine the behaviour, which Webots will be told about. Webots will perform whatever motions are necessary through teleporting.
    Behaviour Test
@ysims ysims added the enhancement New feature or request label May 26, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

1 participant