Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add eigen utils to convert mesh 3d vertices to oriented box #224

Merged
merged 17 commits into from
Sep 7, 2021

Conversation

AmrElsersy
Copy link
Contributor

Signed-off-by: AmrElsersy [email protected]

🎉 New feature

Summary

Adding helper functions to convert 3d vertices of a mesh to an oriented bounding box to be used in the bounding box camera
rendering#334

Checklist

  • Signed all commits for DCO
  • Added tests
  • Added example and/or tutorial
  • Updated documentation (as needed)
  • Updated migration guide (as needed)
  • codecheck passed (See contributing)
  • All tests passed (See test coverage)
  • While waiting for a review on your PR, please help review another open pull request to support the maintainers

@github-actions github-actions bot added 🏢 edifice Ignition Edifice 🏯 fortress Ignition Fortress 🏰 citadel Ignition Citadel 🔮 dome Ignition Dome labels Aug 18, 2021
@adlarkin adlarkin self-requested a review August 18, 2021 21:04
eigen3/src/Util_TEST.cc Outdated Show resolved Hide resolved
eigen3/src/Util_TEST.cc Outdated Show resolved Hide resolved
eigen3/src/Util_TEST.cc Outdated Show resolved Hide resolved
eigen3/include/ignition/math/eigen3/Util.hh Outdated Show resolved Hide resolved
eigen3/include/ignition/math/eigen3/Util.hh Outdated Show resolved Hide resolved
eigen3/include/ignition/math/eigen3/Util.hh Outdated Show resolved Hide resolved
eigen3/include/ignition/math/eigen3/Util.hh Outdated Show resolved Hide resolved
eigen3/include/ignition/math/eigen3/Util.hh Outdated Show resolved Hide resolved
eigen3/include/ignition/math/eigen3/Util.hh Outdated Show resolved Hide resolved
eigen3/include/ignition/math/eigen3/Util.hh Outdated Show resolved Hide resolved
Copy link
Member

@scpeters scpeters left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

you've resolved a lot of comments but I don't see the corresponding changes in the code

eigen3/include/ignition/math/eigen3/Util.hh Outdated Show resolved Hide resolved
eigen3/include/ignition/math/eigen3/Util.hh Outdated Show resolved Hide resolved
@AmrElsersy
Copy link
Contributor Author

you've resolved a lot of comments but I don't see the corresponding changes in the code

forgot to push :D

Signed-off-by: AmrElsersy <[email protected]>
@codecov
Copy link

codecov bot commented Aug 19, 2021

Codecov Report

Merging #224 (ee7f156) into ign-math6 (b4b6d23) will increase coverage by 0.00%.
The diff coverage is 100.00%.

Impacted file tree graph

@@            Coverage Diff             @@
##           ign-math6     #224   +/-   ##
==========================================
  Coverage      99.21%   99.22%           
==========================================
  Files             65       66    +1     
  Lines           6089     6162   +73     
==========================================
+ Hits            6041     6114   +73     
  Misses            48       48           
Impacted Files Coverage Δ
eigen3/include/ignition/math/eigen3/Util.hh 100.00% <100.00%> (ø)
include/ignition/math/OrientedBox.hh 100.00% <0.00%> (ø)

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update b4b6d23...ee7f156. Read the comment docs.

Copy link
Contributor

@adlarkin adlarkin left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Overall, this looks good. I'm trying to do a few verifications for testing, but I think that this can be merged as long as I don't find anything strange in the tests I run.

eigen3/include/ignition/math/eigen3/Util.hh Outdated Show resolved Hide resolved
eigen3/include/ignition/math/eigen3/Util.hh Outdated Show resolved Hide resolved
eigen3/include/ignition/math/eigen3/Util.hh Outdated Show resolved Hide resolved
eigen3/include/ignition/math/eigen3/Util.hh Outdated Show resolved Hide resolved
eigen3/include/ignition/math/eigen3/Util.hh Outdated Show resolved Hide resolved
eigen3/include/ignition/math/eigen3/Util.hh Show resolved Hide resolved
eigen3/include/ignition/math/eigen3/Util.hh Outdated Show resolved Hide resolved
eigen3/include/ignition/math/eigen3/Util.hh Outdated Show resolved Hide resolved
Signed-off-by: AmrElsersy <[email protected]>
Copy link
Member

@scpeters scpeters left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I made a few suggestions about documentation and naming, suggesting vertices instead of mesh to avoid confusion with ignition::common::Mesh

eigen3/include/ignition/math/eigen3/Util.hh Outdated Show resolved Hide resolved
eigen3/include/ignition/math/eigen3/Util.hh Outdated Show resolved Hide resolved
eigen3/include/ignition/math/eigen3/Util.hh Show resolved Hide resolved
eigen3/include/ignition/math/eigen3/Util.hh Outdated Show resolved Hide resolved
eigen3/include/ignition/math/eigen3/Util.hh Outdated Show resolved Hide resolved
Signed-off-by: AmrElsersy <[email protected]>
Signed-off-by: AmrElsersy <[email protected]>
@osrf-triage
Copy link

This pull request has been mentioned on Gazebo Community. There might be relevant details there:

https://community.gazebosim.org/t/gsoc-2021-machine-learning-extension-to-ignition-gazebo/1070/1

@adlarkin
Copy link
Contributor

I'm trying to do a few verifications for testing

I went ahead and ran the example set of points in the unit test through Open3D's CreateFromPoints method for comparison. Here are the results I got (link to the code can be found here):

Box center is: 
0.423213
0.507995
 1.83459

Box extent (size) is: 
6.53071
4.12782
3.25527

roll: -2.01634
pitch: 0.0058244
yaw: 1.1554

It looks like the box's center and size are similar enough between Open3D and this PR, but the rotation seems like it could be an issue. Of course, this is assuming that Open3D should be considered a "source of truth", but I just wanted to mention it here in case this deserves further investigation. The code I linked shows how I compute roll, pitch and yaw from the rotation matrix that is provided by Open3D (hopefully my calculations are correct). cc @scpeters

@AmrElsersy
Copy link
Contributor Author

@adlarkin
I thinks that is because the part of calculating the orientation in the 2 algorithms is different

their code

    Eigen::Matrix3d R = es.eigenvectors();
    R.col(0) /= R.col(0).norm();
    R.col(1) /= R.col(1).norm();
    R.col(2) /= R.col(2).norm();

    if (evals(1) > evals(0)) {
        std::swap(evals(1), evals(0));
        Eigen::Vector3d tmp = R.col(1);
        R.col(1) = R.col(0);
        R.col(0) = tmp;
    }
    if (evals(2) > evals(0)) {
        std::swap(evals(2), evals(0));
        Eigen::Vector3d tmp = R.col(2);
        R.col(2) = R.col(0);
        R.col(0) = tmp;
    }
    if (evals(2) > evals(1)) {
        std::swap(evals(2), evals(1));
        Eigen::Vector3d tmp = R.col(2);
        R.col(2) = R.col(1);
        R.col(1) = tmp;
    }

But anyway I see them are close (not too much close) but they have rotation in the same directions at least .. and also the rotation is acceptable visually, so we don't need to compare it with Open3D

eigen3/include/ignition/math/eigen3/Util.hh Outdated Show resolved Hide resolved
eigen3/include/ignition/math/eigen3/Util.hh Outdated Show resolved Hide resolved
eigen3/include/ignition/math/eigen3/Util.hh Outdated Show resolved Hide resolved
eigen3/include/ignition/math/eigen3/Util.hh Show resolved Hide resolved
eigen3/include/ignition/math/eigen3/Util.hh Outdated Show resolved Hide resolved
eigen3/src/Util_TEST.cc Show resolved Hide resolved
eigen3/include/ignition/math/eigen3/Util.hh Show resolved Hide resolved
Signed-off-by: AmrElsersy <[email protected]>
@AmrElsersy
Copy link
Contributor Author

AmrElsersy commented Aug 28, 2021

That is our output for multi-links models (double pendolum and cars)

ezgif com-gif-maker (1)

@AmrElsersy
Copy link
Contributor Author

@adlarkin
I thinks that is because the part of calculating the orientation in the 2 algorithms is different

their code

    Eigen::Matrix3d R = es.eigenvectors();
    R.col(0) /= R.col(0).norm();
    R.col(1) /= R.col(1).norm();
    R.col(2) /= R.col(2).norm();

    if (evals(1) > evals(0)) {
        std::swap(evals(1), evals(0));
        Eigen::Vector3d tmp = R.col(1);
        R.col(1) = R.col(0);
        R.col(0) = tmp;
    }
    if (evals(2) > evals(0)) {
        std::swap(evals(2), evals(0));
        Eigen::Vector3d tmp = R.col(2);
        R.col(2) = R.col(0);
        R.col(0) = tmp;
    }
    if (evals(2) > evals(1)) {
        std::swap(evals(2), evals(1));
        Eigen::Vector3d tmp = R.col(2);
        R.col(2) = R.col(1);
        R.col(1) = tmp;
    }

That was the output when I tried that code of Open3D for calculating Rotation instead of the current algorithm
It was bad

ezgif com-gif-maker (2)

Copy link
Contributor

@adlarkin adlarkin left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just a few small comments, but I think this is almost ready to merge!

eigen3/include/ignition/math/eigen3/Util.hh Outdated Show resolved Hide resolved
eigen3/src/Util_TEST.cc Outdated Show resolved Hide resolved
eigen3/src/Util_TEST.cc Show resolved Hide resolved
@scpeters
Copy link
Member

have we considered specifying the orientation of the bounding box and then simply expanding its dimensions to include all the passed vertices? this would avoid the weird box rotations at the cost of generating non-minimal boxes

@adlarkin
Copy link
Contributor

have we considered specifying the orientation of the bounding box and then simply expanding its dimensions to include all the passed vertices? this would avoid the weird box rotations at the cost of generating non-minimal boxes

I think the challenge is that in some scenarios, we have the vertices of an object, but not the orientation of the object. For example, in gazebosim/gz-rendering#334, I believe that we have all of the vertices from the bounding boxes provided by Ogre, but I don't believe that we have a way of getting the "combined orientation" from all of the rendering visuals (I could be wrong about this).

However, that being said, I think that it would be good to provide users with the option to pass in an orientation to verticesToOrientedBox. Perhaps we can add a input parameter that's a pointer to an orientation: if the pointer is null, we ignore it and run the algorithm as-is, but if it isn't null, then we take the orientation into account. Does that sound reasonable?

Signed-off-by: AmrElsersy <[email protected]>
Copy link
Contributor

@adlarkin adlarkin left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code looks good! I'm going to wait to see what @scpeters thinks about #224 (comment) before merging this.

@scpeters
Copy link
Member

scpeters commented Sep 4, 2021

However, that being said, I think that it would be good to provide users with the option to pass in an orientation to verticesToOrientedBox. Perhaps we can add a input parameter that's a pointer to an orientation: if the pointer is null, we ignore it and run the algorithm as-is, but if it isn't null, then we take the orientation into account. Does that sound reasonable?

I would just add a separate API that accepts a vector of vertices as well as a Quaternion, but this PR is fine without it. We could add it later if desired

@adlarkin adlarkin merged commit 4ac81c0 into gazebosim:ign-math6 Sep 7, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
🏰 citadel Ignition Citadel 🔮 dome Ignition Dome 🏢 edifice Ignition Edifice 🏯 fortress Ignition Fortress
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants