Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Reimplement NMS using morphological operations #1192

Merged
merged 1 commit into from
Jan 2, 2024

Conversation

borongyuan
Copy link
Contributor

This borrows from the PyTorch implementation in SuperGlue, but uses this corrected version. This implementation is relatively simple, and since OpenCV has various optimizations for morphological operations, it will theoretically have better performance. But I haven't done any performance comparison testing yet. Since this is performed on heatmap, the NMS in SuperPoint Torch can be changed to PyTorch implementation. This way we can remove this part from util2d and introduce ANMS later.

@matlabbe matlabbe merged commit 096592e into introlab:master Jan 2, 2024
6 checks passed
@matlabbe
Copy link
Member

matlabbe commented Jan 2, 2024

The NMS version for SuperPoint Torch you are referring is this one?

std::vector<cv::KeyPoint> keypoints_no_nms;
for (int i = 0; i < kpts_cpu.size(0); i++) {
if(mask.empty() || mask.at<unsigned char>(kpts_cpu[i][0].item<int>(), kpts_cpu[i][1].item<int>()) != 0)
{
float response = prob_cpu[kpts_cpu[i][0]][kpts_cpu[i][1]].item<float>();
keypoints_no_nms.push_back(cv::KeyPoint(kpts_cpu[i][1].item<float>(), kpts_cpu[i][0].item<float>(), 8, -1, response));
}
}
detected_ = true;
if (nms_ && !keypoints_no_nms.empty()) {
int border = 0;
int dist_thresh = minDistance_;
int height = img.rows;
int width = img.cols;
std::vector<cv::KeyPoint> keypoints;
cv::Mat descEmpty;
util2d::NMS(keypoints_no_nms, descEmpty, keypoints, descEmpty, border, dist_thresh, width, height);
if(keypoints.size()>1)
{
return keypoints;
}
return std::vector<cv::KeyPoint>();
}

I am not using that much that code right now, but if you use it and know how to update it, you can make another PR.

@borongyuan borongyuan deleted the oak-d-pro-w-dev branch January 3, 2024 05:16
@borongyuan
Copy link
Contributor Author

I am not using that much that code right now, but if you use it and know how to update it, you can make another PR.

I don't use this part very much either. But I will test the new environment after upgrading my Jetson system to JetPack 6 in the near future and update this part.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants