Implementation of Quasi Dense Stereo algorithm. #1941

dimitrisPs · 2018-12-10T19:48:39Z

I have implemented a class that establishes dense pairwise correspondences from a stereo image pair, and I wish to contribute it to the official repository. The class extracts a sparse set of correspondences from a stereo image pair using pyramidal Lucas-Kanade Flow algorithm based on Shi and Tomasi features, this way the class can process non-rectified images. To interpolate a dense set of correspondences the class uses the Quasi Dense Stereo algorithm proposed by M. Lhuillier and Long Quan et. al. [https://ieeexplore.ieee.org/document/905620].

Please let me know if you have any further questions or comments.

Thank You

alalek

I believe this algorithm should be added into stereo module instead of creation of new one.

alalek · 2018-12-11T17:52:05Z

modules/qds/include/opencv2/qds/quasiDenseStereo.hpp

+    CV_PROP_RW cv::Point2i p1;
+    CV_PROP_RW float	corr;
+
+    CV_WRAP bool operator < (const Match & rhs) const


C++ operators can't be wrapped directly.
Use CV_WRAP_AS(less) to wrap under different name.

modules/qds/include/opencv2/qds/quasiDenseStereo.hpp

modules/qds/samples/export_param_file.cpp

…e compilation issues.

dimitrisPs · 2018-12-12T16:27:36Z

Yes, I can add it there, just tell me how do you want me to integrate it. For now, will try and remove the warnings that your build-bot exports.

alalek

Migration to "stereo" module:

Extract public API part (which is going to be used by samples or high-level tests) from all headers and put it all into (opencv2/stereo/quasiDenseStereo.hpp). Merge documentation (documentation is generated from "public" headers (include/) only).
BTW, It is better to avoid case in filenames - quasi_dense_stereo.hpp

Include "opencv2/stereo/quasiDenseStereo.hpp" into "opencv2/stereo.hpp". Like this (for ximgproc module)

Merge .bib files and move tutorials

Update dependencies in stereo/CMakeLists.txt
opencv_imgcodecs - should not be used. Algorithms should accept cv::Mat, instead of file names. Most part of image data comes from cameras.
Avoid entries duplication in OPTIONAL section.

Bindings integration can be done later (current state probably doesn't work well and there is Python sample). Just don't use WRAP or _W variants of macros. CV_EXPORTS is enough.
It would be nice to add some minimal test for primary API.

modules/qds/include/opencv2/qds/defaults.hpp

alalek · 2018-12-13T01:47:23Z

modules/qds/include/opencv2/qds/quasiDenseStereo.hpp

+
+    PropagationParameters	Param;
+
+protected:


Split interface (include/) and implementation (src/).

I am not sure I understand how you want me to change the structure. I think that this version of the class is already split. Could you please provide me with more information on how you want me to split the class?.

Most of algorithms use following scheme:

// hpp class Algo { public: virtual void doSomething() = 0; virtual void setSomeProperty() = 0; virtual float getSomeProperty() = 0; ... // no protected or private members // no class fields }; // can be defined inside the class - "static Ptr<Algo> Algo::create(...)" static Ptr<Algo> createAlgo(...);

// cpp class AlgoImpl : public Algo { // implement all methods }; Ptr<Algo> createAlgo(...) { return makePtr<AlgoImpl>(...); }

modules/qds/include/opencv2/qds/quasiDenseStereo.hpp

alalek · 2018-12-13T01:49:11Z

modules/qds/misc/python/pyopencv_quasiDenseStereo.hpp

@@ -0,0 +1,18 @@
+#ifdef HAVE_OPENCV_QDS
+#include "opencv2/core/saturate.hpp"


Why saturate header is needed here?

My mistake, I had trouble wrapping for python and I copied a snippet from pyopencv_linemod.hpp without checking if the header is needed.

modules/qds/include/opencv2/qds/quasiDenseStereo.hpp

…omp.hpp

dimitrisPs · 2018-12-14T12:33:19Z

Thank you for your input. I made changes to qds module. If your build-bot won't output any errors/warnings, I will start the integration to stereo module the way you described it.

alalek · 2018-12-18T10:19:58Z

modules/stereo/include/opencv2/stereo/quasi_dense_stereo.hpp

+     * in case of video processing.
+     * @sa loadParameters
+     */
+    CV_EXPORTS virtual int loadParameters(cv::String filepath="") = 0;


remove CV_EXPORTS from members. It is already applied for whole class.

dimitrisPs · 2018-12-18T11:04:13Z

I will fix the errors within the next hour and I will upload it again.

alalek

Well done!

Please take a look on comments below.

alalek · 2018-12-19T12:09:39Z

modules/stereo/doc/stereo.bib

+isbn="978-3-642-15705-9"
+}
+
+


Please remove extra space before "}" and extra empty lines

alalek · 2018-12-19T12:11:01Z

modules/stereo/include/opencv2/stereo/quasi_dense_stereo.hpp

+
+};
+
+typedef std::priority_queue<Match, std::vector<Match>, std::less<Match> > t_matchPriorityQueue;


Not used in public API. Move to "src/" private headers.
Move #include <queue> too.

alalek · 2018-12-19T12:11:33Z

modules/stereo/include/opencv2/stereo/quasi_dense_stereo.hpp

+     * in case of video processing.
+     * @sa loadParameters
+     */
+    virtual int loadParameters(cv::String filepath="") = 0;


=""

default value is not needed here

alalek · 2018-12-19T12:11:46Z

modules/stereo/include/opencv2/stereo/quasi_dense_stereo.hpp

+     * @note This method can be used to generate a template file for tuning the class.
+     * @sa loadParameters
+     */
+    virtual int saveParameters(cv::String filepath="./qds_parameters.yaml") = 0;


Please remove default value too

alalek · 2018-12-19T12:12:59Z

modules/stereo/include/opencv2/stereo/quasi_dense_stereo.hpp

+    virtual cv::Mat getDisparity(uint8_t disparityLvls=50) = 0;
+
+
+    static cv::Ptr<QuasiDenseStereo> create(cv::Size monoImgSize, cv::String paramFilepath ="");


= cv::String()

alalek · 2018-12-19T12:16:05Z

modules/stereo/include/opencv2/stereo/quasi_dense_stereo.hpp

@@ -0,0 +1,234 @@
+/*
+By downloading, copying, installing or using the software you agree to this license.
+If you do not agree to this license, do not download, install,


Please use short OpenCV license header:

// This file is part of OpenCV project. // It is subject to the license terms in the LICENSE file found in the top-level directory // of this distribution and at http://opencv.org/license.html.

https://github.com/opencv/opencv/wiki/Coding_Style_Guide

You may add "author" information after these 3 lines.

alalek · 2018-12-19T12:18:21Z

modules/stereo/tutorials/quasi_dense_stereo.markdown

+-   Compute dense Stereo correspondences.
+
+-----------
+[Source Code](../samples/dense_disparity.cpp)


Link is broken: http://pullrequest.opencv.org/buildbot/export/pr_contrib/1941/docs/d3/d7d/quasi_dense_stereo.html

Use @include or @snippet or GitHub link (opencv/opencv master branch)

alalek · 2018-12-19T12:19:24Z

modules/stereo/tutorials/quasi_dense_stereo.markdown

+```
+Mat rightImg, leftImg;
+leftImg = imread("./imgLeft.png", IMREAD_COLOR);
+rightImg = imread("./imgRight.png", IMREAD_COLOR);


Use @snippet instead of code duplication.

alalek · 2018-12-19T12:21:34Z

modules/stereo/src/quasi_dense_stereo.cpp

+    {
+        Match tmpMatch;
+        dMatches.clear();
+    //	dMatches.resize(dMatchesLen);


dMatches.reserve

…header.

… tutorials in new directories. Modify samples and tutorials to use snippet and include Doxygen commands.

…DMatch build-in type.

alalek

Well done!

alalek · 2018-12-21T09:55:44Z

modules/stereo/src/quasi_dense_stereo.cpp

+/*
+By downloading, copying, installing or using the software you agree to this license.
+If you do not agree to this license, do not download, install,
+copy or use the software.


Please use the short license header here too.

alalek

Thank you 👍

ZhuLingfeng1993 · 2021-02-13T12:45:21Z

@dimitrisPs
I test quasi dense stereo algorithm with opencv4.5.1.
I use images in testdata folder in stereo module to test the sample code file dense_disparity.cpp, but I get a disparity map with wrong result. Can you share your test image and configuration?

dimitrisPs · 2021-02-13T13:27:20Z

@ZhuLingfeng1993
Unfortunately, I do not have a copy of the test code I used back then.
Most likely, the reason you do not get correct disparity results is that you are using the getDisparity() function to get disparities. Inside this function, the disparity values get normalized to span the full range between 0 and 255. This was done because when I implemented the code I was working with planar scenes and I needed a fast way of visualizing normalized disparity images. To get proper disparity maps you can use the getDenseMatches() which will return the location of corresponding points and use those to create a disparity map.
I am fully aware that the getDisparity has a confusing name, for what it does, I plan to commit changes in the following days.

ZhuLingfeng1993 · 2021-02-16T10:05:04Z

@dimitrisPs
Thank you for your reply. I read the source code of getDisparity() and understand your reply. And I get the correct disparity map finally, thank you for your contribution.

dimitrisPs · 2021-02-16T19:11:39Z

@ZhuLingfeng1993
That's good to hear... I already made a pull request which removes the normalisation and added an accuracy test, proving that the algorithm works out of the box on Middlebury. Hopefully it will get merged and you will be able to get correct disparities using the getDisparity directly.

initial commit.

502770f

alalek reviewed Dec 11, 2018

View reviewed changes

dimitrisPs added 5 commits December 12, 2018 12:18

Remove license header.

e3301f4

Fix python wrap flags

529f8eb

Change std::string to cv::String, in function declarations, to resolv…

5952b83

…e compilation issues.

Add python wrapper extending header

6bb29e0

Fix python wrapper conflicts

928a47f

Fix implicit type conversions

17dde34

alalek reviewed Dec 13, 2018

View reviewed changes

dimitrisPs added 5 commits December 13, 2018 17:08

Change C API types and enums to C++.

3f90782

Remove redundant included headers and move wanted headers to src/prec…

8809463

…omp.hpp

Remove saturate header

791e823

Remove unnecessary python wrapping flags

8062754

Removed defaults parameter header

0a4f040

dimitrisPs added 6 commits December 17, 2018 19:19

Split declaration and implementation of the class using Pimpl.

19e42b0

Fix to comply with new public API.

1d92960

Remove unnecessary modules

b81a070

Fix maybe-uninitialized warnings on linux

e3c9eec

Migration to stereo module

2738d12

Remove CV_PROP_RW flag.

2bb9994

alalek reviewed Dec 18, 2018

View reviewed changes

Remove CV_EXPORTS flags from class members.

e2963f0

Fix: Removed misplaced flag

4d04ae7

alalek reviewed Dec 19, 2018

View reviewed changes

dimitrisPs added 3 commits December 19, 2018 14:39

Remove empty lines.

2c3b480

Move queue to private headers.

e2cdd27

Fix default arguments of public methods.

a7d0556

dimitrisPs added 5 commits December 19, 2018 16:09

Add authors information and switch to the compact version of license …

2d8d4ca

…header.

Reorganize and fix markdown files. Create a table of content and move…

bb769ad

… tutorials in new directories. Modify samples and tutorials to use snippet and include Doxygen commands.

Change argument name dMatch->denseMatch, to avoid confusion with cv::…

326d6d7

…DMatch build-in type.

Remove duplicate snippet.

5775768

Fix: change vector resize to reserve.

045ad89

alalek reviewed Dec 27, 2018

View reviewed changes

Fix: replace extensive license header with the compact version.

4a29108

alalek approved these changes Dec 28, 2018

View reviewed changes

alalek merged commit b1e9dd5 into opencv:master Dec 28, 2018

sotsuka-fixstars mentioned this pull request Jan 7, 2019

Add cudastereo semi global matching (SGM) implementation fixstars/opencv_contrib#1

Merged

9 tasks

ZhuLingfeng1993 mentioned this pull request Feb 13, 2021

solved issue 2813 #2819

Merged

6 tasks

dimitrisPs mentioned this pull request Feb 13, 2021

Remove normalization in cv::stereo::QuasiDenseStereo getDisparity(). #2869

Merged

6 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Implementation of Quasi Dense Stereo algorithm. #1941

Implementation of Quasi Dense Stereo algorithm. #1941

dimitrisPs commented Dec 10, 2018

alalek left a comment

alalek Dec 11, 2018

dimitrisPs commented Dec 12, 2018

alalek left a comment

alalek Dec 13, 2018

dimitrisPs Dec 14, 2018

mshabunin Dec 14, 2018

alalek Dec 13, 2018

dimitrisPs Dec 13, 2018

dimitrisPs commented Dec 14, 2018

alalek Dec 18, 2018

dimitrisPs commented Dec 18, 2018

alalek left a comment

alalek Dec 19, 2018

alalek Dec 19, 2018

alalek Dec 19, 2018

alalek Dec 19, 2018

alalek Dec 19, 2018

alalek Dec 19, 2018

alalek Dec 19, 2018

alalek Dec 19, 2018

alalek Dec 19, 2018

alalek left a comment

alalek Dec 21, 2018

alalek left a comment

ZhuLingfeng1993 commented Feb 13, 2021

dimitrisPs commented Feb 13, 2021

ZhuLingfeng1993 commented Feb 16, 2021

dimitrisPs commented Feb 16, 2021

		@@ -0,0 +1,18 @@
		#ifdef HAVE_OPENCV_QDS
		#include "opencv2/core/saturate.hpp"


		};

		typedef std::priority_queue<Match, std::vector<Match>, std::less<Match> > t_matchPriorityQueue;

		virtual cv::Mat getDisparity(uint8_t disparityLvls=50) = 0;


		static cv::Ptr<QuasiDenseStereo> create(cv::Size monoImgSize, cv::String paramFilepath ="");

		isbn="978-3-642-15705-9"
		}

Implementation of Quasi Dense Stereo algorithm. #1941

Implementation of Quasi Dense Stereo algorithm. #1941

Conversation

dimitrisPs commented Dec 10, 2018

alalek left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

dimitrisPs commented Dec 12, 2018

alalek left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

dimitrisPs commented Dec 14, 2018

Choose a reason for hiding this comment

dimitrisPs commented Dec 18, 2018

alalek left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

alalek left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

alalek left a comment

Choose a reason for hiding this comment

ZhuLingfeng1993 commented Feb 13, 2021

dimitrisPs commented Feb 13, 2021

ZhuLingfeng1993 commented Feb 16, 2021

dimitrisPs commented Feb 16, 2021