Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Refactor bindings. #346

Merged
merged 33 commits into from
Sep 25, 2023
Merged

Refactor bindings. #346

merged 33 commits into from
Sep 25, 2023

Conversation

DinoBektesevic
Copy link
Member

@DinoBektesevic DinoBektesevic commented Sep 24, 2023

With all the recent refactoring I thought I'd join in on the fun.
Here is an initial pass through the code, making it (hopefully) more compliant to the google's C++ style guide. The refactoring:

  • snake_cases the C++ method names
  • CapitalCamelCases classes, structs, types and enums
  • makes the bindings reflect the C++ code (this is not strictly the best way, but at least they match now)
  • separates the Python documentation into separate files
  • cleans up bindings.cpp by separating the bindings into factory methods located next to the implementation
  • renames KBMOSearch into StackSearch to match the Python bindings (see full 7b2d479 commit message for reasoning, this isn't set in stone)
  • Updates all Python code and documentation references
  • removes various getDataRef variants, replacing them with .data() where necessary
  • fixes various minor oddities in naming conventions (f.e $v_x$ was spelled x_v)

I just noticed that GH shows all the renamed files as new files. Moving files with git, registered them as renames. Rest assured that per-line commits and provenance is still there, you can verify for yourself. This seems to be more a quirk of GitHub not being able to correctly display it. I think this occurred because I moved them and then edited them before committing the move as its own commit. If people feel very passionate about it, I can go back and rebase and split up the git mv's into separate commits, but, again, like I said if you click view file and check out git blame even GH will display the full history. Git locally doesn't seem to suffer from this.

Main does not build

I think our build on main is broken. At least it doesn't work on baldur. Both nvcc and nvidia-smi are visible in the environment and addition of print statement and running pip in verbose mode shows that it's building "with the GPU". However, there are tests that are missing skipIf protection, a print statement that wasn't scoped to pybind, but also for some reason the constexpr for HAS_GPU doesn't evaluate correctly. The IPO checks throw a warning too.

This PR "fixes" the broken build in so much that it fixes the bare print statement and adds protection to wherever search is called in tests so they don't error out. What it doesn't fix is the HAS_GPU flag, which evaluates to False. I haven't investigated the reasons in order to finish the refactor, but this should be a priority. Steps to reproduce on baldur:

(kbmod) [] /local/.../dinob$: git clone --recursive [email protected]:dirac-institute/kbmod.git kbmod_test
(kbmod) [] /local/.../dinob$: cd kbmod_test
(kbmod) [] /local/.../dinob$: pip install .
[... cut for brevity ...]
      -- Configuring done
      CMake Warning (dev) at /usr/share/cmake3/Modules/FindPython/Support.cmake:2571 (add_library):
        Policy CMP0069 is not set: INTERPROCEDURAL_OPTIMIZATION is enforced when
        enabled.  Run "cmake --help-policy CMP0069" for policy details.  Use the
        cmake_policy command to set the policy and suppress this warning.
      
        INTERPROCEDURAL_OPTIMIZATION property will be ignored for target 'search'.
      Call Stack (most recent call first):
        /usr/share/cmake3/Modules/FindPython3.cmake:313 (__Python3_add_library)
        lib/pybind11/tools/pybind11NewTools.cmake:189 (python3_add_library)
        CMakeLists.txt:40 (pybind11_add_module)
      This warning is for project developers.  Use -Wno-dev to suppress it.
      
      -- Generating done
      CMake Warning:
        Manually-specified variables were not used by the project:
      
          CMAKE_C_COMPILER_AR
          CMAKE_C_COMPILER_RANLIB
          PYTHON_EXECUTABLE
          SDISTVERSION
      
      
      -- Build files have been written to: /tmp/tmpmtq3m8c7.build-temp/kbmod.search
      Scanning dependencies of target search
      [ 50%] Building CXX object CMakeFiles/search.dir/src/kbmod/search/bindings.cpp.o
      In file included from /local/tmp/dinob/kbmod_test/src/kbmod/search/bindings.cpp:9:
      /local/tmp/dinob/kbmod_test/src/kbmod/search/KBMOSearch.cpp: In member function 'std::vector<search::RawImage> search::KBMOSearch::coaddedScienceStamps(std::vector<search::trajectory>&, std::vector<std::vector<bool> >&, const search::StampParameters&, bool)':
      /local/tmp/dinob/kbmod_test/src/kbmod/search/KBMOSearch.cpp:381:9: error: 'print' was not declared in this scope
        381 |         print("WARNING: GPU is not enabled. Performing co-adds on the CPU.");

Even when fixed and confirmed the build executes with the GPU found by running pip install -v . and adding the print statements to setup.py

(kbmod) [] /local/.../dinob$:  python
Python 3.11.5 (main, Sep 11 2023, 13:54:46) [GCC 11.2.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import kbmod
>>> kbmod.search.HAS_GPU
False

@DinoBektesevic DinoBektesevic force-pushed the refactor_bindings branch 2 times, most recently from 5c46d9b to e9e5876 Compare September 25, 2023 06:55
@DinoBektesevic DinoBektesevic force-pushed the refactor_bindings branch 2 times, most recently from 17c096d to 470ace2 Compare September 25, 2023 07:04
I forgot to split up this commit into
i) prefactoring method names
ii) splitting bindings
iii) fixing docs
iv) fixing Python references

but it's all been done.
KBMOSearch has been renamed StackSearch to match Python bindings.
I have no preference regarding the naming of the class, it can
rename KBMOSearch - as long as it's consistent with its Python
bindings.

In general no other class had method names as confusing or as
disparate from its Python bindings so in this merge, I've
renamed multiple of its methods - which was not the case for
other classes in this refactor. The following renames
occured:

* get_image_stack -> get_imagestack
* scienceStampsForViz, science_viz_stamps -> get_stamps
* [science, sci, mean, median, coadd, summed]_[sci, science]_stamp[s] -> [mean, median, coadd, summed]_stamp[s]
* get_traj_pos, getTrajPos -> get_trajectory_position
* get_mult_traj_pos, getMultTrajPos -> get_trajectory_positions
* [psi, phi]_curves -> get_[psi, phi]_curves

Same like with the class name, I'm not married for either particualar
individual variation on the topic, but the **only** kind of indirection
that should occur in the bindings is to make the interface feel and
act more Pythonic, by for example wrapping [get, set]_method using a
pybind11::def_property and then not exposing the individual getters
and setters, or for example, wrapping print, stringify and similar
methods in the dunder str, dunder repr methods and then also not
exposing the internals in the binding code.
Rename the struct to make it compliant with style guide.
Fix the missmatched binding names.
Fix all the tests.
Rebasing after #345 left some old references, to renamed methods,
that I must have missed to correct in the rebase.

Added documentation entry for the new methods.
Removed the documentation entries for methods removed in #345.
It seems there were more than a few items I missed renaming during
the rebase and I fat-fingered a couple of , instead of a .

Fixes bindings for add_pixel_interp in RawImage.
Copy link
Contributor

@jeremykubica jeremykubica left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A few small comments, but no changes needed.

i++;
}

radius = i - 1; // This value is good for
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is there more to this comment? Or was it a question?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great question, it first appeared in 506d347 6 years ago and has been carried along ever since...

src/kbmod/search/psf.cpp Outdated Show resolved Hide resolved
src/kbmod/search/psf.h Show resolved Hide resolved
src/kbmod/search/image_stack.h Show resolved Hide resolved
@DinoBektesevic DinoBektesevic merged commit eeb62d5 into main Sep 25, 2023
1 check passed
@jeremykubica jeremykubica mentioned this pull request Sep 28, 2023
@jeremykubica jeremykubica deleted the refactor_bindings branch February 23, 2024 19:24
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants