Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

With a Mixed Python/Rust project, only Rust functions appear in Python Wheel. #1649

Closed
1 of 2 tasks
DanielChaseButterfield opened this issue Jun 6, 2023 · 5 comments
Closed
1 of 2 tasks

Comments

@DanielChaseButterfield
Copy link

Bug Description

This may seem like a duplicate of #885, but that uses a different version of Maturin, and the solution for that thread is not the solution for my issue. I am trying to build into a conda virtual environment in WSL, and my pip wheel that is created by running maturin develop does not have any of my python functions or classes in it, just the functions defined by the Rust module. I have a mixed Python/Rust project with the following structure:

actag
├── Cargo.toml
├── python
│   └── actag
│       ├── __init__.py
│       ├── actag_src.py
│       ├── adaptive_threshold.py
│       ├── contour_identification.py
│       ├── median_filter.py
│       ├── quad_fitting.py
│       ├── sonar_parameters.py
│       ├── tag_decoding.py
│       └── tag_parameters.py
├── pyproject.toml
├── README.md
└── src
    ├── contours.py
    ├── filters.rs
    ├── lib.rs
    ├── quad_fitting.rs
    ├── tag_detection.rs
    └── tag_parameters.rs

My pyproject.toml contains:

[build-system]
requires = ["maturin>=0.13,<0.14"]
build-backend = "maturin"

[tool.maturin]
python-source = "python"

[project]
name = "actag"
requires-python = ">=3.9"
classifiers = [
    "Programming Language :: Rust",
    "Programming Language :: Python :: Implementation :: CPython",
    "Programming Language :: Python :: Implementation :: PyPy",
]

My Cargo.toml contains:

[package]
name = "actag"
version = "1.0.0"
edition = "2021"

# See more keys and their definitions at https://doc.rust-lang.org/cargo/reference/manifest.html
[lib]
name = "actag"
crate-type = ["cdylib", "lib"]

[dependencies]
ndarray = "0.15.4"
num = "0.4"
libm = "0.1.4"
geo = "0.22.1"
num_cpus = "1.13.1"
crossbeam = "0.8"
indicatif = "0.17.0"
itertools-num = "0.1.3"
nalgebra = "0.31.1"
lstsq = "0.4.0"
float-cmp = "0.9.0"
randomkit = "0.1"
substring = "1.4.5"
rug = "=1.18.0"

[dev-dependencies]
criterion = "0.3"

[[bench]]
name = "my_benchmark"
harness = false

[dependencies.pyo3]
version = "0.16.5"

[features]
abi3-py37 = ["pyo3/abi3-py37"]
extension-module = ["pyo3/extension-module"]
default = ["extension-module"]

My init.py file contains:

from .actag_src import AcTag

__doc__ = actag_src.__doc__
if hasattr(actag_src, "__all__"):
	__all__ = actag_src.__all__

My actag_src.py file contains an AcTag class. It also imports the actag module generated by the Rust side of the code.

With this setup, I can successfully build the pip wheel and install it with maturin develop, but I get the following error when I try to use the AcTag class from the python code:

Traceback (most recent call last):
  File "/home/dlittleman/actag/test.py", line 5, in <module>
    detector = actag.AcTag(min_range=0.1, max_range=1.5, horizontal_aperture=1.0472, tag_family="AcTag24h10", 
AttributeError: module 'actag' has no attribute 'AcTag'

However, I can see all of the Rust functions when I run print(dir(actag)):

['__all__', '__builtins__', '__cached__', '__doc__', '__file__', '__loader__', '__name__', '__package__', '__path__', '__spec__', 'actag', 'adaptive_threshold', 'adaptive_threshold_multithread', 'bilateral_filter', 'bilateral_filter_multithread', 'check_quad_for_tag_python_wrap', 'convert_tag_detections_to_range_azi_locations', 'decode_tags', 'fit_quadrilaterals_to_contours', 'get_contours', 'get_data_bit_locations', 'get_intersection_of_lines', 'get_random_point_and_fit_line_python_wrap', 'least_squares_line_fit_python_wrap', 'median_filter', 'median_filter_multithread', 'parse_apriltag_family', 'plot_contours']

Your maturin version (maturin --version)

0.13.0

Your Python version (python -V)

3.9.16

Your pip version (pip -V)

23.0.1

What bindings you're using

pyo3

Does cargo build work?

  • Yes, it works

If on windows, have you checked that you aren't accidentally using unix path (those with the forward slash /)?

  • Yes

Steps to Reproduce

  1. Install Conda and Initialize a conda environment: conda create -n tester python=3.9
  2. Activate the environment: conda activate tester
  3. Install maturin: pip install maturin
  4. Download the following repository: https://github.com/DanielChaseButterfield/maturinIssueDemonstration
  5. Run maturin develop --release from the main directory of the repository.
  6. Create a test.py file somewhere else with the following file contents:
import actag

detector = actag.AcTag(min_range=0.1, max_range=1.5, horizontal_aperture=1.0472, tag_family="AcTag24h10", 
                     tag_size=0.130628571428644, median_filter_kernel_radius=4, adaptive_threshold_kernel_radius=8, 
                     adaptive_threshold_offset=1, quads_use_same_random_vals=True)
  1. Run python test.py
@DanielChaseButterfield DanielChaseButterfield added the bug Something isn't working label Jun 6, 2023
@messense
Copy link
Member

messense commented Jun 7, 2023

Have you tried maturin 1.0 yet? No bugfixes will be backported to the old 0.13 versions.

@DanielChaseButterfield
Copy link
Author

I didn't know maturin 1.0 existed! I've now updated, and I can now see my phython functions and classes. However, now I can't see my Rust functions. This is what print(dir(actag)) returns now:

['AcTag', '__builtins__', '__cached__', '__doc__', '__file__', '__loader__', '__name__', '__package__', '__path__', '__spec__', 'actag_src', 'adaptive_threshold', 'contour_identification', 'median_filter', 'quad_fitting', 'sonar_parameters', 'tag_decoding', 'tag_parameters']

Am I correct in assuming that both the python and rust functions should be available?

@messense
Copy link
Member

messense commented Jun 7, 2023

No, Rust functions/classes are contained in actag (the name of your Rust extension module) module, you can re-export it to top level module if you want.

@messense messense closed this as completed Jun 7, 2023
@messense messense removed the bug Something isn't working label Jun 7, 2023
@messense
Copy link
Member

messense commented Jun 7, 2023

Please refer to https://www.maturin.rs/project_layout.html#mixed-rustpython-project for more information.

@DanielChaseButterfield
Copy link
Author

Okay, thank you! I got it working by adding this to the init.py file:

import actag as actag_rust

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants