Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add multithreading support to Multi* geometries #1265

Merged
merged 8 commits into from
Nov 6, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions geo-types/CHANGES.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,8 @@

## Unreleased
* Add rstar compatibility for MultiPolygon
* Add multi-threading support to Multi* geometries.
* Feature-gated ('multithreading'), disabled by default, enabled by default when geo-types is used by geo

## 0.7.13

Expand Down
2 changes: 2 additions & 0 deletions geo-types/Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,7 @@ edition = "2021"
[features]
default = ["std"]
std = ["approx?/std", "num-traits/std", "serde?/std"]
multithreading = ["rayon"]
# Prefer `use-rstar` feature rather than enabling rstar directly.
# rstar integration relies on the optional approx crate, but implicit features cannot yet enable other features.
# See: https://doc.rust-lang.org/nightly/cargo/reference/unstable.html#namespaced-features
Expand All @@ -25,6 +26,7 @@ use-rstar_0_11 = ["rstar_0_11", "approx"]
use-rstar_0_12 = ["rstar_0_12", "approx"]

[dependencies]
rayon = { version = "1.10.0", optional = true }
approx = { version = ">= 0.4.0, < 0.6.0", optional = true, default-features = false }
arbitrary = { version = "1.2.0", optional = true }
num-traits = { version = "0.2", default-features = false, features = ["libm"] }
Expand Down
47 changes: 47 additions & 0 deletions geo-types/src/geometry/multi_line_string.rs
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,8 @@ use alloc::vec::Vec;
#[cfg(any(feature = "approx", test))]
use approx::{AbsDiffEq, RelativeEq};
use core::iter::FromIterator;
#[cfg(feature = "multithreading")]
use rayon::prelude::*;

/// A collection of
/// [`LineString`s](line_string/struct.LineString.html). Can
Expand Down Expand Up @@ -118,6 +120,36 @@ impl<T: CoordNum> MultiLineString<T> {
}
}

#[cfg(feature = "multithreading")]
impl<T: CoordNum + Send> IntoParallelIterator for MultiLineString<T> {
type Item = LineString<T>;
type Iter = rayon::vec::IntoIter<LineString<T>>;

fn into_par_iter(self) -> Self::Iter {
self.0.into_par_iter()
}
}

#[cfg(feature = "multithreading")]
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The consuming into implementation makes sense to me.

And I can see how the ref and mut flavors below provide necessary non-consuming functionality. But what does it look like to use it? Can you include some example code in your PR that uses these new methods to make the new interface easier to evaluate?

My guess is that you'd need to do something like

(& my_multi_line_string).into_par_iter()
and
(&mut my_multi_line_string).into_par_iter()
?

Which is a bit rough. I'm afraid people will just end up my_multi_line_string.clone().into_par_iter() out of confusion.

Would it get better if we got rid of these two ref/mut implementations and instead implemented rayon::IntoParallelRefIterator and rayon::IntoParallelRefMutIterator which seem purpose built for this?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hm. These work with the current impls:

    #[test]
    fn test_multithreading_linestring() {
        let multi: MultiLineString<i32> = wkt! {
            MULTILINESTRING((0 0,2 0,1 2,0 0), (10 10,12 10,11 12,10 10))
        };
        let mut multimut: MultiLineString<i32> = wkt! {
            MULTILINESTRING((0 0,2 0,1 2,0 0), (10 10,12 10,11 12,10 10))
        };
        let _ = multi.par_iter().for_each(|_p| ());
        let _ = multimut.par_iter_mut().for_each(|_p| ());
        let _ = &multi.into_par_iter().for_each(|_p| ());
        let _ = &mut multimut.par_iter_mut().for_each(|_p| ());
    }

I admittedly didn't see the rayon::IntoParallelRefIterator and just adapted our existing iterators to rayon::IntoParallelIterator, but I'm confused by

This trait is automatically implemented for I where &I: IntoParallelIterator. In most cases, users will want to implement IntoParallelIterator rather than implement this trait directly.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh splendid! They're already implemented by the blanket impl!

I'm glad that users will be able to do:

let _ = multi.par_iter().for_each(|_p| ());
let _ = multimut.par_iter_mut().for_each(|_p| ());

vs. the slightly more arcane:

let _ = &multi.into_par_iter().for_each(|_p| ());
let _ = &mut multimut.into_par_iter().for_each(|_p| ());

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh yeah the latter two were just to make sure "everything" worked. Thanks for the sanity check!

impl<'a, T: CoordNum + Sync> IntoParallelIterator for &'a MultiLineString<T> {
type Item = &'a LineString<T>;
type Iter = rayon::slice::Iter<'a, LineString<T>>;

fn into_par_iter(self) -> Self::Iter {
self.0.par_iter()
}
}

#[cfg(feature = "multithreading")]
impl<'a, T: CoordNum + Send + Sync> IntoParallelIterator for &'a mut MultiLineString<T> {
type Item = &'a mut LineString<T>;
type Iter = rayon::slice::IterMut<'a, LineString<T>>;

fn into_par_iter(self) -> Self::Iter {
self.0.par_iter_mut()
}
}

#[cfg(any(feature = "approx", test))]
impl<T> RelativeEq for MultiLineString<T>
where
Expand Down Expand Up @@ -199,6 +231,21 @@ mod test {
use super::*;
use crate::{line_string, wkt};

#[cfg(feature = "multithreading")]
#[test]
urschrei marked this conversation as resolved.
Show resolved Hide resolved
fn test_multithreading_linestring() {
let multi: MultiLineString<i32> = wkt! {
MULTILINESTRING((0 0,2 0,1 2,0 0), (10 10,12 10,11 12,10 10))
};
let mut multimut: MultiLineString<i32> = wkt! {
MULTILINESTRING((0 0,2 0,1 2,0 0), (10 10,12 10,11 12,10 10))
};
multi.par_iter().for_each(|_p| ());
multimut.par_iter_mut().for_each(|_p| ());
let _ = &multi.into_par_iter().for_each(|_p| ());
let _ = &mut multimut.par_iter_mut().for_each(|_p| ());
}

#[test]
fn test_iter() {
let multi: MultiLineString<i32> = wkt! {
Expand Down
32 changes: 32 additions & 0 deletions geo-types/src/geometry/multi_point.rs
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,8 @@ use approx::{AbsDiffEq, RelativeEq};
use alloc::vec;
use alloc::vec::Vec;
use core::iter::FromIterator;
#[cfg(feature = "multithreading")]
use rayon::prelude::*;

/// A collection of [`Point`s](struct.Point.html). Can
/// be created from a `Vec` of `Point`s, or from an
Expand Down Expand Up @@ -85,6 +87,36 @@ impl<'a, T: CoordNum> IntoIterator for &'a mut MultiPoint<T> {
}
}

#[cfg(feature = "multithreading")]
impl<T: CoordNum + Send> IntoParallelIterator for MultiPoint<T> {
type Item = Point<T>;
type Iter = rayon::vec::IntoIter<Point<T>>;

fn into_par_iter(self) -> Self::Iter {
self.0.into_par_iter()
}
}

#[cfg(feature = "multithreading")]
impl<'a, T: CoordNum + Sync> IntoParallelIterator for &'a MultiPoint<T> {
type Item = &'a Point<T>;
type Iter = rayon::slice::Iter<'a, Point<T>>;

fn into_par_iter(self) -> Self::Iter {
self.0.par_iter()
}
}

#[cfg(feature = "multithreading")]
impl<'a, T: CoordNum + Send + Sync> IntoParallelIterator for &'a mut MultiPoint<T> {
type Item = &'a mut Point<T>;
type Iter = rayon::slice::IterMut<'a, Point<T>>;

fn into_par_iter(self) -> Self::Iter {
self.0.par_iter_mut()
}
}

impl<T: CoordNum> MultiPoint<T> {
pub fn new(value: Vec<Point<T>>) -> Self {
Self(value)
Expand Down
49 changes: 49 additions & 0 deletions geo-types/src/geometry/multi_polygon.rs
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,10 @@ use alloc::vec;
use alloc::vec::Vec;
#[cfg(any(feature = "approx", test))]
use approx::{AbsDiffEq, RelativeEq};

use core::iter::FromIterator;
#[cfg(feature = "multithreading")]
use rayon::prelude::*;

/// A collection of [`Polygon`s](struct.Polygon.html). Can
/// be created from a `Vec` of `Polygon`s, or from an
Expand Down Expand Up @@ -75,6 +78,36 @@ impl<'a, T: CoordNum> IntoIterator for &'a mut MultiPolygon<T> {
}
}

#[cfg(feature = "multithreading")]
impl<T: CoordNum + Send> IntoParallelIterator for MultiPolygon<T> {
type Item = Polygon<T>;
type Iter = rayon::vec::IntoIter<Polygon<T>>;

fn into_par_iter(self) -> Self::Iter {
self.0.into_par_iter()
}
}

#[cfg(feature = "multithreading")]
impl<'a, T: CoordNum + Sync> IntoParallelIterator for &'a MultiPolygon<T> {
type Item = &'a Polygon<T>;
type Iter = rayon::slice::Iter<'a, Polygon<T>>;

fn into_par_iter(self) -> Self::Iter {
self.0.par_iter()
}
}

#[cfg(feature = "multithreading")]
impl<'a, T: CoordNum + Send + Sync> IntoParallelIterator for &'a mut MultiPolygon<T> {
type Item = &'a mut Polygon<T>;
type Iter = rayon::slice::IterMut<'a, Polygon<T>>;

fn into_par_iter(self) -> Self::Iter {
self.0.par_iter_mut()
}
}

impl<T: CoordNum> MultiPolygon<T> {
/// Instantiate Self from the raw content value
pub fn new(value: Vec<Polygon<T>>) -> Self {
Expand Down Expand Up @@ -250,6 +283,22 @@ mod test {
}
}

#[cfg(feature = "multithreading")]
#[test]
fn test_par_iter() {
let multi = MultiPolygon::new(vec![
polygon![(x: 0, y: 0), (x: 2, y: 0), (x: 1, y: 2), (x:0, y:0)],
polygon![(x: 10, y: 10), (x: 12, y: 10), (x: 11, y: 12), (x:10, y:10)],
]);
let mut multimut = MultiPolygon::new(vec![
polygon![(x: 0, y: 0), (x: 2, y: 0), (x: 1, y: 2), (x:0, y:0)],
polygon![(x: 10, y: 10), (x: 12, y: 10), (x: 11, y: 12), (x:10, y:10)],
]);
multi.par_iter().for_each(|_p| ());
let _ = &multimut.par_iter_mut().for_each(|_p| ());
let _ = &multi.into_par_iter().for_each(|_p| ());
let _ = &mut multimut.par_iter_mut().for_each(|_p| ());
}
#[test]
fn test_iter_mut() {
let mut multi = MultiPolygon::new(vec![
Expand Down
2 changes: 2 additions & 0 deletions geo-types/src/lib.rs
Original file line number Diff line number Diff line change
Expand Up @@ -62,6 +62,8 @@
//! The following optional [Cargo features] are available:
//!
//! - `std`: Enables use of the full `std` library. Enabled by default.
//! - `multithreading`: Enables multi-threaded iteration over `Multi*` geometries. **Disabled**
//! by default but **enabled** by `geo`'s default features.
//! - `approx`: Allows geometry types to be checked for approximate equality with [approx]
//! - `arbitrary`: Allows geometry types to be created from unstructured input with [arbitrary]
//! - `serde`: Allows geometry types to be serialized and deserialized with [Serde]
Expand Down
2 changes: 1 addition & 1 deletion geo/Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,7 @@ default = ["earcutr", "spade", "multithreading"]
use-proj = ["proj"]
proj-network = ["use-proj", "proj/network"]
use-serde = ["serde", "geo-types/serde"]
multithreading = ["i_overlay/allow_multithreading"]
multithreading = ["i_overlay/allow_multithreading", "geo-types/multithreading"]

[dependencies]
earcutr = { version = "0.4.2", optional = true }
Expand Down
3 changes: 2 additions & 1 deletion geo/src/lib.rs
Original file line number Diff line number Diff line change
Expand Up @@ -191,7 +191,8 @@
//! - Allows geometry types to be serialized and deserialized with [Serde].
//! - ☐ Disabled by default.
//! - `multithreading`:
//! - Enables multithreading support for the `i_overlay` crate.
//! - Enables multithreading support for the `i_overlay` crate (via Rayon), and activates the `multithreading` flag
//! in `geo-types`, enabling multi-threaded iteration over `Multi*` geometries.
//! - ☑ Enabled by default.
//!
//! # Ecosystem
Expand Down
Loading