Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

refactor OkWrap to not call .into_py(py) #3595

Merged
merged 1 commit into from
Nov 25, 2023
Merged

Conversation

davidhewitt
Copy link
Member

Motivated by https://github.com/PyO3/pyo3/pull/3588/files#r1403933922

The goal here is to keep the OkWrap trait as simple as it can possibly be.

@wyfo what do you think? It conflicts a little bit with #3588, so that'd need rebasing if this merged, but I like how this keeps OkWrap simple.

@davidhewitt davidhewitt added the CI-skip-changelog Skip checking changelog entry label Nov 24, 2023
@wyfo
Copy link
Contributor

wyfo commented Nov 24, 2023

I didn't dare to do this refactor as part of the async work, but I fully agree with this simplification

src/impl_/wrap.rs Outdated Show resolved Hide resolved
src/impl_/wrap.rs Show resolved Hide resolved
Comment on lines +52 to +68
pub fn map_result_into_ptr<T: IntoPy<PyObject>>(
py: Python<'_>,
result: PyResult<T>,
) -> PyResult<*mut ffi::PyObject> {
result.map(|obj| obj.into_py(py).into_ptr())
}

/// This is a follow-up function to `OkWrap::wrap` that converts the result into
/// a safe wrapper.
pub fn map_result_into_py<T: IntoPy<PyObject>>(
py: Python<'_>,
result: PyResult<T>,
) -> PyResult<PyObject> {
result.map(|err| err.into_py(py))
}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
pub fn map_result_into_ptr<T: IntoPy<PyObject>>(
py: Python<'_>,
result: PyResult<T>,
) -> PyResult<*mut ffi::PyObject> {
result.map(|obj| obj.into_py(py).into_ptr())
}
/// This is a follow-up function to `OkWrap::wrap` that converts the result into
/// a safe wrapper.
pub fn map_result_into_py<T: IntoPy<PyObject>>(
py: Python<'_>,
result: PyResult<T>,
) -> PyResult<PyObject> {
result.map(|err| err.into_py(py))
}
pub fn map_result_into_ptr<T, E>(
py: Python<'_>,
result: Result<T, E>,
) -> PyResult<*mut ffi::PyObject>
where
T: IntoPy<PyObject>,
PyErr: From<E>,
{
Ok(result?.into_py(py).into_ptr())
}
/// This is a follow-up function to `OkWrap::wrap` that converts the result into
/// a safe wrapper.
pub fn map_result_into_py<T, E>(
py: Python<'_>,
result: Result<T, E>,
) -> PyResult<PyObject>
where
T: IntoPy<PyObject>,
PyErr: From<E>,
{
Ok(result?.into_py(py))
}

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I guess it's a bit of a style thing, I slightly prefer the combinators over ? in these one-line implementations.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The suggestion is not about the ? but using Result<T, E> instead of PyResult<T>. If you use map_err instead of ?, you can in fact relax the bound to E: Into<PyErr> instead of PyErr: From<E>, and that's maybe better (I should also relax the bound in async support implementation by the way)

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah I see. I take your point (and I did initially accept Result<T, E>) but having the E: Into<PyErr> bound introduced on this function and doing .map_err in here instead of directly in the macro generated code means this internal function map_result_into_ptr appears in the compiler error.

I didn't think that was useful (that was the UI test regression noted in the comment below) so I kept with just PyResult for now as all we need.

I nearly changed the async support but actually I think it's better if I don't touch that in this PR so that it conflicts less with what you've got stacked up.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[...] means this internal function map_result_into_ptr appears in the compiler error.

Makes sense indeed.

@@ -33,7 +33,7 @@ fn test_compile_errors() {
t.compile_fail("tests/ui/invalid_pymethod_receiver.rs");
t.compile_fail("tests/ui/missing_intopy.rs");
// adding extra error conversion impls changes the output
#[cfg(all(target_os = "linux", not(any(feature = "eyre", feature = "anyhow"))))]
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why this modification?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's a bit of a drive-by but I was working on macOS today and I broke CI in the first attempt because this UI test regressed. Turns out the macos and linux outputs are the same.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

(I think) I added this because the output is different on Windows. The error message for this lists all the variants, and windows has additional error classes.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yep that's what I recall too, which is the why I left windows in the not(any()) list

@wyfo
Copy link
Contributor

wyfo commented Nov 24, 2023

👍 for me

@davidhewitt
Copy link
Member Author

Thanks again for review, as this touches more code which other contributors initially helped design I'll wait to merge a little bit in case of other opinions. But not too long, as you'll need to rebase off this 🙂

@davidhewitt davidhewitt added this pull request to the merge queue Nov 25, 2023
Merged via the queue into PyO3:main with commit 9f66846 Nov 25, 2023
34 checks passed
@davidhewitt davidhewitt deleted the ok-wrap branch November 25, 2023 06:20
facebook-github-bot pushed a commit to facebook/buck2 that referenced this pull request May 3, 2024
Summary:
NOTE: If your `hg bisect` brings you here & the error you are seeing looks like ` expected Result<&PyAny, PyErr>, found Result<Bound<'_, PyAny>, PyErr>` then see these  [migration notes](https://pyo3.rs/v0.21.0/migration.html#from-020-to-021) for the fix or click on `fbcode/security/ace/pyo3/authz.rs` or similar files from bellow to see what the fix is!

In order to upgrade `pyo3` to [`0.21.1`](https://github.com/PyO3/pyo3/releases/tag/v0.21.1), the following had to take place:

## [PyO3]
* Address [migration notes](https://pyo3.rs/v0.21.0/migration.html#from-020-to-021) for `Bound<'py, T>`
* Address [#3595](PyO3/pyo3#3595) - this is done in a crude way for now since there are many call sites depending on `fbcode/dba/rust/common/service_address/py/pyo3_conversion_helper.rs` which would require a more thorough review.
* Address [#3821](PyO3/pyo3#3821) - `pyo3-build-config` is now dependent on and used by PyO3 macros. Currently, the only thing that gets checked is `abi3` compatibility. To address that, we introduce a fixup for that automatically generate `pyo3-build-config*.txt` configuration files, given an `fbsource` Python version. We are tryin to stay as close to `pyo3-build-config*.txt` spec as possible even though only a single bool from that file is ever since there is validation for the other fields but also to future proof future changes. By also generating this file ourselves, we prevent PyO3 from attempting to locate a Python interpreter some other way which seems to start leaking into the PyO3 API as an implementation choice already.

## [PyPi + Rust]
* Upgrade `orjson` to [`3.10.1`](https://github.com/ijl/orjson/releases/tag/3.10.1)
  * Added `README.md` notes for future upgrades
* Upgrade `py-polars` to [`0.20.22`](https://github.com/pola-rs/polars/releases/tag/py-0.20.22)
  * Removed `py-polars` and `polars` from `target_os = "windows"`. The `third-party/pypi/polars` Python extension has only been supported for Mac and Linux for a while now so its only natural to do that on the Rust side as well. What is more, `polars-util` is bringing in `stacker = = "0.1.14"` which does not build on Windows mostly because its using a much more recent version of `libc` than we use in `third-party/rust` (see P1228807344)
* Upgrade `pydantic-core` to [`2.18.2`](https://github.com/pydantic/pydantic-core/releases/tag/v2.18.2)
  *  Removed old `third-party/pypi/pydantic-core` versions
* Upgrade `safetensors` to [`0.4.3`](https://github.com/huggingface/safetensors/releases/tag/v0.4.3)
* Patch `third-party/pypi/cryptography/41.0.7` to account for PyO3's [#2975](PyO3/pyo3#2975) (`0.19.0`) where `pyo3::once_cell` was renamed to `pyo3::sync` (see D56826865)
* Upgrade `tokenizers` and `tokenizers-python` to [`0.19.1`](https://github.com/huggingface/tokenizers/releases/tag/v0.19.1)
  * Removed old `third-party/pypi/tokenizers` versions
  * Fixed `third-party/pypi/tokenizers/BUCK`
  * Migrated `third-party/pypi/tokenizers/0.19.1/BUCK` to mirror other Python packages that bind to Rust crates e.g. `libcst`, `polars` etc.
  * Removed Windows support from `third-party/pypi/tokenizers`

## [Rust]
* Upgrade `indexmap` to [`2.2.6`](https://github.com/indexmap-rs/indexmap/releases/tag/2.2.6)
  * Both latest `pydantic-core` and `c2pa` depend on `serde_json > 1.0.112` which brings in `indexmap = 2.2.1`. The latter has deprecated `.take` and `.remove` on both `IndexMap` and `IndexSet` leading to a bunch of errors (see bellow), all addressed:

```bash
error: use of deprecated method `indexmap::set::IndexSet::<T, S>::take`: `take` disrupts the set order -- use `swap_take` or `shift_take` for explicit behavior.
  --> fbcode/hphp/hack/src/package/types.rs:76:16
   |
76 |         self.0.take(value)
   |                ^^^^
   |
   = note: `-D deprecated` implied by `-D warnings`
   = help: to override `-D warnings` add `#[allow(deprecated)]`
```

Reviewed By: capickett

Differential Revision: D56671179

fbshipit-source-id: 3ae69c069b7f005570c1a06d37194cf056282a18
facebook-github-bot pushed a commit to facebook/sapling that referenced this pull request May 3, 2024
Summary:
NOTE: If your `hg bisect` brings you here & the error you are seeing looks like ` expected Result<&PyAny, PyErr>, found Result<Bound<'_, PyAny>, PyErr>` then see these  [migration notes](https://pyo3.rs/v0.21.0/migration.html#from-020-to-021) for the fix or click on `fbcode/security/ace/pyo3/authz.rs` or similar files from bellow to see what the fix is!

In order to upgrade `pyo3` to [`0.21.1`](https://github.com/PyO3/pyo3/releases/tag/v0.21.1), the following had to take place:

## [PyO3]
* Address [migration notes](https://pyo3.rs/v0.21.0/migration.html#from-020-to-021) for `Bound<'py, T>`
* Address [#3595](PyO3/pyo3#3595) - this is done in a crude way for now since there are many call sites depending on `fbcode/dba/rust/common/service_address/py/pyo3_conversion_helper.rs` which would require a more thorough review.
* Address [#3821](PyO3/pyo3#3821) - `pyo3-build-config` is now dependent on and used by PyO3 macros. Currently, the only thing that gets checked is `abi3` compatibility. To address that, we introduce a fixup for that automatically generate `pyo3-build-config*.txt` configuration files, given an `fbsource` Python version. We are tryin to stay as close to `pyo3-build-config*.txt` spec as possible even though only a single bool from that file is ever since there is validation for the other fields but also to future proof future changes. By also generating this file ourselves, we prevent PyO3 from attempting to locate a Python interpreter some other way which seems to start leaking into the PyO3 API as an implementation choice already.

## [PyPi + Rust]
* Upgrade `orjson` to [`3.10.1`](https://github.com/ijl/orjson/releases/tag/3.10.1)
  * Added `README.md` notes for future upgrades
* Upgrade `py-polars` to [`0.20.22`](https://github.com/pola-rs/polars/releases/tag/py-0.20.22)
  * Removed `py-polars` and `polars` from `target_os = "windows"`. The `third-party/pypi/polars` Python extension has only been supported for Mac and Linux for a while now so its only natural to do that on the Rust side as well. What is more, `polars-util` is bringing in `stacker = = "0.1.14"` which does not build on Windows mostly because its using a much more recent version of `libc` than we use in `third-party/rust` (see P1228807344)
* Upgrade `pydantic-core` to [`2.18.2`](https://github.com/pydantic/pydantic-core/releases/tag/v2.18.2)
  *  Removed old `third-party/pypi/pydantic-core` versions
* Upgrade `safetensors` to [`0.4.3`](https://github.com/huggingface/safetensors/releases/tag/v0.4.3)
* Patch `third-party/pypi/cryptography/41.0.7` to account for PyO3's [#2975](PyO3/pyo3#2975) (`0.19.0`) where `pyo3::once_cell` was renamed to `pyo3::sync` (see D56826865)
* Upgrade `tokenizers` and `tokenizers-python` to [`0.19.1`](https://github.com/huggingface/tokenizers/releases/tag/v0.19.1)
  * Removed old `third-party/pypi/tokenizers` versions
  * Fixed `third-party/pypi/tokenizers/BUCK`
  * Migrated `third-party/pypi/tokenizers/0.19.1/BUCK` to mirror other Python packages that bind to Rust crates e.g. `libcst`, `polars` etc.
  * Removed Windows support from `third-party/pypi/tokenizers`

## [Rust]
* Upgrade `indexmap` to [`2.2.6`](https://github.com/indexmap-rs/indexmap/releases/tag/2.2.6)
  * Both latest `pydantic-core` and `c2pa` depend on `serde_json > 1.0.112` which brings in `indexmap = 2.2.1`. The latter has deprecated `.take` and `.remove` on both `IndexMap` and `IndexSet` leading to a bunch of errors (see bellow), all addressed:

```bash
error: use of deprecated method `indexmap::set::IndexSet::<T, S>::take`: `take` disrupts the set order -- use `swap_take` or `shift_take` for explicit behavior.
  --> fbcode/hphp/hack/src/package/types.rs:76:16
   |
76 |         self.0.take(value)
   |                ^^^^
   |
   = note: `-D deprecated` implied by `-D warnings`
   = help: to override `-D warnings` add `#[allow(deprecated)]`
```

Reviewed By: capickett

Differential Revision: D56671179

fbshipit-source-id: 3ae69c069b7f005570c1a06d37194cf056282a18
facebook-github-bot pushed a commit to facebook/ocamlrep that referenced this pull request May 3, 2024
Summary:
NOTE: If your `hg bisect` brings you here & the error you are seeing looks like ` expected Result<&PyAny, PyErr>, found Result<Bound<'_, PyAny>, PyErr>` then see these  [migration notes](https://pyo3.rs/v0.21.0/migration.html#from-020-to-021) for the fix or click on `fbcode/security/ace/pyo3/authz.rs` or similar files from bellow to see what the fix is!

In order to upgrade `pyo3` to [`0.21.1`](https://github.com/PyO3/pyo3/releases/tag/v0.21.1), the following had to take place:

## [PyO3]
* Address [migration notes](https://pyo3.rs/v0.21.0/migration.html#from-020-to-021) for `Bound<'py, T>`
* Address [#3595](PyO3/pyo3#3595) - this is done in a crude way for now since there are many call sites depending on `fbcode/dba/rust/common/service_address/py/pyo3_conversion_helper.rs` which would require a more thorough review.
* Address [#3821](PyO3/pyo3#3821) - `pyo3-build-config` is now dependent on and used by PyO3 macros. Currently, the only thing that gets checked is `abi3` compatibility. To address that, we introduce a fixup for that automatically generate `pyo3-build-config*.txt` configuration files, given an `fbsource` Python version. We are tryin to stay as close to `pyo3-build-config*.txt` spec as possible even though only a single bool from that file is ever since there is validation for the other fields but also to future proof future changes. By also generating this file ourselves, we prevent PyO3 from attempting to locate a Python interpreter some other way which seems to start leaking into the PyO3 API as an implementation choice already.

## [PyPi + Rust]
* Upgrade `orjson` to [`3.10.1`](https://github.com/ijl/orjson/releases/tag/3.10.1)
  * Added `README.md` notes for future upgrades
* Upgrade `py-polars` to [`0.20.22`](https://github.com/pola-rs/polars/releases/tag/py-0.20.22)
  * Removed `py-polars` and `polars` from `target_os = "windows"`. The `third-party/pypi/polars` Python extension has only been supported for Mac and Linux for a while now so its only natural to do that on the Rust side as well. What is more, `polars-util` is bringing in `stacker = = "0.1.14"` which does not build on Windows mostly because its using a much more recent version of `libc` than we use in `third-party/rust` (see P1228807344)
* Upgrade `pydantic-core` to [`2.18.2`](https://github.com/pydantic/pydantic-core/releases/tag/v2.18.2)
  *  Removed old `third-party/pypi/pydantic-core` versions
* Upgrade `safetensors` to [`0.4.3`](https://github.com/huggingface/safetensors/releases/tag/v0.4.3)
* Patch `third-party/pypi/cryptography/41.0.7` to account for PyO3's [#2975](PyO3/pyo3#2975) (`0.19.0`) where `pyo3::once_cell` was renamed to `pyo3::sync` (see D56826865)
* Upgrade `tokenizers` and `tokenizers-python` to [`0.19.1`](https://github.com/huggingface/tokenizers/releases/tag/v0.19.1)
  * Removed old `third-party/pypi/tokenizers` versions
  * Fixed `third-party/pypi/tokenizers/BUCK`
  * Migrated `third-party/pypi/tokenizers/0.19.1/BUCK` to mirror other Python packages that bind to Rust crates e.g. `libcst`, `polars` etc.
  * Removed Windows support from `third-party/pypi/tokenizers`

## [Rust]
* Upgrade `indexmap` to [`2.2.6`](https://github.com/indexmap-rs/indexmap/releases/tag/2.2.6)
  * Both latest `pydantic-core` and `c2pa` depend on `serde_json > 1.0.112` which brings in `indexmap = 2.2.1`. The latter has deprecated `.take` and `.remove` on both `IndexMap` and `IndexSet` leading to a bunch of errors (see bellow), all addressed:

```bash
error: use of deprecated method `indexmap::set::IndexSet::<T, S>::take`: `take` disrupts the set order -- use `swap_take` or `shift_take` for explicit behavior.
  --> fbcode/hphp/hack/src/package/types.rs:76:16
   |
76 |         self.0.take(value)
   |                ^^^^
   |
   = note: `-D deprecated` implied by `-D warnings`
   = help: to override `-D warnings` add `#[allow(deprecated)]`
```

Reviewed By: capickett

Differential Revision: D56671179

fbshipit-source-id: 3ae69c069b7f005570c1a06d37194cf056282a18
facebook-github-bot pushed a commit to facebookincubator/reindeer that referenced this pull request May 3, 2024
Summary:
NOTE: If your `hg bisect` brings you here & the error you are seeing looks like ` expected Result<&PyAny, PyErr>, found Result<Bound<'_, PyAny>, PyErr>` then see these  [migration notes](https://pyo3.rs/v0.21.0/migration.html#from-020-to-021) for the fix or click on `fbcode/security/ace/pyo3/authz.rs` or similar files from bellow to see what the fix is!

In order to upgrade `pyo3` to [`0.21.1`](https://github.com/PyO3/pyo3/releases/tag/v0.21.1), the following had to take place:

## [PyO3]
* Address [migration notes](https://pyo3.rs/v0.21.0/migration.html#from-020-to-021) for `Bound<'py, T>`
* Address [#3595](PyO3/pyo3#3595) - this is done in a crude way for now since there are many call sites depending on `fbcode/dba/rust/common/service_address/py/pyo3_conversion_helper.rs` which would require a more thorough review.
* Address [#3821](PyO3/pyo3#3821) - `pyo3-build-config` is now dependent on and used by PyO3 macros. Currently, the only thing that gets checked is `abi3` compatibility. To address that, we introduce a fixup for that automatically generate `pyo3-build-config*.txt` configuration files, given an `fbsource` Python version. We are tryin to stay as close to `pyo3-build-config*.txt` spec as possible even though only a single bool from that file is ever since there is validation for the other fields but also to future proof future changes. By also generating this file ourselves, we prevent PyO3 from attempting to locate a Python interpreter some other way which seems to start leaking into the PyO3 API as an implementation choice already.

## [PyPi + Rust]
* Upgrade `orjson` to [`3.10.1`](https://github.com/ijl/orjson/releases/tag/3.10.1)
  * Added `README.md` notes for future upgrades
* Upgrade `py-polars` to [`0.20.22`](https://github.com/pola-rs/polars/releases/tag/py-0.20.22)
  * Removed `py-polars` and `polars` from `target_os = "windows"`. The `third-party/pypi/polars` Python extension has only been supported for Mac and Linux for a while now so its only natural to do that on the Rust side as well. What is more, `polars-util` is bringing in `stacker = = "0.1.14"` which does not build on Windows mostly because its using a much more recent version of `libc` than we use in `third-party/rust` (see P1228807344)
* Upgrade `pydantic-core` to [`2.18.2`](https://github.com/pydantic/pydantic-core/releases/tag/v2.18.2)
  *  Removed old `third-party/pypi/pydantic-core` versions
* Upgrade `safetensors` to [`0.4.3`](https://github.com/huggingface/safetensors/releases/tag/v0.4.3)
* Patch `third-party/pypi/cryptography/41.0.7` to account for PyO3's [#2975](PyO3/pyo3#2975) (`0.19.0`) where `pyo3::once_cell` was renamed to `pyo3::sync` (see D56826865)
* Upgrade `tokenizers` and `tokenizers-python` to [`0.19.1`](https://github.com/huggingface/tokenizers/releases/tag/v0.19.1)
  * Removed old `third-party/pypi/tokenizers` versions
  * Fixed `third-party/pypi/tokenizers/BUCK`
  * Migrated `third-party/pypi/tokenizers/0.19.1/BUCK` to mirror other Python packages that bind to Rust crates e.g. `libcst`, `polars` etc.
  * Removed Windows support from `third-party/pypi/tokenizers`

## [Rust]
* Upgrade `indexmap` to [`2.2.6`](https://github.com/indexmap-rs/indexmap/releases/tag/2.2.6)
  * Both latest `pydantic-core` and `c2pa` depend on `serde_json > 1.0.112` which brings in `indexmap = 2.2.1`. The latter has deprecated `.take` and `.remove` on both `IndexMap` and `IndexSet` leading to a bunch of errors (see bellow), all addressed:

```bash
error: use of deprecated method `indexmap::set::IndexSet::<T, S>::take`: `take` disrupts the set order -- use `swap_take` or `shift_take` for explicit behavior.
  --> fbcode/hphp/hack/src/package/types.rs:76:16
   |
76 |         self.0.take(value)
   |                ^^^^
   |
   = note: `-D deprecated` implied by `-D warnings`
   = help: to override `-D warnings` add `#[allow(deprecated)]`
```

Reviewed By: capickett

Differential Revision: D56671179

fbshipit-source-id: 3ae69c069b7f005570c1a06d37194cf056282a18
facebook-github-bot pushed a commit to facebookexperimental/reverie that referenced this pull request May 3, 2024
Summary:
NOTE: If your `hg bisect` brings you here & the error you are seeing looks like ` expected Result<&PyAny, PyErr>, found Result<Bound<'_, PyAny>, PyErr>` then see these  [migration notes](https://pyo3.rs/v0.21.0/migration.html#from-020-to-021) for the fix or click on `fbcode/security/ace/pyo3/authz.rs` or similar files from bellow to see what the fix is!

In order to upgrade `pyo3` to [`0.21.1`](https://github.com/PyO3/pyo3/releases/tag/v0.21.1), the following had to take place:

## [PyO3]
* Address [migration notes](https://pyo3.rs/v0.21.0/migration.html#from-020-to-021) for `Bound<'py, T>`
* Address [#3595](PyO3/pyo3#3595) - this is done in a crude way for now since there are many call sites depending on `fbcode/dba/rust/common/service_address/py/pyo3_conversion_helper.rs` which would require a more thorough review.
* Address [#3821](PyO3/pyo3#3821) - `pyo3-build-config` is now dependent on and used by PyO3 macros. Currently, the only thing that gets checked is `abi3` compatibility. To address that, we introduce a fixup for that automatically generate `pyo3-build-config*.txt` configuration files, given an `fbsource` Python version. We are tryin to stay as close to `pyo3-build-config*.txt` spec as possible even though only a single bool from that file is ever since there is validation for the other fields but also to future proof future changes. By also generating this file ourselves, we prevent PyO3 from attempting to locate a Python interpreter some other way which seems to start leaking into the PyO3 API as an implementation choice already.

## [PyPi + Rust]
* Upgrade `orjson` to [`3.10.1`](https://github.com/ijl/orjson/releases/tag/3.10.1)
  * Added `README.md` notes for future upgrades
* Upgrade `py-polars` to [`0.20.22`](https://github.com/pola-rs/polars/releases/tag/py-0.20.22)
  * Removed `py-polars` and `polars` from `target_os = "windows"`. The `third-party/pypi/polars` Python extension has only been supported for Mac and Linux for a while now so its only natural to do that on the Rust side as well. What is more, `polars-util` is bringing in `stacker = = "0.1.14"` which does not build on Windows mostly because its using a much more recent version of `libc` than we use in `third-party/rust` (see P1228807344)
* Upgrade `pydantic-core` to [`2.18.2`](https://github.com/pydantic/pydantic-core/releases/tag/v2.18.2)
  *  Removed old `third-party/pypi/pydantic-core` versions
* Upgrade `safetensors` to [`0.4.3`](https://github.com/huggingface/safetensors/releases/tag/v0.4.3)
* Patch `third-party/pypi/cryptography/41.0.7` to account for PyO3's [#2975](PyO3/pyo3#2975) (`0.19.0`) where `pyo3::once_cell` was renamed to `pyo3::sync` (see D56826865)
* Upgrade `tokenizers` and `tokenizers-python` to [`0.19.1`](https://github.com/huggingface/tokenizers/releases/tag/v0.19.1)
  * Removed old `third-party/pypi/tokenizers` versions
  * Fixed `third-party/pypi/tokenizers/BUCK`
  * Migrated `third-party/pypi/tokenizers/0.19.1/BUCK` to mirror other Python packages that bind to Rust crates e.g. `libcst`, `polars` etc.
  * Removed Windows support from `third-party/pypi/tokenizers`

## [Rust]
* Upgrade `indexmap` to [`2.2.6`](https://github.com/indexmap-rs/indexmap/releases/tag/2.2.6)
  * Both latest `pydantic-core` and `c2pa` depend on `serde_json > 1.0.112` which brings in `indexmap = 2.2.1`. The latter has deprecated `.take` and `.remove` on both `IndexMap` and `IndexSet` leading to a bunch of errors (see bellow), all addressed:

```bash
error: use of deprecated method `indexmap::set::IndexSet::<T, S>::take`: `take` disrupts the set order -- use `swap_take` or `shift_take` for explicit behavior.
  --> fbcode/hphp/hack/src/package/types.rs:76:16
   |
76 |         self.0.take(value)
   |                ^^^^
   |
   = note: `-D deprecated` implied by `-D warnings`
   = help: to override `-D warnings` add `#[allow(deprecated)]`
```

Reviewed By: capickett

Differential Revision: D56671179

fbshipit-source-id: 3ae69c069b7f005570c1a06d37194cf056282a18
facebook-github-bot pushed a commit to facebook/hhvm that referenced this pull request May 3, 2024
Summary:
NOTE: If your `hg bisect` brings you here & the error you are seeing looks like ` expected Result<&PyAny, PyErr>, found Result<Bound<'_, PyAny>, PyErr>` then see these  [migration notes](https://pyo3.rs/v0.21.0/migration.html#from-020-to-021) for the fix or click on `fbcode/security/ace/pyo3/authz.rs` or similar files from bellow to see what the fix is!

In order to upgrade `pyo3` to [`0.21.1`](https://github.com/PyO3/pyo3/releases/tag/v0.21.1), the following had to take place:

## [PyO3]
* Address [migration notes](https://pyo3.rs/v0.21.0/migration.html#from-020-to-021) for `Bound<'py, T>`
* Address [#3595](PyO3/pyo3#3595) - this is done in a crude way for now since there are many call sites depending on `fbcode/dba/rust/common/service_address/py/pyo3_conversion_helper.rs` which would require a more thorough review.
* Address [#3821](PyO3/pyo3#3821) - `pyo3-build-config` is now dependent on and used by PyO3 macros. Currently, the only thing that gets checked is `abi3` compatibility. To address that, we introduce a fixup for that automatically generate `pyo3-build-config*.txt` configuration files, given an `fbsource` Python version. We are tryin to stay as close to `pyo3-build-config*.txt` spec as possible even though only a single bool from that file is ever since there is validation for the other fields but also to future proof future changes. By also generating this file ourselves, we prevent PyO3 from attempting to locate a Python interpreter some other way which seems to start leaking into the PyO3 API as an implementation choice already.

## [PyPi + Rust]
* Upgrade `orjson` to [`3.10.1`](https://github.com/ijl/orjson/releases/tag/3.10.1)
  * Added `README.md` notes for future upgrades
* Upgrade `py-polars` to [`0.20.22`](https://github.com/pola-rs/polars/releases/tag/py-0.20.22)
  * Removed `py-polars` and `polars` from `target_os = "windows"`. The `third-party/pypi/polars` Python extension has only been supported for Mac and Linux for a while now so its only natural to do that on the Rust side as well. What is more, `polars-util` is bringing in `stacker = = "0.1.14"` which does not build on Windows mostly because its using a much more recent version of `libc` than we use in `third-party/rust` (see P1228807344)
* Upgrade `pydantic-core` to [`2.18.2`](https://github.com/pydantic/pydantic-core/releases/tag/v2.18.2)
  *  Removed old `third-party/pypi/pydantic-core` versions
* Upgrade `safetensors` to [`0.4.3`](https://github.com/huggingface/safetensors/releases/tag/v0.4.3)
* Patch `third-party/pypi/cryptography/41.0.7` to account for PyO3's [#2975](PyO3/pyo3#2975) (`0.19.0`) where `pyo3::once_cell` was renamed to `pyo3::sync` (see D56826865)
* Upgrade `tokenizers` and `tokenizers-python` to [`0.19.1`](https://github.com/huggingface/tokenizers/releases/tag/v0.19.1)
  * Removed old `third-party/pypi/tokenizers` versions
  * Fixed `third-party/pypi/tokenizers/BUCK`
  * Migrated `third-party/pypi/tokenizers/0.19.1/BUCK` to mirror other Python packages that bind to Rust crates e.g. `libcst`, `polars` etc.
  * Removed Windows support from `third-party/pypi/tokenizers`

## [Rust]
* Upgrade `indexmap` to [`2.2.6`](https://github.com/indexmap-rs/indexmap/releases/tag/2.2.6)
  * Both latest `pydantic-core` and `c2pa` depend on `serde_json > 1.0.112` which brings in `indexmap = 2.2.1`. The latter has deprecated `.take` and `.remove` on both `IndexMap` and `IndexSet` leading to a bunch of errors (see bellow), all addressed:

```bash
error: use of deprecated method `indexmap::set::IndexSet::<T, S>::take`: `take` disrupts the set order -- use `swap_take` or `shift_take` for explicit behavior.
  --> fbcode/hphp/hack/src/package/types.rs:76:16
   |
76 |         self.0.take(value)
   |                ^^^^
   |
   = note: `-D deprecated` implied by `-D warnings`
   = help: to override `-D warnings` add `#[allow(deprecated)]`
```

Reviewed By: capickett

Differential Revision: D56671179

fbshipit-source-id: 3ae69c069b7f005570c1a06d37194cf056282a18
facebook-github-bot pushed a commit to facebook/errpy that referenced this pull request May 3, 2024
Summary:
NOTE: If your `hg bisect` brings you here & the error you are seeing looks like ` expected Result<&PyAny, PyErr>, found Result<Bound<'_, PyAny>, PyErr>` then see these  [migration notes](https://pyo3.rs/v0.21.0/migration.html#from-020-to-021) for the fix or click on `fbcode/security/ace/pyo3/authz.rs` or similar files from bellow to see what the fix is!

In order to upgrade `pyo3` to [`0.21.1`](https://github.com/PyO3/pyo3/releases/tag/v0.21.1), the following had to take place:

## [PyO3]
* Address [migration notes](https://pyo3.rs/v0.21.0/migration.html#from-020-to-021) for `Bound<'py, T>`
* Address [#3595](PyO3/pyo3#3595) - this is done in a crude way for now since there are many call sites depending on `fbcode/dba/rust/common/service_address/py/pyo3_conversion_helper.rs` which would require a more thorough review.
* Address [#3821](PyO3/pyo3#3821) - `pyo3-build-config` is now dependent on and used by PyO3 macros. Currently, the only thing that gets checked is `abi3` compatibility. To address that, we introduce a fixup for that automatically generate `pyo3-build-config*.txt` configuration files, given an `fbsource` Python version. We are tryin to stay as close to `pyo3-build-config*.txt` spec as possible even though only a single bool from that file is ever since there is validation for the other fields but also to future proof future changes. By also generating this file ourselves, we prevent PyO3 from attempting to locate a Python interpreter some other way which seems to start leaking into the PyO3 API as an implementation choice already.

## [PyPi + Rust]
* Upgrade `orjson` to [`3.10.1`](https://github.com/ijl/orjson/releases/tag/3.10.1)
  * Added `README.md` notes for future upgrades
* Upgrade `py-polars` to [`0.20.22`](https://github.com/pola-rs/polars/releases/tag/py-0.20.22)
  * Removed `py-polars` and `polars` from `target_os = "windows"`. The `third-party/pypi/polars` Python extension has only been supported for Mac and Linux for a while now so its only natural to do that on the Rust side as well. What is more, `polars-util` is bringing in `stacker = = "0.1.14"` which does not build on Windows mostly because its using a much more recent version of `libc` than we use in `third-party/rust` (see P1228807344)
* Upgrade `pydantic-core` to [`2.18.2`](https://github.com/pydantic/pydantic-core/releases/tag/v2.18.2)
  *  Removed old `third-party/pypi/pydantic-core` versions
* Upgrade `safetensors` to [`0.4.3`](https://github.com/huggingface/safetensors/releases/tag/v0.4.3)
* Patch `third-party/pypi/cryptography/41.0.7` to account for PyO3's [#2975](PyO3/pyo3#2975) (`0.19.0`) where `pyo3::once_cell` was renamed to `pyo3::sync` (see D56826865)
* Upgrade `tokenizers` and `tokenizers-python` to [`0.19.1`](https://github.com/huggingface/tokenizers/releases/tag/v0.19.1)
  * Removed old `third-party/pypi/tokenizers` versions
  * Fixed `third-party/pypi/tokenizers/BUCK`
  * Migrated `third-party/pypi/tokenizers/0.19.1/BUCK` to mirror other Python packages that bind to Rust crates e.g. `libcst`, `polars` etc.
  * Removed Windows support from `third-party/pypi/tokenizers`

## [Rust]
* Upgrade `indexmap` to [`2.2.6`](https://github.com/indexmap-rs/indexmap/releases/tag/2.2.6)
  * Both latest `pydantic-core` and `c2pa` depend on `serde_json > 1.0.112` which brings in `indexmap = 2.2.1`. The latter has deprecated `.take` and `.remove` on both `IndexMap` and `IndexSet` leading to a bunch of errors (see bellow), all addressed:

```bash
error: use of deprecated method `indexmap::set::IndexSet::<T, S>::take`: `take` disrupts the set order -- use `swap_take` or `shift_take` for explicit behavior.
  --> fbcode/hphp/hack/src/package/types.rs:76:16
   |
76 |         self.0.take(value)
   |                ^^^^
   |
   = note: `-D deprecated` implied by `-D warnings`
   = help: to override `-D warnings` add `#[allow(deprecated)]`
```

Reviewed By: capickett

Differential Revision: D56671179

fbshipit-source-id: 3ae69c069b7f005570c1a06d37194cf056282a18
facebook-github-bot pushed a commit to facebook/dotslash that referenced this pull request May 3, 2024
Summary:
NOTE: If your `hg bisect` brings you here & the error you are seeing looks like ` expected Result<&PyAny, PyErr>, found Result<Bound<'_, PyAny>, PyErr>` then see these  [migration notes](https://pyo3.rs/v0.21.0/migration.html#from-020-to-021) for the fix or click on `fbcode/security/ace/pyo3/authz.rs` or similar files from bellow to see what the fix is!

In order to upgrade `pyo3` to [`0.21.1`](https://github.com/PyO3/pyo3/releases/tag/v0.21.1), the following had to take place:

## [PyO3]
* Address [migration notes](https://pyo3.rs/v0.21.0/migration.html#from-020-to-021) for `Bound<'py, T>`
* Address [#3595](PyO3/pyo3#3595) - this is done in a crude way for now since there are many call sites depending on `fbcode/dba/rust/common/service_address/py/pyo3_conversion_helper.rs` which would require a more thorough review.
* Address [#3821](PyO3/pyo3#3821) - `pyo3-build-config` is now dependent on and used by PyO3 macros. Currently, the only thing that gets checked is `abi3` compatibility. To address that, we introduce a fixup for that automatically generate `pyo3-build-config*.txt` configuration files, given an `fbsource` Python version. We are tryin to stay as close to `pyo3-build-config*.txt` spec as possible even though only a single bool from that file is ever since there is validation for the other fields but also to future proof future changes. By also generating this file ourselves, we prevent PyO3 from attempting to locate a Python interpreter some other way which seems to start leaking into the PyO3 API as an implementation choice already.

## [PyPi + Rust]
* Upgrade `orjson` to [`3.10.1`](https://github.com/ijl/orjson/releases/tag/3.10.1)
  * Added `README.md` notes for future upgrades
* Upgrade `py-polars` to [`0.20.22`](https://github.com/pola-rs/polars/releases/tag/py-0.20.22)
  * Removed `py-polars` and `polars` from `target_os = "windows"`. The `third-party/pypi/polars` Python extension has only been supported for Mac and Linux for a while now so its only natural to do that on the Rust side as well. What is more, `polars-util` is bringing in `stacker = = "0.1.14"` which does not build on Windows mostly because its using a much more recent version of `libc` than we use in `third-party/rust` (see P1228807344)
* Upgrade `pydantic-core` to [`2.18.2`](https://github.com/pydantic/pydantic-core/releases/tag/v2.18.2)
  *  Removed old `third-party/pypi/pydantic-core` versions
* Upgrade `safetensors` to [`0.4.3`](https://github.com/huggingface/safetensors/releases/tag/v0.4.3)
* Patch `third-party/pypi/cryptography/41.0.7` to account for PyO3's [#2975](PyO3/pyo3#2975) (`0.19.0`) where `pyo3::once_cell` was renamed to `pyo3::sync` (see D56826865)
* Upgrade `tokenizers` and `tokenizers-python` to [`0.19.1`](https://github.com/huggingface/tokenizers/releases/tag/v0.19.1)
  * Removed old `third-party/pypi/tokenizers` versions
  * Fixed `third-party/pypi/tokenizers/BUCK`
  * Migrated `third-party/pypi/tokenizers/0.19.1/BUCK` to mirror other Python packages that bind to Rust crates e.g. `libcst`, `polars` etc.
  * Removed Windows support from `third-party/pypi/tokenizers`

## [Rust]
* Upgrade `indexmap` to [`2.2.6`](https://github.com/indexmap-rs/indexmap/releases/tag/2.2.6)
  * Both latest `pydantic-core` and `c2pa` depend on `serde_json > 1.0.112` which brings in `indexmap = 2.2.1`. The latter has deprecated `.take` and `.remove` on both `IndexMap` and `IndexSet` leading to a bunch of errors (see bellow), all addressed:

```bash
error: use of deprecated method `indexmap::set::IndexSet::<T, S>::take`: `take` disrupts the set order -- use `swap_take` or `shift_take` for explicit behavior.
  --> fbcode/hphp/hack/src/package/types.rs:76:16
   |
76 |         self.0.take(value)
   |                ^^^^
   |
   = note: `-D deprecated` implied by `-D warnings`
   = help: to override `-D warnings` add `#[allow(deprecated)]`
```

Reviewed By: capickett

Differential Revision: D56671179

fbshipit-source-id: 3ae69c069b7f005570c1a06d37194cf056282a18
facebook-github-bot pushed a commit to facebook/fbthrift that referenced this pull request May 3, 2024
Summary:
NOTE: If your `hg bisect` brings you here & the error you are seeing looks like ` expected Result<&PyAny, PyErr>, found Result<Bound<'_, PyAny>, PyErr>` then see these  [migration notes](https://pyo3.rs/v0.21.0/migration.html#from-020-to-021) for the fix or click on `fbcode/security/ace/pyo3/authz.rs` or similar files from bellow to see what the fix is!

In order to upgrade `pyo3` to [`0.21.1`](https://github.com/PyO3/pyo3/releases/tag/v0.21.1), the following had to take place:

## [PyO3]
* Address [migration notes](https://pyo3.rs/v0.21.0/migration.html#from-020-to-021) for `Bound<'py, T>`
* Address [#3595](PyO3/pyo3#3595) - this is done in a crude way for now since there are many call sites depending on `fbcode/dba/rust/common/service_address/py/pyo3_conversion_helper.rs` which would require a more thorough review.
* Address [#3821](PyO3/pyo3#3821) - `pyo3-build-config` is now dependent on and used by PyO3 macros. Currently, the only thing that gets checked is `abi3` compatibility. To address that, we introduce a fixup for that automatically generate `pyo3-build-config*.txt` configuration files, given an `fbsource` Python version. We are tryin to stay as close to `pyo3-build-config*.txt` spec as possible even though only a single bool from that file is ever since there is validation for the other fields but also to future proof future changes. By also generating this file ourselves, we prevent PyO3 from attempting to locate a Python interpreter some other way which seems to start leaking into the PyO3 API as an implementation choice already.

## [PyPi + Rust]
* Upgrade `orjson` to [`3.10.1`](https://github.com/ijl/orjson/releases/tag/3.10.1)
  * Added `README.md` notes for future upgrades
* Upgrade `py-polars` to [`0.20.22`](https://github.com/pola-rs/polars/releases/tag/py-0.20.22)
  * Removed `py-polars` and `polars` from `target_os = "windows"`. The `third-party/pypi/polars` Python extension has only been supported for Mac and Linux for a while now so its only natural to do that on the Rust side as well. What is more, `polars-util` is bringing in `stacker = = "0.1.14"` which does not build on Windows mostly because its using a much more recent version of `libc` than we use in `third-party/rust` (see P1228807344)
* Upgrade `pydantic-core` to [`2.18.2`](https://github.com/pydantic/pydantic-core/releases/tag/v2.18.2)
  *  Removed old `third-party/pypi/pydantic-core` versions
* Upgrade `safetensors` to [`0.4.3`](https://github.com/huggingface/safetensors/releases/tag/v0.4.3)
* Patch `third-party/pypi/cryptography/41.0.7` to account for PyO3's [#2975](PyO3/pyo3#2975) (`0.19.0`) where `pyo3::once_cell` was renamed to `pyo3::sync` (see D56826865)
* Upgrade `tokenizers` and `tokenizers-python` to [`0.19.1`](https://github.com/huggingface/tokenizers/releases/tag/v0.19.1)
  * Removed old `third-party/pypi/tokenizers` versions
  * Fixed `third-party/pypi/tokenizers/BUCK`
  * Migrated `third-party/pypi/tokenizers/0.19.1/BUCK` to mirror other Python packages that bind to Rust crates e.g. `libcst`, `polars` etc.
  * Removed Windows support from `third-party/pypi/tokenizers`

## [Rust]
* Upgrade `indexmap` to [`2.2.6`](https://github.com/indexmap-rs/indexmap/releases/tag/2.2.6)
  * Both latest `pydantic-core` and `c2pa` depend on `serde_json > 1.0.112` which brings in `indexmap = 2.2.1`. The latter has deprecated `.take` and `.remove` on both `IndexMap` and `IndexSet` leading to a bunch of errors (see bellow), all addressed:

```bash
error: use of deprecated method `indexmap::set::IndexSet::<T, S>::take`: `take` disrupts the set order -- use `swap_take` or `shift_take` for explicit behavior.
  --> fbcode/hphp/hack/src/package/types.rs:76:16
   |
76 |         self.0.take(value)
   |                ^^^^
   |
   = note: `-D deprecated` implied by `-D warnings`
   = help: to override `-D warnings` add `#[allow(deprecated)]`
```

Reviewed By: capickett

Differential Revision: D56671179

fbshipit-source-id: 3ae69c069b7f005570c1a06d37194cf056282a18
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
CI-skip-changelog Skip checking changelog entry
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants