Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merge 'upstream/master' (76b7f53) #67

Closed

Conversation

EricCousineau-TRI
Copy link
Collaborator

@EricCousineau-TRI EricCousineau-TRI commented Nov 2, 2023

This change is Reviewable

Ralf W. Grosse-Kunstleve and others added 30 commits July 14, 2023 14:53
* Boilerplate version number update after v2.11.0 release.

* Small updates to release.rst
…ind#4753)

* Remove GIL checks

* Update common.h

* Add flag

* style: pre-commit fixes

* Update pytypes.h

* style: pre-commit fixes

* Update common.h

* style: pre-commit fixes

* Update pytypes.h

* style: pre-commit fixes

* Update common.h

* style: pre-commit fixes

---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
…cy. (pybind#4757)

* Add command to check for vX.Y.Z tag vs pybind11/_version.py consistency. Piggy-backing hints for converting changelog to release message.

* Simpler way of double-checking version number in sources vs. git tag

* Even simpler.

* Fix rst rendering (hopefully) and remove stray dots.

* [ci skip] Trying more to make GitHub rst renderer show this nicely.

* [ci skip] Fix up RST rendering issues.

Lesson learned:

This is NOT GOOD:

```
- Bullet nesting level 1.
    - Bullet nesting level 2.
```

This is BETTER:

```
- Bullet nesting level 1.
  - Bullet nesting level 2.
```

Also consistently adding empty lines between bullet points, to make the .rst
file easier to read.

Also piggy-backing a few very minor enhancements.
* Copy clang 17 compatibility fixes from PR pybind#4762 to a separate PR.

* Add gcc:13 C++20

* Add silkeh/clang:16-bullseye C++20
updates:
- [github.com/psf/black: 23.3.0 → 23.7.0](psf/black@23.3.0...23.7.0)
- [github.com/astral-sh/ruff-pre-commit: v0.0.276 → v0.0.281](astral-sh/ruff-pre-commit@v0.0.276...v0.0.281)
- [github.com/asottile/blacken-docs: 1.14.0 → 1.15.0](adamchainz/blacken-docs@1.14.0...1.15.0)

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
…tuptools (pybind#4774)

* docs: Remove upper bound on pybind11 in example pyproject.toml for setuptools

* Update docs/compiling.rst

---------

Co-authored-by: Henry Schreiner <[email protected]>
* Provide better type hints for a variety of generic types

* Makes better documentation
* tuple, dict, list, set, function

* Move to py::typing

* style: pre-commit fixes

* Update copyright line with correct year and actual author. The author information was copy-pasted from the git log output.

---------

Co-authored-by: Ralf W. Grosse-Kunstleve <[email protected]>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Update from CentOS to Ubuntu and to a recent version
of NVHPC (former: PGI).

Co-authored-by: Ralf W. Grosse-Kunstleve <[email protected]>
…pybind#4772)

* Copy clang 17 compatibility fixes from PR pybind#4762 to a separate PR.

* static py::exception<> -> static py::handle

* Add `py::set_error()` but also try the suggestion of @malfet (pytorch/pytorch#106401 (review)).

* clang 17 compatibility fixes (pybind#4767)

* Copy clang 17 compatibility fixes from PR pybind#4762 to a separate PR.

* Add gcc:13 C++20

* Add silkeh/clang:16-bullseye C++20

* chore(deps): update pre-commit hooks (pybind#4770)

updates:
- [github.com/psf/black: 23.3.0 → 23.7.0](psf/black@23.3.0...23.7.0)
- [github.com/astral-sh/ruff-pre-commit: v0.0.276 → v0.0.281](astral-sh/ruff-pre-commit@v0.0.276...v0.0.281)
- [github.com/asottile/blacken-docs: 1.14.0 → 1.15.0](adamchainz/blacken-docs@1.14.0...1.15.0)

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>

* docs: Remove upper bound on pybind11 in example pyproject.toml for setuptools (pybind#4774)

* docs: Remove upper bound on pybind11 in example pyproject.toml for setuptools

* Update docs/compiling.rst

---------

Co-authored-by: Henry Schreiner <[email protected]>

* Provide better type hints for a variety of generic types (pybind#4259)

* Provide better type hints for a variety of generic types

* Makes better documentation
* tuple, dict, list, set, function

* Move to py::typing

* style: pre-commit fixes

* Update copyright line with correct year and actual author. The author information was copy-pasted from the git log output.

---------

Co-authored-by: Ralf W. Grosse-Kunstleve <[email protected]>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>

* Use `py::set_error()` everywhere possible (only one special case, in common.h).
Overload `py::set_error(py::handle, py::handle)`.
Change back to `static py::handle exc = ... .release();`
Deprecate `py::exception<>::operator()`

* Add `PYBIND11_WARNING_DISABLE` for INTEL and MSVC (and sort alphabetically).

* `PYBIND11_WARNING_DISABLE_INTEL(10441)` does not work.

For ICC only, falling back to the recommended `py::set_error()` to keep the testing simple.

It is troublesome to add `--diag-disable=10441` specifically for test_exceptions.cpp, even that is non-ideal because it covers the entire file, not just the one line we need it for, and the value of exercising the trivial deprecated `operator()` on this one extra platform is practically zero.

* Fix silly oversight.

* NVHPC 23.5.0 generates deprecation warnings. They are currently not treated as errors, but falling back to using `py::set_error()` to not have to deal with that distraction.

---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Keto D. Zhang <[email protected]>
Co-authored-by: Henry Schreiner <[email protected]>
Co-authored-by: Dustin Spicuzza <[email protected]>
* Copy of recommded readthedocs configuration file v2

* [ci skip] It is now requirements (not requirements_file)
The previous sentence had an extra "a" before "several", which isn't right.
* Remove newlines from docstring signature

* Jean/dev (#1)

Replace newlines in arg values with spaces

* style: pre-commit fixes

* Don't use std::find_if for C++ 11 compatibility

* Avoid implicit char to bool conversion

* Test default arguments for line breaks

* style: pre-commit fixes

* Separate Eigen tests

* style: pre-commit fixes

* Fix merge

* Try importing numpy

* Avoid unreferenced variable in catch block

* style: pre-commit fixes

* Update squash function

* Reduce try block

* Additional test cases

* style: pre-commit fixes

* Put statement inside braces

* Move string into function body

* Rename repr for better readability. Make constr explicit.

* Add multiline string default argument test case

* style: pre-commit fixes

* Add std namespace, do not modify string repr

* Test for all space chars, test str repr not modified

---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* fix: Enum __str__ function name

* tests: Test enum.__str__.__doc__
* fix: Render `py::function` as `Callable`

* style: pre-commit fixes

---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* Change variable name `it` to `curr_overl` to improve readability, add `assert()` in the only place where `curr_overl` could become `nullptr`.

* Move the `assert()` to where Coverty generates a false positive.

* variable name as suggested by @henryiii
* chore(deps): update pre-commit hooks

updates:
- [github.com/astral-sh/ruff-pre-commit: v0.0.281 → v0.0.287](astral-sh/ruff-pre-commit@v0.0.281...v0.0.287)
- [github.com/pre-commit/mirrors-mypy: v1.4.1 → v1.5.1](pre-commit/mirrors-mypy@v1.4.1...v1.5.1)
- [github.com/asottile/blacken-docs: 1.15.0 → 1.16.0](adamchainz/blacken-docs@1.15.0...1.16.0)
- [github.com/Lucas-C/pre-commit-hooks: v1.5.1 → v1.5.4](Lucas-C/pre-commit-hooks@v1.5.1...v1.5.4)
- [github.com/PyCQA/pylint: v3.0.0a6 → v3.0.0a7](pylint-dev/pylint@v3.0.0a6...v3.0.0a7)

* style: pre-commit fixes

* Update pyproject.toml

---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Henry Schreiner <[email protected]>
* chore(deps): bump actions/checkout from 1 to 4

Bumps [actions/checkout](https://github.com/actions/checkout) from 1 to 4.
- [Release notes](https://github.com/actions/checkout/releases)
- [Changelog](https://github.com/actions/checkout/blob/main/CHANGELOG.md)
- [Commits](actions/checkout@v1...v4)

---
updated-dependencies:
- dependency-name: actions/checkout
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <[email protected]>

* Update .github/workflows/ci.yml

* actions/checkout@v1 for centos:7

* Fix oversight: centos:7 actually works with actions/checkout@v3

---------

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Henry Schreiner <[email protected]>
Co-authored-by: Ralf W. Grosse-Kunstleve <[email protected]>
…4834)

* feature: Support move-only iterators in `py::make_*iterator`

* fix: Missing static assertion message

* fixup: Missing `explicit` in single argument constructors

* fix: Simplify tests: make existing iterator move-only

* fix: Missing `noexcept`
* fix: Add capitalize render name of `py::buffer` and `py::sequence`

* fix: Render `py::handle` same way as `py::object`

* tests: Fix tests `handle` -> `object`

* tests: Test capitaliation of `py::sequence` and `py::buffer`

* style: pre-commit fixes

* fix: Render `py::object` as `Any`

* Revert "fix: Render `py::object` as `Any`"

This reverts commit 7861dcf.

---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Ralf W. Grosse-Kunstleve <[email protected]>
* Bug fix: `result[0]` called if `result.empty()`

* Add unit test that fails without the fix.
henryiii and others added 21 commits September 15, 2023 17:59
* cmake: split extension

Into suffix and debug postfix. Pybind11 is currently treating both as
suffix, which is problematic when something else defines the
DEBUG_POSTFIX because they will be concatenated.

pybind11_extension sets SUFFIX to _d.something and if DEBUG_POSTFIX is
set to _d.

    _d + _d.something = _d_d.something

The issue has been reported at:

pybind#4699

* style: pre-commit fixes

* fix(cmake): support postfix for old FindPythonInterp mode too

Signed-off-by: Henry Schreiner <[email protected]>

---------

Signed-off-by: Henry Schreiner <[email protected]>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Henry Schreiner <[email protected]>
This change is fixing a Coverity AUTO_CAUSES_COPY issues.
* MAINT: Include numpy._core imports

* style: pre-commit fixes

* Apply review comments

* style: pre-commit fixes

* Add no-inline attribute

* Select submodule name based on numpy version

* style: pre-commit fixes

* Update pre-commit check

* Add error_already_set and simplify if statement

* Update .pre-commit-config.yaml

Co-authored-by: Ralf W. Grosse-Kunstleve <[email protected]>

---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Ralf W. Grosse-Kunstleve <[email protected]>
* chore(deps): update pre-commit hooks

updates:
- [github.com/psf/black-pre-commit-mirror: 23.7.0 → 23.9.1](psf/black-pre-commit-mirror@23.7.0...23.9.1)
- [github.com/astral-sh/ruff-pre-commit: v0.0.287 → v0.0.292](astral-sh/ruff-pre-commit@v0.0.287...v0.0.292)
- [github.com/codespell-project/codespell: v2.2.5 → v2.2.6](codespell-project/codespell@v2.2.5...v2.2.6)
- [github.com/shellcheck-py/shellcheck-py: v0.9.0.5 → v0.9.0.6](shellcheck-py/shellcheck-py@v0.9.0.5...v0.9.0.6)
- [github.com/PyCQA/pylint: v3.0.0a7 → v3.0.0](pylint-dev/pylint@v3.0.0a7...v3.0.0)

* Update .pre-commit-config.yaml

* style: pre-commit fixes

* Update .pre-commit-config.yaml

---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Henry Schreiner <[email protected]>
…#4875)

Bumps [seanmiddleditch/gha-setup-ninja](https://github.com/seanmiddleditch/gha-setup-ninja) from 3 to 4.
- [Release notes](https://github.com/seanmiddleditch/gha-setup-ninja/releases)
- [Commits](seanmiddleditch/gha-setup-ninja@v3...v4)

---
updated-dependencies:
- dependency-name: seanmiddleditch/gha-setup-ninja
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
….h) (pybind#4877)

* LazyInitializeAtLeastOnceDestroyNever v1

* Go back to using `union` as originally suggested by jbms@. The trick (also suggested by jbms@) is to add empty ctor + dtor.

* Revert "Go back to using `union` as originally suggested by jbms@. The trick (also suggested by jbms@) is to add empty ctor + dtor."

This reverts commit e7b8c4f.

* Remove `#include <stdalign.h>`

* `include\pybind11/numpy.h(24,10): fatal error C1083: Cannot open include file: 'stdalign.h': No such file or directory`

* @tkoeppe wrote: this is a C interop header (and we're not writing C)

* Suppress gcc 4.8.5 (CentOS 7) warning.

```
include/pybind11/eigen/../numpy.h:63:53: error: dereferencing type-punned pointer will break strict-aliasing rules [-Werror=strict-aliasing]
         return *reinterpret_cast<T *>(value_storage_);
                                                     ^
```

* Replace comments:

Document PRECONDITION.

Adopt comment suggested by @tkoeppe: pybind#4877 (comment)

* Adopt suggestion by @tkoeppe:

* pybind#4877 (comment)

* https://godbolt.org/z/Wa79nKz6e

* Add `PYBIND11_CONSTINIT`, but it does not work for the current use cases:

```
g++ -o pybind11/tests/test_numpy_array.os -c -std=c++20 -fPIC -fvisibility=hidden -O0 -g -Wall -Wextra -Wconversion -Wcast-qual -Wdeprecated -Wundef -Wnon-virtual-dtor -Wunused-result -Werror -isystem /usr/include/python3.11 -isystem /usr/include/eigen3 -DPYBIND11_STRICT_ASSERTS_CLASS_HOLDER_VS_TYPE_CASTER_MIX -DPYBIND11_ENABLE_TYPE_CASTER_ODR_GUARD_IF_AVAILABLE -DPYBIND11_TEST_BOOST -Ipybind11/include -I/usr/local/google/home/rwgk/forked/pybind11/include -I/usr/local/google/home/rwgk/clone/pybind11/include /usr/local/google/home/rwgk/forked/pybind11/tests/test_numpy_array.cpp
```

```
In file included from /usr/local/google/home/rwgk/forked/pybind11/tests/test_numpy_array.cpp:10:
/usr/local/google/home/rwgk/forked/pybind11/include/pybind11/numpy.h: In static member function ‘static pybind11::detail::npy_api& pybind11::detail::npy_api::get()’:
/usr/local/google/home/rwgk/forked/pybind11/include/pybind11/numpy.h:258:82: error: ‘constinit’ variable ‘api_init’ does not have a constant initializer
  258 |         PYBIND11_CONSTINIT static LazyInitializeAtLeastOnceDestroyNever<npy_api> api_init;
      |                                                                                  ^~~~~~~~
```

```
In file included from /usr/local/google/home/rwgk/forked/pybind11/tests/test_numpy_array.cpp:10:
/usr/local/google/home/rwgk/forked/pybind11/include/pybind11/numpy.h: In static member function ‘static pybind11::object& pybind11::dtype::_dtype_from_pep3118()’:
/usr/local/google/home/rwgk/forked/pybind11/include/pybind11/numpy.h:697:13: error: ‘constinit’ variable ‘imported_obj’ does not have a constant initializer
  697 |             imported_obj;
      |             ^~~~~~~~~~~~
```

* Revert "Add `PYBIND11_CONSTINIT`, but it does not work for the current use cases:"

This reverts commit f07b28b.

* Reapply "Add `PYBIND11_CONSTINIT`, but it does not work for the current use cases:"

This reverts commit 36be645.

* Add Default Member Initializer on `value_storage_` as suggested by @tkoeppe:

pybind#4877 (comment)

This fixes the errors reported under commit f07b28b.

* Fix copy-paste-missed-a-change mishap in commit 88cec11.

* Semi-paranoid placement new (based on pybind#4877 (comment)).

* Move PYBIND11_CONSTINIT to detail/common.h

* Move code to the right places, rename new class and some variables.

* Fix oversight: update tests/extra_python_package/test_files.py

* Get the name right first.

* Use `std::call_once`, `std::atomic`, following a pattern developed by @tkoeppe

* Make the API more self-documenting (and possibly more easily reusable).

* google-clang-tidy IWYU fixes

* Rewrite comment as suggested by @tkoeppe

* Update test_exceptions.cpp and exceptions.rst

* Fix oversight in previous commit: add `PYBIND11_CONSTINIT`

* Make `get_stored()` non-const for simplicity.

As suggested by @tkoeppe: not seeing any reasonable use in which `get_stored` has to be const.

* Add comment regarding `KeyboardInterrupt` behavior, based heavily on information provided by @jbms.

* Add `assert(PyGILState_Check())` in `gil_scoped_release` ctor (simple & non-simple implementation) as suggested by @EthanSteinberg.

* Fix oversight in previous commit (missing include cassert).

* Remove use of std::atomic, leaving comments with rationale, why it is not needed.

* Rewrite comment re `std:optional` based on deeper reflection (aka 2nd thoughts).

* Additional comment with the conclusion of a discussion under PR pybind#4877.

* pybind#4877 (comment)

* Small comment changes suggested by @tkoeppe.
…pybind#4880)

* Update pytest (which removes their dependency on py)

The py library through 1.11.0 for Python allows remote attackers to conduct a ReDoS (Regular expression Denial of Service) attack via a Subversion repository with crafted info data, because the InfoSvnCommand argument is mishandled.

The particular codepath in question is the regular expression at py._path.svnurl.InfoSvnCommand.lspattern and is only relevant when dealing with subversion (svn) projects. Notably the codepath is not used in the popular pytest project. The developers of the pytest package have released version 7.2.0 which removes their dependency on py. Users of pytest seeing alerts relating to this advisory may update to version 7.2.0 of pytest to resolve this issue. See pytest-dev/py#287 (comment) for additional context.

* Added conditions so that we keep using 7.0.0 on python 3.6
* Workaround NVCC parse failure in `cast_op`

There is a bug in some CUDA versions (observed in CUDA 12.1 and 11.7 w/ GCC 12.2),
that makes `cast_op` fail to compile:
  `cast.h:45:120: error: expected template-name before ‘<’ token`

Defining the nested type as an alias and using it allows this to work
without any change in semantics.

Fixes pybind#4606

* style: pre-commit fixes

* Add comments to result_t referencing PR

---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
According to https://docs.python.org/3.13/whatsnew/3.13.html:

Add PyThreadState_GetUnchecked() function: similar to
PyThreadState_Get(), but don't kill the process with a fatal error if
it is NULL. The caller is responsible to check if the result is
NULL. Previously, the function was private and known as
_PyThreadState_UncheckedGet().
* [minor] Add a missing space

Add a missing space to error message

* Add space after period, always add newline.
Add 3.12 to the default `Python_ADDITIONAL_VERSIONS`.
# Conflicts:
#	.github/workflows/ci.yml
#	.github/workflows/format.yml
#	.github/workflows/pip.yml
#	docs/changelog.rst
#	include/pybind11/detail/common.h
#	include/pybind11/pytypes.h
#	pybind11/_version.py
Addresses compilation error for C++11
Copy link
Collaborator Author

@EricCousineau-TRI EricCousineau-TRI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@jwnimmer-tri I plan to YOLO merge this in too.
You OK with that?

Reviewable status: 0 of 57 files reviewed, all discussions resolved

@jwnimmer-tri
Copy link

I saw your other question, and I'm still trying to decide what is best.

For sure we always want Drake's repository external to be pinned to the tip of this repo's drake branch.

The question is, would we prefer to keep this repo merged up only to stable upstream releases, or merged up to bleeding-edge commits? I'm on the fence about that one.

I don't think CI here is relevant at all. Drake CI is the only CI that matters. So the question is really about do we want to ship (our patched version of) pybind11 upstream stable releases, or (our patch version of) pybind11 unstable master branch?

@jwnimmer-tri
Copy link

So the question is really about do we want to ship (our patched version of) pybind11 upstream stable releases, or (our patch version of) pybind11 unstable master branch?

And for that, I'll defer to your best guess. Generally I would lean to keeping with numbered releases (plus maybe some cherry-picks), but since we already have a bunch of local patches, possibly it's a moot point.

Copy link
Collaborator Author

@EricCousineau-TRI EricCousineau-TRI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

My preference for bleeding edge is really just to appease CI, and I see your point that this CI is maybe useful, but Drake CI is the one that matters.

I'm good w/ just merging in stable versions, and otherwise cherry pick what we need.

Will go ahead and close this out.

Reviewable status: 0 of 57 files reviewed, all discussions resolved

@EricCousineau-TRI
Copy link
Collaborator Author

Closing this branch, pending release of stable version.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.