WIP: Dummy PR to check maint-15.0.0 status #39545
Closed
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
### Rationale for this change We sometimes need to use a more modern cmake, before this change although we downloaded a functioning cmake on macos, we didn't have the correct path for it. ### What changes are included in this PR? Resolves #38811 so that cmake is useable when downloaded on macos. This also restores the local source build jobs to be testing that source builds work (which is what the Ci jobs say they are doing). I believe these jobs started using binaries when we overhauled the build system last release. ### Are these changes tested? Yes, in CI with the local (source) install jobs in crossbow) ### Are there any user-facing changes? * Closes: #38811 Authored-by: Jonathan Keane <[email protected]> Signed-off-by: Jacob Wujciak-Jens <[email protected]>
…CI build (#39498) Update version checks and assertions of pyarrow array equality for pandas failing tests on the CI: [test-conda-python-3.10-pandas-nightly](https://github.com/ursacomputing/crossbow/actions/runs/7391976015/job/20109720695) * Closes: #39437 Lead-authored-by: AlenkaF <[email protected]> Co-authored-by: Alenka Frim <[email protected]> Co-authored-by: Joris Van den Bossche <[email protected]> Signed-off-by: Joris Van den Bossche <[email protected]>
…9522) ### Rationale for this change With CMake > 3.28 the generated Makefile fails on the jemalloc_ep due to 'bad file descriptor'. ### What changes are included in this PR? For a sequential build for jemalloc by setting -j1. ### Are these changes tested? CI ### Are there any user-facing changes? No. * Closes: #39517 Authored-by: Jacob Wujciak-Jens <[email protected]> Signed-off-by: Jacob Wujciak-Jens <[email protected]>
### Rationale for this change The CRAN check on `fedora clang devel` builds with clang against libc++ and has a system re2 installed that was build with C++11 ABI which causes linking to fail due to the [abi:cxx11]-symbol annotation on the system version. A user could manually use the bundled build or path hint a clang version of the library. To avoid extra work for the CRAN maintainers we can just default to the bundled build. The re2 build is small enough that users building from source will not really feel the difference and can still opt to use the system re2 via `EXTRA_CMAKE_FLAGS`. ### What changes are included in this PR? Default to use our bundled build to prevent the problems. ### Are these changes tested? On a local dev container replicating the cran env. ### Are there any user-facing changes? Source build now default to use the bundled re2 version, this can be overridden. Authored-by: Jacob Wujciak-Jens <[email protected]> Signed-off-by: Jacob Wujciak-Jens <[email protected]>
…nd test_view (#39534) Skipping dask tests `test_dataframe.py::test_describe_empty` and `test_dataframe.py::test_view` on our CI to stop the nightly dask test jobs to fail. * Closes: #39531 Authored-by: AlenkaF <[email protected]> Signed-off-by: AlenkaF <[email protected]>
…ements for the 15.x release branch (#39538) ### Rationale for this change PyArrow wheels for the 15.0.0 release will not be compatible with future numpy 2.0 packages, therefore it is recommended to add this upper pin now for _releases_. We will keep the more flexible pin on the development branch (by reverting this commit on main, but so it can be cherry-picked in the release branch) * Closes: #39537 Authored-by: Joris Van den Bossche <[email protected]> Signed-off-by: Joris Van den Bossche <[email protected]>
### Rationale for this change Removing usage of `np.core`, as that is deprecated and will be removed in numpy 2.0. For this specific case, we can just hardcode the list of data types instead of using a numpy api (this list doesn't typically change). * Closes: #39533 Authored-by: Joris Van den Bossche <[email protected]> Signed-off-by: Joris Van den Bossche <[email protected]>
raulcd
requested review from
assignUser,
kou,
paleolimbot and
thisisnic
as code owners
github-actions
bot
added
Component: R
Component: C++
Component: Python
awaiting committer review
This comment was marked as outdated.
This comment was marked as outdated.
This comment was marked as outdated.
This comment was marked as outdated.
|
@ursabot please benchmark |
|
Benchmark runs are scheduled for commit b5d26f8. Watch https://buildkite.com/apache-arrow and https://conbench.ursa.dev for updates. A comment will be posted here when the runs are complete. |
|
Thanks for your patience. Conbench analyzed the 6 benchmarking runs that have been run so far on PR commit b5d26f8. There were 2 benchmark results indicating a performance regression:
The full Conbench report has more details. |
### Rationale for this change For Iceberg we want to add metadata type the type (the field-id), therefore we need to pass in the type analog to what we do for `ListArray.from_arrays(self, offsets, values, DataType type=None, MemoryPool pool=None, mask=None)`. ### What changes are included in this PR? Updated a keyword argument for the `type`, and make sure that the the static method to create the MapType is exposed from the cpp side. ### Are these changes tested? I've added a simple test. ### Are there any user-facing changes? * Closes: #39515 Authored-by: Fokko Driesprong <[email protected]> Signed-off-by: AlenkaF <[email protected]>
… integration tests (#39502) Integration verification tasks are currently failing on CI. Install jpype and build JNI c-data to run integration tests Yes via archery No * Closes: #38470 Lead-authored-by: Raúl Cumplido <[email protected]> Co-authored-by: Sutou Kouhei <[email protected]> Signed-off-by: Sutou Kouhei <[email protected]>
|
Revision: dffef2c Submitted crossbow builds: ursacomputing/crossbow @ maint-15.0.0-java-jars-0
|
This comment was marked as outdated.
This comment was marked as outdated.
### Rationale for this change The version set currently on the maintenance branch is incorrect for Java BOM. ### What changes are included in this PR? Suggested changes to set specifically version for BOM and maven. ### Are these changes tested? I will trigger java-jars via archery but I think this is currently only reproducible on the maintenance branch. So we will have to merge and validate there. ### Are there any user-facing changes? No * Closes: #39564 Authored-by: Raúl Cumplido <[email protected]> Signed-off-by: Raúl Cumplido <[email protected]>
|
Revision: 69174f8 Submitted crossbow builds: ursacomputing/crossbow @ maint-15.0.0-nightly-tests-1
|
|
Revision: 69174f8 Submitted crossbow builds: ursacomputing/crossbow @ maint-15.0.0-nightly-packaging-1
|
|
Revision: 69174f8 Submitted crossbow builds: ursacomputing/crossbow @ maint-15-nightly-verification-0
|
…x macOS build with conda (#39589) ### Rationale for this change CI job has been failing since we added integration tests. ### What changes are included in this PR? Add `CGO_ENABLED=1` to go build cdata_integration on the verification script. ### Are these changes tested? Yes via archery. ### Are there any user-facing changes? No * Closes: #39588 Authored-by: Raúl Cumplido <[email protected]> Signed-off-by: Raúl Cumplido <[email protected]>
|
@github-actions crossbow submit verify-rc-source-csharp-linux-conda-latest-amd64 |
|
Revision: eb6076f Submitted crossbow builds: ursacomputing/crossbow @ actions-874dc77e12
|
See #39601 ### Are these changes tested? Existing CI should pass. This should also pass on macbuilder without downloading cmake, and if hardcoding `download_ok <- FALSE`, it should exit cleanly and informatively. ### Are there any user-facing changes? Define "user". * Closes: #39601 Authored-by: Neal Richardson <[email protected]> Signed-off-by: Jacob Wujciak-Jens <[email protected]>
### Rationale for this change Resolves #39584 ### What changes are included in this PR? We now only check the checksum after the download succeeded, and try to be quieter about it when we do. We also use bundled boost and lz4 source on macos by default (to avoid system versions of each on cran that seem to have issues) ### Are these changes tested? I submitted a download-malignant (and verbose) build to [CRAN's macbuilder](https://mac.r-project.org/macbuilder/results/1705088784-991a5beacf4ec26e/) and it succeeds. ### Are there any user-facing changes? In principle the macos source build is slightly altered + we have a cleaner path when file downloads fail. But both of these should be relatively non-impactful since most macos users are getting binaries from CRAN. Most importantly it helps us stay on CRAN. **This PR contains a "Critical Fix".** * Closes: #39584 Lead-authored-by: Jonathan Keane <[email protected]> Co-authored-by: Jacob Wujciak-Jens <[email protected]> Signed-off-by: Jacob Wujciak-Jens <[email protected]>
### Rationale for this change CMake is now a sysreq and we don't want to default to using nightly builds in CI ### Are these changes tested? Crossbos * Closes: #39624 Authored-by: Jacob Wujciak-Jens <[email protected]> Signed-off-by: Jacob Wujciak-Jens <[email protected]>
### What changes are included in this PR? The verification script is modified to look for the versions of .NET now supported by the package. ### Are these changes tested? Manually tested the verification command. * Closes: #39598 Authored-by: Curt Hagenlocher <[email protected]> Signed-off-by: Curt Hagenlocher <[email protected]>
…ring (#39632) ### Rationale for this change `ParquetFileFragment` stores a `SchemaManifest` that has a raw pointer to a `SchemaDescriptor`. The `SchemaDescriptor` is originally provided by a `FileMetadata` instance but, in some cases, the `FileMetadata` instance can be destroyed while the `ParquetFileFragment` is still in use. This can typically lead to bugs or crashes. ### What changes are included in this PR? Ensure that `ParquetFileFragment` keeps an owning pointer to the `FileMetadata` instance that provides its `SchemaManifest`'s schema descriptor. ### Are these changes tested? An assertion is added that would fail deterministically in the Python test suite. ### Are there any user-facing changes? No. * Closes: #39562 Authored-by: Antoine Pitrou <[email protected]> Signed-off-by: Antoine Pitrou <[email protected]>
raulcd
requested review from
domoritz,
trxcllnt and
westonpace
as code owners
|
Closing 15.0.0 PR for maintenance branch. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
DO NOT MERGE.
This PR is to track some crossbow jobs to validate status of maintenance branch before creating the first RC for 15.0.0.