Update to version 48 #47

Dandandan · 2023-10-27T11:41:29Z

Which issue does this PR close?

Closes #.

Rationale for this change

What changes are included in this PR?

Are there any user-facing changes?

* Impl RBR for Box * Require send to create a FFI stream

* Add Scalar/Datum abstraction (apache#1047) * Add dyn Array

* Adding sort_primitives benchmark * Adding sort_primitives improvements * Fix lints * Remove all unsafe code and handle offset cases * Incorporate review comments * Remove unneeded returns

The trait currently does not have many default implementations, but it does have a lot of methods. This PR adds default implementations for all methods returning Status::unimplemented to fix apache#4372

* Add Datum based arithmetic kernels (apache#3999) * Fix benchmark * Review feedback

…scalar (apache#4492)

Updates the requirements on [proc-macro2](https://github.com/dtolnay/proc-macro2) to permit the latest version. - [Release notes](https://github.com/dtolnay/proc-macro2/releases) - [Commits](dtolnay/proc-macro2@1.0.63...1.0.64) --- updated-dependencies: - dependency-name: proc-macro2 dependency-type: direct:production ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

…ache#4499)

* Add negate kernels (apache#4488) * Fix doc * Add Inteval tests * Review feedback

Bumps [actions/labeler](https://github.com/actions/labeler) from 4.2.0 to 4.3.0. - [Release notes](https://github.com/actions/labeler/releases) - [Commits](actions/labeler@v4.2.0...v4.3.0) --- updated-dependencies: - dependency-name: actions/labeler dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* fix incorrect buffer size limiting in parquet async writer * Format * Review feedback --------- Co-authored-by: zhangli20 <[email protected]> Co-authored-by: Raphael Taylor-Davies <[email protected]>

…pache#4493) * Add arithmetic tests (apache#4480) * Fix test * Review feedback

* object_store: Add `Box<dyn ObjectStore>` tests * object_store: Extract `as_ref_impl!()` macro * object_store: Implement `ObjectStore` for `Arc`

Bumps [actions/upload-pages-artifact](https://github.com/actions/upload-pages-artifact) from 1 to 2. - [Release notes](https://github.com/actions/upload-pages-artifact/releases) - [Commits](actions/upload-pages-artifact@v1...v2) --- updated-dependencies: - dependency-name: actions/upload-pages-artifact dependency-type: direct:production update-type: version-update:semver-major ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Support Date - Date (apache#4383) * Review feedback

* Cleanup cast_primitive_to_list * Review feedback

…fig` (apache#4516) * object_store: Export `ClientConfigKey` * object_store: Add `HttpBuilder::with_config`

…he#4527)

* Use Parser for cast kernel (apache#4512) * Support parsing unpadded dates * Review feedback * Tweak bound

* Prepare arrow 44.0.0 * Final tweaks

…e#4532)

Updates the requirements on [proc-macro2](https://github.com/dtolnay/proc-macro2) to permit the latest version. - [Release notes](https://github.com/dtolnay/proc-macro2/releases) - [Commits](dtolnay/proc-macro2@1.0.64...1.0.66) --- updated-dependencies: - dependency-name: proc-macro2 dependency-type: direct:production ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Add RowConverter::append (apache#4479) * Add overwrite test

* feat: Support FixedSizedListArray for length kernel * fix clippy * update comment * avoid unsafe * reduce useless trait * remove T

Updates the requirements on [zstd](https://github.com/gyscos/zstd-rs) to permit the latest version. - [Release notes](https://github.com/gyscos/zstd-rs/releases) - [Commits](gyscos/zstd-rs@v0.12.0...v0.13.0) --- updated-dependencies: - dependency-name: zstd dependency-type: direct:production ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* feat: add method for async read bloomfilter * fix: compatible for bloom filter length * test: add unit tests for read bloom filter * fix: format code for unit test

* Support Arbitrary JSON values in JSON Reader (apache#4905) * Review feedback * Clippy * Docs

* Default connection and request timeouts of 5 seconds * Clippy * Allow disabling timeouts

Updates the requirements on [pyo3](https://github.com/pyo3/pyo3) to permit the latest version. - [Release notes](https://github.com/pyo3/pyo3/releases) - [Changelog](https://github.com/PyO3/pyo3/blob/main/CHANGELOG.md) - [Commits](PyO3/pyo3@v0.19.0...v0.20.0) --- updated-dependencies: - dependency-name: pyo3 dependency-type: direct:production ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

…SignedJwt (apache#4926) * Support service_account in ApplicationDefaultCredentials * Use SelfSignedJwt for Service Accounts * Update CI * Apply suggestions from code review Co-authored-by: Marco Neumann <[email protected]> --------- Co-authored-by: Marco Neumann <[email protected]>

* Add `FileWriter` schema getter * Make schema getter consistent with the parquet implementation

…arsed as `Null` rather than `""`) (apache#4942) * fix: add null_regex for string type in csv * Update arrow-csv/src/reader/mod.rs Co-authored-by: Raphael Taylor-Davies <[email protected]> --------- Co-authored-by: Raphael Taylor-Davies <[email protected]>

…pache#4922) * Support ETag in InMemory (apache#4879) * Add LocalFileSystem Etag * Review feedback * Review feedback

…pache#4943) * Assume records not split across pages (apache#4921) * More test * Add PageReader::at_record_boundary * Fix flush partial

* Remove nested async and fallibility from ObjectStore::list * Clippy * Update limit test * Update docs

…ache#4933)

* feat: support parsing for parquet writer option Signed-off-by: fan <[email protected]> * fix clippy warning Signed-off-by: fan <[email protected]> * add tests Signed-off-by: fan <[email protected]> * follow reviews Signed-off-by: fan <[email protected]> * fix only support lower and uppercase Signed-off-by: fan <[email protected]> --------- Signed-off-by: fan <[email protected]>

* Add ReaderBuilder::with_header * Update test

wjones127 and others added 30 commits July 3, 2023 15:00

feat: support RecordBatchReader on boxed trait objects (apache#4475)

5ea197d

* Impl RBR for Box * Require send to create a FFI stream

Add Scalar/Datum abstraction (apache#1047) (apache#4393)

9ee36b2

* Add Scalar/Datum abstraction (apache#1047) * Add dyn Array

Improve in-place primitive sorts by 13-67% (apache#4473)

aac3aa9

* Adding sort_primitives benchmark * Adding sort_primitives improvements * Fix lints * Remove all unsafe code and handle offset cases * Incorporate review comments * Remove unneeded returns

Add default implementations to the FlightSqlService trait (apache#4485)

08a57e9

The trait currently does not have many default implementations, but it does have a lot of methods. This PR adds default implementations for all methods returning Status::unimplemented to fix apache#4372

Add Datum based arithmetic kernels (apache#3999) (apache#4465)

ee2c292

* Add Datum based arithmetic kernels (apache#3999) * Fix benchmark * Review feedback

support FixedSizeBinary types in eq_dyn_binary_scalar/neq_dyn_binary_…

9649d08

…scalar (apache#4492)

add num-complex requirement (apache#4482)

66aab5a

object_store/InMemory: Add fork() fn and deprecate clone() fn (ap…

e5378be

…ache#4499)

Add negate kernels (apache#4488) (apache#4494)

20af94b

* Add negate kernels (apache#4488) * Fix doc * Add Inteval tests * Review feedback

ci: verify MSRV on CI (apache#4490)

72b2ec5

fix incorrect buffer size limiting in parquet async writer (apache#4478)

6f22313

* fix incorrect buffer size limiting in parquet async writer * Format * Review feedback --------- Co-authored-by: zhangli20 <[email protected]> Co-authored-by: Raphael Taylor-Davies <[email protected]>

Add Datum Arithmetic tests, Fix Interval Substraction (apache#4480) (a…

6bbf2f0

…pache#4493) * Add arithmetic tests (apache#4480) * Fix test * Review feedback

object_store: Implement ObjectStore for Arc (apache#4502)

8da2f97

* object_store: Add `Box<dyn ObjectStore>` tests * object_store: Extract `as_ref_impl!()` macro * object_store: Implement `ObjectStore` for `Arc`

Support Date - Date (apache#4383) (apache#4504)

8bcb3fc

* Support Date - Date (apache#4383) * Review feedback

Cleanup cast_primitive_to_list (apache#4511)

e169355

* Cleanup cast_primitive_to_list * Review feedback

object_store: Export ClientConfigKey and add `HttpBuilder::with_con…

c044464

…fig` (apache#4516) * object_store: Export `ClientConfigKey` * object_store: Add `HttpBuilder::with_config`

fix: clippy problem (apache#4521)

43712ac

Handle empty S3 payloads (apache#4514) (apache#4518)

edeb7bb

Fix AsyncArrowWriter flush for large buffer sizes (apache#4526) (apac…

f951c8f

…he#4527)

Use Parser for cast kernel (apache#4512) (apache#4513)

6909db8

* Use Parser for cast kernel (apache#4512) * Support parsing unpadded dates * Review feedback * Tweak bound

Prepare arrow 44.0.0 (apache#4528)

8f44472

* Prepare arrow 44.0.0 * Final tweaks

use new num version instead of explicit num-complex dependency (apach…

13fdfa4

…e#4532)

add a validity slice access for boolean array builders (apache#4536)

be8bd13

Add RowConverter::append (apache#4479) (apache#4541)

276d8c5

* Add RowConverter::append (apache#4479) * Add overwrite test

Initial loongarch port (apache#4538)

47f288c

feat: Support FixedSizedListArray for length kernel (apache#4520)

730941f

* feat: Support FixedSizedListArray for length kernel * fix clippy * update comment * avoid unsafe * reduce useless trait * remove T

dependabot bot and others added 22 commits October 12, 2023 15:27

feat: add method for async read bloom filter (apache#4917)

6e49f31

* feat: add method for async read bloomfilter * fix: compatible for bloom filter length * test: add unit tests for read bloom filter * fix: format code for unit test

Support Arbitrary JSON values in JSON Reader (apache#4905) (apache#4911)

90bc5ec

* Support Arbitrary JSON values in JSON Reader (apache#4905) * Review feedback * Clippy * Docs

Add GetOptions::head (apache#4931)

bb8e42f

Allow opting out of request signing (apache#4927) (apache#4929)

57cd094

Default connection and request timeouts of 5 seconds (apache#4928)

31bc84c

* Default connection and request timeouts of 5 seconds * Clippy * Allow disabling timeouts

Add FileWriter schema getter (apache#4940)

ce2a958

* Add `FileWriter` schema getter * Make schema getter consistent with the parquet implementation

Generate ETags for InMemory and LocalFileSystem (apache#4879) (a…

ab87abd

…pache#4922) * Support ETag in InMemory (apache#4879) * Add LocalFileSystem Etag * Review feedback * Review feedback

Assume Pages Delimit Records When Offset Index Loaded (apache#4921) (a…

d4d11fe

…pache#4943) * Assume records not split across pages (apache#4921) * More test * Add PageReader::at_record_boundary * Fix flush partial

Remove Nested async and Fallibility from ObjectStore::list (apache#4930)

fa7a61a

* Remove nested async and fallibility from ObjectStore::list * Clippy * Update limit test * Update docs

Fix object_store docs (apache#4947)

511ac44

Expose SubstraitPlan structure in arrow_flight::sql (apache#4932) (ap…

952cd2e

…ache#4933)

Add ReaderBuilder::with_header for csv reader (apache#4949)

4964d84

* Add ReaderBuilder::with_header * Update test

Prepare arrow 48.0.0 (apache#4948)

6e332b8

Respect ARROW_TEST_DATA in apache-avro tests (apache#4950)

51ac6fe

Cherry-pick change

89726de

Cherry-pick change

397d50a

Fixup

dbdbb6b

github-actions bot added arrow arrow-flight object-store parquet parquet-derive labels Oct 27, 2023

Dandandan closed this Oct 30, 2023

Dandandan reopened this Nov 1, 2023

Dandandan changed the base branch from master to v43 November 1, 2023 12:40

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Update to version 48 #47

Update to version 48 #47

Dandandan commented Oct 27, 2023

Update to version 48 #47

Are you sure you want to change the base?

Update to version 48 #47

Conversation

Dandandan commented Oct 27, 2023

Which issue does this PR close?

Rationale for this change

What changes are included in this PR?

Are there any user-facing changes?