Website: Add blog post for 18.0.0

_posts/2024-10-16-18.0.0-release.md

@@ -0,0 +1,195 @@
+---
+layout: post
+title: "Apache Arrow 18.0.0 Release"
+date: "2024-10-16 00:00:00"
+author: pmc
+categories: [release]
+---
+<!--
+{% comment %}
+Licensed to the Apache Software Foundation (ASF) under one or more
+contributor license agreements.  See the NOTICE file distributed with
+this work for additional information regarding copyright ownership.
+The ASF licenses this file to you under the Apache License, Version 2.0
+(the "License"); you may not use this file except in compliance with
+the License.  You may obtain a copy of the License at
+
+http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+{% endcomment %}
+-->
+
+
+The Apache Arrow team is pleased to announce the 18.0.0 release. This covers
+over 3 months of development work and includes [**XXX resolved issues**][1]
+on [**YYY distinct commits**][2] from [**ZZZ distinct contributors**][2].
+See the [Install Page](https://arrow.apache.org/install/)
+to learn how to get the libraries for your platform.
+
+The release notes below are not exhaustive and only expose selected highlights
+of the release. Many other bugfixes and improvements have been made: we refer
+you to the [complete changelog][3].
+
+## Community
+
+Since the 17.0.0 release, JJJJJ has been invited to be committer.
+No new members have joined the Project Management Committee (PMC).
+
+Thanks for your contributions and participation in the project!
+
+## Columnar format
+
+The Arrow columnar format now allows 32-bit and 64-bit decimal data, in
+addition to the already existing 128-bit and 256-bit decimal data types
+(GH-43956).
+
+## Linux packages notes
+
+Azure file system is enabled.
+
+## C Data Interface notes
+
+
+## Arrow Flight RPC notes
+
+**Flight UCX is deprecated.** We plan to remove this experiment in the next couple of releases.
+
+The Java implementation now transparently handles compressed Arrow data when reading, instead of requiring explicit configuration. (GH-43469)
+
+The Ruby bindings now support implementing DoPut on the server. (GH-43814)
+
+## C++ notes
+
+The default memory pool has changed to mimalloc on all platforms (GH-43254).
+Previously, jemalloc was used by default on Linux. Using mimalloc by default
+provides a more consistent experience accross different platforms, and
+makes configuration easier. It is expected that this might either increase
+or decrease performance on user workloads that use the default memory pool;
+please benchmark accordingly. Jemalloc can still be selected by setting
+the `ARROW_DEFAULT_MEMORY_POOL` environment variable to "jemalloc".
+
+A new class `arrow::ArrayStatistics` has been added to encode basic statistics
+about an Arrow array. It provides a source-agnostic representation for statistics
+provided by third-party sources such as Parquet files (GH-41909).
+
+The new Decimal32 and Decimal64 types have been made available (GH-43956).
+
+Several canonical extension types have been implemented:
+- the [Opaque](https://arrow.apache.org/docs/dev/format/CanonicalExtensions.html#opaque) extension type (GH-43454);
+- the [8-bit boolean](https://arrow.apache.org/docs/dev/format/CanonicalExtensions.html#bit-boolean) extension type (GH-17682);
+- the [UUID](https://arrow.apache.org/docs/dev/format/CanonicalExtensions.html#uuid) extension type (GH-15058);
+- the [JSON](https://arrow.apache.org/docs/dev/format/CanonicalExtensions.html#json) extension type (GH-32538).
+
+### Acero
+
+- Enhanced the row-oriented representation by widening the offset type from 32-bit to 64-bit, resolving crashes and data corruption in aggregation and hash join on large datasets due to offset overflow (GH-43495).
+- Improved ordered aggregation performance by reducing complexity from `O(n*m)` to `O(n)`, where `n` is the number of rows and `m` the number of segments in the batch (GH-44052).
+
+### Compute
+
+Casting between string-like and string-view-like types has been implemented (GH-42247).
+
+### Dataset
+
+
+### Filesystems
+
+Writing small files to S3 now uses a single S3 API call instead of three
+(GH-40557). Files larger than 5 MB still go through the regular multipart
+upload mechanism.
+
+Background writes are now implemented and enabled by default for the Azure
+filesystem, dramatically improving the performance of writing to remote files
+(GH-40036).
+
+Finalization of the S3 filesystem layer should hopefully be more robust (GH-44071).
+
+### Gandiva
+
+LLVM 19.1 is now supported (GH-44222).
+
+### GPU
+
+
+### IPC
+
+The seed corpus used for fuzzing the IPC reader has been improved, hopefully
+helping make the IPC reader even more robust against corrupt or malicious
+IPC streams (GH-38041).
+
+### Parquet
+
+A new command line utility `parquet-dump-footer` allows dumping the Thrift-encoded
+footer metadata of a Parquet file, optionally scrubbing confidential data
+(GH-42102). This is part of the effort to collect real-world Parquet metadata
+so as to evaluate the efficiency of future improvements to the Parquet format.
+Please see https://github.com/apache/parquet-benchmark for instructions to submit
+footers representative of your own workloads.
+
+### Substrait
+
+
+## C# notes
+
+- Partial support has been added for LargeBinary, LargeString and LargeList. The column sizes cannot exceed 2 GB in length. (GH-43266).
+- Changes to Flight support were made for better control and compatibility, and to allow Flight Server to be hosted in pre-Kestrel versions of .NET (GH-43907, GH-43672, GH-41347).
+- Support has been added for newly-defined types decimal32 and decimal64 (GH-44271).
+- The import of sliced arrays through the C Data interface now works correctly. (GH-43267)
+## Java notes
+
+**Java 8 is no longer supported.** (GH-38051)
+
+**Gandiva may not work in this release.** For details, please see [GH-43576](https://github.com/apache/arrow/issues/43576).
+
+Basic support for RunEndEncoded was added (GH-39982). The ListView/StringView vector implementations are now more complete, including C Data support (multiple issues).
+
+Several APIs have been updated to accept `long` for addresses in preparation for FFM/large buffer support (GH-43902). We no longer expose `sun.misc.Unsafe` (GH-43479). We no longer ship the `shaded` flight-core JARs (GH-43217).
+
+More options were added to the Dataset ScanOptions API (GH-28866).
+
+## JavaScript notes
+
+
+## Python notes
+
+
+## R notes
+
+For more on what’s in the 18.0.0 R package, see the [R changelog][4].
+
+## Ruby and C GLib notes
+
+### Ruby
+
+* Add workaround for install failure due to `re2.pc` on Ubuntu 20.04: [GH-41396](https://github.com/apache/arrow/issues/41396)
+* Add support for `0` decimal value: [GH-43877](https://github.com/apache/arrow/issues/43877)
+
+C GLib related improvements are also available in Ruby. 
+
+### C GLib
+
+* Add support for Azure file system: [GH-43738](https://github.com/apache/arrow/issues/43738)
+* FlightRPC: Add support for DoPut: [GH-41056](https://github.com/apache/arrow/issues/41056)
+* FlightRPC: Add support for timeout: [GH-44178](https://github.com/apache/arrow/issues/44178)
+* Parquet: Add support for writing a record batch: [GH-40860](https://github.com/apache/arrow/issues/40860)
+* Add support for pull style IPC stream format decoder: [GH-40493](https://github.com/apache/arrow/issues/40493)
+
+## Rust notes and Go notes
+
+The Rust and Go projects have moved to separate repositories outside the
+main Arrow monorepo. For notes on the latest release of the Rust
+implementation, see the latest [Arrow Rust changelog][5].
+For notes on the latest release of the Go implementation, see the latest
+[Arrow Go changelog][6]
+
+[1]: https://github.com/apache/arrow/milestone/64?closed=1
+[2]: {{ site.baseurl }}/release/18.0.0.html#contributors
+[3]: {{ site.baseurl }}/release/18.0.0.html#changelog
+[4]: {{ site.baseurl }}/docs/r/news/
+[5]: https://github.com/apache/arrow-rs/tags
+[6]: https://github.com/apache/arrow-go/tags