feat: 1.0.0 libraries release (tracking issue)

[lidavidm]

Broadly:

• Include C#/.NET in releases
• Bring drivers on par with each other roughly (particularly for PostgreSQL and SQLite, which are extremely divergent codebases)
• Ensure more functional and correctness testing
• Ensure some basic performance testing

Rust is being explicitly punted on without a contributor.

Hopefully this would be what would be 0.12.0 or 0.13.0 (roughly, Summer 2024)

This isn't a complete list of tasks, just me scanning the backlog of things I think would be good to get done and adding a couple more thoughts.

Bindings

• Release C#/.NET
• csharp: add basic CI pipelines #750
• python: make sure SIGINT properly interrupts everything #1484
• c: Move to C++17 #1431
• go/adbc/drivermgr: CGO Wrapper for adbc_driver_manager does not Implement all ADBC Methods #1291
• c: fix include paths for adbc.h #1150
• c: make build warning-clean on MSVC #847
• java: add Checker Framework #624
• go: extend driver framework to cover connections #1105
• c,go,java: provide a driver 'framework' or 'base' instead of entirely separate drivers #924

Documentation

• docs: document exact versions of software, particularly for wheels #1463

Drivers

Arrow Flight SQL

• go/adbc/driver/flightsql: When CookiesMiddleware is enabled, DO_GET requests have a different set of cookies #1194
• java: bring Flight SQL driver on par with Go Flight SQL driver #745
• go/adbc/driver/flightsql: add session management  #1557
• go/adbc/driver/flightsql: support new result set expiration #1452

JDBC Adapter

PostgreSQL

• c/driver/postgresql: Support enum type #1483
• c/driver/postgresql: ensure rest of primitive types are properly supported #933
• [C] libpq driver improvements #81
• c/driver/postgresql: support falling back to non-COPY path #855

Snowflake

• go/adbc/driver/snowflake: driver cannot support query "SHOW TABLES" #1454
• Performance testing
• Correctness testing
• go/adbc/driver/snowflake: Parameterized SQL queries -> NotSupportedError #1144
• go/adbc/driver/snowflake: quote/escape names in ingestion #1064
• go/adbc/driver/snowflake: NUMBER(n, 0) values get truncated #1277

SQLite3

• c/driver/sqlite: ADBC_INGEST_OPTION_MODE_REPLACE is an "Invalid statement option" #1355

[jduo]

Would updating to Arrow 15 be part of this?

[CurtHagenlocher]

Do we think that this would be with the existing 1.1 specification or should we also consider any items in #8 as targets for a 1.2 spec and 1.0 release?

[lidavidm]

Would updating to Arrow 15 be part of this?

@jduo this isn't a complete list, I didn't list general maintenance tasks (Arrow 15 I expect to happen for the next release)

Do we think that this would be with the existing 1.1 specification or should we also consider any items in #8 as targets for a 1.2 spec and 1.0 release?

I think it would extend the timeline but if there is interest we should start drafting the proposal.

[judahrand]

Would it be reasonable to add handling prepared statements for Postgres to this release?

[lidavidm]

@judahrand would you mind filing an issue about what's missing so I can link it here? (I know the support is incomplete but it's been a while since I've dug into the implementation)

[lidavidm]

Ah now I see the other comments you've made

[lidavidm]

I linked #81 and #855 but if you'd still like to file an issue with more details I can link it here too

[lidavidm]

Removing #1143 because it needs upstream support first.

[xinyiZzz]

Would Spark and Flink have their own driver? thanks.

I found an open source spark-flight-connector, but it has not update for a long time.
(https://github.com/qwshen/spark-flight-connector)

I saw that Dremio introduced a plugin in 2023, it allow hive/spark to read and write Dremio. I hope Apache Doris can also have this capability.
(https://george-jen.gitbook.io/data-science-and-apache-spark/dremio-data-lake-engine-apache-arrow-flight-connector-with-spark-machine-learning, https://github.com/dremio-hub/dremio-flight-connector)

[lidavidm]

If people are willing to contribute drivers for Spark/Flink, that may be possible. That will not happen for 1.0.0.

[xinyiZzz]

ok, thanks reply.

If people are willing to contribute drivers for Spark/Flink, that may be possible. That will not happen for 1.0.0.

[lidavidm]

I've added to the discussion on dev@ about versioning given I'd like to put out 1.0.0 releases of some components here (Snowflake, Flight SQL, SQLite) but not everything else (PostgreSQL still needs some work, I believe, and the Java bindings are just too untested). It would mean refactoring the release scripts and the next couple releases would probably take a good deal of manual work, though.

[CurtHagenlocher]

It would be good to iron out the exact scope of what a 1.0.0 would include. I'd like (for instance) to make some breaking changes to some C# APIs to clean them up a bit but I've been putting that off as lower priority. I also want to implement the 1.1 spec. So if 1.0.0 is going to include some of the C# components then I'll need to get to work... .

CC: @davidhcoe

[lidavidm]

Okay. I'll give breaking up versioning a shot since I don't want to rush you into that work.

[lidavidm]

I started on this in #1762 so C# will be 0.12.0. That way we can move to 1.0.0 whenever you feel the API is ready and there's no rush on your side. Testing this out pre-release is going to be a bit of a pain, but that's alright.

@kou you've already seen it but I'll be counting on you for review/よろしくお願いします 🙏

[kou]

Sure! I'll review #1762 carefully!

[lidavidm]

While there are still issues listed in the parent, I don't think anything is exactly a blocker. I'd like to kick off the release either at the end of the week or at the start of next week. (5/3-5/6)

[lidavidm]

@paleolimbot I've currently marked R packages to have a next version number of "0.12.0", but we can make it "1.0.0" if you'd like