Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[C++] Thrift mirrors are outdated #44607

Closed
2 tasks done
assignUser opened this issue Nov 1, 2024 · 3 comments · Fixed by #44685
Closed
2 tasks done

[C++] Thrift mirrors are outdated #44607

assignUser opened this issue Nov 1, 2024 · 3 comments · Fixed by #44685
Assignees
@amoeba
Labels
Component: C++ Type: bug
Milestone
18.1.0

Comments

@assignUser
Copy link
Member

assignUser commented Nov 1, 2024 •
edited by amoeba
Loading

Some observations:

  • downloads.apache.org only has archives of Thrift 0.20.0 and 0.21.0 (current latest stable is 0.21.0)
  • Apache appears to now (since 2021?) have a CDN: https://dlcdn.apache.org/thrift/. That also only has archives of 0.20.0 and 0.21.0.

https://apache.org/history/mirror-history.html gives more context and this sentence may explain why this list of URLs degraded over time:

Any organization that has been hosting an Apache download mirror site, and that finds it useful for its own purposes, may continue to host the mirror site.

So I think by continuing to use these mirror URLs we were living on borrowed time. The CDN is now the recommended way for users to download content from ASF projects.

So I think we actually need to do two things here:

  • Change the URL to just use the ASF CDN
  • Update our Thrift version in cpp/thirdparty/versions.txt to 0.20.0 or greater. Arrow C++ builds cleanly on my system when using Brew-provided Thrift 0.21.0 so it seems like this may be doable.

Originally posted by @amoeba in #44523 (comment)
@assignUser assignUser added this to the 18.1.0 milestone Nov 1, 2024
@amoeba amoeba self-assigned this Nov 1, 2024
@amoeba
Copy link
Member

amoeba commented Nov 4, 2024

For extra context here, we originally found this in this CI job as part of pre-submission testing in #44523.

@amoeba amoeba mentioned this issue Nov 9, 2024
Merged
@amoeba amoeba closed this as completed in d7e982c Nov 11, 2024
@amoeba
Copy link
Member

amoeba commented Nov 11, 2024

Issue resolved by pull request 44685
#44685

@amoeba
Copy link
Member

amoeba commented Nov 11, 2024

Note for posterity: We opted not to upgrade all the way to Thrift 0.21.0 here because Thrift 0.21.0 requires an upgrade to Boost 1.86.0 and that version of Boost shipped a buggy bcp which we use in cpp/build-support/trim-boost.sh to repackage Boost. We can upgrade Thrift and Boost once a fixed Boost release has been published.

amoeba added a commit that referenced this issue Nov 11, 2024
@amoeba
GH-44607: [C++][Dev] Update bundled Thrift, update mirrors to use CDN (…
2354c2f 
…#44685)

### Rationale for this change

Builds with bundled Thrift could fail because of permanently offline download mirror URLs. ASF now provides [a CDN](https://dlcdn.apache.org/).

### What changes are included in this PR?

1. Updates the pinned version for bundled Thrift from 0.16.0 to 0.20.0. We couldn't just update the URLs because the ASF CDN only hosts the last two versions of Thrift (0.20, 0.21.0). We didn't upgrade to Thrift 0.21.0 in this PR because it requires Boost 1.86.0 and we found incompatibilities with Boost 1.86.0 and `cpp/build-support/trim-boost.sh` which we use to publish a trimmed-down distribution of Boost for our build system to use.
2. Removes the set of URLs we were using to download Thrift in favor of just using the main CDN URL. We'll see how this works in practice.

### Are these changes tested?

Yes. I configured a build with `...-DThrift_SOURCE=BUNDLED...` and ran the full C++ test suite. I think further testing can be done in CI.

### Are there any user-facing changes?

No.

Closes #44607
* GitHub Issue: #44607

Authored-by: Bryce Mecum <[email protected]>
Signed-off-by: Bryce Mecum <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@amoeba amoeba

Labels
Component: C++ Type: bug
Projects
None yet
Milestone
18.1.0
Development

Successfully merging a pull request may close this issue.

GH-44607: [C++][Dev] Update bundled Thrift, update mirrors to use CDN amoeba/arrow
2 participants
@amoeba @assignUser