Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

SOLR-17406: Introduce Version Catalogs #2706

Merged

Conversation

malliaridis
Copy link
Contributor

@malliaridis malliaridis commented Sep 11, 2024

https://issues.apache.org/jira/browse/SOLR-17406

Description

With the version catalogs all versions and references to them are managed in a single file and updated from there.

Solution

This PR introduces version catalogs and removes the palantir consistent versions plugin, since it is no longer needed.

For syncing versions a new gradle file (dependencies.gradle) is introduced, where the version of a dependency (either transitive or non-transitive) is set via constraints. This part of the changes should be improved in the future if possible to reduce the maintenance overhead.

This PR is split from #2646.

This PR is following the practices from apache/lucene#13484.

⚠️ IMPORTANT ⚠️ Merging this PR will likely break Solrbot functionality and will require additional actions.

Note that this PR is not tested for backwards compatibility (9x).

Tests

No tests should be affected by this.

Checklist

Please review the following and check all that apply:

  • I have reviewed the guidelines for How to Contribute and my code conforms to the standards described there to the best of my ability.
  • I have created a Jira issue and added the issue ID to my pull request title.
  • I have given Solr maintainers access to contribute to my PR branch. (optional but recommended, not available for branches on forks living under an organisation)
  • I have developed this patch against the main branch.
  • I have run ./gradlew check.
  • I have added tests for my changes.
  • I have added documentation for the Reference Guide

@github-actions github-actions bot removed the scripts label Oct 16, 2024
@malliaridis
Copy link
Contributor Author

I think it would be a good idea to first resolve the current solrbot PRs before aligning the versions. I am already looking into some of the PRs to see how they can be resolved.

…sion-catalogs

# Conflicts:
#	build-tools/scriptDepVersions.gradle
#	build.gradle
#	gradle/validation/check-environment.gradle
#	solr/core/build.gradle
#	solr/modules/hadoop-auth/build.gradle
#	solr/modules/hdfs/build.gradle
#	solr/server/build.gradle
#	solr/solrj-zookeeper/build.gradle
#	versions.lock
#	versions.props
@malliaridis malliaridis force-pushed the SOLR-17406/rebase-version-catalogs branch from 788d305 to 6caff99 Compare November 8, 2024 18:12
@malliaridis malliaridis marked this pull request as ready for review November 9, 2024 17:35
@malliaridis
Copy link
Contributor Author

The current state should differ now only in hamcrest-core, which is being merged with hamcrest in the dependencies.gradle.

Copy link
Contributor

@dsmiley dsmiley left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What prompted you to update the NOTICE files of existing dependencies? -- very thorough of you but it would not have occurred to me. I found a couple misattributed LGPL references to correct but no other issue.

I ran the bats/integration tests and docker tests, and build the ref guide, just to exercise some parts of our build that might not otherwise in the PR validation. I think we should merge this very soon. A shout-out of the change arriving to our dev list would be helpful for general awareness and its impacts (e.g. dependency upgrades are now different and won't backport).

Comment on lines 165 to 173
* license/LICENSE.jboss-marshalling.txt (GNU LGPL 2.1)
* license/LICENSE.jboss-marshalling.txt (Apache License 2.0)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I checked this -- looks accurate. Then I got curious and searched out entire licenses folder for "LGPL". netty-transport-native-epoll-NOTICE.txt (and unix-common-NOTICE.txt) also makes a reference to LGPL but it's via the same jboss-marshalling line you are correcting here. So I think they can also be updated.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The updates are originally coming from #2702. I looked up the repository and noticed a few changes there. It's probably best to make it a required step of the dependency update procedure. It should also be quite simple to find and copy-paste the files if they are on GitHub (or part of the jar, which is often not the case).

Apparently I missed to update the license and notice file of the rest of the netty dependencies. Since they come all from the same repository, it is probably best to merge the files into netty-LICENSE.txt and netty-NOTICE-txt by simply removing the other files. Should be part of SOLR-15929 though.

…sion-catalogs

# Conflicts:
#	solr/core/src/test/org/apache/solr/handler/admin/IndexSizeEstimatorTest.java
#	versions.lock
#	versions.props
Copy link
Contributor

@gus-asf gus-asf left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just looked at some of the instructions didn't have time to review all, but the upgrade steps look like they could use some clarification, and I think there's an obsolete section in dependencies.txt that you are updating instead of removing.

dev-docs/dependency-upgrades.adoc Show resolved Hide resolved
help/dependencies.txt Show resolved Hide resolved
dev-docs/dependency-upgrades.adoc Outdated Show resolved Hide resolved
@malliaridis
Copy link
Contributor Author

Thanks @gus-asf for your review and input. Your questions are pointing out some of our upcoming TODOs. I'll leave some comments about the current state on each of your comments, hopefully clarifying and answering your questions.

FYI the documentation will be updated periodically in the near future as we work to clean up and improve specific processes.

@malliaridis
Copy link
Contributor Author

I have updated the documentation to hopefully be clearer, but it is still not perfect yet. I have created https://issues.apache.org/jira/browse/SOLR-17563 to address outdated documentation.

@malliaridis malliaridis merged commit c9d3885 into apache:main Nov 15, 2024
5 of 6 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

7 participants