Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Implement VersionCollection in Java #34050

Merged
Merged
Show file tree
Hide file tree
Changes from 3 commits
Commits
Show all changes
25 commits
Select commit Hold shift + click to select a range
0e475b0
Add test for 7.0.0 index compatability
alpar-t Sep 21, 2018
17b9734
Add java unused implementation of VersionCollection
alpar-t Sep 24, 2018
8ab7df8
Start using Java implementation of version collection
alpar-t Sep 25, 2018
b08b3b6
Self review
alpar-t Sep 25, 2018
bd03af5
Determine unreleased versions by gruping major and minor versions
alpar-t Sep 27, 2018
f0ce80f
Check all versions not just the index compatible ones
alpar-t Sep 27, 2018
4f5ef6d
remove unused inport
alpar-t Sep 27, 2018
ca79ab9
rename version collection
alpar-t Sep 28, 2018
fbf8f3e
Move version check core logic in version collection
alpar-t Sep 28, 2018
c72cc95
improove docs
alpar-t Sep 28, 2018
ff69345
Remove unused import
alpar-t Sep 28, 2018
4d54ac8
remove unused index property
alpar-t Sep 28, 2018
d01c914
Merge remote-tracking branch 'origin/master' into remove-version-qual…
alpar-t Oct 1, 2018
6962565
Remove TreeSet
alpar-t Oct 1, 2018
555ef78
Fix javadoc
alpar-t Oct 1, 2018
a27505f
Merge branch 'master' into remove-version-qualifier-snapshot-build
alpar-t Oct 4, 2018
c2041d2
Merge remote-tracking branch 'origin/master' into remove-version-qual…
alpar-t Oct 5, 2018
5d11cd4
PR review
alpar-t Oct 30, 2018
dc328a7
Change indent so git diff is less confused
alpar-t Oct 30, 2018
179e54c
PR review
alpar-t Oct 30, 2018
7d4a144
Merge remote-tracking branch 'origin/master' into remove-version-qual…
alpar-t Oct 30, 2018
7252607
fix javadoc
alpar-t Oct 30, 2018
8032307
unused imports
alpar-t Oct 30, 2018
c576684
pull out changes to ASCIDOC
alpar-t Oct 31, 2018
47a7b79
PR review
alpar-t Oct 31, 2018
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
32 changes: 13 additions & 19 deletions build.gradle
Original file line number Diff line number Diff line change
Expand Up @@ -103,10 +103,6 @@ subprojects {
* in a branch if there are only betas and rcs in the branch so we have
* *something* to test against. */
VersionCollection versions = new VersionCollection(file('server/src/main/java/org/elasticsearch/Version.java').readLines('UTF-8'))
if (versions.currentVersion != VersionProperties.elasticsearch) {
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The check is moved into version collections.
I tried to move checks and decisions there for better visibility.

throw new GradleException("The last version in Versions.java [${versions.currentVersion}] does not match " +
"VersionProperties.elasticsearch [${VersionProperties.elasticsearch}]")
}

// build metadata from previous build, contains eg hashes for bwc builds
String buildMetadataValue = System.getenv('BUILD_METADATA')
Expand Down Expand Up @@ -154,7 +150,8 @@ task verifyVersions {
* Versions not marked as released don't get the same testing and we want
* to make sure that we flip all unreleased versions to released as soon
* as possible after release. */
Set<Version> actualVersions = new TreeSet<>(bwcVersions.indexCompatible.findAll { false == it.snapshot })
Set<Version> actualVersions = new TreeSet<>(bwcVersions.indexCompatible)
actualVersions.removeAll(bwcVersions.getUnreleased())

// Finally, compare!
if (knownVersions.equals(actualVersions) == false) {
Expand Down Expand Up @@ -251,20 +248,17 @@ subprojects {
"org.elasticsearch.plugin:percolator-client:${version}": ':modules:percolator',
"org.elasticsearch.plugin:rank-eval-client:${version}": ':modules:rank-eval',
]

bwcVersions.snapshotProjectNames.each { snapshotName ->
Version snapshot = bwcVersions.getSnapshotForProject(snapshotName)
if (snapshot != null ) {
String snapshotProject = ":distribution:bwc:${snapshotName}"
project(snapshotProject).ext.bwcVersion = snapshot
ext.projectSubstitutions["org.elasticsearch.distribution.deb:elasticsearch:${snapshot}"] = snapshotProject
ext.projectSubstitutions["org.elasticsearch.distribution.rpm:elasticsearch:${snapshot}"] = snapshotProject
ext.projectSubstitutions["org.elasticsearch.distribution.zip:elasticsearch:${snapshot}"] = snapshotProject
if (snapshot.onOrAfter('6.3.0')) {
ext.projectSubstitutions["org.elasticsearch.distribution.deb:elasticsearch-oss:${snapshot}"] = snapshotProject
ext.projectSubstitutions["org.elasticsearch.distribution.rpm:elasticsearch-oss:${snapshot}"] = snapshotProject
ext.projectSubstitutions["org.elasticsearch.distribution.zip:elasticsearch-oss:${snapshot}"] = snapshotProject
}
// substitute unreleased versiopns with projects that check out and build locally
bwcVersions.forEachUnreleased { VersionCollection.UnreleasedVersionDescription unreleasedVersion ->
Version snapshot = unreleasedVersion.version
String snapshotProject = ":distribution:bwc:build-unreleased-version-${unreleasedVersion.index}"
ext.projectSubstitutions["org.elasticsearch.distribution.deb:elasticsearch:${snapshot}"] = snapshotProject
ext.projectSubstitutions["org.elasticsearch.distribution.rpm:elasticsearch:${snapshot}"] = snapshotProject
ext.projectSubstitutions["org.elasticsearch.distribution.zip:elasticsearch:${snapshot}"] = snapshotProject
if (snapshot.onOrAfter('6.3.0')) {
ext.projectSubstitutions["org.elasticsearch.distribution.deb:elasticsearch-oss:${snapshot}"] = snapshotProject
ext.projectSubstitutions["org.elasticsearch.distribution.rpm:elasticsearch-oss:${snapshot}"] = snapshotProject
ext.projectSubstitutions["org.elasticsearch.distribution.zip:elasticsearch-oss:${snapshot}"] = snapshotProject
}
}

Expand Down

This file was deleted.

13 changes: 9 additions & 4 deletions buildSrc/src/main/java/org/elasticsearch/gradle/Version.java
Original file line number Diff line number Diff line change
Expand Up @@ -21,6 +21,10 @@ public final class Version implements Comparable<Version> {
private static final Pattern pattern =
Pattern.compile("(\\d)+\\.(\\d+)\\.(\\d+)(-alpha\\d+|-beta\\d+|-rc\\d+)?(-SNAPSHOT)?");

public Version(int major, int minor, int revision) {
this(major, minor, revision, "", false);
}

public Version(int major, int minor, int revision, String suffix, boolean snapshot) {
Objects.requireNonNull(major, "major version can't be null");
Objects.requireNonNull(minor, "minor version can't be null");
Expand Down Expand Up @@ -136,10 +140,7 @@ public boolean equals(Object o) {
Version version = (Version) o;
return major == version.major &&
minor == version.minor &&
revision == version.revision &&
id == version.id &&
snapshot == version.snapshot &&
Objects.equals(suffix, version.suffix);
revision == version.revision;
}

@Override
Expand Down Expand Up @@ -176,4 +177,8 @@ public String getSuffix() {
public int compareTo(Version other) {
return Integer.compare(getId(), other.getId());
}

public Version numbersOnly() {
return new Version(getMajor(), getMinor(), getRevision());
}
}
239 changes: 239 additions & 0 deletions buildSrc/src/main/java/org/elasticsearch/gradle/VersionCollection.java
Original file line number Diff line number Diff line change
@@ -0,0 +1,239 @@
/*
* Licensed to Elasticsearch under one or more contributor
* license agreements. See the NOTICE file distributed with
* this work for additional information regarding copyright
* ownership. Elasticsearch licenses this file to you under
* the Apache License, Version 2.0 (the "License"); you may
* not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing,
* software distributed under the License is distributed on an
* "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
* KIND, either express or implied. See the License for the
* specific language governing permissions and limitations
* under the License.
*/
package org.elasticsearch.gradle;

import java.util.ArrayList;
import java.util.Collections;
import java.util.Iterator;
import java.util.List;
import java.util.Optional;
import java.util.SortedSet;
import java.util.TreeSet;
import java.util.function.Consumer;
import java.util.function.Predicate;
import java.util.regex.Matcher;
import java.util.regex.Pattern;
import java.util.stream.Collectors;
import java.util.stream.IntStream;

public class VersionCollection {

private final Pattern LINE_PATTERN = Pattern.compile(
"\\W+public static final Version V_(\\d+)_(\\d+)_(\\d+)(_alpha\\d+|_beta\\d+|_rc\\d+)? .*"
);
private final SortedSet<Version> versions = new TreeSet<>();
private Version currentVersion;

public class UnreleasedVersionDescription {
alpar-t marked this conversation as resolved.
Show resolved Hide resolved
private final Version version;
private final Integer index;
private final String branch;

public UnreleasedVersionDescription(Integer index, Version version, String branch) {
this.version = version;
this.index = index;
this.branch = branch;
}

public Version getVersion() {
alpar-t marked this conversation as resolved.
Show resolved Hide resolved
return version;
}

public Integer getIndex() {
return index;
}

public String getBranch() {
return branch;
}
}

protected VersionCollection(List<String> versionLines, Version currentVersionProperty) {
versionLines.stream()
.map(LINE_PATTERN::matcher)
.filter(Matcher::matches)
.map(match -> new Version(
Integer.parseInt(match.group(1)),
Integer.parseInt(match.group(2)),
Integer.parseInt(match.group(3)),
(match.group(4) == null ? "" : match.group(4)).replace('_', '-'),
false
))
.filter(((Predicate<? super Version>) Version::isSnapshot).negate())
.filter(version -> version.getSuffix().isEmpty() || version.numbersOnly().equals(currentVersionProperty.numbersOnly()))
.forEach(version -> {
if (versions.add(version) == false) {
throw new IllegalStateException("Found duplicate while parsing all versions: " + version);
}
});
if (versions.isEmpty()) {
throw new IllegalArgumentException("Could not parse any versions");
}

currentVersion = versions.last();
if (currentVersionProperty.equals(currentVersion) == false) {
throw new IllegalStateException(
"Parsed versions latest version does not match the one configured in build properties. " +
"Parsed latest version is " + currentVersion + " but the build has " +
currentVersionProperty
);
}

SortedSet<Version> versionsMoreThanTowMajorsAgo = versions
.headSet(new Version(currentVersion.getMajor() - 1, 0, 0, "", false));
if (versionsMoreThanTowMajorsAgo.isEmpty() == false) {
throw new IllegalStateException(
"Found unexpected parsed versions, more than two majors ago: " + versionsMoreThanTowMajorsAgo
);
}

List<Integer> distinctMajors = versions.stream()
.map(Version::getMajor)
.distinct()
.collect(Collectors.toList());
if (distinctMajors.size() != 2) {
throw new IllegalStateException("Expected to have exactly 2 parsed major bout found: " + distinctMajors);
}

getUnreleased().forEach(version -> {
versions.remove(version);
versions.add(
new Version(
version.getMajor(), version.getMinor(), version.getRevision(), version.getSuffix(), true
)
);
});
}

public void forEachUnreleased(Consumer<UnreleasedVersionDescription> consumer) {
List<Version> unreleasedVersionsList = new ArrayList<>(getUnreleased());
IntStream.range(0, unreleasedVersionsList.size())
.filter(index -> unreleasedVersionsList.get(index).equals(currentVersion) == false)
.forEach(index ->
consumer.accept(new UnreleasedVersionDescription(
index,
unreleasedVersionsList.get(index),
getBranchFor(unreleasedVersionsList.get(index))
))
);
}

protected String getBranchFor(Version unreleasedVersion) {
SortedSet<Version> unreleased = getUnreleased();
if (unreleased.contains(unreleasedVersion)) {
if (unreleasedVersion.getMinor() == 0 && unreleasedVersion.getRevision() == 0) {
return "master";
}
if (unreleasedVersion.getRevision() != 0) {
return unreleasedVersion.getMajor() + "." + unreleasedVersion.getMinor();
}
Optional<Version> nextUnreleased = getVersionAfter(unreleased, unreleasedVersion);
if (nextUnreleased.isPresent() && nextUnreleased.get().getMajor() == unreleasedVersion.getMajor()) {
return unreleasedVersion.getMajor() + "." + unreleasedVersion.getMinor();
}
return unreleasedVersion.getMajor() + ".x";
} else {
throw new IllegalStateException(
"Can't get branch of released version: " + unreleasedVersion + " unreleased versions are: " + unreleased
);
}
}

public SortedSet<Version> getUnreleased() {
SortedSet<Version> unreleased = new TreeSet<>();
// The current version is being worked, is always unreleased
unreleased.add(currentVersion);
// The tip of the previous major will also be unreleased
unreleased.add(getLatestBugfixBeforeMajor(currentVersion));
// Initial releases of minors and majors will have more unreleased versions before them
if (currentVersion.getRevision() == 0) {
Version greatestPreviousMinor = getLatestBugfixBeforeMinor(currentVersion);
unreleased.add(greatestPreviousMinor);
Version greatestMinorBehindByTwo = getLatestBugfixBeforeMinor(greatestPreviousMinor);
if (greatestPreviousMinor.getRevision() == 0) {
// So we have x.y.0 and x.(y-1).0 with nothing in-between. This happens when x.y.0 is feature freeze.
// We'll add x.y.1 as soon as x.y.0 releases so this is correct.
unreleased.add(greatestMinorBehindByTwo);
}
}
if (unreleased.size() > 4) {
throw new IllegalStateException("Expected no more than 4 unreleased version but found: " + unreleased);
}
return unreleased;
}

private Version getLatestBugfixBeforeMajor(Version version) {
return getVersionBefore(new Version(version.getMajor(), 0, 0));
}

private Version getLatestBugfixBeforeMinor(Version version) {
return getVersionBefore(new Version(version.getMajor(), version.getMinor(), 0));
}

public VersionCollection(List<String> versionLines) {
this(versionLines, VersionProperties.getElasticsearch());
}

public SortedSet<Version> getIndexCompatible() {
return Collections.unmodifiableSortedSet(
versions
.tailSet(new Version(currentVersion.getMajor() - 1, 0, 0))
.headSet(currentVersion)
);
}

public SortedSet<Version> getWireCompatible() {
final Version lastPrevMajor = getLatestBugfixBeforeMajor(currentVersion);
return Collections.unmodifiableSortedSet(
versions
.tailSet(new Version(lastPrevMajor.getMajor(), lastPrevMajor.getMinor(), 0))
.headSet(currentVersion)
);
}

public SortedSet<Version> getSnapshotsIndexCompatible() {
SortedSet<Version> unreleasedIndexCompatible = new TreeSet<>(getIndexCompatible());
unreleasedIndexCompatible.retainAll(getUnreleased());
return Collections.unmodifiableSortedSet(unreleasedIndexCompatible);
}

public SortedSet<Version> getSnapshotsWireCompatible() {
SortedSet<Version> unreleasedWireCompatible = new TreeSet<>(getWireCompatible());
unreleasedWireCompatible.retainAll(getUnreleased());
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

retainAll with 2 lists is going to be quadratic. The unreleased list should be sorted. The same logic suggested above on finding index/wire compatible can be used here.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I really like how well this reads, and the number of versions is finite in practice.

return Collections.unmodifiableSortedSet(unreleasedWireCompatible);
}

private Version getVersionBefore(Version e) {
return versions
.headSet(e).last();
}

private Optional<Version> getVersionAfter(SortedSet<Version> versions, Version version) {
Iterator<Version> iterator = versions
.tailSet(version).iterator();
if (iterator.hasNext()) {
iterator.next();
if (iterator.hasNext()) {
return Optional.of(iterator.next());
}
}
return Optional.empty();
}

}
Loading