Target platforms should not grow or change unexpectedly.
Below is the process to follow for sanely changing the target platform to empower developers and keep churn to a minimum.
Goals:
-
Avoid unnecessary churn (failed builds)
-
Avoid adding unnecessary or incomplete dependencies
-
Document why and when changes occur
Process:
-
Send email to [email protected] requesting feedback. State expected feedback window (usually 1 week).
-
Await feedback from build, affected team leads and project lead(s). For urgent changes, 1 project lead is OK.
-
Apply the pull-request
Use https://jira.jboss.org/jira/browse/JBIDE to open the issue.
Type of bug:
-
Feature Request if this change is introducing or removing a new feature/set of plugins
-
Bug if this is just to fix/update existing plugins
Component: target-platform + releated components if applicable (i.e. gwt)
Assignee: (default)
Fix version: Set it to the corresponding JBoss Tools release
Description:
Template (copy into JIRA):
*Reason:* ... *Project page/sources:* ... *Version:* .... *License and owner:* ... *Original p2 repo:* http://third-party-vendor.com/path/to/updates-site/ *JBoss mirror:* Leave empty *Include Sources:* Yes|No *Affected JBoss Tools components:* ... (components that will depend on this change, jbosstools-whatever, discovery/central, JBoss Tools update-site...) *Include in JBDS:* Yes|No *Type of dependency:* testing|central-only|distribution *List of bundles added/removed:* {code} suggested diff on the .target files {code} If applicable: *non-p2 repository:* ... *non-p2 ids:* ...
Provide enough detail to explain why the change is needed, and what is required to verify. If you do not know the answer use "Unknown"; if not applicable, use "N/A".
-
Reason: Why is the request needed, typical reasons are: "Required by new feature described in JBIDE-xyz", "Eclipse releasetrain update from Kepler M5 to M6" or "Removal because it conflicts with JBTIS and give bug describe at JBIDE-xyz"
-
License and owner: Which license are these features available under and who are the maintainer? i.e. EPL, Google.
-
Original p2 repo: which repository are the plugins/features from? eg., http://dl.google.com/eclipse/plugin/4.2
-
JBoss mirror: URL to use, eg. http://downloads.jboss.org/jbosstools/updates/requirements/mars/201506111000-RC4/
-
If no mirror already exists, you can:
-
leave this blank, or
-
-
-
Include Sources: Are source bundles included in the mirror, or are sources available elsewhere? Use "included" or explain.
-
Affected projects: Which projects are this known to affect, both inside and outside JBoss Tools core.
-
Include in JBDS: Incubating (unstable) features should only be included in JBoss Tools.
-
Type of dependency: is this just for testing (incl. in TP, but not in distribution), central-only (incl. in Central ONLY, not in the JBoss Tools or JBDS target platform), distribution (incl. in installer & update sites)
-
List of bundles added/removed: p2diff report or pull-request preferred; if not available, please list features/plugins in alpha-sorted order
If your requirement is not available in an update site (p2 repository), eg. only available as maven artifact, please include more info:
-
non-p2 repository: the exact repository URL (eg., Maven central or jboss repository) where the artifact exists
-
non-p2 ids: list the exact ids, eg., for Maven state GAV (Group, Artifact, Version)
Non-p2 artifacts will be added to JBoss Tools Locus as p2 artifacts, so they can be included in the target platform.
Following instructions cover the minimal edits needed for a review (a final Pull request needs updates to both jbosstools and devstudio target platform as needed)
git clone https://github.com/jbosstools/jbosstools-target-platforms.git
cd jbosstools-target-platforms
git checkout <relevant version.x>
For the Integration Stack:
git clone https://github.com/jbosstools/jbosstools-integration-stack.git
cd jbosstools-integration-stack
git checkout <relevant version.x or master>
To build, you require specific versions of Java and Maven. Also, there is some Maven setup. The How to Build JBoss Tools with Maven 3 document will guide you through that setup.
Note that you can only build the multiple/pom.xml locally since the unified/pom.xml refers to a remote location on download.jboss.org or devstudio.redhat.com which may or may not exist yet.
This command will run the build, but will NOT download the contents of the target platform to disk:
$ mvn clean verify -f multiple/pom.xml
If you want to download the contents of the target platform to disk, do this:
$ mvn clean verify -f multiple/pom.xml -Pmultiple2repo
If you want to run the build and fetch source bundles at the same time as other bundles are being resolved, do this:
$ mvn clean verify -f multiple/pom.xml -Pmultiple2repo -Dmirror-target-to-repo.includeSources=true
If you want to build both the multiple and unified target platforms (including mirroring the actual features & plugins onto disk), you need to point the unified pom at your locally created target platform:
$ mvn clean verify -f pom.xml -Pmultiple2repo -DtargetRepositoryUrl=file:///path/to/jbosstools-target-platforms/jbosstools/multiple/target/jbosstools-multiple.target.repo/
Or to just build the unified .target (after first building the multiple .target):
$ mvn clean verify -f unified/pom.xml -DtargetRepositoryUrl=file:///path/to/jbosstools-target-platforms/jbosstools/multiple/target/jbosstools-multiple.target.repo/
Caution
|
Partial target platforms such as those for JBoss Central cannot be materialized with their sources included. If you require sources, you must use a complete (merged) target platform which includes ALL upstream dependencies, but bear in mind this will produce a MUCH larger p2 repo, including all upstream dependencies and those dependencies' sources, too. |
If you want to run the build and not fail if there’s a problem w/ validation, do this:
$ mvn clean verify -Pmultiple2repo -Dvalidate-target-platform.failOnError=false
If you just want to check if things compiles/builds you can run:
$ mvn clean verify -Pmultiple2repo -DskipTest=true
But do not push changes without having the new and existing unit tests pass!
Edit jbosstools/multiple/jbosstools-multiple.target
. You may update an existing location or add a new one:
<location includeAllPlatforms="false" includeMode="slicer" type="InstallableUnit" includeSource="true">
<repository location="http://download.jboss.org/jbosstools/updates/requirements/tm/4.1.0.201606052351/"/>
<unit id="org.eclipse.rse.terminals.feature.group" version="3.8.0.201505221634"/>
</location>
Caution
|
UNSTATED transitive dependencies could be resolved from any of the available URLs in the target platform. To avoid resolving the wrong (too old, too new) version of a dependency, make sure that you clearly STATE all dependencies in the <location> that contains on the IU. Using 0.0.0 may resolve to the wrong version. For example, if docker tools depends on a specific version of httpclient, make sure that the version needed is available on the <location> from which you’re trying to resolve it. <location includeAllPlatforms="false" includeMode="slicer" type="InstallableUnit" includeSource="true">
<repository location="http://download.jboss.org/jbosstools/updates/requirements/orbit/S20170306214312/"/>
<unit id="org.apache.commons.httpclient" version="3.1.0.v201012070820"/>
<unit id="org.apache.httpcomponents.httpclient" version="4.5.2.v20170210-0925"/>
...
</location> |
For JBoss Central, edit jbtcentraltarget/multiple/jbtcentral-multiple.target
and/or jbtearlyaccesstarget/multiple/jbtearlyaccess-multiple.target
.
For the Integration Stack, edit the applicable .target files in target-platform/
.
Note
|
If you do not have a mirrored download.jboss.org location use the vendor’s URL, so that the .target file can be easily tested. |
Now build and validate the changed target platform.
When moving from one version of the target to another, the steps are:
-
If this is a new target platform (rather than changes to an existing SNAPSHOT) increment the target platform versions contained in all the pom.xml and *.target files.
-
Manually update the URLs contained in the *.target files (as noted above).
-
Validate the target platform. You can use a script to automate these next 4 steps.
Caution-
Regenerate the IU versions, using org.jboss.tools.tycho-plugins:target-platform-utils, and validate the results.
-
mvn -U org.jboss.tools.tycho-plugins:target-platform-utils:fix-versions -DtargetFile=jbosstools/multiple/jbosstools-multiple.target
-
-
Resolve & verify the new target platform, optionally including sources (which will produce a MUCH larger on-disk footprint).
-
mvn (clean) install -Pmultiple2repo
or -
mvn (clean) install -Pmultiple2repo -Dmirror-target-to-repo.includeSources=true
-
-
Install the new target platform into a clean Eclipse JEE bundle to verify that everything can be installed, using a headless p2.director script. More info.
-
Produce a p2diff report.
-
-
If applicable, test your locally built target platform in another build.
-
mvn verify -Dtpc.targetKind=multiple -Dtpc.version=<version-you-built>
-
Note that
tpc.version
is the version listed in thepom.xml
file of the target platform ( eg.,4.40.0.Beta4-SNAPSHOT
) andtpc.targetKind
is set tomultiple
since the default (unified
) will not work before the target platform (.target and site) are published to Nexus and download.jboss.org -
If everything builds OK, can be installed (see install validation step in Update versions of IUs in the .target files), and you can use the new functionallity needed by the change, commit your change to a topic branch and submit a pull request for your update.
-
-
-
Follow the release guidelines for how to commit to a topic branch, and announce target platform changes.
-
Check in updated target files & push to the branch.
As noted above, you can create a p2diff report to clarify what IUs will change if your PR is applied. For example:
git clone https://github.com/irbull/p2diff.git
pushd p2diff/org.eclipse.equinox.p2.example.p2diff.releng/ && mvn install && popd
pushd p2diff/org.eclipse.equinox.p2.example.p2diff.packaging/target/products/org.eclipse.equinox.p2.example.p2diff.product
cd linux/gtk/x86_64 # or your preferred platform
./p2diff \
file://${HOME}/tru/jbosstools-target-platforms/jbosstools/multiple/target_with_com.google/jbosstools-multiple.target.repo/ \
file://${HOME}/tru/jbosstools-target-platforms/jbosstools/multiple/target/jbosstools-multiple.target.repo/
popd
< com.google.gwt.eclipse.sdkbundle.feature.feature.group [2.5.1]
< com.google.gwt.eclipse.core [3.4.2.v201310081834-rel-r43]
< com.google.gdt.eclipse.core [3.4.2.v201310081834-rel-r43]
< com.google.gdt.eclipse.platform [3.4.2.v201310081834-rel-r43]
< com.google.gwt.eclipse.sdkbundle.feature.feature.jar [2.5.1]
< com.google.gwt.eclipse.oophm [3.4.2.v201310081834-rel-r43]
< com.google.gwt.eclipse.sdkbundle [2.5.1]
=== Summary ===
file:///${HOME}/tru/jbosstools-target-platforms/jbosstools/multiple/target_with_com.google/jbosstools-multiple.target.repo/ contains 7 unique IUs
In most cases, the change can impact multiple components so the whole team should be notified about the change request to verify the suggested change doesn’t break them.
Send feedback request email to [email protected], listing the PR, JIRA link(s), steps to use, and additional info if desired.
Past examples:
Using the template below will get you started, but you will still need to fill in these details:
-
PR # and JIRA(s)
-
p2diff report
Template:
# configure these to generate the email body
TARGET_PLATFORM_VERSION=4.50.0.Beta1-SNAPSHOT
TOPIC_BRANCH=JBIDE-19776 # use 4.50.x or similar branch if PR is already applied
JIRA_NUM=JBIDE-19776
GITHUB_USER=mickaelistra # or nickboldt, etc. Use "origin" if PR is already applied
DEADLINE="within 7 days" # or "within 2 days" or "within 24 hours" or "as soon as possible"
PR_NUM=142
PR_URL=https://github.com/jbosstools/jbosstools-target-platforms/pull/${PR_NUM}/
P2DIFF="https://issues.jboss.org/secure/attachment/12389427/jbosstools.p2diff.txt"
echo "
Subject:
ACTION REQUIRED: Update target platform ${TARGET_PLATFORM_VERSION}
or
Proposed change to target platform ${TARGET_PLATFORM_VERSION}
Body:
Here is a proposal for a change to the JBoss Tools and Red Hat CodeReady Studio ${TARGET_PLATFORM_VERSION} target platforms.
${PR_URL}
It consists in the following change(s):
* ${JIRA_NUM}: Create and use Mars M7 target-platform
p2diff reports:
${P2DIFF}
Please review the above PR(s), as it will be applied *${DEADLINE}*.
You can use the following to build & test the target-platform locally against your component(s).
Build target-platform:
cd /path/to/jbosstools-target-platforms/jbosstools/multiple
git checkout ${PR_URL}
mvn clean install
Or, without hub:
cd /path/to/jbosstools-target-platforms/jbosstools/multiple
git fetch origin pull/${PR_NUM}/head && git checkout FETCH_HEAD
mvn clean install
If you'd like to build the "unified" target platform too:
cd /path/to/jbosstools-target-platforms/jbosstools/unified
mvn clean install -DtargetRepositoryUrl=file:///path/to/jbosstools-target-platforms/jbosstools/multiple/jbosstools-multiple.target.repo/
Then, to test the new "multiple" target platform against your component's build:
cd /path/to/your/jbosstools-component
mvn clean verify -Dtpc.version=${TARGET_PLATFORM_VERSION} -Dtpc.targetKind=multiple
Or, to test the new "unified" target platform against your component's build:
cd /path/to/your/jbosstools-component
mvn clean verify -Dtpc.version=${TARGET_PLATFORM_VERSION}
--
If you want to perform a scripted install of the entire target platform
into your local Eclipse or JBDS instance, you can now do so with this
script:
https://github.com/jbosstools/jbosstools-build-ci/blob/master/util/installFromTarget.sh
Usage is documented in the above script, and in this README:
https://github.com/jbosstools/jbosstools-target-platforms/tree/master#updating-versions-of-ius-in-target-files
--
More info:
(if applicable)
"
You can also use a Jenkins job to send the mail.
Then Await feedback: Generally, developers should have 7 days to review a proposed change. However, for more urgent changes (such as to fix a broken contribution from a previous PR or to remove something on which no projects still depend), 1 or 2 days may be sufficient. For components that are known to be potentially affected, directly ping their developers via the mailing-list or IRC to make sure they try it.
Be sure to state the deadline in the email so devs & project leads know the urgency.
When the following conditions are met:
-
Only one component is affected
-
The requester is the lead of the only affected component
-
There is no IU added/removed
-
It’s only an update of a project known to be snapshot
Then, we can assume that the requester’s feedback is enough, and ask for review directly on the Jira. If requesters gives a +1 for a given PR.
When the change is applied and that TP has been rebuilt and made available, send a mail to jbosstools-dev to notify about that change.
You can also use a Jenkins job to send the mail.