Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add prepare artifacts job; fixup AzDO fetch script #1679

Merged
merged 3 commits into from
Jul 24, 2020

Conversation

dagood
Copy link
Member

@dagood dagood commented Jul 22, 2020

Add a simple prepare-artifacts job to take over manual work preparing previously-source-built tarballs.

Fix up the AzDO fetch script: simplify and document for the scenario it's actually used for: downloading onto fresh machines for manual testing.


I worked on this to avoid manual steps for 3.1.106, and polished it a little more for this PR for next time.

Fixes #1676

@dagood dagood requested review from dseefeld and crummel July 22, 2020 16:53
@dagood dagood self-assigned this Jul 22, 2020
"$(nonportableSourceBuiltStageDir)" \
"$(portableSourceBuiltStageDir)" \
-iname 'Private.SourceBuilt.Artifacts.*.tar.gz' \
-exec tar -xf {} -C "$(allSourceBuiltStageDir)" \;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What about PackageVersions.props file? If last one in wins, it will not reflect the combined files. It seems like this needs to be coupled with #1638

Copy link
Member Author

@dagood dagood Jul 22, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is (the general process of) how @crummel made the previously-source-built tarball we're using now, and I don't see any PVPs, so I assume it worked out... I'm not sure.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, it's not strictly correct, but it works out because we don't use runtime package versions for anything right now. If we're productizing it, it might be worth looking into a bit. There's probably two things we care about:

  • getting all package versions into PackageVersions.props: I think something like adding unique lines from each individual PackageVersions.props to the output PackageVersions.props would work decently well here and probably shouldn't screw up too much. This currently doesn't matter but would cover us if we ever needed to depend on, e.g., a runtime package's version.
  • not having any conflicting package versions: in theory, this shouldn't happen. Package names and versions should be consistent across RIDs, and I don't think we've seen any real-life cases where this has gone wrong. I can't think offhand of a cheap way to detect this other than a bunch of sed nonsense, but I think it would be worth filing an issue for a future improvement.

I don't think either of these are blockers and #1638 should obviate them anyway.

@crummel
Copy link
Contributor

crummel commented Jul 22, 2020

Do you have the tarball from before you disabled for pull requests? I checked for the build but couldn't find it, I was just going to make sure it looked reasonable.

@dagood
Copy link
Member Author

dagood commented Jul 22, 2020

@dagood
Copy link
Member Author

dagood commented Jul 24, 2020

osx job failure (libicu not found in smoke-test) tracked by #1685, seems impossible to be related to this PR's changes. Ignoring for merge.

@dagood dagood merged commit 76ed853 into dotnet:release/3.1 Jul 24, 2020
@dagood dagood deleted the prepare-prev-sb-tb branch July 24, 2020 18:07
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Automatically create merged previously-source-built tar.gz in CI builds
3 participants