Release notes not complete due to current VSTS API limitation

[ReneSchumacher]

VSTS Extension you are using

GenerateReleaseNotes - affects xplat version as well

Where are you running it?

VSTS

Version of Extension/Task

2.x (latest published)

Expected behaviour and actual behaviour

Generated release notes should have all commits and work items associated with a release. Due to a current VSTS API limitation, the API listing the work items between two builds only inspects the first 200 commits that are linked to the build. Thus, if the number of commits between builds is greater than 200 and work items are linked to the 201st or earlier commit, those work items are missing in the release notes. This also affects the work item view in VSTS itself.

Steps to reproduce the problem

Create a commit with a linked work item. Create 200 additional commits without any linked work items. Run a release and generate release notes, the work item is not listed.

[ReneSchumacher]

@rfennell I have been discussing the issue with our product teams and I would like to create a work around for the issue (either a private one or - more preferably - a public one via PR).

My idea is to not change the default behavior of the task and instead introduce a variable that enables the workaround behavior. Once the issue has been fixed by our VSTS team, you'd just have to remove the additional code and everything works as previously designed.

You're opinion?

[rfennell]

@ReneSchumacher that sounds like a great plan, a checkbox parameter on the advanced menu perhaps to control the usage.

On point, I can see how this would effect the XPatform version, which just does a straight single API call to get the commit/wi details, but not sure how it also effect the PowerShell version, as this builds list of items by first getting a build list then getting the items for each and there is an advanced option to limit the number of WI/Commits per build. I guess you might be meaning that this number control though defaulting to 50 really has an upper limit of 200.

Always happy to consider PR, especially when they address critical issues

[ReneSchumacher]

I have to admit I didn't spend much time in the PowerShell version. Afaik, one of our consulting services teams is using the PowerShell version in one of their projects and they're running into this issue. The build API has this limitation hardcoded so if you ask VSTS for a build with associated work items it only returns work items linked to the last 200 commits of the build.

A quick review of the PowerShell code shows that you enumerate all builds and then add all linked work items to your global work item list. However, the list of work items in the build might already be incomplete. I don't see any easy way to fix the PowerShell version unless we change it to use different APIs, but I'd like to fix the cross-plat version. Actually, I already have a fix but I'm unable to build the task on my machine :(

[ReneSchumacher]

Yikes, forget what I wrote. I took a closer look at the PowerShell version now. The issue lies in the Get-BuildWorkItems method because the used API has the mentioned limitation. I can fix that as well, although it will be slightly more complicated.

I just realized that the PowerShell version can only be used in a build while the cross-plat version can only be used in a release. Is that by design?

[rfennell]

A bit of context...

The PowerShell version can be used in both a build or release.It all came out on on PowerShell script I wrote to dump associated items for a build. It got repeatedly extended to become a task, then an extension then one that support releases too. This was all home grown logic

When I needed release notes on XPlatform I decide not to move my own logic to Node, but to use the MS API as done by the VSTS UI. Hence this version only supports Releases

[ReneSchumacher]

I see. I just looked at the current version of the PowerShell's task.json and according to that the task is only visible in build (https://github.com/rfennell/vNextBuild/blob/3daa09583fe85a94a3d402cbe13282925736db0a/Extensions/GenerateReleaseNotes/GenerateReleaseNotesTask/task.json#L10)

[rfennell]

Has been fixed by @ReneSchumacher PR #352

[sbsprg]

Ran in to an issue with the fix above

2020-09-01T21:47:53.3864794Z Comparing 456 against 459
2020-09-01T21:47:53.4871926Z Comparing 453 against 459
2020-09-01T21:47:53.5722387Z Found required stage __default in the completed and successful state in build 453
2020-09-01T21:47:53.5725384Z Getting the details between 453 and 459
2020-09-01T21:47:53.6719355Z Using workaround for build API limitation (see issue #349)
2020-09-01T21:47:53.9415199Z (node:6184) UnhandledPromiseRejectionWarning: TypeError: Cannot read property 'forEach' of null
2020-09-01T21:47:53.9423235Z     at Object.<anonymous> (C:\DevOps-Agents\INVO-TMU\_work\_tasks\XplatGenerateReleaseNotes_7b0a5887-75c4-4ade-a915-f92a93c934cb\3.8.3\Issue349Workaround.js:29:30)
2020-09-01T21:47:53.9436105Z     at Generator.next (<anonymous>)
2020-09-01T21:47:53.9438849Z     at fulfilled (C:\..._work\_tasks\XplatGenerateReleaseNotes_7b0a5887-75c4-4ade-a915-f92a93c934cb\3.8.3\Issue349Workaround.js:5:58)
2020-09-01T21:47:53.9586957Z     at process._tickCallback (internal/process/next_tick.js:68:7)
2020-09-01T21:47:53.9662633Z (node:6184) UnhandledPromiseRejectionWarning: Unhandled promise rejection. This error originated either by throwing inside of an async function without a catch block, or by rejecting a promise which was not handled with .catch(). (rejection id: 1)
2020-09-01T21:47:53.9703811Z (node:6184) [DEP0018] DeprecationWarning: Unhandled promise rejections are deprecated. In the future, promise rejections that are not handled will terminate the Node.js process with a non-zero exit code.
2020-09-01T21:47:53.9726231Z ##[section]Finishing: Generate Release Notes based on Release Comparison API

What is the error condition that causes this?

[rfennell]

It looks as if the gitClient.getCommitsBatch() call is returning a null so there is no way to loop across it. I will add a null check and a bit of extra logging. I will try to get it out tomorrow

Note that this fix will only be available in the V3 release as 3.10.2