Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: Initial version of Save-AzDevOpsBuild #183

Merged
merged 20 commits into from
Jul 14, 2021

Conversation

fgheysels
Copy link
Member

@fgheysels fgheysels commented Jun 23, 2021

A first version is created of a function that allows you to retain a DevOps build indefnitely.

This is based on the 'Builds' API of Azure DevOps, which allow you to update a build and set a 'keepForever' flag.

This results in a lease that is being created for the designated build. (I've also investigated the Leases API but then you have to specify an owner for that lease, whereas here you do not have to do that).

The Powershell function takes 2 arguments:

  • ProjectId: The Id of the project for the pipeline of which you want to retain a build. When called from a DevOps pipeline, you can make use of the $(System.TeamProjectId) predefined variable
  • BuildId: The Id of the build that you want to retain. When called from a DevOps pipeline, you can use the $(Build.BuildId) variable.

It relies on Azure DevOps variables to determine the URL of the project and to get an access-token that is necessary to do the API request

This relates to #182

@fgheysels
Copy link
Member Author

Not sure if I need some other changes to incorporate this change.
I guess I'll need to include the name of this function in the 'FunctionsToExport' variable in the psd1 file ? Are there any other changes required ?

exit 1
}

exit 0
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm absolutely no Powershell expert, so if you have any improvements, recommendations, .... please let me know :)

@fgheysels fgheysels changed the title Initial version of Set-RetainBuildFlag feature: Initial version of Set-RetainBuildFlag Jun 23, 2021
@fgheysels fgheysels changed the title feature: Initial version of Set-RetainBuildFlag feat: Initial version of Set-RetainBuildFlag Jun 23, 2021
@fgheysels
Copy link
Member Author

I'm also open for suggestions for a better name for this function :)

Copy link
Member

@stijnmoreels stijnmoreels left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great addition! Thanks @fgheysels , great work. 🥇
I would ask you to add some unit tests:

  • Stubbed response's status code to verify that the scripts fails
  • Passing information results in an expected request

And an integration test, but we can take this up for you if you want:

  • Setting the build flag on the current build,
    • checking it's there in the assertion,
    • and removing it back in the tear-down of the test.

src/Arcus.Scripting.DevOps/Arcus.Scripting.DevOps.psm1 Outdated Show resolved Hide resolved
src/Arcus.Scripting.DevOps/Scripts/Set-RetainBuildFlag.ps1 Outdated Show resolved Hide resolved

$requestBody = $retentionPayload | ConvertTo-Json -Depth 100

$requestUri = "https://dev.azure.com/$OrganizationName/$ProjectId/_apis/build/builds/" + $BuildId + "?api-version=6.0"
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It may be important to also make sure that legacy Azure DevOps systems can benefit from this (legacy = {organizationName}.visualstudio.com).
As I see it, you can benefit from the environment variable $env:SYSTEM_TEAMFOUNDATIONCOLLECTIONURI to retrieve the root URL where all Azure DevOps API calls can be made.

Context: https://docs.microsoft.com/en-us/azure/devops/extend/develop/work-with-urls?view=azure-devops&tabs=http

On the other hand, we may consider using a client package for this. But that may be taking things to far.
https://docs.microsoft.com/en-us/azure/devops/integrate/concepts/dotnet-client-libraries?view=azure-devops

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is the API version also something we should parameterize? Default can be 6.0.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I didn't want to parameterize the complete 'base' uri, as I thought it would make this more complex to pass this in as a parameter.
Unless you rely on this environment variable inside the script and do not pass it via a param.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

To be honest, I don't see a point right now in parametrizing the API version ? @mbraekman ?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What we're typically doing, in case it's applicable, is set it as a parameter but default it to a specific version (as Stijn indicates). (for example in Enable-AzLogicApp)
The reason for this is that based on this approach the same script can easily be used to use the latest/another specific version of the API, without requiring any changes in the script.

However, while API versions change very often in the case of ARM templates, for instance, this doesn't seem to be the case for DevOps-related APIs. So, not quite sure if in this case, it would be very useful.
--> So not really required in this case.

Regarding the base-URI, I would rely on the built-in variables as much as possible, as this requires less input from the client using the script, increasing user-friendliness. But, even so, we should still offer users the possibility to provide their own custom values, just in case.
--> use built-in variable.

Copy link
Member Author

@fgheysels fgheysels Jun 25, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

But then I'd suggest that we do not pass in the organizationname parameter, but put the DevOps variable which contains the Uri inside the script ? This has the side effect that this script is only usable from within an Azure DevOps pipeline; that's why I opted for passing in the organisationName as a parameter, and not to rely on the System.CollectionUri variable which defines this base-url.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, exactly. So, we could opt for using the built-in base URL but that this is the default value for a custom provided URL?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think this would make it more useable. You'll have to pass in an url, consumer needs to know if it requires a trailing slash or not (ok, you can handle that in your script), but imho, it makes the whole thing a bit less user-friendly

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

src/Arcus.Scripting.DevOps/Scripts/Set-RetainBuildFlag.ps1 Outdated Show resolved Hide resolved
src/Arcus.Scripting.DevOps/Scripts/Set-RetainBuildFlag.ps1 Outdated Show resolved Hide resolved
src/Arcus.Scripting.DevOps/Scripts/Set-RetainBuildFlag.ps1 Outdated Show resolved Hide resolved
@stijnmoreels
Copy link
Member

Not sure if I need some other changes to incorporate this change.
I guess I'll need to include the name of this function in the 'FunctionsToExport' variable in the psd1 file ? Are there any other changes required ?

No, as I see it, all the necessary changes were made. Great job! 👍

@fgheysels fgheysels requested a review from stijnmoreels June 25, 2021 18:56
Copy link
Member

@stijnmoreels stijnmoreels left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great! Now we only need some tests. Could you provide some unit tests? We'll make sure that there's some integration test then if that's too much work for you.

A test I'm thinking on, is that the returned status code determines whether or not the script fails.

@fgheysels
Copy link
Member Author

Great! Now we only need some tests. Could you provide some unit tests? We'll make sure that there's some integration test then if that's too much work for you.

A test I'm thinking on, is that the returned status code determines whether or not the script fails.

I've provided two tests in the Azure DevOps context. Don't know if that 's ok or if you want to have a different per script.
Also, I'm struggling a bit with running the tests locally.

I've tried Invoke-Pester -Path <file> but got the error that my script is not a script module. After some searching, I found out that I can install Pester instead of using the built-in version of Pester, but still running into errors.
Some guidance on how to run the tests locally would be nice. (See also issue #188)

@fgheysels fgheysels changed the title feat: Initial version of Set-RetainBuildFlag feat: Initial version of Save-AzDevOpsBuild Jul 7, 2021
@fgheysels
Copy link
Member Author

We'll make sure that there's some integration test then if that's too much work for you.

It would be nice if you could provide some integration tests indeed :)

@stijnmoreels
Copy link
Member

We'll make sure that there's some integration test then if that's too much work for you.

It would be nice if you could provide some integration tests indeed :)

Ok, we'll look into that.

@fgheysels
Copy link
Member Author

We'll make sure that there's some integration test then if that's too much work for you.

It would be nice if you could provide some integration tests indeed :)

Ok, we'll look into that.

Is that necessary to complete this PR ? I actually would like to use this asap :)

Copy link
Member

@stijnmoreels stijnmoreels left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We'll make sure that there's some integration test then if that's too much work for you.

It would be nice if you could provide some integration tests indeed :)

Ok, we'll look into that.

Is that necessary to complete this PR ? I actually would like to use this asap :)

I guess not, we can make a preview package for you once this PR is completed. We can then take up the integration test ourselves. One remark I have is that it would be best to streamline this DevOps script with the other script we have:
https://github.com/arcus-azure/arcus.scripting/blob/master/src/Arcus.Scripting.DevOps/Scripts/Set-AzDevOpsArmOutputs.ps1
Take a look at how we did handled the subscription-specific information there. It can help resolve some discussions we had in this PR. Thanks again!

Oh, and I saw that while renaming the script, you didn't update the project file?

Once this is done, we can complete this, right @mbraekman ?

Copy link
Member

@stijnmoreels stijnmoreels left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@fgheysels
Copy link
Member Author

fgheysels commented Jul 9, 2021

One remark I have is that it would be best to streamline this DevOps script with the other script we have:
https://github.com/arcus-azure/arcus.scripting/blob/master/src/Arcus.Scripting.DevOps/Scripts/Set-AzDevOpsArmOutputs.ps1
Take a look at how we did handled the subscription-specific information there. It can help resolve some discussions we had in this PR. Thanks again!

You mean making use of the DevOps variables like System.CollectionUri (https://docs.microsoft.com/en-us/azure/devops/pipelines/build/variables?view=azure-devops&tabs=yaml#system-variables-devops-services) I can do that, but imho it will make the script less usable (and maybe also harder to test ? )

Next to that, I can also rely on System.AccessToken and bake that into the script instead of having it to pass it in as a parameter, but also, this results in tight coupling the script with Azure Devops Pipelines. You'll not be able to use it in any other way. (Same goes for the Collectionuri variable.

Don't you think the script will be to tightly coupled to Azure DevOps, as we're going to use DevOps specific variables inside the script ?

@fgheysels fgheysels requested a review from stijnmoreels July 12, 2021 17:54
@stijnmoreels
Copy link
Member

One remark I have is that it would be best to streamline this DevOps script with the other script we have:
https://github.com/arcus-azure/arcus.scripting/blob/master/src/Arcus.Scripting.DevOps/Scripts/Set-AzDevOpsArmOutputs.ps1
Take a look at how we did handled the subscription-specific information there. It can help resolve some discussions we had in this PR. Thanks again!

You mean making use of the DevOps variables like System.CollectionUri (https://docs.microsoft.com/en-us/azure/devops/pipelines/build/variables?view=azure-devops&tabs=yaml#system-variables-devops-services) I can do that, but imho it will make the script less usable (and maybe also harder to test ? )

Next to that, I can also rely on System.AccessToken and bake that into the script instead of having it to pass it in as a parameter, but also, this results in tight coupling the script with Azure Devops Pipelines. You'll not be able to use it in any other way. (Same goes for the Collectionuri variable.

Don't you think the script will be to tightly coupled to Azure DevOps, as we're going to use DevOps specific variables inside the script ?

Yeah, you definitely have a point. I think it's mainly for Azure DevOps intended, but let's ask @mbraekman bc I'm not sure 😅. I'll approve my part.

@stijnmoreels
Copy link
Member

Thanks again for your patience and time with this one! 🥇

@stijnmoreels stijnmoreels removed their assignment Jul 13, 2021
@mbraekman
Copy link
Contributor

One remark I have is that it would be best to streamline this DevOps script with the other script we have:
https://github.com/arcus-azure/arcus.scripting/blob/master/src/Arcus.Scripting.DevOps/Scripts/Set-AzDevOpsArmOutputs.ps1
Take a look at how we did handled the subscription-specific information there. It can help resolve some discussions we had in this PR. Thanks again!

You mean making use of the DevOps variables like System.CollectionUri (https://docs.microsoft.com/en-us/azure/devops/pipelines/build/variables?view=azure-devops&tabs=yaml#system-variables-devops-services) I can do that, but imho it will make the script less usable (and maybe also harder to test ? )
Next to that, I can also rely on System.AccessToken and bake that into the script instead of having it to pass it in as a parameter, but also, this results in tight coupling the script with Azure Devops Pipelines. You'll not be able to use it in any other way. (Same goes for the Collectionuri variable.
Don't you think the script will be to tightly coupled to Azure DevOps, as we're going to use DevOps specific variables inside the script ?

Yeah, you definitely have a point. I think it's mainly for Azure DevOps intended, but let's ask @mbraekman bc I'm not sure 😅. I'll approve my part.

As discussed, these scripts are meant to be used by/in Azure DevOps, so tightly coupling is not a problem, as long as we provide clear documentation on how/what it does.

Copy link
Contributor

@mbraekman mbraekman left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Only 1 thing missing: documentation ;)

@fgheysels
Copy link
Member Author

Only 1 thing missing: documentation ;)

Added!

@fgheysels fgheysels merged commit dd0d81d into arcus-azure:master Jul 14, 2021
@fgheysels fgheysels deleted the frgh/feature/retain_build branch July 14, 2021 15:37
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants