Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

new build scripts for release pipeline #1442

Merged
merged 16 commits into from
Apr 29, 2020
Merged

new build scripts for release pipeline #1442

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
16 commits
Select commit Hold shift + click to select a range
c3c7fdd
change build location when in VSTS
JamesWTruher Apr 7, 2020
272039b
copy newtonsoft assembly from proper location
JamesWTruher Apr 7, 2020
67cc341
Add support for verbose output during build
JamesWTruher Apr 7, 2020
db8bed0
update verbose behavior
JamesWTruher Apr 7, 2020
2243b62
better code to handle verbose
JamesWTruher Apr 7, 2020
8ed8171
file for signing analyzer files
JamesWTruher Apr 8, 2020
5ff3bff
tweaks to catalog creation
JamesWTruher Apr 9, 2020
eacda3b
Add suppression of UseCompatibleCommands rule for function that build…
JamesWTruher Apr 13, 2020
c391c7a
Improve message when there are failures in use compatible commands
JamesWTruher Apr 13, 2020
dc4653b
change logic for whether catalog can be created based on whether new-…
JamesWTruher Apr 13, 2020
52e5818
Add catalog signing configuration
JamesWTruher Apr 13, 2020
5d5ae33
fix typo in catalog name
JamesWTruher Apr 13, 2020
2bbd2d5
use 400 for signing
JamesWTruher Apr 13, 2020
1462e1b
Address PR feedback
JamesWTruher Apr 17, 2020
2fb32d3
add the suppression back to the function which creates the filecatalog
JamesWTruher Apr 21, 2020
f04b6a1
Create suppression file for CredScan so it does not get triggered.
JamesWTruher Apr 23, 2020
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion Tests/Rules/UseCompatibleCommands.Tests.ps1
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -400,7 +400,7 @@ Describe 'UseCompatibleCommands' {
}

$diagnostics = Invoke-ScriptAnalyzer -Path "$PSScriptRoot/../../" -IncludeRule $script:RuleName -Settings $settings
$diagnostics.Count | Should -Be 0 -Because ($diagnostics.RuleName -join ', ')
$diagnostics.Count | Should -Be 0 -Because ($diagnostics.Message -join ', ')
}
}

Expand Down
30 changes: 25 additions & 5 deletions build.ps1
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -34,13 +34,24 @@ param(
[switch] $InProcess,

[Parameter(ParameterSetName='Bootstrap')]
[switch] $Bootstrap
[switch] $Bootstrap,

[Parameter(ParameterSetName='BuildAll')]
[switch] $Catalog

)

BEGIN {
$verboseWanted = $false
if ( $PSBoundParameters['Verbose'] ) {
$verboseWanted = $PSBoundParameters['Verbose'].ToBool()
}
}

END {
Import-Module -Force (Join-Path $PSScriptRoot build.psm1)
Import-Module -Force (Join-Path $PSScriptRoot build.psm1) -verbose:$false
if ( $Clean -or $Clobber ) {
Remove-Build
Remove-Build -verbose:$false
if ( $PSCmdlet.ParameterSetName -eq "Clean" ) {
return
}
Expand All @@ -49,10 +60,19 @@ END {
$setName = $PSCmdlet.ParameterSetName
switch ( $setName ) {
"BuildAll" {
Start-ScriptAnalyzerBuild -All -Configuration $Configuration
$buildArgs = @{
All = $true
Configuration = $Configuration
Verbose = $verboseWanted
Catalog = $false
}
if ( $Catalog ) {
$buildArgs['Catalog'] = $true
}
Start-ScriptAnalyzerBuild @buildArgs
}
"BuildDocumentation" {
Start-ScriptAnalyzerBuild -Documentation
Start-ScriptAnalyzerBuild -Documentation -Verbose:$Verbose
}
"BuildOne" {
$buildArgs = @{
Expand Down
80 changes: 68 additions & 12 deletions build.psm1
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -114,7 +114,7 @@ function Start-DocumentationBuild
{
throw "Cannot find markdown documentation folder."
}
Import-Module platyPS
Import-Module platyPS -Verbose:$false
if ( -not (Test-Path $outputDocsPath)) {
$null = New-Item -Type Directory -Path $outputDocsPath -Force
}
Expand Down Expand Up @@ -150,7 +150,9 @@ function Start-ScriptAnalyzerBuild
[ValidateSet("Debug", "Release")]
[string]$Configuration = "Debug",

[switch]$Documentation
[switch]$Documentation,

[switch]$Catalog
)

BEGIN {
Expand All @@ -163,27 +165,36 @@ function Start-ScriptAnalyzerBuild
$foundVersion = Get-InstalledCLIVersion
Write-Warning "No suitable dotnet CLI found, requires version '$requiredVersion' found only '$foundVersion'"
}
$verboseWanted = $false
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why is this needed? PowerShell has the $VerbosePreference variable for that and should automatically propagate the usage of -Verbose to down to the other called cmdlets (splatting is the only exception where this does not work as far as I am aware)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

because I want finer grained control, I don't want verbose everywhere. I tried it without and the logs are super large and I would rather be more targeted (no reason to be verbose on import-module for example).

if ( $PSBoundParameters['Verbose'] ) {
$verboseWanted = $PSBoundParameters['Verbose'].ToBool()
}
}
END {

# Build docs either when -Documentation switch is being specified or the first time in a clean repo
$documentationFileExists = Test-Path (Join-Path $PSScriptRoot 'out\PSScriptAnalyzer\en-us\Microsoft.Windows.PowerShell.ScriptAnalyzer.dll-Help.xml')
if ( $Documentation -or -not $documentationFileExists )
{
Start-DocumentationBuild
Write-Verbose -Verbose:$verboseWanted -Message "Start-DocumentationBuild"
Start-DocumentationBuild -Verbose:$verboseWanted
}

if ( $All )
{
# Build all the versions of the analyzer
foreach($psVersion in 3..7) {
Start-ScriptAnalyzerBuild -Configuration $Configuration -PSVersion $psVersion
Start-ScriptAnalyzerBuild -Configuration $Configuration -PSVersion $psVersion -Verbose:$verboseWanted
}
if ( $Catalog ) {
New-Catalog -Location $script:destinationDir
}
return
}

if (-not $profilesCopied)
{
Write-Verbose -Verbose:$verboseWanted -Message "Copy-CompatibilityProfiles"
Copy-CompatibilityProfiles
# Set the variable in the caller's scope, so this will only happen once
Set-Variable -Name profilesCopied -Value $true -Scope 1
Expand Down Expand Up @@ -253,12 +264,24 @@ function Start-ScriptAnalyzerBuild
# The Rules project has a dependency on the Engine therefore just building the Rules project is enough
try {
Push-Location $projectRoot/Rules
Write-Progress "Building ScriptAnalyzer for PSVersion '$PSVersion' using framework '$framework' and configuration '$Configuration'"
$message = "Building ScriptAnalyzer for PSVersion '$PSVersion' using framework '$framework' and configuration '$Configuration'"
Write-Verbose -Verbose:$verboseWanted -Message "$message"
Write-Progress "$message"
if ( -not $script:DotnetExe ) {
$script:DotnetExe = Get-DotnetExe
}
$buildOutput = & $script:DotnetExe build --framework $framework --configuration "$buildConfiguration" 2>&1
$dotnetArgs = "build",
"--framework",
$framework,
"--configuration",
"$buildConfiguration"
if ( $env:TF_BUILD ) {
$dotnetArgs += "--output"
$dotnetArgs += "${PSScriptRoot}\bin\${buildConfiguration}\${framework}"
}
$buildOutput = & $script:DotnetExe $dotnetArgs 2>&1
if ( $LASTEXITCODE -ne 0 ) { throw "$buildOutput" }
Write-Verbose -Verbose:$verboseWanted -message "$buildOutput"
}
catch {
Write-Warning $_
Expand All @@ -271,24 +294,57 @@ function Start-ScriptAnalyzerBuild

Publish-File $itemsToCopyCommon $script:destinationDir

$itemsToCopyBinaries = @(
"$projectRoot\Engine\bin\${buildConfiguration}\${Framework}\Microsoft.Windows.PowerShell.ScriptAnalyzer.dll",
"$projectRoot\Rules\bin\${buildConfiguration}\${Framework}\Microsoft.Windows.PowerShell.ScriptAnalyzer.BuiltinRules.dll"
"$projectRoot\Rules\bin\${buildConfiguration}\${framework}\Microsoft.PowerShell.CrossCompatibility.dll"
)
if ( $env:TF_BUILD ) {
$itemsToCopyBinaries = @(
"$projectRoot\bin\${buildConfiguration}\${Framework}\Microsoft.Windows.PowerShell.ScriptAnalyzer.dll",
"$projectRoot\bin\${buildConfiguration}\${Framework}\Microsoft.Windows.PowerShell.ScriptAnalyzer.BuiltinRules.dll"
"$projectRoot\bin\${buildConfiguration}\${framework}\Microsoft.PowerShell.CrossCompatibility.dll"
)
}
else {
$itemsToCopyBinaries = @(
"$projectRoot\Engine\bin\${buildConfiguration}\${Framework}\Microsoft.Windows.PowerShell.ScriptAnalyzer.dll",
"$projectRoot\Rules\bin\${buildConfiguration}\${Framework}\Microsoft.Windows.PowerShell.ScriptAnalyzer.BuiltinRules.dll"
"$projectRoot\Rules\bin\${buildConfiguration}\${framework}\Microsoft.PowerShell.CrossCompatibility.dll"
)
}
Publish-File $itemsToCopyBinaries $destinationDirBinaries

$settingsFiles = Get-Childitem "$projectRoot\Engine\Settings" | ForEach-Object -MemberName FullName
Publish-File $settingsFiles (Join-Path -Path $script:destinationDir -ChildPath Settings)

if ($framework -eq 'net452') {
Copy-Item -path "$projectRoot\Rules\bin\${buildConfiguration}\${framework}\Newtonsoft.Json.dll" -Destination $destinationDirBinaries
if ( $env:TF_BUILD ) {
$nsoft = "$projectRoot\bin\${buildConfiguration}\${framework}\Newtonsoft.Json.dll"
}
else {
$nsoft = "$projectRoot\Rules\bin\${buildConfiguration}\${framework}\Newtonsoft.Json.dll"
}
Copy-Item -path $nsoft -Destination $destinationDirBinaries
}

Pop-Location
}
}

function New-Catalog
{
[System.Diagnostics.CodeAnalysis.SuppressMessageAttribute('PSUseCompatibleCommands', '')]
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is there a specific diagnostic here that was appearing?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

One of the stages in the sign build was to run all the tests, and one of the tests found new-filecatalog not being present in all versions, but I hadn't picked up the latest master. Also, I removed the test phase from the build since it happens in CI. I can pull this line.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nope - I need to put it back - now it's failing in the CI build

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No worries -- perhaps we can open an issue to track removing this?

param ( [Parameter()]$Location )
$newFileCatalog = Get-Command -ErrorAction SilentlyContinue New-FileCatalog
if ($null -eq $newFileCatalog) {
Write-Warning "New-FileCatalog not found, not creating catalog"
return
}
try {
Push-Location $Location
New-FileCatalog -CatalogFilePath PSScriptAnalyzer.cat -Path .
}
finally {
Pop-Location
}
}

# TEST HELPERS
# Run our tests
function Test-ScriptAnalyzer
Expand Down
43 changes: 43 additions & 0 deletions tools/releaseBuild/AssemblySignConfig.xml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,43 @@
<?xml version="1.0" encoding="utf-8" ?>
<SignConfigXML>
<!-- ****Begin**** BothDual - Dual (Sha256 and Sha1) AuthicodeDual) and should be StrongName, but we will add this in 6.1.0 ******** -->
<job platform="" configuration="" dest="__OUTPATHROOT__\signed" jobname="PowerShell Script Analyzer" approvers="vigarg;gstolt">
<file src="__INPATHROOT__\out\PSScriptAnalyzer\1.19.0\Microsoft.Windows.PowerShell.ScriptAnalyzer.BuiltinRules.dll" signType="400" dest="__OUTPATHROOT__\PSScriptAnalyzer\1.19.0\Microsoft.Windows.PowerShell.ScriptAnalyzer.BuiltinRules.dll" />
<file src="__INPATHROOT__\out\PSScriptAnalyzer\1.19.0\Microsoft.Windows.PowerShell.ScriptAnalyzer.dll" signType="400" dest="__OUTPATHROOT__\PSScriptAnalyzer\1.19.0\Microsoft.Windows.PowerShell.ScriptAnalyzer.dll" />
<file src="__INPATHROOT__\out\PSScriptAnalyzer\1.19.0\Microsoft.PowerShell.CrossCompatibility.dll" signType="400" dest="__OUTPATHROOT__\PSScriptAnalyzer\1.19.0\Microsoft.PowerShell.CrossCompatibility.dll" />
<file src="__INPATHROOT__\out\PSScriptAnalyzer\1.19.0\PSScriptAnalyzer.psd1" signType="400" dest="__OUTPATHROOT__\PSScriptAnalyzer\1.19.0\PSScriptAnalyzer.psd1" />
<file src="__INPATHROOT__\out\PSScriptAnalyzer\1.19.0\PSScriptAnalyzer.psm1" signType="400" dest="__OUTPATHROOT__\PSScriptAnalyzer\1.19.0\PSScriptAnalyzer.psm1" />
<file src="__INPATHROOT__\out\PSScriptAnalyzer\1.19.0\ScriptAnalyzer.format.ps1xml" signType="400" dest="__OUTPATHROOT__\PSScriptAnalyzer\1.19.0\ScriptAnalyzer.format.ps1xml" />
<file src="__INPATHROOT__\out\PSScriptAnalyzer\1.19.0\ScriptAnalyzer.types.ps1xml" signType="400" dest="__OUTPATHROOT__\PSScriptAnalyzer\1.19.0\ScriptAnalyzer.types.ps1xml" />
<file src="__INPATHROOT__\out\PSScriptAnalyzer\1.19.0\Settings\CmdletDesign.psd1" signType="400" dest="__OUTPATHROOT__\PSScriptAnalyzer\1.19.0\Settings\CmdletDesign.psd1" />
<file src="__INPATHROOT__\out\PSScriptAnalyzer\1.19.0\Settings\CodeFormatting.psd1" signType="400" dest="__OUTPATHROOT__\PSScriptAnalyzer\1.19.0\Settings\CodeFormatting.psd1" />
<file src="__INPATHROOT__\out\PSScriptAnalyzer\1.19.0\Settings\CodeFormattingAllman.psd1" signType="400" dest="__OUTPATHROOT__\PSScriptAnalyzer\1.19.0\Settings\CodeFormattingAllman.psd1" />
<file src="__INPATHROOT__\out\PSScriptAnalyzer\1.19.0\Settings\CodeFormattingOTBS.psd1" signType="400" dest="__OUTPATHROOT__\PSScriptAnalyzer\1.19.0\Settings\CodeFormattingOTBS.psd1" />
<file src="__INPATHROOT__\out\PSScriptAnalyzer\1.19.0\Settings\CodeFormattingStroustrup.psd1" signType="400" dest="__OUTPATHROOT__\PSScriptAnalyzer\1.19.0\Settings\CodeFormattingStroustrup.psd1" />
<file src="__INPATHROOT__\out\PSScriptAnalyzer\1.19.0\Settings\DSC.psd1" signType="400" dest="__OUTPATHROOT__\PSScriptAnalyzer\1.19.0\Settings\DSC.psd1" />
<file src="__INPATHROOT__\out\PSScriptAnalyzer\1.19.0\Settings\PSGallery.psd1" signType="400" dest="__OUTPATHROOT__\PSScriptAnalyzer\1.19.0\Settings\PSGallery.psd1" />
<file src="__INPATHROOT__\out\PSScriptAnalyzer\1.19.0\Settings\ScriptFunctions.psd1" signType="400" dest="__OUTPATHROOT__\PSScriptAnalyzer\1.19.0\Settings\ScriptFunctions.psd1" />
<file src="__INPATHROOT__\out\PSScriptAnalyzer\1.19.0\Settings\ScriptingStyle.psd1" signType="400" dest="__OUTPATHROOT__\PSScriptAnalyzer\1.19.0\Settings\ScriptingStyle.psd1" />
<file src="__INPATHROOT__\out\PSScriptAnalyzer\1.19.0\Settings\ScriptSecurity.psd1" signType="400" dest="__OUTPATHROOT__\PSScriptAnalyzer\1.19.0\Settings\ScriptSecurity.psd1" />
</job>
<job platform="" configuration="" dest="__OUTPATHROOT__\signed" jobname="PowerShell Script Analyzer PSv6" approvers="vigarg;gstolt">
<file src="__INPATHROOT__\out\PSScriptAnalyzer\1.19.0\PSv6\Microsoft.Windows.PowerShell.ScriptAnalyzer.BuiltinRules.dll" signType="400" dest="__OUTPATHROOT__\PSScriptAnalyzer\1.19.0\PSv6\Microsoft.Windows.PowerShell.ScriptAnalyzer.BuiltinRules.dll" />
<file src="__INPATHROOT__\out\PSScriptAnalyzer\1.19.0\PSv6\Microsoft.Windows.PowerShell.ScriptAnalyzer.dll" signType="400" dest="__OUTPATHROOT__\PSScriptAnalyzer\1.19.0\PSv6\Microsoft.Windows.PowerShell.ScriptAnalyzer.dll" />
<file src="__INPATHROOT__\out\PSScriptAnalyzer\1.19.0\PSv6\Microsoft.PowerShell.CrossCompatibility.dll" signType="400" dest="__OUTPATHROOT__\PSScriptAnalyzer\1.19.0\PSv6\Microsoft.PowerShell.CrossCompatibility.dll" />
</job>
<job platform="" configuration="" dest="__OUTPATHROOT__\signed" jobname="PowerShell Script Analyzer PSv7" approvers="vigarg;gstolt">
<file src="__INPATHROOT__\out\PSScriptAnalyzer\1.19.0\PSv7\Microsoft.Windows.PowerShell.ScriptAnalyzer.BuiltinRules.dll" signType="400" dest="__OUTPATHROOT__\PSScriptAnalyzer\1.19.0\PSv7\Microsoft.Windows.PowerShell.ScriptAnalyzer.BuiltinRules.dll" />
<file src="__INPATHROOT__\out\PSScriptAnalyzer\1.19.0\PSv7\Microsoft.Windows.PowerShell.ScriptAnalyzer.dll" signType="400" dest="__OUTPATHROOT__\PSScriptAnalyzer\1.19.0\PSv7\Microsoft.Windows.PowerShell.ScriptAnalyzer.dll" />
<file src="__INPATHROOT__\out\PSScriptAnalyzer\1.19.0\PSv7\Microsoft.PowerShell.CrossCompatibility.dll" signType="400" dest="__OUTPATHROOT__\PSScriptAnalyzer\1.19.0\PSv7\Microsoft.PowerShell.CrossCompatibility.dll" />
</job>
<job platform="" configuration="" dest="__OUTPATHROOT__\signed" jobname="PowerShell Script Analyzer PSv4" approvers="vigarg;gstolt">
<file src="__INPATHROOT__\out\PSScriptAnalyzer\1.19.0\PSv4\Microsoft.Windows.PowerShell.ScriptAnalyzer.BuiltinRules.dll" signType="400" dest="__OUTPATHROOT__\PSScriptAnalyzer\1.19.0\PSv4\Microsoft.Windows.PowerShell.ScriptAnalyzer.BuiltinRules.dll" />
<file src="__INPATHROOT__\out\PSScriptAnalyzer\1.19.0\PSv4\Microsoft.Windows.PowerShell.ScriptAnalyzer.dll" signType="400" dest="__OUTPATHROOT__\PSScriptAnalyzer\1.19.0\PSv4\Microsoft.Windows.PowerShell.ScriptAnalyzer.dll" />
<file src="__INPATHROOT__\out\PSScriptAnalyzer\1.19.0\PSv4\Microsoft.PowerShell.CrossCompatibility.dll" signType="400" dest="__OUTPATHROOT__\PSScriptAnalyzer\1.19.0\PSv4\Microsoft.PowerShell.CrossCompatibility.dll" />
</job>
<job platform="" configuration="" dest="__OUTPATHROOT__\signed" jobname="PowerShell Script Analyzer PSv3" approvers="vigarg;gstolt">
<file src="__INPATHROOT__\out\PSScriptAnalyzer\1.19.0\PSv3\Microsoft.Windows.PowerShell.ScriptAnalyzer.BuiltinRules.dll" signType="400" dest="__OUTPATHROOT__\PSScriptAnalyzer\1.19.0\PSv3\Microsoft.Windows.PowerShell.ScriptAnalyzer.BuiltinRules.dll" />
<file src="__INPATHROOT__\out\PSScriptAnalyzer\1.19.0\PSv3\Microsoft.Windows.PowerShell.ScriptAnalyzer.dll" signType="400" dest="__OUTPATHROOT__\PSScriptAnalyzer\1.19.0\PSv3\Microsoft.Windows.PowerShell.ScriptAnalyzer.dll" />
<file src="__INPATHROOT__\out\PSScriptAnalyzer\1.19.0\PSv3\Microsoft.PowerShell.CrossCompatibility.dll" signType="400" dest="__OUTPATHROOT__\PSScriptAnalyzer\1.19.0\PSv3\Microsoft.PowerShell.CrossCompatibility.dll" />
</job>
</SignConfigXML>
6 changes: 6 additions & 0 deletions tools/releaseBuild/CatalogSignConfig.xml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
<?xml version="1.0" encoding="utf-8" ?>
<SignConfigXML>
<job platform="" configuration="" dest="__OUTPATHROOT__\signed" jobname="PowerShell Script Analyzer File Catalog" approvers="vigarg;gstolt">
<file src="__INPATHROOT__\PSScriptAnalyzer\1.19.0\PSScriptAnalyzer.cat" signType="400" dest="__OUTPATHROOT__\PSScriptAnalyzer\1.19.0\PSScriptAnalyzer.cat" />
</job>
</SignConfigXML>
15 changes: 15 additions & 0 deletions tools/releaseBuild/CredScan.Suppressions.json
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,15 @@
{
"tool": "Credential Scanner",
"suppressions": [
{ "file": "\\Engine\\Settings\\desktop-4.0-windows.json",
"_justification": "The file contains the list of all parameters of a cmdlet but no passwords are actually present." },
{ "file": "\\Engine\\Settings\\desktop-3.0-windows.json",
"_justification": "The file contains the list of all parameters of a cmdlet but no passwords are actually present." },
{ "file": "\\Engine\\Settings\\desktop-5.1.14393.206-windows.json",
"_justification": "The file contains the list of all parameters of a cmdlet but no passwords are actually present." },
{ "file": "\\Tests\\Engine\\RuleSuppression.tests.ps1",
"_justification": "The parameter password is used in function declaration for test but is not called and no password is present." }
]
}