[Feature]: Enforce valid .NET version types in evergreen objects

[BronsonMagnan]

What is your feature request?

It would be nice if Evergreen could filter scrub malformed version numbers that do not conform to the .NET version class.
Two examples in the list of Evergreen applications now are these:
X:\evergreen\stealthpuppyWindowsCustomisedDefaults\2406.01.173\x86 (the leading 0 in the minor version number is not permitted and should be cleaned up)
X:\evergreen\QGIS\latest\3.38.0-1 (non '.' separators are not permitted)
These types of issues cause problems for automation solutions that are leveraging the .NET version class for performing comparisons. I think extending that class on the developer side is a bridge too far, and the ideally the problem should be fixed at the supplier, but a compromise would be to have Evergreen filter and replace non conforming version numbers

Have you tested against the current version?

• Yes, I've updated Evergreen and tested against the current version.

Have you reviewed the documentation?

• Documentation at: https://stealthpuppy.com/evergreen/

[BronsonMagnan]

OneDrive also does this:
X:\evergreen\MicrosoftOneDrive\Enterprise\24.010.0114.0003\x64

[BronsonMagnan]

Arcrobat does leading zeros.
X:\evergreen\AdobeAcrobatProStdDC\24.002.20857\x64

[BronsonMagnan]

A sample for the filter could look like this:

function Format-Version () {
     [cmdletBinding()]
     [OutputType([string[]])]
     param(
          [parameter(valueFromPipeLine=$true)]
          [string[]]$Version
     )
     begin {
          $badSeparator = "[\d+\.]{0,2}(\d+([\-\/\+\~\:\\\\_])\d+)[\.\d+]{0,2}"
          $leadingZeros = "\b(0\d+|(\d+\.0\d+)|(\d+\.\d+\.0\d+)|(\d+\.\d+\.\d+\.0\d+)|(\d+\.0\d+\.\d+)|(\d+\.0\d+\.\d+\.0\d+)|(\d+\.\d+\.0\d+\.\d+))\b"
     }
     process {
          foreach ($Item in $Version) {
               if ($Item -match $badSeparator) { 
                    $badChar = $matches[2]
                    $Item = $Item.replace($badchar,'.')
                    $Item = Format-Version -Version $Item
               }
               if ($Item -match $leadingZeros) { 
                    $Item = ([Version]$Item).tostring()
               }
               Write-Output $item
          }

     }
     end {

     }
}

[JimMoyle]

I'd say reporting back the unscrubbed number is best and then consumers could scrub themselves as you demonstrate if needed.

[BronsonMagnan]

Jim do you have to deal with this problem in production? If so, how are you handling it?

[JimMoyle]

I'd generally normalise the version number after getting it to do comparisons later, but the reason I say don't do it in evergreen is you never know what people have written, they may be parsing the number any unusual way on a per app basis when they are consuming the output of evergreen.

It may be that someone at the ISV decides randomly to put the string 'release' in the version or do dot versions in 6 places. There are so many failure cases that you can't build a regex for, and they are totally out of the control of Evergreen itself.

You could try type conversion, then build regex replacements something like the below in a catch block if it fails, maybe adding your regex too, but I'd say it will always have issues.

$verCount = $Version.ToString().Split('.').Count
 switch ($true) {
     { $verCount -eq 4 } { $outputVer = $Version ;break}
     { $verCount -lt 4 } { 
          $outputVer = (4 - $verCount)..1 | ForEach-Object {
               $Version.ToString() + '.0'
           } 
                break
     }
     { $verCount -gt 4 } {
          $outputVer = $Version.ToString() -replace "^(\d+(?:\.\d+){0,3})(?:\.\d+)*$", $matches[1]
          break
     }
            Default {}
}

I'd say this would work mostly, but the issues raised on this repo would significantly increase when vendors just did weird shit with their versioning.

Given the lack of control over the source data I'd just pass it through to the module user.

Given it's not my decision, it's Aaron's, but that's what I'd do in his position.

[BronsonMagnan]

they may be parsing the number any unusual way on a per app basis when they are consuming the output of evergreen

I would say that this is the direct result of unsanitized version numbers. At least that is my case.

[aaronparker]

I'm hesitant to modify the version number output as it comes from the vendor too much. Below is an example of some of the version numbers - some of these could be fixed up, .e.g removing jdk, and v but these examples are on a per-app basis.

Some of the examples are obviously terrible version numbers - how do we determine how they should be modified?

8u422+6
22.0.1+8
v22-build1
21.0.4.7.1
17.0.12.7.1
jdk-11.0.23+9
7.1.1-35
2024b
129.0b3
2023.2.20f1

[aaronparker]

We could take that Format-Version example and include it as a function in Evergreen, so that you could run:

Get-EvergreenApp -Name "MicrosoftOneDrive" | Format-Version

[DanGough]

I do my own version sanitisation so that I can be in control of how it works and have the ability to quickly fix stuff when vendors do something unusual. Also because I get versions from multiple sources other than Evergreen. But it would definitely be of benefit to have something built in too.

Rather than returning a string that can be cast as a version, how about returning a VersionObject property that has already been cast?

BTW I also convert to a SQLVersion, which pads out each field to have up to 9 digits, which enables version sorting and comparison in SQL statements when these values are stored in a database. E.g. 1.2.3 becomes 100000000200000000300000000000000000. Not totally relevant to this topic, but just sharing an idea I had that solved a problem of mine!

[BronsonMagnan]

Get-EvergreenApp -Name "MicrosoftOneDrive" | Format-Version | Save-EvergreenApp ... would be perfect, sweet normalization.

[BronsonMagnan]

Aaron, probably a good idea to have the format function move the original vendor version to a new field in the object, in case it needed to be referenced for some reason. I imagine users will do filtering cmdlets after the format version step. Replacing the original version field seems like the way to go though, so the least amount of stuff needs to change.

[BronsonMagnan]

I figured this out, it maintains all sortability, and normalized to a 4 segment version number

function Normalize-Segment {
    param (
        [string]$segment
    )

    if ($segment -match '^\d+$') {
        return [int]$segment
    } else {
        $normalized = 0
        foreach ($char in $segment.ToCharArray()) {
            $normalized = $normalized * 100 + [int][char]$char
        }
        return $normalized
    }
}

function Convert-ToStandardVersion {
    param (
        [string]$version
    )

    $segments = $version -split '[.\-_+]'
    $normalizedSegments = @()

    foreach ($segment in $segments) {
        $normalizedSegments += @(Normalize-Segment -segment $segment)
    }

    if ($normalizedSegments.Count -gt 4) {
        $normalizedSegments = $normalizedSegments[0..2] + ($normalizedSegments[3..($normalizedSegments.Count - 1)] | Measure-Object -Sum).Sum
    }

    while ($normalizedSegments.Count -lt 4) {
        $normalizedSegments += 0
    }

    return ($normalizedSegments -join ".")
}

# Example
Convert-ToStandardVersion -version "8u422+6"         #5717525050.6.0.0 
Convert-ToStandardVersion -version "22.0.1+8"        #22.0.1.8 
Convert-ToStandardVersion -version "v22-build1"      #1185050.991806090049.0.0 
Convert-ToStandardVersion -version "21.0.4.7.1"      #21.0.4.8 
Convert-ToStandardVersion -version "17.0.12.7.1"     #17.0.12.8 
Convert-ToStandardVersion -version "jdk-11.0.23+9"   #1070107.11.0.32 
Convert-ToStandardVersion -version "7.1.1-35"        #7.1.1.35 
Convert-ToStandardVersion -version "2024b"           #5048505298.0.0.0 
Convert-ToStandardVersion -version "129.0b3"         #129.489851.0.0 
Convert-ToStandardVersion -version "2023.2.20f1"     #2023.2.50490249.0 

[aaronparker]

I've added ConvertTo-DotNetVersionClass which will do this normalisation plus conversion to a .NET Version class. Note that conversion to a .NET class will fail on some version numbers, so the raw string will be returned in those instances.

[aaronparker]

Examples:

Convert a version number string and return the .NET Version class representation

ConvertTo-DotNetVersionClass -Version "jdk-11.0.23+9" | fl

Major         : 1070107
Minor         : 11
Build         : 0
Revision      : 32
MajorRevision : 0
MinorRevision : 32

[aaronparker]

Convert a version number string which fails to convert to the .NET Version class representation, so return as a string:

ConvertTo-DotNetVersionClass -Version "v22-build1"

WARNING: Failed to convert version string 'v22-build1' to a .NET Version class.
1185050.991806090049.0.0

[aaronparker]

Find the latest version of AdoptiumTemurin22 and convert the version number:

Get-EvergreenApp -Name AdoptiumTemurin22 | Where-Object { $_.ImageType -eq "jre" } | ConvertTo-DotNetVersionClass

Major  Minor  Build  Revision
-----  -----  -----  --------
22     0      2      9

[aaronparker]

Find the latest version of AdoptiumTemurin22 and construct an object that includes the converted version number:

Get-EvergreenApp -Name AdoptiumTemurin22 | ForEach-Object {
    [PSCustomObject]@{
        Version      = ConvertTo-DotNetVersionClass -Version $_.Version
        ImageType    = $_.ImageType
        Architecture = $_.Architecture
        URI          = $_.URI
    }
}

Version      : 22.0.2.9
ImageType    : jre
Architecture : x64
URI          : https://github.com/adoptium/temurin22-binaries/releases/download/jdk-22.0.2%2B9/OpenJDK22U-jre_x64_windows_hotspot_22.0.2_9.msi

Version      : 22.0.2.9
ImageType    : jdk
Architecture : x64
URI          : https://github.com/adoptium/temurin22-binaries/releases/download/jdk-22.0.2%2B9/OpenJDK22U-jdk_x64_windows_hotspot_22.0.2_9.msi