Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

FileTransformV1: Upgrade 7zip #19284

Merged
merged 2 commits into from
Nov 15, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
423 changes: 173 additions & 250 deletions Tasks/FileTransformV1/package-lock.json

Large diffs are not rendered by default.

2 changes: 1 addition & 1 deletion Tasks/FileTransformV1/package.json
Original file line number Diff line number Diff line change
Expand Up @@ -20,7 +20,7 @@
"@types/mocha": "^5.2.7",
"@types/node": "^16.11.39",
"@types/q": "1.0.7",
"azure-pipelines-tasks-webdeployment-common": "^4.219.1",
"azure-pipelines-tasks-webdeployment-common": "^4.230.6",
"q": "1.4.1"
},
"devDependencies": {
Expand Down
2 changes: 1 addition & 1 deletion Tasks/FileTransformV1/task.json
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,7 @@
],
"version": {
"Major": 1,
"Minor": 220,
"Minor": 231,
"Patch": 0
},
"instanceNameFormat": "File Transform: $(Package)",
Expand Down
2 changes: 1 addition & 1 deletion Tasks/FileTransformV1/task.loc.json
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,7 @@
],
"version": {
"Major": 1,
"Minor": 220,
"Minor": 231,
"Patch": 0
},
"instanceNameFormat": "ms-resource:loc.instanceNameFormat",
Expand Down
4 changes: 2 additions & 2 deletions _generated/FileTransformV1.versionmap.txt
Original file line number Diff line number Diff line change
@@ -1,2 +1,2 @@
Default|1.220.0
Node20_229_3|1.230.1
Default|1.231.0
Node20_229_3|1.231.1
94 changes: 94 additions & 0 deletions _generated/FileTransformV1/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,94 @@
# File transform

## Overview

Use this task to apply file transformations and variable substitutions on configuration and parameters files. For details of how translations are processed, see [File transforms and variable substitution reference](https://docs.microsoft.com/en-us/azure/devops/pipelines/tasks/transforms-variable-substitution?view=azure-devops).

## File transformations

* At present file transformations are supported for only XML files.

* To apply XML transformation to configuration files (*.config) you must specify a newline-separated list of transformation file rules using the syntax:

`-transform <path to the transform file> -xml <path to the source file> -result <path to the result file>`

* File transformations are useful in many scenarios, particularly when you are deploying to an App service and want to add, remove or modify configurations for different environments (such as Dev, Test, or Prod) by following the standard Web.config Transformation Syntax.

* You can also use this functionality to transform other files, including Console or Windows service application configuration files (for example, FabrikamService.exe.config).

* Config file transformations are run before variable substitutions.

## Variable substitution

* At present only XML and JSON file formats are supported for variable substitution.

* Tokens defined in the target configuration files are updated and then replaced with variable values.

* Variable substitutions are run after config file transformations.

* Variable substitution is applied for only the JSON keys predefined in the object hierarchy. It does not create new keys.

* Only custom variables defined in build/release pipelines are used in substitution. Default/system defined pipeline variables are excluded.

* If same variables are defined in the release pipeline and in the stage, then the stage variables will supersede the release pipeline variables.

* Predefined or build Variables are skipped during variable substitution. Hence, variables having prefix among ['agent.', 'azure_http_user_agent', 'build.', 'common.', 'release.', 'system.', 'tf_'] are ignored during variable substitution.

### Examples

If you need XML transformation to run on all the configuration files named with pattern .Production.config, the transformation rule should be specified as:

`-transform **\*.Production.config -xml **\*.config`

If you have a configuration file named based on the stage name in your pipeline, you can use:

`-transform **\*.$(Release.EnvironmentName).config -xml **\*.config`

To substitute JSON variables that are nested or hierarchical, specify them using JSONPath expressions. For example, to replace the value of ConnectionString in the sample below, you must define a variable as Data.DefaultConnection.ConnectionString in the build or release pipeline (or in a stage within the release pipeline).

```
{
"Data": {
"DefaultConnection": {
"ConnectionString": "Server=(localdb)\SQLEXPRESS;Database=MyDB;Trusted_Connection=True"
}
}
}
```

### YAML snippet

```
# File transform
# Replace tokens with variable values in XML or JSON configuration files
- task: FileTransform@1
inputs:
#folderPath: '$(System.DefaultWorkingDirectory)/**/*.zip'
#enableXmlTransform: # Optional
#xmlTransformationRules: '-transform **\*.Release.config -xml **\*.config-transform **\*.$(Release.EnvironmentName).config -xml **\*.config' # Optional
#fileType: # Optional. Options: xml, json
#targetFiles: # Optional
```

### Arguments

Package or folder -> folderPath File path to the package or a folder. Variables ( Build | Release ), wildcards are supported. For example, `$ (System.DefaultWorkingDirectory)/*/.zip`. For zipped folders, the contents are extracted to the TEMP location, transformations executed, and the results zipped in original artifact location.

XML transformation
enableXmlTransform -> Enable this option to apply XML transformations based on the rules specified below. Config transforms run prior to any variable substitution. XML transformations are supported only for the Windows platform.

Transformation rules
xmlTransformationRules -> Provide a newline-separated list of transformation file rules using the syntax
`-transform <path to="" the transform file> -xml <path to the source configuration file> -result <path to the result file>`
The result file path is optional and, if not specified, the source configuration file will be replaced with the transformed result file.

File format
fileType -> Specify the file format on which substitution is to be performed. Variable substitution runs after any configuration transforms. For XML, Variables defined in the build or release pipelines will be matched against the token ('key' or 'name') entries in the appSettings, applicationSettings, and connectionStrings sections of any config file and parameters.xml file.

Target files
targetFiles -> Provide a newline-separated list of files for variable substitution. Files names must be specified relative to the root folder.

* **File transformation and variable substitution:** Refer to following links:
* [XML transformation](https://docs.microsoft.com/en-us/vsts/build-release/tasks/transforms-variable-substitution?view=vsts#xml-transformation)
* [XML variable substitution](https://docs.microsoft.com/en-us/vsts/build-release/tasks/transforms-variable-substitution?view=vsts#xml-variable-substitution)
* [JSON variable substitution](https://docs.microsoft.com/en-us/vsts/build-release/tasks/transforms-variable-substitution?view=vsts#json-variable-substitution)
Original file line number Diff line number Diff line change
@@ -0,0 +1,39 @@
{
"loc.friendlyName": "Dateitransformation",
"loc.helpMarkDown": "Aufgabe zur Dateitransformation und Variablenersetzung: Aktualisieren Sie Token in Ihren XML-basierten Konfigurationsdateien, und ersetzen Sie diese Token durch Variablenwerte. <br/>Aktuell werden nur XML- und JSON-Dateiformate für die Variablenersetzung unterstützt. </br> [Weitere Informationen zu dieser Aufgabe](https://docs.microsoft.com/azure/devops/pipelines/tasks/utility/file-transform)",
"loc.description": "Hiermit werden in XML- oder JSON-Konfigurationsdateien Token durch Variablenwerte ersetzt.",
"loc.instanceNameFormat": "Dateitransformation: $(Package)",
"loc.group.displayName.VariableSubstitution": "Variablenersetzung",
"loc.input.label.folderPath": "Paket oder Ordner",
"loc.input.help.folderPath": "Dateipfad zu einem Paket oder einem Ordner.<br />Variablen ([Build](https://docs.microsoft.com/vsts/pipelines/build/variables) | [Release](https://docs.microsoft.com/vsts/pipelines/release/variables#default-variables)) und Platzhalter werden unterstützt. <br/> Beispiel: $(System.DefaultWorkingDirectory)/\\*\\*/\\*.zip.",
"loc.input.label.enableXmlTransform": "XML-Transformation",
"loc.input.help.enableXmlTransform": "Vor der Variablenersetzung werden Konfigurationstransformationen durchgeführt. <br/>XML-Transformationen werden nur für die Windows-Plattform unterstützt.",
"loc.input.label.xmlTransformationRules": "Transformationsregeln",
"loc.input.help.xmlTransformationRules": "Geben Sie eine Liste mit Transformationsdateiregeln an, bei der jeder Eintrag in einer neuen Zeile enthalten ist. Verwenden Sie hierbei diese Syntax: <br/>-transform <PfadzuTransformationsdatei> -xml <PfadzuQuellkonfigurationsdatei>",
"loc.input.label.fileType": "Dateiformat",
"loc.input.help.fileType": "Geben Sie das Dateiformat an, für das die Ersetzung durchgeführt werden soll. <br/>Für XML werden in den Build- oder Releasepipelines definierte Variablen anhand der Einträge \"key\" oder \"name\" in den Abschnitten \"appSettings\", \"applicationSettings\" und \"connectionStrings\" einer beliebigen Konfigurationsdatei und \"parameters.xml\" abgeglichen. Die Variablenersetzung wird nach den Konfigurationstransformationen durchgeführt.<br/>Um JSON-Variablen zu ersetzen, die geschachtelt oder hierarchisch aufgebaut sind, geben Sie diese mithilfe von JSONPath-Ausdrücken an. <br/> <br/>Um beispielsweise den Wert von \"ConnectionString\" im Beispiel unten zu ersetzen, müssen Sie eine Variable als \"Data.DefaultConnection.ConnectionString\" in der Build- oder Releasepipeline (oder der Umgebung der Releasepipeline) definieren. <br/> {<br/>&nbsp;&nbsp;\"Data\": {<br/>&nbsp;&nbsp;&nbsp;&nbsp;\"DefaultConnection\": {<br/>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;\"ConnectionString\": \"Server=(localdb)\\SQLEXPRESS;Database=MyDB;Trusted_Connection=True\"<br/>&nbsp;&nbsp;&nbsp;&nbsp;}<br/>&nbsp;&nbsp;}<br/> }<br/> Die Variablenersetzung wird nach den Konfigurationstransformationen durchgeführt. <br/><br/> Hinweis: Bei der Ersetzung werden nur benutzerdefinierte Variablen verwendet, die in Build-/Releasepipelines definiert sind. In der Pipeline definierte Standard- oder Systemvariablen werden ausgeschlossen. <br/>Hinweis: Wenn dieselben Variablen für die Releasepipeline und die Stufe definiert werden, haben die Stufenvariablen Vorrang vor den Variablen der Releasepipeline.",
"loc.input.label.targetFiles": "Zieldateien",
"loc.input.help.targetFiles": "Geben Sie eine Liste mit Dateien für die Ersetzung der Variablenwerte an, bei der jede Datei in einer neuen Zeile enthalten ist. Die Dateinamen müssen relativ zum Stammordner angegeben werden.",
"loc.messages.JSONvariablesubstitutionappliedsuccessfully": "Die JSON-Variablenersetzung wurde erfolgreich angewendet.",
"loc.messages.XMLvariablesubstitutionappliedsuccessfully": "XML-Variablenersetzung erfolgreich angewendet.",
"loc.messages.XDTTransformationsappliedsuccessfully": "Die XML-Transformationen wurden erfolgreich angewendet.",
"loc.messages.CannotPerformXdtTransformationOnNonWindowsPlatform": "XML-Transformationen können auf einer Nicht-Windows-Plattform nicht ausgeführt werden.",
"loc.messages.XdtTransformationErrorWhileTransforming": "XML-Transformationsfehler beim Transformieren von \"%s\" unter Verwendung von \"%s\".",
"loc.messages.JSONParseError": "Die JSON-Datei konnte nicht analysiert werden: %s. Fehler: %s",
"loc.messages.NOJSONfilematchedwithspecificpattern": "Keine JSON-Datei stimmte mit dem angegebenen Muster überein: %s",
"loc.messages.FailedToApplyTransformation": "Die Transformation für das angegebene Paket kann nicht angewendet werden.",
"loc.messages.MissingArgumentsforXMLTransformation": "Unvollständige oder fehlende Argumente. Erwartetes Format: -transform <Transformationsdatei> -xml <Quelldatei> -result <Zieldatei>. Transformations- und Quelldatei sind Pflichteingaben.",
"loc.messages.SubstitutingValueonKey": "Der Wert für den Schlüssel \"%s\" wird ersetzt.",
"loc.messages.SubstitutingValueonKeyWithNumber": "Wert für Schlüssel \"%s\" wird durch (Zahl) Wert ersetzt: %s",
"loc.messages.SubstitutingValueonKeyWithBoolean": "Der Wert für den Schlüssel \"%s\" wird durch den (booleschen) Wert \"%s\" ersetzt.",
"loc.messages.SubstitutingValueonKeyWithObject": "Wert für Schlüssel \"%s\" wird durch (Objekt) Wert ersetzt: %s",
"loc.messages.SubstitutingValueonKeyWithString": "Wert für Schlüssel \"%s\" wird durch (Zeichenfolge) Wert ersetzt: %s",
"loc.messages.JSONvariableSubstitution": "Die JSON-Variablenersetzung für \"%s\" wird angewendet.",
"loc.messages.ApplyingXDTtransformation": "Die XDT-Transformation wird aus Transformationsdatei \"%s\" > Quelldatei \"%s\" angewendet.",
"loc.messages.SubstitutionForXmlNode": "Ersetzung für XML-Knoten wird verarbeitet: %s",
"loc.messages.UpdatingKeyWithTokenValue": "Der Wert für den Schlüssel \"%s\" wird durch den Tokenwert ersetzt: %s",
"loc.messages.SubstitutingConnectionStringValue": "Der connectionString-Wert für connectionString \"%s\" wird durch den Tokenwert ersetzt: %s ",
"loc.messages.VariableSubstitutionInitiated": "Die Variablenersetzung in der Konfigurationsdatei wurde initialisiert: %s",
"loc.messages.ConfigFileUpdated": "Die Konfigurationsdatei \"%s\" wurde aktualisiert.",
"loc.messages.SkippedUpdatingFile": "Die Aktualisierung der Datei wurde übersprungen: %s"
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,39 @@
{
"loc.friendlyName": "File transform",
"loc.helpMarkDown": "File transformation and variable substitution task: Update tokens in your XML based configuration files and then replaces those tokens with variable values. <br/>Currently only XML, JSON file formats are supported for variable substitution. </br> [Learn more about this task](https://docs.microsoft.com/azure/devops/pipelines/tasks/utility/file-transform)",
"loc.description": "Replace tokens with variable values in XML or JSON configuration files",
"loc.instanceNameFormat": "File Transform: $(Package)",
"loc.group.displayName.VariableSubstitution": "Variable Substitution",
"loc.input.label.folderPath": "Package or folder",
"loc.input.help.folderPath": "File path to the package or a folder.<br />Variables ( [Build](https://docs.microsoft.com/vsts/pipelines/build/variables) | [Release](https://docs.microsoft.com/vsts/pipelines/release/variables#default-variables)), wildcards are supported. <br/> For example, $(System.DefaultWorkingDirectory)/\\*\\*/\\*.zip.",
"loc.input.label.enableXmlTransform": "XML transformation",
"loc.input.help.enableXmlTransform": "Config transforms will be run prior to the Variable Substitution.<br/>XML transformations are supported only for Windows platform.",
"loc.input.label.xmlTransformationRules": "Transformation rules",
"loc.input.help.xmlTransformationRules": "Provide new line separated list of transformation file rules using the syntax: <br/>-transform <pathToTransformFile> -xml <pathToSourceConfigurationFile>",
"loc.input.label.fileType": "File format",
"loc.input.help.fileType": "Provide file format on which substitution has to be perfformed<br/>For XML, Variables defined in the build or release pipelines will be matched against the 'key' or 'name' entries in the appSettings, applicationSettings, and connectionStrings sections of any config file and parameters.xml. Variable Substitution is run after config transforms.<br/>To substitute JSON variables that are nested or hierarchical, specify them using JSONPath expressions. <br/> <br/> For example, to replace the value of ‘ConnectionString’ in the sample below, you need to define a variable as ‘Data.DefaultConnection.ConnectionString’ in the build or release pipeline (or release pipeline's environment). <br/> {<br/>&nbsp;&nbsp;\"Data\": {<br/>&nbsp;&nbsp;&nbsp;&nbsp;\"DefaultConnection\": {<br/>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;\"ConnectionString\": \"Server=(localdb)\\SQLEXPRESS;Database=MyDB;Trusted_Connection=True\"<br/>&nbsp;&nbsp;&nbsp;&nbsp;}<br/>&nbsp;&nbsp;}<br/> }<br/> Variable Substitution is run after configuration transforms. <br/><br/> Note: only custom variables defined in build/release pipelines are used in substitution. Default/system defined pipeline variables are excluded. <br/>Note: If same variables are defined in the release pipeline and in the stage, then the stage variables will supersede the release pipeline variables.",
"loc.input.label.targetFiles": "Target files",
"loc.input.help.targetFiles": "Provide new line separated list of files to substitute the variable values. Files names are to be provided relative to the root folder.",
"loc.messages.JSONvariablesubstitutionappliedsuccessfully": "JSON variable substitution applied successfully.",
"loc.messages.XMLvariablesubstitutionappliedsuccessfully": "XML variable substitution applied successfully.",
"loc.messages.XDTTransformationsappliedsuccessfully": "XML Transformations applied successfully",
"loc.messages.CannotPerformXdtTransformationOnNonWindowsPlatform": "Cannot perform XML transformations on a non-Windows platform.",
"loc.messages.XdtTransformationErrorWhileTransforming": "XML transformation error while transforming %s using %s.",
"loc.messages.JSONParseError": "Unable to parse JSON file: %s. Error: %s",
"loc.messages.NOJSONfilematchedwithspecificpattern": "NO JSON file matched with specific pattern: %s.",
"loc.messages.FailedToApplyTransformation": "Unable to apply transformation for the given package.",
"loc.messages.MissingArgumentsforXMLTransformation": "Incomplete or missing arguments. Expected format -transform <transform file> -xml <source file> -result <destinamtion file>. Transformation and source file are mandatory inputs.",
"loc.messages.SubstitutingValueonKey": "Substituting value on key: %s",
"loc.messages.SubstitutingValueonKeyWithNumber": "Substituting value on key %s with (number) value: %s",
"loc.messages.SubstitutingValueonKeyWithBoolean": "Substituting value on key %s with (boolean) value: %s",
"loc.messages.SubstitutingValueonKeyWithObject": "Substituting value on key %s with (object) value: %s",
"loc.messages.SubstitutingValueonKeyWithString": "Substituting value on key %s with (string) value: %s",
"loc.messages.JSONvariableSubstitution": "Applying JSON variable substitution for %s",
"loc.messages.ApplyingXDTtransformation": "Applying XDT Transformation from transformation file %s -> source file %s ",
"loc.messages.SubstitutionForXmlNode": "Processing substitution for xml node : %s",
"loc.messages.UpdatingKeyWithTokenValue": "Updating value for key= %s with token value: %s",
"loc.messages.SubstitutingConnectionStringValue": "Substituting connectionString value for connectionString = %s with token value: %s ",
"loc.messages.VariableSubstitutionInitiated": "Initiated variable substitution in config file : %s",
"loc.messages.ConfigFileUpdated": "Config file : %s updated.",
"loc.messages.SkippedUpdatingFile": "Skipped Updating file: %s"
}
Loading