`azurerm_data_factory_pipeline`: fix `headers` acceptable value #24921

ziyeqf · 2024-02-18T09:16:40Z

magodo

Thanks for this PR!
I have left some additional mostly minor comments that once addressed this should be good to merge 👍

internal/services/datafactory/azuresdkhacks/pipelines.go

magodo · 2024-02-19T05:34:24Z

LGTM!

tombuildsstuff · 2024-02-19T09:00:24Z

@ziyeqf rather than doing this and needing to continue patching around this in the future - is hashicorp/go-azure-sdk usable as a replacement here? https://github.com/hashicorp/go-azure-sdk/blob/main/resource-manager/datafactory/2018-06-01/pipelines/method_createorupdate.go

ziyeqf · 2024-02-20T02:13:28Z

@tombuildsstuff I checked it, but it seems go-azure-sdk does not have the needed variant of activity
go-azure-sdk: https://github.com/hashicorp/go-azure-sdk/blob/4f74876e2154eae6add03690707943f92daad264/resource-manager/automation/2015-10-31/activity/model_activityproperties.go#L12
kermit:

// WebActivity web activity.
type WebActivity struct {
	// WebActivityTypeProperties - Web activity properties.
	*WebActivityTypeProperties `json:"typeProperties,omitempty"`
	// LinkedServiceName - Linked service reference.
	LinkedServiceName *LinkedServiceReference `json:"linkedServiceName,omitempty"`
	// Policy - Activity policy.
	Policy *ActivityPolicy `json:"policy,omitempty"`
	// AdditionalProperties - Unmatched properties from the message are deserialized this collection
	AdditionalProperties map[string]interface{} `json:""`
	// Name - Activity name.
	Name *string `json:"name,omitempty"`
	// Description - Activity description.
	Description *string `json:"description,omitempty"`
	// State - Activity state. This is an optional property and if not provided, the state will be Active by default. Possible values include: 'ActivityStateActive', 'ActivityStateInactive'
	State ActivityState `json:"state,omitempty"`
	// OnInactiveMarkAs - Status result of the activity when the state is set to Inactive. This is an optional property and if not provided when the activity is inactive, the status will be Succeeded by default. Possible values include: 'ActivityOnInactiveMarkAsSucceeded', 'ActivityOnInactiveMarkAsFailed', 'ActivityOnInactiveMarkAsSkipped'
	OnInactiveMarkAs ActivityOnInactiveMarkAs `json:"onInactiveMarkAs,omitempty"`
	// DependsOn - Activity depends on condition.
	DependsOn *[]ActivityDependency `json:"dependsOn,omitempty"`
	// UserProperties - Activity user properties.
	UserProperties *[]UserProperty `json:"userProperties,omitempty"`
	// Type - Possible values include: 'TypeBasicActivityTypeActivity', 'TypeBasicActivityTypeExecuteWranglingDataflow', 'TypeBasicActivityTypeSparkJob', 'TypeBasicActivityTypeSynapseNotebook', 'TypeBasicActivityTypeScript', 'TypeBasicActivityTypeExecuteDataFlow', 'TypeBasicActivityTypeAzureFunctionActivity', 'TypeBasicActivityTypeDatabricksSparkPython', 'TypeBasicActivityTypeDatabricksSparkJar', 'TypeBasicActivityTypeDatabricksNotebook', 'TypeBasicActivityTypeDataLakeAnalyticsUSQL', 'TypeBasicActivityTypeAzureMLExecutePipeline', 'TypeBasicActivityTypeAzureMLUpdateResource', 'TypeBasicActivityTypeAzureMLBatchExecution', 'TypeBasicActivityTypeGetMetadata', 'TypeBasicActivityTypeWebActivity', 'TypeBasicActivityTypeLookup', 'TypeBasicActivityTypeAzureDataExplorerCommand', 'TypeBasicActivityTypeDelete', 'TypeBasicActivityTypeSQLServerStoredProcedure', 'TypeBasicActivityTypeCustom', 'TypeBasicActivityTypeExecuteSSISPackage', 'TypeBasicActivityTypeHDInsightSpark', 'TypeBasicActivityTypeHDInsightStreaming', 'TypeBasicActivityTypeHDInsightMapReduce', 'TypeBasicActivityTypeHDInsightPig', 'TypeBasicActivityTypeHDInsightHive', 'TypeBasicActivityTypeCopy', 'TypeBasicActivityTypeExecution', 'TypeBasicActivityTypeWebHook', 'TypeBasicActivityTypeAppendVariable', 'TypeBasicActivityTypeSetVariable', 'TypeBasicActivityTypeFilter', 'TypeBasicActivityTypeValidation', 'TypeBasicActivityTypeUntil', 'TypeBasicActivityTypeFail', 'TypeBasicActivityTypeWait', 'TypeBasicActivityTypeForEach', 'TypeBasicActivityTypeSwitch', 'TypeBasicActivityTypeIfCondition', 'TypeBasicActivityTypeExecutePipeline', 'TypeBasicActivityTypeContainer'
	Type TypeBasicActivity `json:"type,omitempty"`
}

tombuildsstuff · 2024-02-20T16:20:45Z

internal/services/datafactory/azuresdkhacks/pipelines.go

+	if err := validation.Validate([]validation.Validation{
+		{
+			TargetValue: resourceGroupName,
+			Constraints: []validation.Constraint{
+				{Target: "resourceGroupName", Name: validation.MaxLength, Rule: 90, Chain: nil},
+				{Target: "resourceGroupName", Name: validation.MinLength, Rule: 1, Chain: nil},
+				{Target: "resourceGroupName", Name: validation.Pattern, Rule: `^[-\w\._\(\)]+$`, Chain: nil},
+			},
+		},
+		{
+			TargetValue: factoryName,
+			Constraints: []validation.Constraint{
+				{Target: "factoryName", Name: validation.MaxLength, Rule: 63, Chain: nil},
+				{Target: "factoryName", Name: validation.MinLength, Rule: 3, Chain: nil},
+				{Target: "factoryName", Name: validation.Pattern, Rule: `^[A-Za-z0-9]+(?:-[A-Za-z0-9]+)*$`, Chain: nil},
+			},
+		},
+		{
+			TargetValue: pipelineName,
+			Constraints: []validation.Constraint{
+				{Target: "pipelineName", Name: validation.MaxLength, Rule: 260, Chain: nil},
+				{Target: "pipelineName", Name: validation.MinLength, Rule: 1, Chain: nil},
+				{Target: "pipelineName", Name: validation.Pattern, Rule: `^[A-Za-z0-9_][^<>*#.%&:\\+?/]*$`, Chain: nil},
+			},
+		},
+		{
+			TargetValue: pipeline,
+			Constraints: []validation.Constraint{{
+				Target: "pipeline.Pipeline", Name: validation.Null, Rule: true,
+				Chain: []validation.Constraint{
+					{
+						Target: "pipeline.Pipeline.Concurrency", Name: validation.Null, Rule: false,
+						Chain: []validation.Constraint{{Target: "pipeline.Pipeline.Concurrency", Name: validation.InclusiveMinimum, Rule: int64(1), Chain: nil}},
+					},
+				},
+			}},
+		},
+	}); err != nil {
+		return result, validation.NewError("datafactory.PipelinesClient", "CreateOrUpdate", err.Error())
+	}


can we remove the validation here?

katbyte

Thanks @ziyeqf - LGTM 🎧

…icorp#24921) * fix hashicorp#24758 * golint * add comment to workaround * update comment

<Actions> <action id="f410411e63aff4bb73a81c2aec1d373cf8a903e63b30dee2006b0030d8a94cc8"> <h3>Bump Terraform `azurerm` provider version</h3> <details id="1d9343c012f5434ac9fe8a98135bae3667b399259be16d9b14302ea3bd424a24"> <summary>Update Terraform lock file</summary> <p>changes detected:
	"hashicorp/azurerm" updated from "3.92.0" to "3.93.0" in file ".terraform.lock.hcl"</p> <details> <summary>3.93.0</summary> <pre>Changelog retrieved from:
	https://github.com/hashicorp/terraform-provider-azurerm/releases/tag/v3.93.0
* **New Data Source**: `azurerm_express_route_circuit_peering` ([#24971](hashicorp/terraform-provider-azurerm#24971 **New Data Source**: `azurerm_storage_table_entities` ([#24973](hashicorp/terraform-provider-azurerm#24973 **New Resource**: `azurerm_dev_center_catalog` ([#24833](hashicorp/terraform-provider-azurerm#24833 **New Resource**: `azurerm_system_center_virtual_machine_manager_server` ([#24278](https://github.com/hashicorp/terraform-provider-azurerm/issues/24278))

BUG FIXES:

* `azurerm_key_vault` - conditionally polling the Data Plane endpoint when `public_network_access_enabled` is set to false ([#23823](hashicorp/terraform-provider-azurerm#23823 `azurerm_storage_account` - allow the `identity.type` property to be `SystemAssigned, UserAssigned` when using a Customer Managed Key ([#24923](hashicorp/terraform-provider-azurerm#24923 `azurerm_automation_account` - prevent the `identity.identity_ids` User Assigned identity being set when not specified in config ([#24977](https://github.com/hashicorp/terraform-provider-azurerm/issues/24977))

ENHANCEMENTS:

* dependencies: updating to `v0.20240221.1170458` of `hashicorp/go-azure-sdk` ([#24967](hashicorp/terraform-provider-azurerm#24967 dependencies: refactor `azurerm_spring_cloud_configuration_service` to use `go-azure-sdk` ([#24918](hashicorp/terraform-provider-azurerm#24918 provider: support or the feature flag `virtual_machine_scale_set.reimage_on_manual_upgrade` ([#22975](hashicorp/terraform-provider-azurerm#22975 `sentinel`: updating to use the transport layer from `hashicorp/go-azure-sdk` rather than `Azure/go-autorest` ([#24962](hashicorp/terraform-provider-azurerm#24962 `sqlvirtualmachines`: updating to use the transport layer from `hashicorp/go-azure-sdk` rather than `Azure/go-autorest` ([#24912](hashicorp/terraform-provider-azurerm#24912 `nginx` : updating to use `2024-01-01-preview` ([#24868](hashicorp/terraform-provider-azurerm#24868 `azurerm_cosmosdb_account` - support for the `backup.tier` property ([#24595](hashicorp/terraform-provider-azurerm#24595 `azurerm_linux_virtual_machine` - the `virtual_machine_scale_set_id` proeprty can now be changed without creating a new resource ([#24768](hashicorp/terraform-provider-azurerm#24768 `azurerm_machine_learning_workspace` - support for the `managed_network.isolation_mode` property ([#24951](hashicorp/terraform-provider-azurerm#24951 `azurerm_private_dns_resolver_inbound_endpoint` - support the `static` value for the `private_ip_allocation_method` property ([#24952](hashicorp/terraform-provider-azurerm#24952 `azurerm_postgresql_flexible_server` - expose the `storage_tier` field ([#24892](hashicorp/terraform-provider-azurerm#24892 `azurerm_redis_cache` - support for the `preferred_data_persistence_auth_method` property ([#24370](hashicorp/terraform-provider-azurerm#24370 `azurerm_servicebus_namespace` - support for the `premium_messaging_partitions` property ([#24676](hashicorp/terraform-provider-azurerm#24676 `azurerm_windows_virtual_machine` - the `virtual_machine_scale_set_id` proeprty can now be changed without creating a new resource ([#24768](https://github.com/hashicorp/terraform-provider-azurerm/issues/24768))

BUG FIXES:

* `azurerm_cognitive_deployment` - the `version_upgrade_option` property can not be updated without creating a new resource ([#24922](hashicorp/terraform-provider-azurerm#24922 `azurerm_data_protection_backup_vault` - support or the `soft_delete` and `retention_duration_in_days` properties ([#24775](hashicorp/terraform-provider-azurerm#24775 `azurerm_data_factory_pipeline` - correctly handle incorrect header values ([#24921](hashicorp/terraform-provider-azurerm#24921 `azurerm_kusto_cluster` - `optimized_auto_scale` is now updated after `sku` has been updated ([#24906](hashicorp/terraform-provider-azurerm#24906 `azurerm_key_vault_certificate` - will now only update the `lifetime_action` of the certificate block unless otherwise required ([#24755](hashicorp/terraform-provider-azurerm#24755 `azurerm_linux_virtual_machine_scale_set` - correctly include `public_ip_prefix_id` during updates ([#24939](hashicorp/terraform-provider-azurerm#24939 `azurerm_postgresql_flexible_server` - the `customer_managed_key.key_vault_key_id` property is now required ([#24981](hashicorp/terraform-provider-azurerm#24981 `azurerm_nginx_deployment` - changing the `sku` property now creates a new resource ([#24905](hashicorp/terraform-provider-azurerm#24905 `azurerm_orchestrated_virtual_machine_scale_set` - the `disk_size_gb` and `lun` parameters of `data_disks` are optional now ([#24944](hashicorp/terraform-provider-azurerm#24944 `azurerm_storage_account` - change order of API calls to be GET-then-PUT ratehr then PATCHES ([#23935](hashicorp/terraform-provider-azurerm#23935 `azurerm_storage_account` - improve the validation around the `immutability_policy` being used with `blob_properties` ([#24938](hashicorp/terraform-provider-azurerm#24938 `azurerm_security_center_setting` - prevent a bug when name is `SENTINEL` ([#24497](hashicorp/terraform-provider-azurerm#24497 `azurerm_windows_virtual_machine_scale_set` - correctly include `public_ip_prefix_id` during updates ([#24939](https://github.com/hashicorp/terraform-provider-azurerm/issues/24939))




</pre> </details> </details> <a href="https://infra.ci.jenkins.io/job/updatecli/job/azure/job/main/19/">Jenkins pipeline link</a> </action> </Actions> --- <table> <tr> <td width="77"> <img src="https://www.updatecli.io/images/updatecli.png" alt="Updatecli logo" width="50" height="50"> </td> <td> <p> Created automatically by <a href="https://www.updatecli.io/">Updatecli</a> </p> <details><summary>Options:</summary> <br /> <p>Most of Updatecli configuration is done via <a href="https://www.updatecli.io/docs/prologue/quick-start/">its manifest(s)</a>.</p> <ul> <li>If you close this pull request, Updatecli will automatically reopen it, the next time it runs.</li> <li>If you close this pull request and delete the base branch, Updatecli will automatically recreate it, erasing all previous commits made.</li> </ul> <p> Feel free to report any issues at <a href="https://github.com/updatecli/updatecli/issues">github.com/updatecli/updatecli</a>.<br /> If you find this tool useful, do not hesitate to star <a href="https://github.com/updatecli/updatecli/stargazers">our GitHub repository</a> as a sign of appreciation, and/or to tell us directly on our <a href="https://matrix.to/#/#Updatecli_community:gitter.im">chat</a>! </p> </details> </td> </tr> </table> Co-authored-by: Jenkins Infra Bot (updatecli) <[email protected]>

…icorp#24921) * fix hashicorp#24758 * golint * add comment to workaround * update comment

github-actions · 2024-04-22T02:09:18Z

I'm going to lock this pull request because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active contributions.
If you have found a problem that seems related to this change, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.

fix hashicorp#24758

9e7c8b2

github-actions bot added service/data-factory size/XL labels Feb 18, 2024

ziyeqf mentioned this pull request Feb 18, 2024

Error occurred unmarshalling JSON #24758

Closed

1 task

github-actions bot added the waiting-response label Feb 18, 2024

golint

5d71927

github-actions bot removed the waiting-response label Feb 19, 2024

magodo requested changes Feb 19, 2024

View reviewed changes

internal/services/datafactory/azuresdkhacks/pipelines.go Show resolved Hide resolved

github-actions bot added the waiting-response label Feb 19, 2024

add comment to workaround

7b2040e

github-actions bot removed the waiting-response label Feb 19, 2024

update comment

f4c6bf3

ziyeqf marked this pull request as ready for review February 19, 2024 05:40

tombuildsstuff added question waiting-response labels Feb 19, 2024

github-actions bot removed the waiting-response label Feb 20, 2024

tombuildsstuff reviewed Feb 20, 2024

View reviewed changes

github-actions bot added the waiting-response label Feb 20, 2024

katbyte approved these changes Feb 20, 2024

View reviewed changes

katbyte merged commit f970438 into hashicorp:main Feb 20, 2024
31 checks passed

github-actions bot added this to the v3.93.0 milestone Feb 20, 2024

katbyte added a commit that referenced this pull request Feb 20, 2024

Update CHANGELOG.md #24921

76cbe12

rizkybiz pushed a commit to rizkybiz/terraform-provider-azurerm that referenced this pull request Feb 21, 2024

azurerm_data_factory_pipeline: fix headers acceptable value (hash…

e6a5c40

…icorp#24921) * fix hashicorp#24758 * golint * add comment to workaround * update comment

rizkybiz pushed a commit to rizkybiz/terraform-provider-azurerm that referenced this pull request Feb 21, 2024

Update CHANGELOG.md hashicorp#24921

5635fac

rizkybiz pushed a commit to rizkybiz/terraform-provider-azurerm that referenced this pull request Feb 29, 2024

azurerm_data_factory_pipeline: fix headers acceptable value (hash…

4c2de32

…icorp#24921) * fix hashicorp#24758 * golint * add comment to workaround * update comment

rizkybiz pushed a commit to rizkybiz/terraform-provider-azurerm that referenced this pull request Feb 29, 2024

Update CHANGELOG.md hashicorp#24921

d062b13

github-actions bot locked as resolved and limited conversation to collaborators Apr 22, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

`azurerm_data_factory_pipeline`: fix `headers` acceptable value #24921

`azurerm_data_factory_pipeline`: fix `headers` acceptable value #24921

ziyeqf commented Feb 18, 2024 •

edited

Loading

magodo left a comment

magodo commented Feb 19, 2024

tombuildsstuff commented Feb 19, 2024

ziyeqf commented Feb 20, 2024

tombuildsstuff Feb 20, 2024

katbyte left a comment

github-actions bot commented Apr 22, 2024

azurerm_data_factory_pipeline: fix headers acceptable value #24921

azurerm_data_factory_pipeline: fix headers acceptable value #24921

Conversation

ziyeqf commented Feb 18, 2024 • edited Loading

magodo left a comment

Choose a reason for hiding this comment

magodo commented Feb 19, 2024

tombuildsstuff commented Feb 19, 2024

ziyeqf commented Feb 20, 2024

tombuildsstuff Feb 20, 2024

Choose a reason for hiding this comment

katbyte left a comment

Choose a reason for hiding this comment

github-actions bot commented Apr 22, 2024

`azurerm_data_factory_pipeline`: fix `headers` acceptable value #24921

`azurerm_data_factory_pipeline`: fix `headers` acceptable value #24921

ziyeqf commented Feb 18, 2024 •

edited

Loading