Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Added PackageAction Set for Update-AzSynapseSparkPool to support removing and adding packages in one action #21579

Merged
merged 4 commits into from
Apr 20, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions src/Synapse/Synapse/ChangeLog.md
Original file line number Diff line number Diff line change
Expand Up @@ -20,6 +20,7 @@

## Upcoming Release
* Updated Azure.Core to 1.31.0.
* Added PackageAction `Set` for `Update-AzSynapseSparkPool` to support removing and adding packages in one action

## Version 2.3.0
* Upgraded Azure.Analytics.Synapse.Artifacts to 1.0.0-preview.17
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -267,9 +267,9 @@ public override void ExecuteCmdlet()

if (this.IsParameterBound(c => c.PackageAction) && this.IsParameterBound(c => c.Package))
{
if (this.PackageAction == SynapseConstants.PackageActionType.Add)
if (this.PackageAction == SynapseConstants.PackageActionType.Add || this.PackageAction == SynapseConstants.PackageActionType.Set)
{
if (existingSparkPool.CustomLibraries == null)
if (existingSparkPool.CustomLibraries == null || this.PackageAction == SynapseConstants.PackageActionType.Set)
{
existingSparkPool.CustomLibraries = new List<LibraryInfo>();
}
Expand Down
3 changes: 2 additions & 1 deletion src/Synapse/Synapse/Models/SynapseConstants.cs
Original file line number Diff line number Diff line change
Expand Up @@ -332,7 +332,8 @@ public class RepositoryType
public enum PackageActionType
{
Add,
Remove
Remove,
Set
}
public enum UserAssignedManagedIdentityActionType
{
Expand Down
22 changes: 18 additions & 4 deletions src/Synapse/Synapse/help/Update-AzSynapseSparkPool.md
Original file line number Diff line number Diff line change
Expand Up @@ -169,28 +169,42 @@ The first command retrieves an Apache Spark pool in Azure Synapse Analytics. The

### Example 14
```powershell
$workspace_packages = Get-AzSynapseWorkspacePackage -WorkspaceName ContosoWorkspace

$pool = Get-AzSynapseSparkPool -ResourceGroupName ContosoResourceGroup -WorkspaceName ContosoWorkspace -Name ContosoSparkPool
$library_names = $pool.WorkspacePackages | Where-Object {$_.name -notlike "new_package-*"} | ForEach-Object {$_.name}
$library_names += "new_package-2.0-py3-none-any.whl"

$new_pool_packages = @($workspace_packages | Where-Object {$_.name -in $library_names})
Update-AzSynapseSparkPool -ResourceGroupName ContosoResourceGroup -WorkspaceName ContosoWorkspace -Name ContosoSparkPool -PackageAction Set -Package $new_pool_packages
```

The first command retrieves the packages available in the workspace. The second command group retrieves the spark pool to get the packages currently linked to this pool and removes all versions of the package starting with `new_package-` from the retrieved list. The new version of the package is then added to this list. In the third group of commands the package list, containing only package names, is tranformed into a list of workspace packages by filtering the list of available workspace_packages accordingly and is then linked to the spark pool.

### Example 15
```powershell
$config = Get-AzSynapseSparkConfiguration -WorkspaceName ContosoWorkspace -Name ContosoSparkConfig1
Update-AzSynapseSparkPool -WorkspaceName ContosoWorkspace -Name ContosoSparkPool -Tag @{"key" = "value"} -NodeCount 5 -NodeSize Medium -SparkConfiguration $configs
```

This command updates an Apache Spark pool in Azure Synapse Analytics and specify a Spark configuration for the Spark pool.

### Example 15
### Example 16
```powershell
Update-AzSynapseSparkPool -WorkspaceName ContosoWorkspace -Name ContosoSparkPool -NodeSize small -ForceApplySetting
```

This command updates an Apache Spark pool in Azure Synapse Analytics, set NodeSize to small for the spark pool and force stop any running jobs in the Spark pool to apply this new setting.

### Example 16
### Example 17
```powershell
$pool = Get-AzSynapseSparkPool -ResourceGroupName ContosoResourceGroup -WorkspaceName ContosoWorkspace -Name ContosoSparkPool
$pool | Update-AzSynapseSparkPool -PackageAction Remove -Package $pool.WorkspacePackages -ForceApplySetting
```

The first command retrieves an Apache Spark pool in Azure Synapse Analytics. The second command removes all workspace packages that are linked to that Apache Spark pool and force stop any running jobs in the Spark pool to apply this new setting.

### Example 17
### Example 18
```powershell
Update-AzSynapseSparkPool -WorkspaceName ContosoWorkspace -Name ContosoSparkPool -EnableIsolatedCompute $true -NodeSize XXXLarge
```
Expand Down Expand Up @@ -480,7 +494,7 @@ Package action must be specified when you add or remove a workspace package from
Type: Microsoft.Azure.Commands.Synapse.Models.SynapseConstants+PackageActionType
Parameter Sets: (All)
Aliases:
Accepted values: Add, Remove
Accepted values: Add, Remove, Set

Required: False
Position: Named
Expand Down