Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

PowerShell cmdlets #1407

Closed
wants to merge 25 commits into from
Closed

PowerShell cmdlets #1407

wants to merge 25 commits into from

Conversation

denelon
Copy link
Contributor

@denelon denelon commented Aug 28, 2021


#674

Microsoft Reviewers: Open in CodeFlow

@denelon denelon requested a review from a team as a code owner August 28, 2021 00:18
@github-actions

This comment has been minimized.

@github-actions

This comment has been minimized.

@github-actions

This comment has been minimized.

@github-actions

This comment has been minimized.

Copy link

@megamorf megamorf left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added my feedback 👍

doc/specs/#674 - PowerShell cmdlets.md Outdated Show resolved Hide resolved
doc/specs/#674 - PowerShell cmdlets.md Show resolved Hide resolved


```PowerShell
Get-WinGetSource | convertto-json -compress

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Since this would entirely rely on the output object structure this approach could break easily. It might make more sense to handle this via a -Raw parameter, e.g.

Get-WinGetSource -Raw

Copy link

@doctordns doctordns Aug 29, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The cmdlets should emit .NET Objects (for consumption by other PowerShell cmdlets) along with JSON only where necessary.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

NOte - if the intention is to handle cmdlets using Crescendo - then this can be met, provided the team do a good job at writing the necessary output handelers. And for the record, with Cresendo, this is NOT an easy task unless you are outstandingly good with regular expressions.

With that said, the cmdlets must emit objects - and some of those objects can contain json where json is relevant.

doc/specs/#674 - PowerShell cmdlets.md Show resolved Hide resolved
```
**Parameters**

-Output

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

-Path and -LiteralPath might be better parameter names for -Output. See Export-Csv

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@megamorf what is the difference between the two?

Copy link

@jantari jantari Aug 30, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Per convention, -Path accepts and interprets wildcards and globs. LiteralPath does not. There's probably official docs for this as it's very standard

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Correct. The official documentation says the following:

Unlike Path, the value of the LiteralPath parameter is used exactly as it is typed. No characters are interpreted as wildcards. If the path includes escape characters, use single quotation marks. Single quotation marks tell PowerShell not to interpret any characters as escape sequences.

Why is that important? Let's consider that you have a folder with square brackets or an asterisk in its name. -Path will treat that as an expression rather than a true representantion of the path. Users have learned to use -LiteralPath when they literally want PowerShell to treat the user specified path as it is.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

OK, so rather than "-Output" I believe what I'm reading is we should support both "-Path" and "-LiteralPath". Is it safe to assume PowerShell handles the wildcards when "-Path" is used?

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So far as I can recall, -Path takes a wildcarded string and the cmdlet does the variable expansion.

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

OK, so rather than "-Output" I believe what I'm reading is we should support both "-Path" and "-LiteralPath". Is it safe to assume PowerShell handles the wildcards when "-Path" is used?

AFAIK the -Path string is passed to the so called "provider" and the interpretation of wildcards is up to the implementation of that provider. PowerShells "path-related" cmdlets can interact with multiple providers, e.g. the Certificate Store, ActiveDirectory, the registry OR the filesystem. So the meaning of * may be different for an NTFS filesystem path / provider implementation than for the registry or for an ActiveDirectory LDAP path. It would be best to speak to the professionals, PowerShell team, about their preference here. Another existing cmdlet that creates a file and writes content to it is Set-Content - it too supports -Path and -LiteralPath but when trying to use it with a registry-path I got an error that the functionality is not implemented by the provider - similar to how winget wouldn't support exporting a packageset to the registry or certificate store I imagine:

Set-Content -path "HKCU:\Software\XD" -Value 'kek'
Set-Content: Cannot use interface. The IContentCmdletProvider interface is not implemented by this provider.

doc/specs/#674 - PowerShell cmdlets.md Outdated Show resolved Hide resolved
Comment on lines +100 to +109
>Note: The current behavior for `winget show vscode` would look like the example below.

```PowerShell
Find-WinGetPackage -Moniker vscode | Get-WinGetPackageVersion -Detail
```

>Note: The current behavior for `winget show vscode --versions` would look like the example below.
```PowerShell
Find-WinGetPackage -Moniker vscode | Get-WinGetPackageVersion -All
```

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is valid PowerShell code but needlessly complicated from a user experience perspective. Many users will expect these actions to be done with a single command. It'll be interesting to see what feedback you'll get here :-)

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Find-Package should, uhh, find packages and return all the information. C/F Find-Module. Please do not introduce needless complexity where it can be avoided.

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I disagree. AFAIK winget repositories use a cached index to speed up searches. It makes a ton of sense to return only the minimal, cached set of data first and then get all the information (which is an expensive operation) only for the package you're really interested in. Other existing cmdlets such as Get-ADUser and Get-WindowsDriver take a similar approach for performance and server-load reasons.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If the approach is to return a minimal amount of properties, then two things:

  1. The Find-WinGetPackage should return enough properties, by default, to be useful. As you mantion, like Get-ADuser etc.
  2. If there are more properties not returned by default - then use a -Property parameter to specify which optional properties to return.
    In this case, I'd argue that the version of a package is important and probably should be returned by default.

There are two issues therefore: what properties get returned by default and how to get more. I do not support adding cmdlets in the proposed manner.

Just checking, since it's not in this spec file, these cmdlets will be targeting Windows PowerShell (5.1), not PowerShell Core, right? We can't guarantee that PowerShell Core is installed since it's not shipped with Windows (and it isn't usually installed on systems that aren't used for development unless you work at a really hip company).

I sure hope not. It would be crazy, this long after PowerShell 7 has shipped to be so backwards as to ONLY target WIndows PowerShell.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I sure hope not. It would be crazy, this long after PowerShell 7 has shipped to be so backwards as to ONLY target WIndows PowerShell.

I'd disagree. For all that Microsoft want PowerShell 7 to be the future, I don't know a single enterprise that's using it, precisely because it's not a backwards compatible, drop in replacement for PowerShell 5. In every current, supported version of Windows, PowerShell 7 is a side by side addon and the chance of MS actually ever releasing an in place upgrade from PS5 to PS7+ is slim to none, so PowerShell 5 is going to continue to be what is used by enterprise admins for years to come.
So yeah, IMHO this module absolutely needs to be PS 5.1 compatible.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@JustSomeGuyNZ I agree with your point that there are companies that will never allow PS 7 to be deployed. I do not get the reason to be honest. I consult customers to deploy it widely just for the same of improved commandlets like test-connection etc.
It's an MSI package, easy to deploy with a small mst. What is their issues with PS 7? Do you have more information about it? I just know it is a controversy because of patchmanagement.org and also some ConfigMgr MEMCM pals.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's not a drop in replacement, and it breaks backwards compatibility. In my experience enterprise IT is pretty resistant to change that breaks backwards compatibility and unforced changes lol.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've been engaging the PowerShell team in the discussions with our design, and the crescendo team to aid in rapid prototyping. We will follow their guidance with respect to which versions of PowerShell behave differently so we can maintain the greatest compatibility level. I am certainly not a PowerShell expert, so I'm learning from the experts on that front. I wanted to get this draft out quickly so we could get community feedback. I'm building on the list of concerns brought up here so they can be clearly addressed in the document.

This is excellent news and much to be welcomed. Knowing what I do, Crescendo should provide a good way forward and helps to move this forward and makes the best out of the decision to do this all in C++. But we are where we are, and Crescendo does look promising as a way forward. It also gives the winget team to re-write certain cmdlets in C# where that makes business sense.

As I stated a while ago, creating the object model is very important. With Crescendo, as I understand it, you use regular expressions to create objects that the Crescendo-generated cmdlet emits. As I have said elsewhere, this is going to be a very difficult task to do well and completely - string to object translation is not an exact science. Thus my first concern is the objects emitted. It should be a goal for winget-ps to emit objects that are as rich and functional as those emitted by other modules and usable in rich cross-silo scenarios.

The second concern is over verb usage. As mentioned a long time ago, the subcommand of winget do not line up completely with PowerShell standard verb naming. For example, the show command in winget should probably be get. And hash should be something else. I would like to see the command to verb translation table - and please, please, please only uses approved verbs in all cmdlets.

I look forward to seeing how this progresses.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've been going through the process of learning the PowerShell verbs and continuing to expand the draft to others so we can get a sense of the best verbs and behaviors. First impressions are very valuable in this area. There is definitely some impedance to overcome between the .exe cli syntax and PowerShell. We're heads down on the v1.1 release, and then I'll have more time to dedicate to fleshing this out. We have been discussing the rich object model needed for a good PowerShell experience.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The verb choice is a vital component of the sacred vow. IT Pros would expect to use certain verbs and certain parameter names names. To create a truly great Powershell module means using approved verbs. :-)

@jantari
Copy link

jantari commented Aug 28, 2021

Looks like a solid first draft overall.

I have noticed there is no mention of ParameterSets, but some cmdlets parameters should probably be mutually exclusive e.g. these three wherever they are used:

-PackageIdentifier
-Name
-Moniker

I think a user should not be able to specify more than one of these for any given command.

@denelon
Copy link
Contributor Author

denelon commented Oct 14, 2021

It looks like we're going to go with Get-WinGetPackage for "list" a.k.a. installed packages and Find-WinGetPackages for "search" a.k.a. packages available in sources.

We're still debating some options for "show". We're not really displaying an entire manifest, but we're returning an object that is a bit of a subset of a manifest. We're still looking at parameters and verbs that might fit the bill better in this scenario.

@weeyin83
Copy link
Contributor

It looks like we're going to go with Get-WinGetPackage for "list" a.k.a. installed packages and Find-WinGetPackages for "search" a.k.a. packages available in sources.

This combo would be my preferred config. 👍

@denelon
Copy link
Contributor Author

denelon commented Oct 14, 2021

How about "Find-WinGetInstaller" for winget show?

@jantari
Copy link

jantari commented Oct 14, 2021

We're still debating some options for "show". We're not really displaying an entire manifest, but we're returning an object that is a bit of a subset of a manifest. We're still looking at parameters and verbs that might fit the bill better in this scenario.

You could do something like Get-WinGetManifest for retrieving the entire available dataset/manifest. At that point it's a different object you're returning so why not a different noun? Since the noun in PowerShell sort of is supposed to describe what you'll be working with.

This would also open the door for cmdlets like Test-WinGetManifest (for winget validate)


EDIT - Additional thoughts:

Frankly, I think if we have Get-WinGetPackage and Find-WinGetPackage that return the properties most people commonly want and then a cmdlet like Get-WinGetManifest to return all the gory details the usecase for an intermediary Show-WinGetPackage is small anyway. Anytime you are building cmdlets to retrieve the same object at three detail-levels the syntax isn't going to be the ideal. The original subcommands winget search and winget show are already hardly a paragon of pulchritude as far as their naming goes.

@weeyin83
Copy link
Contributor

How about "Find-WinGetInstaller" for winget show?

I like it, but wondering if Find-WinGetManifest might fit better? Or is that too controversial since you will only be returning a bit of the manifest?

@Trenly
Copy link
Contributor

Trenly commented Oct 14, 2021

How about "Find-WinGetInstaller" for winget show?

I like it, but wondering if Find-WinGetManifest might fit better? Or is that too controversial since you will only be returning a bit of the manifest?

I would think that Find-WinGetManifest or Find-WinGetPackageDetail would be better, since you're not getting details about the installer. I personally prefer the Find-WinGetPackageDetail since it is a more natural extension of Find-WinGetPackage and more adequately describes that it is getting the package details from the source. Whether the details are the version lists, or the portion of the manifest, or the dependencies, they would fall under the package details.

@jantari
Copy link

jantari commented Oct 14, 2021

For the "medium detail level" cmdlet (aka winget show - in between Find-WinGetPackage which gets basic information and retrieving the entire manifest), I am fine with any of these:

Show-WinGetPackage

Pros:

  • Same verb as "winget show"
  • Verb use somewhat similarly precedented by Show-NetFirewallRule (although that's more about visualization/formatting)

Cons:

  • Use of Show verb somewhat uncommon
  • Cmdlet name doesn't communicate well what exactly it'll do

Find-WinGetPackage -Detailed

Pros:

  • Unifies the similar winget search and show commands in one cmdlet
  • Use of parameter -Detailed is precedented by Get-Help.
  • Since -Detailed would be a [switch] parameter it could be toggled easily like -Detailed:$false / -Detailed:$true

Cons:

  • If the objects returned from Find-WinGetPackage with and without -Detailed will be too dissimilar it would not be intuitive to get them from the same cmdlet. However if it's going to just be the difference between winget search and winget show I think it'd be fine

Find-WinGetPackageDetail

Pros:

  • maybe more immediately discoverable than a parameter on Find-WinGetPackage since it shows in Get-Command -Module <wingetmodule>
  • some users may also just prefer a dedicated cmdlet.

Cons:

  • Cmdlet couldn't be fully tab-completed when using the PSReadLine completion-behavior called "Complete" because tab-completion would stop at "Find-WinGetPackage" since that's a valid cmdlet too, until the user manually types the qualifying D for tab completion to pick back up. From experience, this gets annoying for fast typers 😉

I wouldn't want to use the noun "WinGetManifest" for an object that isn't actually the entire package manifest - that'd just be confusing and make cmdlet names even weirder once we have cmdlets that do actually have to interact with the full manifest (like for validating or uploading a package manifest).

@Trenly
Copy link
Contributor

Trenly commented Oct 14, 2021

  • Cmdlet couldn't be fully tab-completed because tab-completion would stop at "Find-WinGetPackage" since that's a valid cmdlet too, until the user manually types the qualifying D for tab completion to pick back up. From experience, this gets annoying for fast typers 😉

I think it would be able to be? Doesn't powershell just cycle through available completion options?

ezgif-1-d0db8a1380aa

@jantari
Copy link

jantari commented Oct 14, 2021

@Trenly oh uh, yea - I forgot that it has that horrible completion behavior by default (it takes way too long to tab through stuff, and it's easy to tab past what you wanted). I have always adjusted my completion setting to be bash-like with:
Set-PSReadLineKeyHandler -Key Tab -Function Complete
It's a common configuration I think, as most users are unhappy with PowerShells/PSReadLines odd default setting.

But yea, you are right. It wouldn't matter for the default completion behavior. Me and the other users who use the "Complete" setting though would still appreciate the different cmdlet names :P

I updated my comment above

@denelon
Copy link
Contributor Author

denelon commented Oct 14, 2021

We're going to have "Get-WinGetManifest" for operations against a REST source. winget show is really pulling in a subset of a manifest isolated to the locale, machine architecture, and scope. We were thinking the object was more about the installer than a manifest.

@JohnMcPMS
Copy link
Member

We're going to have "Get-WinGetManifest" for operations against a REST source. winget show is really pulling in a subset of a manifest isolated to the locale, machine architecture, and scope. We were thinking the object was more about the installer than a manifest.

Remember to not be limited by how the current winget CLI operates. PowerShell is more like the COM API than the command line, and it will be far easier to make them work together the closer they are. I realize that makes an intermediate Crescendo based exploration more difficult to implement fully, but maybe it can be half measures until fully hooked up with COM. For instance, a Manifest object exposing a subset of the fields, but also all of the installers. For now you can only have one with Crescendo, but still make it an array that we can put all of them into it in the future.

Also, the source type should not ever be a consideration for API design at this level. They are all black box repositories of packages. Yes, there might be some special flags that only affect a subset of types, but that is acceptable so long as the underlying code is handling it, not this layer.

@denelon
Copy link
Contributor Author

denelon commented Oct 15, 2021

There may have been a lack of clarity on my part. We're not designing the PowerShell API based on the constraints of the Windows Package Manager. We're designing the API to work in the future anticipating changes in the Client. The functionality we're providing in this early "alpha" stage will take advantage of the portions of the system available today, and will be extended as improvements in the COM interface are made.

We're anticipating some performance hits in some areas due to the current implementation when we might need several round trip calls to populate the object model in some areas, and in some cases where we truncate strings in the table formatter, PowerShell pipelining will not work as expected in every case, so we're putting comments/documentation in place to call those out. We're also looking for possible approaches in PowerShell to at least be able to detect when this is happening so users can decide how to move forward util we have full fidelity.

@doctordns
Copy link

It seems to me that instead of emitting mangled strings, there should be an option at least for Crescendo, that send the output ion JSON, and enables Crescendo to do the conversion into objects.

When will the team publish the object model?

@denelon
Copy link
Contributor Author

denelon commented Oct 18, 2021

We still have to take what we get from the client (strings in most cases) and build objects. Crescendo ended up not being quite as useful as we had hoped due to the difference between the command line verbs/switches and PowerShell. One exception was the winget source export command returned JSON.

Since we ended up with four modules, we're going to create a new repository to get the PowerShell build pipelines set up, then we will move the "child" modules back to their respective repositories.

You can see the work in progress on my fork.

@leoniDEV
Copy link

@denelon I see your repo and I find 2 big issues:

  1. don't use Write-Host!

  2. Collect all the resultant objects in an array and then return the array, defeat the purpose to support the pipeline

@doctordns
Copy link

Regarding @leoniDEV's comment - it looks like the Write-Host is on an error path just before throwing an exception, so it would work in the normal case (well as I read it). There has to be a better way do throw that exception though.

And as an aside: both as a tool for debugging and a tool for letting the IT pro have more useful info, consider populating the functions with Write-Verbose and expose operations inside. The user never has to see them so it's no big deal leaving them in. And when you are debugging, especially with Crescendo functions, verbose output would be useful from time to time.

@RDMacLachlan
Copy link
Member

@denelon I see your repo and I find 2 big issues:

  1. don't use Write-Host!
  2. Collect all the resultant objects in an array and then return the array, defeat the purpose to support the pipeline

Already started making that shift in the Module 😊 returning informative objects for reference in a failure.

@denelon
Copy link
Contributor Author

denelon commented Nov 3, 2021

The Microsoft.WinGet.Source PowerShell module was released at https://github.com/microsoft/winget-cli-restsource/releases as a preview version. I've created a discussion to gather feedback.

@megamorf
Copy link

megamorf commented Nov 3, 2021

The Microsoft.WinGet.Source PowerShell module was released at https://github.com/microsoft/winget-cli-restsource/releases as a preview version. I've created a discussion to gather feedback.

That's great to hear. I'll check the code base and cmdlet behaviour this weekend and add my feedback 👍

@denelon
Copy link
Contributor Author

denelon commented Nov 3, 2021

We will be releasing the other modules containing cmdlets via another open source repository here at GitHub. They will be migrated to their respective repositories as they become a bit more complete. We will keep the other repository for the parent module designed to import all three of the modules we've defined:

Microsoft.WinGet

  • Microsoft.WinGet.Client
  • Microsoft.WinGet.Create
  • Microsoft.WinGet.Source

We're currently dotting the "I"s and crossing the "T"s to release the project.

Update: we've decided to keep the modules in the associated repository:

Microsoft.WinGet winget-cli

  • Microsoft.WinGet.Client winget-cli
  • Microsoft.WinGet.Create winget-create
  • Microsoft.WinGet.Source winget-cli-restsource

@denelon denelon closed this Dec 1, 2021
@denelon denelon deleted the PowerShell-cmdlets branch December 1, 2021 21:31
@floh96
Copy link
Contributor

floh96 commented Dec 2, 2021

@denelon why did you not merge the spec pr?

@denelon
Copy link
Contributor Author

denelon commented Dec 2, 2021

@floh96 I broke my branches yesterday. I just submitted #1760 yesterday after restoring one of them. I'll try to get this one back as well.

@denelon denelon restored the PowerShell-cmdlets branch December 2, 2021 18:27
@denelon denelon reopened this Dec 2, 2021
Copy link
Contributor

@ryfu-msft ryfu-msft left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

:shipit:

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.