[Question] BaGet as PowerShell module repository

[OCram85]

❓ Question

Did anyone managed to run BaGet as PowerShell module repository sucessfully?

📑 Details

It seems like there are several discussions about the usage of BaGet and PowerShell in general.

Especially finding and installing packages via PowerShell (PackageManagement & PowerShellGet) seems to be unstable.

BaGet would be the first PackageRespository type using NuGet v3 API. Could the issues relate to this?

💣 Testing Details

• Server hosts http://baget:5555
  • latest BaGet docker image
• Client
  • Windows PowerShell 5.1 Desktop
    • PowerShellGet 2.0.4
    • PackageManagement 1.2.4
  • Ubuntu PowerShell 6.2.3 Core

🖼 Screenshots

🔖 Refs

• Unable to install modules using the powershellget install-module commandlet #199
• Unable to install module hosted on a private baget server nuget feed (v3) OneGet/oneget#430

[tiksn]

Right now we have the same issue

[OCram85]

@loic-sharma: We really need this to be fixed 😀

Is there any way to help? - Maybe we could provide additional analysis data of specific test scenarios?

[OCram85]

BUMP.

@loic-sharma: Any updates about this?

[loic-sharma]

Hi, I apologize for the very long response. This requires supporting the legacy NuGet V2 APIs, which is tracked by #43. @Mizipzor and @jgagnaire made good progress on this (see #294 and #178).

I'll admit that I've been dragging my feet here as I feel torn on adding "legacy" APIs to BaGet. However, I believe that supporting Powershell modules and fully replacing NuGet.Server's feature set feels valuable enough. I will tackle this as part of BaGet V1 soon.

[Jaykul]

It's worth pointing out that PowerShellGet v3 is already in progres and doesn't intend to remain fully backward compatible @loic-sharma -- perhaps (assuming it uses the new v3 APIs, since they're commiting to using NuGet.Client) PowerShell is not a good enough reason to re-implement old APIs ...

[softlion]

Especially when powershell core is out and fully incompatible ;) new name: pwsh ! Includes SSH client !!!

[Jaykul]

@softlion except that PowerShell Core is backward compatible, and still ships the same old (v2) NuGet-based PowerShellGet dependency manager, so it doesn't change anything in this regard...

[OCram85]

So If I get this right, the problem could be fixed when:

• PowerShellGet v3 is published
  • because it uses the Nuget v3 API?
• Baget already uses Nuget v3 API
  • nothing to change?

[jeroenlandheer]

I have tried to install PowerShellGet v3 (Beta 4) and my results were not that encouraging 😢

PS> Get-PSResourceRepository

Name          Url                                           Trusted Priority
----          ---                                           ------- --------
MyRepository  https://[REDACTED]/v3/index.json               true    50
PSGallery     https://www.powershellgallery.com/api/v2      false   50

PS> Get-PSResourceRepository MyRepository | fl

Name     : MyRepository
Url      : https://[REDACTED]/v3/index.json
Trusted  : true
Priority : 50


PS> Find-PSResource -Repository MyRepository
No cursor found. Defaulting to 01/01/0001 00:00:00.
Find-PSResource: An invalid request URI was provided. The request URI must either be an absolute URI or BaseAddress must be set.
PS> Set-PSResourceRepository MyRepository -URL https://[REDACTED]/api/v2/package
PS> Find-PSResource -Repository MyRepository
Find-PSResource: 'doctype' is an unexpected token. The expected token is 'DOCTYPE'. Line 1, position 3.
PS> Set-PSResourceRepository MyRepository -URL https://[REDACTED]/api/v2
PS> Find-PSResource -Repository MyRepository
Find-PSResource: 'doctype' is an unexpected token. The expected token is 'DOCTYPE'. Line 1, position 3.
PS> Set-PSResourceRepository MyRepository -URL https://[REDACTED]/api/v3
PS> Find-PSResource -Repository MyRepository
Find-PSResource: 'doctype' is an unexpected token. The expected token is 'DOCTYPE'. Line 1, position 3.
PS> Set-PSResourceRepository MyRepository -URL https://[REDACTED]/api/v3/package
PS> Find-PSResource -Repository MyRepository
Find-PSResource: 'doctype' is an unexpected token. The expected token is 'DOCTYPE'. Line 1, position 3.
PS> Set-PSResourceRepository MyRepository -URL https://[REDACTED]/api/
PS> Find-PSResource -Repository MyRepository
Find-PSResource: 'doctype' is an unexpected token. The expected token is 'DOCTYPE'. Line 1, position 3.
PS>

I've tried several different URL's, none of them really work. Am I missing something?
(The first attempt did give a different message compared to the rest...)

[TylerLeonhardt]

Hello everyone 👋 I work on the PowerShell team and wanted to say a couple things:

1. BaGet is amazing. A really sleek NuGet server AND client that works on .NET Core... I'm in love
2. PowerShellGet v2 is what ships in Windows as a part of Windows PowerShell so needless to say... PSGet v2 is here to stay for a long long time...
3. I was able to get Publish-Module working which is really exciting 😁
4. However, like others here, Find and Install do not work

It would be awesome to see the addition of the last couple of v2 APIs to get Find/Install working 🙂

[brajjan]

I got it to install the module using PowershellGet v3.0.0-beta10
Published the module through v2, have not tried to publish through v3

~ ❯ Find-PSResource -Name NetworkingDsc -Repository Baget

Name          Version Repository Description
----          ------- ---------- -----------
NetworkingDsc 8.1.0.0 Baget      DSC resources for configuring settings related to networking.

~ ❯ Install-PSResource -Name NetworkingDsc -Repository Baget -Verbose -Debug

DEBUG: Parameters passed in >>> Name: 'NetworkingDsc'; Version: ''; Prerelease: 'False'; Repository: 'Baget'; Scope: ''; AcceptLicense: 'False'; Quiet: 'False'; Reinstall: 'False'; TrustRepository: 'False'; NoClobber: 'False';
DEBUG: Entering InstallHelper::ProcessInstallParams
DEBUG: Console is elevated: 'True'
DEBUG: Console is Windows PowerShell: 'False'
DEBUG: Current user scope installation path: 'C:\Users\username\Documents\PowerShell'
DEBUG: All users scope installation path: 'C:\Program Files\PowerShell'
VERBOSE: Scope is: CurrentUser
DEBUG: Checking to see if paths exist
DEBUG: Path: 'C:\Users\username\Documents\PowerShell\Modules'  >>> exists? 'True'
DEBUG: Path: 'C:\Users\username\Documents\PowerShell\Scripts'  >>> exists? 'True'
DEBUG: Path: 'C:\Users\username\Documents\PowerShell\Scripts\InstalledScriptInfos'  >>> exists? 'True'
DEBUG: Attempting to search for packages in 'Baget'
DEBUG: Untrusted repository accepted as trusted source.
VERBOSE: Begin installing package: 'NetworkingDsc'
DEBUG: Successfully able to download package from source to: 'C:\Users\username\AppData\Local\Temp\bc139dfc-f5c4-4057-96ea-fd22bd918bbe'
DEBUG: Deleting 'C:\Users\username\AppData\Local\Temp\bc139dfc-f5c4-4057-96ea-fd22bd918bbe\networkingdsc\8.1.0\networkingdsc.8.1.0.nupkg.sha512'
DEBUG: Deleting 'C:\Users\username\AppData\Local\Temp\bc139dfc-f5c4-4057-96ea-fd22bd918bbe\networkingdsc\8.1.0\networkingdsc.nuspec'
DEBUG: Deleting 'C:\Users\username\AppData\Local\Temp\bc139dfc-f5c4-4057-96ea-fd22bd918bbe\networkingdsc\8.1.0\networkingdsc.8.1.0.nupkg'
DEBUG: Installation path is: 'C:\Users\username\Documents\PowerShell\Modules\NetworkingDsc'
VERBOSE: Full installation path is: 'C:\Users\username\AppData\Local\Temp\bc139dfc-f5c4-4057-96ea-fd22bd918bbe\networkingdsc'
DEBUG: Attempting to move 'C:\Users\username\AppData\Local\Temp\bc139dfc-f5c4-4057-96ea-fd22bd918bbe\networkingdsc' to 'C:\Users\username\Documents\PowerShell\Modules\NetworkingDsc
VERBOSE: Successfully installed package NetworkingDsc
DEBUG: Attempting to delete 'C:\Users\username\AppData\Local\Temp\bc139dfc-f5c4-4057-96ea-fd22bd918bbe'

~ ❯ Get-DscResource -Module NetworkingDsc -Name NetIPInterface

ResourceType  : DSC_NetIPInterface
Name          : NetIPInterface
FriendlyName  : NetIPInterface
Module        : NetworkingDsc
ModuleName    : NetworkingDsc
Version       : 8.1.0
Path          : C:\Users\username\Documents\PowerShell\Modules\NetworkingDsc\8.1.0\DscResources\DSC_NetIPInterface\DSC_NetIPInterface.psm1
ParentPath    : C:\Users\username\Documents\PowerShell\Modules\NetworkingDsc\8.1.0\DscResources\DSC_NetIPInterface
ImplementedAs : PowerShell
CompanyName   : DSC Community
Properties    : {AddressFamily, InterfaceAlias, AdvertiseDefaultRoute, Advertising…}

[VertigoRay]

@brajjan, I see you got it to work with PowerShell Core using DSC. Did it also work for Install-Module or Find-Module? What about PowerShell Desktop (5.1)?

I tried BaGet and LiGet. Was getting the issues on both, but I didn't make the issues over here, yet:

• PowerShellGet v2.2.5, Find-Module errors with "The JSON object returned by the NuGet API does not match expected schema." ai-traders/liget#39
• PowerShellGet v1.0.0.1, Find-Module errors with "No match was found for the specified search criteria and module name 'ActiveDirectory'." ai-traders/liget#40

I'll try to make time today to make similar detailed output for BaGet. We're using sunside/simple-nuget-server instead. I'd prefer BaGet because it's beautiful, as @TylerLeonhardt said. However, we need it functional now. I'll keep an eye on this thread. I hope things get resolved here and we can swap to BaGet in the future.

[TylerLeonhardt]

@VertigoRay there's no frontend for that one is there?

[VertigoRay]

@TylerLeonhardt, you are correct. The meta and downloads are available though. I imagine something could be made relatively easy.

$module = 'ActiveDiretory'
$url = "https://simple-nuget-server.contoso.com/FindPackagesById()?id=%27${module}%27&$skip=0&$top=40
(Invoke-RestMethod $url)[0].properties.DownloadCount.'#text'

I would prefer to not fork that project. I would rather put time into this one. I just need to have time to do that. 😏

[Xeevis]

PowershellGet v3.0.0-beta10 seems to be working fine.

# Install latest prelease version of PowershellGet
Install-Module -Name PowerShellGet -AllowPrerelease -Force
# Register BaGet repository
Register-PSResourceRepository -Name 'BaGet' -Url 'http://127.0.0.1:5555/v3/index.json' -Trusted
# Publish resource
Publish-PSResource -Path E:\PowerShell\PSCalendar -Repository BaGet
# Find module
Find-PSResource PSCalendar -Repository BaGet
# Install module
Install-PSResource PSCalendar -Repository BaGet
# Run module function
Show-Calendar

[MS-LUF]

Hello,
I have tried PowershellGet v3.0.0-beta10. It works in HTTP but not using TLS. Each time the client is trying to communicate without TLS, even if the repo is set with a HTTPS uri... Any idea . Perhaps Xeevis, did you try to expose your repo using a TLS configuration ?

[jeroenlandheer]

@MS-LUF I've ran a couple of tests and I can confirm that using TLS this is not working on Beta 10 atm. (We have BaGet deployed on a linux kubernetes cluster with an NGINX front-end proxy.)

Once a new version of PowerShellGet is released I'll try again.

[TylerLeonhardt]

Might be worth getting an issue going on PowerShellGet?

[MS-LUF]

Might be worth getting an issue going on PowerShellGet?

Already done :)
https://github.com/PowerShell/PowerShellGet/issues/314

It's a shame that in 2020 a web component does not support TLS ! Moreover, TLS is supported for default repository (PSGallery). I don't understand why they have splitted the behavior in their code...

[OCram85]

Hosting a private NuGet service for Powershell is was a hell in 2021. The problem here isn't the server part at all.

Every time I had to connect a new client to the NuGet based service it scared the shit out of me because:

• PowerShell 5.1 and 7+ clients behave differently based on the os and already installed PackageManagement + PowerShellGet module versions
• PowerShellGet seems to be broken because of a (Ping-Endpoint / Invalid Web URI) bug, which prevent register https based repos.
  • Especially because they won't be fixed in version 2. The state of the new PowerShellGetv3 seems to be poorly supported and communicated. So we have to wait until the next major release .
  • The PackageManagement doesn't get updated as well.

[jeroenlandheer]

@OCram85 The issues you describe are very distinct.

• PowerShell 5.x is based on the .Net Framework; PowerShell 7+ on .Net Core (multi-platform) or .net 5/6. Modules created for these versions are very different and not binary compatible. This isn't really related to package management.
• The released versions of PowerShellGet (v2.x) is only compatible with the NuGet v2 protocol, this is causing all these issues we're talking about here in the thread. The new version of PowerShellGet should support NuGet v3 and therefore also integrate better with BaGet.

Hope this clarifies a few things.

[OCram85]

You're absolutely right. My comment wasn't really clear and much more impulsive than it was intended.

Just wanted to describe the situation I'm facing in a heterogen environment with:

• different os (win/linux)
• multiple pwsh versions
• 5.1 (.net based)
• 7+ (.Net Core)
• and even containerized pwsh clients in CI/CD environment.

I've invested weeks in the last years to get these systems hooked up with a private NutGet Server like PowershellGallery (BaGet, Nexus Repository....). In some cases it works instant and others you end up reading PowerShellGet and PackageManagement issues / workarounds.

So to sum up I just want to tell my personal opinion about this situation:

I really like working and developing modules for pwsh. But If you need to deploy your modules internally you're really faced
with some additional challenges.
It seems to me like the PowerShellGet based package management is the only valid way to publish and consume pwsh modules. Unfortunately I don't see much progress in solving the existing issues in PowerShellGet v2, or even a clear roadmap when to expect the PowerShellGet v3.

😄

[commutecomfort]

The following worked for me. It can be a workaround/ solution depending on your requirements:

Download the Nuget client from the following URL. Nuget client is a .exe file. It does not need any installation, not does it need admin rights to execute.

https://learn.microsoft.com/en-us/nuget/install-nuget-client-tools?tabs=windows#install-nugetexe

Then, open a PS window with the folder where nuget.exe was downloaded. Run the following command to add BaGet as a source for NuGet client (nuget.exe).

.\nuget sources Add -Name "BaGet" -Source "http://localhost:5000/v3/index.json"

If the source BaGet has already been added, then run the following command to configure it's settings. (Optional)

.\nuget sources Update -Name "BaGet" -Source "http://localhost:5000/v3/index.json"

Use the following commands to push packages to Baget. These packages are downloaded in the form of .nupkg file from the PowerShell Gallery. The -ApiKey parameter is the value set in appsettings.json for the ApiKey.

.\nuget push -Source BaGet -ApiKey SuperSecret packagemanagement.1.4.8.1.nupkg

.\nuget push -Source BaGet -ApiKey SuperSecret powerstig.4.20.0.nupkg

.\nuget push -Source BaGet -ApiKey SuperSecret get-windowsautopilotinfo.3.9.0.nupkg

Run the following command to install the package on local machine (where you access the BaGet webpage from). This works on Windows. It can possibly work on Linux as well with the equivalent of NuGet.exe for the linux world.

.\nuget install < < name of the package > > -Source BaGet

.\nuget install get-windowsautopilotinfo -Source BaGet

Then, navigate to the directory where the package (PS module) was installed. If you are installing it for -Scope CurrentUser, the directory would be: "C:\users< < username > >\documents\windowspowershell\modules\ < < name of the PS Module > > ".

Now the .psm1 file needs to be imported. Run the following command to do so. You may also browse to the module location with Windows Explorer and look for the name of .psm1 file. Usually, it is the same as the module name. In some cases, you many need to import .psd1 file as well, for the PS module to work.

import-module < <Path to the .psm1 file> >

import-module .\powerstig.4.20.0.psm1
import-module .\get-windowsautopilotinfo.3.9.0.psm1

Run the following command to see the newly added PS cmdlets from the installed module(s).
Get-Command -Module < < Name of the installed module > >

The following is a link to Nuget CLI documentation.
https://learn.microsoft.com/en-us/nuget/reference/cli-reference/cli-ref-sources

[Jaykul]

For what it's worth...

After all these years, the new version (named PSResourceGet) has finally released with support for the v3 feed format. It might be worth testing with that (and opening an issue on the new repo if it doesn't work) 😉

Also, about the compatibility thing... Although backward compatibility is obviously never trivial, writing PowerShell modules that work across multiple versions back to Windows PowerShell (5.1) and on Windows, Linux and OSX is not that hard -- and PSResourceGet for instance, should work across them all.