Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix: Fix download from gh-r #373

Merged
merged 2 commits into from
Sep 18, 2022
Merged

fix: Fix download from gh-r #373

merged 2 commits into from
Sep 18, 2022

Conversation

yutkat
Copy link
Contributor

@yutkat yutkat commented Sep 12, 2022

Description

I tried to update plugins today (September 13, 2022) and the binary download from the GitHub release failed.
gh-r: failed to find the correct GitHub release asset to download.

It seems that the GitHub website has been updated and the method of parsing assets from HTML no longer works.
Check this command and video.
curl https://github.com/BurntSushi/ripgrep/releases
https://user-images.githubusercontent.com/8683947/189714315-fa1d28b0-755a-46f8-9239-5f16291df26b.mp4

I changed the process to get the list of released assets from the API.
This is a temporary patch and we may need to consider a few more things. However, I think it is better than not working.

Related Issue(s)

Reproduce steps

.zshrc

ZPLG_HOME="$HOME/.config/zinit"

if ! test -d "$ZPLG_HOME"; then
  mkdir -p "$ZPLG_HOME"
  chmod g-rwX "$ZPLG_HOME"
  git clone --depth 10 https://github.com/zdharma-continuum/zinit.git ${ZPLG_HOME}/bin
fi

typeset -gAH ZPLGM
ZPLGM[HOME_DIR]="${ZPLG_HOME}"
source "$ZPLG_HOME/bin/zinit.zsh"
autoload -Uz _zinit
(( ${+_comps} )) && _comps[zinit]=_zinit

zinit for \
    from'gh-r' \
    as'program' pick'ripgrep*/rg' \
  BurntSushi/ripgrep

Error

gh-r: failed to find the correct GitHub release asset to download.

How Has This Been Tested?

Types of changes

  • Bug fix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)
  • Breaking change (fix or feature that would cause existing functionality to change)

Checklist:

  • My change requires a change to the documentation.
  • I have updated the documentation accordingly.
  • I have added tests to cover my changes.
  • All new and existing tests passed.

Copy link
Contributor

@tylerw tylerw left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I fixed this same issue locally with the following change, but your proposed fix is probably more robust:

diff --git a/zinit-install.zsh b/zinit-install.zsh
index 531f5299..f0035372 100644
--- a/zinit-install.zsh
+++ b/zinit-install.zsh
@@ -354,7 +354,7 @@ builtin source "${ZINIT[BIN_DIR]}/zinit-side.zsh" || {
 
     (
         if [[ $site = */releases ]] {
-            local url=$site/${ICE[ver]}
+            local url=$site/expanded_assets/${ICE[ver]}
 
             .zinit-get-latest-gh-r-url-part "$user" "$plugin" "$url" || return $?
 
@@ -1456,7 +1456,7 @@ builtin source "${ZINIT[BIN_DIR]}/zinit-side.zsh" || {
     local user=$1 plugin=$2 urlpart=$3
 
     if [[ -z $urlpart ]] {
-        local url=https://github.com/$user/$plugin/releases/$ICE[ver]
+        local url=https://github.com/$user/$plugin/releases/expanded_assets/$ICE[ver]
     } else {
         local url=https://$urlpart
     }

@yutkat
Copy link
Contributor Author

yutkat commented Sep 13, 2022

@tylerw Oh! That's great. I didn't know that page existed.

Your idea looks better, so can you make a new PR instead of me?

@yutkat yutkat closed this Sep 13, 2022
@yutkat
Copy link
Contributor Author

yutkat commented Sep 13, 2022

@tylerw It might not work well when ${ICE[ver]} is empty. When getting the latest version, ${ICE[ver]} will be empty.

https://github.com/BurntSushi/ripgrep/releases/expanded_assets/

@yutkat yutkat reopened this Sep 13, 2022
@yutkat
Copy link
Contributor Author

yutkat commented Sep 13, 2022

My current method works, but it causes a rate limit on the GitHub API, It might be better to get the latest tag somehow and fix this issue tylerw's method.

@yutkat yutkat force-pushed the main branch 2 times, most recently from a7e8507 to 92128f9 Compare September 13, 2022 03:03
@yutkat
Copy link
Contributor Author

yutkat commented Sep 13, 2022

I updated the patch based on tylerw's idea.

I want to make a function to get the latet tag, but I don't know where to place it properly.

@numToStr
Copy link

Also, for some of the binary, zinit is downloading incompatible binary

image

My machine is x86_64 but zinit is downloading arm binaries. @yutkat Does this PR also fixes this issue?

@crivotz
Copy link

crivotz commented Sep 14, 2022

@numToStr I also have the same issue as you, and I use bpick to indicate which binary I want. This PR solves the problem that the bpick could not find the indicated version.

@yutkat
Copy link
Contributor Author

yutkat commented Sep 14, 2022

My machine is x86_64 but zinit is downloading arm binaries. @yutkat Does this PR also fixes this issue?

I haven't changed the binary determination process so that's may a different issue.

@numToStr I also have the same issue as you, and I use bpick to indicate which binary I want.

If it's fixed, why? 🤔

BTW, I'm also using x86_64 and I was able to download fd properly.

@numToStr
Copy link

BTW, I'm also using x86_64 and I was able to download fd properly

@yutkat would you mind sharing your script for fd?

zinit-install.zsh Outdated Show resolved Hide resolved
Co-authored-by: Tyler Wardhaugh <[email protected]>
@crivotz
Copy link

crivotz commented Sep 15, 2022

Just for info, today I tried a fresh zinit installation and the problem with du/rg no longer occurred

@numToStr
Copy link

Yeah, I also just did zinit update everything seems to be working fine again, no errors and no downloading wrong binary.

@mrroll
Copy link

mrroll commented Sep 16, 2022

I can confirm that this PR also fixes an issue I'm having with https://github.com/starship/starship.

My arch is under the More Assets button in the latest release like so:

image

bpick fails to find it and errors out when installing/updating. When I have this PR checked out, it downloads the correct archive.

@yutkat
Copy link
Contributor Author

yutkat commented Sep 16, 2022

@pschmitt @vladdoster Sorry for the mensch. But I think this is a pretty serious issue, so I hope you'll check this out as soon as possible.

@jankatins
Copy link
Contributor

Can confirm, it fixes the starship problem in #374 (comment)

@vladdoster vladdoster merged commit 4a2a120 into zdharma-continuum:main Sep 18, 2022
@vladdoster
Copy link
Member

Big thank you to @yutkat & @tylerw for fixing!!

Sorry for delay in merging :^/.

Cheers!

psprint pushed a commit to psprint/.priv-zinit-fork that referenced this pull request Sep 20, 2022
Recent changes to GitHub HTML structure broke `gh-r` & `bpick` ices method of parsing project release assets.

- `gh-r` utilizes official GitHub REST API to retrieve project releases data
- add `local` scope to `releases_url` variable

# closes zdharma-continuum#374 

Co-authored-by: Tyler Wardhaugh <[email protected]>
github-actions bot pushed a commit that referenced this pull request Nov 7, 2022
# [3.8.0](v3.7.0...v3.8.0) (2022-11-07)

### Bug Fixes

* account for systems where musl is present ([#269](#269)) ([8620574](8620574))
* alist repository for gh-r test ([#305](#305)) ([fb3c082](fb3c082))
* allow zinit to be run from non-interactive scripts ([#227](#227)) ([c3d1bb5](c3d1bb5)), closes [#199](#199)
* broken yaml syntax in issue template ([#355](#355)) ([f729e06](f729e06))
* calico gh-r zunit test ([#356](#356)) ([56fb9e0](56fb9e0))
* change ctags symbols browser key  binding from `ctrl-k` to `alt-Q` ([#387](#387)) ([7f6dc7d](7f6dc7d)), closes [#386](#386)
* Do not try to escape exclamation marks ([#399](#399)) ([0e55b2e](0e55b2e))
* docs workflow should fail if out-of-date ([#278](#278)) ([07cde66](07cde66))
* Don't error if $OPTS is not yet defined in .zinit-compinit call ([44765e0](44765e0))
* filter by runtime detected CPU before compiled CPU ([#304](#304)) ([a4dc13f](a4dc13f)), closes [#287](#287)
* gh-r & plugin zunit tests ([dd12fce](dd12fce))
* gh-r filters i686 (32 bit) for x86_64 ([#226](#226)) ([57f0d82](57f0d82)), closes [#225](#225)
* gh-r logic ignores [36]86 assets ([#235](#235)) ([d60638f](d60638f)), closes [#225](#225) [#246](#246) [#247](#247)
* gh-r removes linux32 assets on 64 bit OS ([1864c0b](1864c0b))
* gh-r retrieves release data GH REST API  ([#373](#373)) ([4a2a120](4a2a120)), closes [#374](#374)
* modify regex in gh-r for assets to not consider for selection ([#244](#244)) ([6ef8439](6ef8439))
* more cleaning up urls ([672ae51](672ae51)), closes [#47](#47)
* names of ctag Make target deps ([#407](#407)) ([9987d5c](9987d5c))
* package are broken again ([24f10f6](24f10f6))
* permissions for PR labeler GH action workflow ([#236](#236)) ([8a0d567](8a0d567))
* read without -r is generally bad. ([00c70a4](00c70a4))
* remove curl option "--tcp-fastopen" which is not always available ([#299](#299)) ([308c9d4](308c9d4))
* remove macOS 10.5 & 11 from test matrix ([c613193](c613193))
* remove use less line ([4f87076](4f87076))
* rename `docs` to `doc` to match doc dir ([#212](#212)) ([3a7dc95](3a7dc95))
* rm linux32 assets in aarch64/arm64 gh-r regex ([#414](#414)) ([529aa20](529aa20))
* syntax error when checking for `realpath` command  ([#259](#259)) ([05559eb](05559eb)), closes [#257](#257)
* trigger for PR labeler GH action workflow ([#237](#237)) ([49af866](49af866))
* typo & triggers in documentation workflow ([#308](#308)) ([161d7c1](161d7c1))
* unmatched "(" in windows gh-r patterns ([#280](#280)) ([1f4ba5a](1f4ba5a))
* update `zdharma` to `zdharma-continuum` ([66b1700](66b1700))
* update docs for new jq-check ([6207427](6207427))
* use [*] inside arbitrary strings. ([73a8c92](73a8c92))
* workflow pkg mgmt due to base OS changes ([195f72d](195f72d))
* ziextract execs discovery regex ([#410](#410)) ([105b38a](105b38a))
* zunit install in GH workflow ([#412](#412)) ([f4787dc](f4787dc))

### Features

* ability to set program for `zinit ls` to use ([#221](#221)) ([bad7af3](bad7af3)), closes [#170](#170)
* add `-a` (actual time) to `zinit times` cmd ([#223](#223)) ([450d3c1](450d3c1))
* add `krew` and `prebuilt-ripgrep` gh-r zunit tests ([#267](#267)) ([f25b4ae](f25b4ae))
* add compile vim from source zunit test ([#232](#232)) ([126528c](126528c))
* add configure"" ice ([#334](#334)) ([40a46c6](40a46c6))
* add GH action to remove old workflow logs ([#248](#248)) ([6647bdc](6647bdc))
* add PR labeler to show what parts of Zinit are changed ([#211](#211)) ([42e83d7](42e83d7))
* add releases via semantic-release ([73542b4](73542b4))
* add releases via semantic-release ([#415](#415)) ([cfa2f0e](cfa2f0e))
* expand linted file types to markdown and shell ([96fe03f](96fe03f))
* **git-process-output:** simplify progress-bar ([#204](#204)) ([c888917](c888917))
* update output messaging to be more informative ([047320a](047320a))
@github-actions
Copy link

github-actions bot commented Nov 7, 2022

🎉 This PR is included in version 3.8.0 🎉

The release is available on GitHub release

Your semantic-release bot 📦🚀

psprint pushed a commit to psprint/.priv-zinit-fork that referenced this pull request Dec 30, 2022
Recent changes to GitHub HTML structure broke `gh-r` & `bpick` ices method of parsing project release assets.

- `gh-r` utilizes official GitHub REST API to retrieve project releases data
- add `local` scope to `releases_url` variable

# closes zdharma-continuum#374 

Co-authored-by: Tyler Wardhaugh <[email protected]>
psprint pushed a commit to psprint/.priv-zinit-fork that referenced this pull request Dec 30, 2022
# [3.8.0](zdharma-continuum/zinit@v3.7.0...v3.8.0) (2022-11-07)

### Bug Fixes

* account for systems where musl is present ([zdharma-continuum#269](zdharma-continuum#269)) ([8620574](zdharma-continuum@8620574))
* alist repository for gh-r test ([zdharma-continuum#305](zdharma-continuum#305)) ([fb3c082](zdharma-continuum@fb3c082))
* allow zinit to be run from non-interactive scripts ([zdharma-continuum#227](zdharma-continuum#227)) ([c3d1bb5](zdharma-continuum@c3d1bb5)), closes [zdharma-continuum#199](zdharma-continuum#199)
* broken yaml syntax in issue template ([zdharma-continuum#355](zdharma-continuum#355)) ([f729e06](zdharma-continuum@f729e06))
* calico gh-r zunit test ([zdharma-continuum#356](zdharma-continuum#356)) ([56fb9e0](zdharma-continuum@56fb9e0))
* change ctags symbols browser key  binding from `ctrl-k` to `alt-Q` ([zdharma-continuum#387](zdharma-continuum#387)) ([7f6dc7d](zdharma-continuum@7f6dc7d)), closes [zdharma-continuum#386](zdharma-continuum#386)
* Do not try to escape exclamation marks ([zdharma-continuum#399](zdharma-continuum#399)) ([0e55b2e](zdharma-continuum@0e55b2e))
* docs workflow should fail if out-of-date ([zdharma-continuum#278](zdharma-continuum#278)) ([07cde66](zdharma-continuum@07cde66))
* Don't error if $OPTS is not yet defined in .zinit-compinit call ([44765e0](zdharma-continuum@44765e0))
* filter by runtime detected CPU before compiled CPU ([zdharma-continuum#304](zdharma-continuum#304)) ([a4dc13f](zdharma-continuum@a4dc13f)), closes [zdharma-continuum#287](zdharma-continuum#287)
* gh-r & plugin zunit tests ([dd12fce](zdharma-continuum@dd12fce))
* gh-r filters i686 (32 bit) for x86_64 ([zdharma-continuum#226](zdharma-continuum#226)) ([57f0d82](zdharma-continuum@57f0d82)), closes [zdharma-continuum#225](zdharma-continuum#225)
* gh-r logic ignores [36]86 assets ([zdharma-continuum#235](zdharma-continuum#235)) ([d60638f](zdharma-continuum@d60638f)), closes [zdharma-continuum#225](zdharma-continuum#225) [zdharma-continuum#246](zdharma-continuum#246) [zdharma-continuum#247](zdharma-continuum#247)
* gh-r removes linux32 assets on 64 bit OS ([1864c0b](zdharma-continuum@1864c0b))
* gh-r retrieves release data GH REST API  ([zdharma-continuum#373](zdharma-continuum#373)) ([4a2a120](zdharma-continuum@4a2a120)), closes [zdharma-continuum#374](zdharma-continuum#374)
* modify regex in gh-r for assets to not consider for selection ([zdharma-continuum#244](zdharma-continuum#244)) ([6ef8439](zdharma-continuum@6ef8439))
* more cleaning up urls ([672ae51](zdharma-continuum@672ae51)), closes [zdharma-continuum#47](zdharma-continuum#47)
* names of ctag Make target deps ([zdharma-continuum#407](zdharma-continuum#407)) ([9987d5c](zdharma-continuum@9987d5c))
* package are broken again ([24f10f6](zdharma-continuum@24f10f6))
* permissions for PR labeler GH action workflow ([zdharma-continuum#236](zdharma-continuum#236)) ([8a0d567](zdharma-continuum@8a0d567))
* read without -r is generally bad. ([00c70a4](zdharma-continuum@00c70a4))
* remove curl option "--tcp-fastopen" which is not always available ([zdharma-continuum#299](zdharma-continuum#299)) ([308c9d4](zdharma-continuum@308c9d4))
* remove macOS 10.5 & 11 from test matrix ([c613193](zdharma-continuum@c613193))
* remove use less line ([4f87076](zdharma-continuum@4f87076))
* rename `docs` to `doc` to match doc dir ([zdharma-continuum#212](zdharma-continuum#212)) ([3a7dc95](zdharma-continuum@3a7dc95))
* rm linux32 assets in aarch64/arm64 gh-r regex ([zdharma-continuum#414](zdharma-continuum#414)) ([529aa20](zdharma-continuum@529aa20))
* syntax error when checking for `realpath` command  ([zdharma-continuum#259](zdharma-continuum#259)) ([05559eb](zdharma-continuum@05559eb)), closes [zdharma-continuum#257](zdharma-continuum#257)
* trigger for PR labeler GH action workflow ([zdharma-continuum#237](zdharma-continuum#237)) ([49af866](zdharma-continuum@49af866))
* typo & triggers in documentation workflow ([zdharma-continuum#308](zdharma-continuum#308)) ([161d7c1](zdharma-continuum@161d7c1))
* unmatched "(" in windows gh-r patterns ([zdharma-continuum#280](zdharma-continuum#280)) ([1f4ba5a](zdharma-continuum@1f4ba5a))
* update `zdharma` to `zdharma-continuum` ([66b1700](zdharma-continuum@66b1700))
* update docs for new jq-check ([6207427](zdharma-continuum@6207427))
* use [*] inside arbitrary strings. ([73a8c92](zdharma-continuum@73a8c92))
* workflow pkg mgmt due to base OS changes ([195f72d](zdharma-continuum@195f72d))
* ziextract execs discovery regex ([zdharma-continuum#410](zdharma-continuum#410)) ([105b38a](zdharma-continuum@105b38a))
* zunit install in GH workflow ([zdharma-continuum#412](zdharma-continuum#412)) ([f4787dc](zdharma-continuum@f4787dc))

### Features

* ability to set program for `zinit ls` to use ([zdharma-continuum#221](zdharma-continuum#221)) ([bad7af3](zdharma-continuum@bad7af3)), closes [zdharma-continuum#170](zdharma-continuum#170)
* add `-a` (actual time) to `zinit times` cmd ([zdharma-continuum#223](zdharma-continuum#223)) ([450d3c1](zdharma-continuum@450d3c1))
* add `krew` and `prebuilt-ripgrep` gh-r zunit tests ([zdharma-continuum#267](zdharma-continuum#267)) ([f25b4ae](zdharma-continuum@f25b4ae))
* add compile vim from source zunit test ([zdharma-continuum#232](zdharma-continuum#232)) ([126528c](zdharma-continuum@126528c))
* add configure"" ice ([zdharma-continuum#334](zdharma-continuum#334)) ([40a46c6](zdharma-continuum@40a46c6))
* add GH action to remove old workflow logs ([zdharma-continuum#248](zdharma-continuum#248)) ([6647bdc](zdharma-continuum@6647bdc))
* add PR labeler to show what parts of Zinit are changed ([zdharma-continuum#211](zdharma-continuum#211)) ([42e83d7](zdharma-continuum@42e83d7))
* add releases via semantic-release ([73542b4](zdharma-continuum@73542b4))
* add releases via semantic-release ([zdharma-continuum#415](zdharma-continuum#415)) ([cfa2f0e](zdharma-continuum@cfa2f0e))
* expand linted file types to markdown and shell ([96fe03f](zdharma-continuum@96fe03f))
* **git-process-output:** simplify progress-bar ([zdharma-continuum#204](zdharma-continuum#204)) ([c888917](zdharma-continuum@c888917))
* update output messaging to be more informative ([047320a](zdharma-continuum@047320a))
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

7 participants