Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jbang-catalog can't be downloaded #49

Open
Bullfrog666 opened this issue Oct 9, 2023 · 24 comments
Open

Jbang-catalog can't be downloaded #49

Bullfrog666 opened this issue Oct 9, 2023 · 24 comments

Comments

@Bullfrog666
Copy link

Since the last update (3.4.2) we've been having an error when building our docker, which launches a back in quarkus :

[jbang] [ERROR] Could not download https://github.com/quarkusio/jbang-catalog/blob/HEAD/jbang-catalog.json
@gsmet
Copy link
Member

gsmet commented Oct 9, 2023

What are you doing exactly to get this error? Thanks.

@Bullfrog666
Copy link
Author

I think the error comes from these lines :

RUN curl -Ls https://sh.jbang.dev/ | bash -s - trust add https://repo1.maven.org/maven2/io/quarkus/quarkus-cli/
RUN curl -Ls https://sh.jbang.dev/ | bash -s - app install --fresh --force quarkus@quarkusio
ENV PATH="${PATH}:/root/.jbang/bin"

But if you want some more context, I can share the complete Dockerfile with you.

@gsmet
Copy link
Member

gsmet commented Oct 9, 2023

OK, I wanted to know if it was a problem with Quarkus CLI. Apparently it's a problem with either JBang or the GitHub infra.

/cc @maxandersen

@Bullfrog666
Copy link
Author

With the docker-compose up --build, we still got these logs :

[+] Building 1.3s (16/16) FINISHED
 => [internal] load build definition from Dockerfile 0.0s 
 => => transferring dockerfile: 721B 0.0s 
 => [internal] load .dockerignore 0.0s 
 => => transferring context: 2B 0.0s 
 => [internal] load metadata for docker.io/library/openjdk:17-ea-slim-buster 1.0s 
 => [internal] load build context 0.0s 
 => => transferring context: 8.00kB 0.0s 
 => [ 1/11] FROM docker.io/library/openjdk:17-ea-slim-buster@sha256:370d2cdc58a1c8e5ddddae16f9467f40f07306be32b42c5ff838bc227e55d79c 0.0s 
 => CACHED [ 2/11] RUN apt update 0.0s 
 => CACHED [ 3/11] RUN apt install -y curl wget 0.0s 
 => CACHED [ 4/11] RUN curl -Ls https://sh.jbang.dev/ | bash -s - trust add https://repo1.maven.org/maven2/io/quarkus/quarkus-cli/ 0.0s 
 => CACHED [ 5/11] RUN curl -Ls https://sh.jbang.dev/ | bash -s - app install --fresh --force quarkus@quarkusio 0.0s 
 => CACHED [ 6/11] RUN mkdir -p /opt 0.0s 
 => CACHED [ 7/11] RUN wget -q https://dlcdn.apache.org/maven/maven-3/3.9.4/binaries/apache-maven-3.9.4-bin.tar.gz -O /opt/apache-maven.tar.gz 0.0s 
 => CACHED [ 8/11] RUN cd /opt && tar xf /opt/apache-maven.tar.gz 0.0s 
 => CACHED [ 9/11] WORKDIR /deployments/quarkus

Where we can find our 2 last lines (steps 4 and 5)
This may mean that the build goes well, but that it's the app startup that gives us the error log.

@maxandersen
Copy link
Member

can you run with --verbose? it should then include a reason. I suspect the issue here is a possible rate limit or local outage of github raw apis.

@Bullfrog666
Copy link
Author

I tried to add --verbose to the line CMD [ "quarkus", "dev", "--verbose" ] in the Dockerfile but it didn't change anything on the logs :/
We thought quarkus call jbang internally so we can't interact with it directly.

@maxandersen
Copy link
Member

ah, try with "-e" or "--errors" then CLI should print info.

@Bullfrog666
Copy link
Author

Nothing I add to the line in Dockerfile seems to change the error log I got.
I got the error log directly in the Docker container console, not in my own console.
Even a --help doesn't work.

@vic1707
Copy link

vic1707 commented Oct 9, 2023

Hello, I work on the same team as @Bullfrog666
I modified our Dockerfile so we can trigger manually the quarkus CLI

image

As you can see none of the arguments seem to do anything :/

Can jbang triggers its verbose mode based on ENV var ?

@maxandersen
Copy link
Member

Do you have the failing docker file or somehow provide reproducer? Does it happen locally too?

@maxandersen
Copy link
Member

@iocanel didbt you recently fix something around not having the cli fail when it cannot download the metadata?

@vic1707
Copy link

vic1707 commented Oct 9, 2023

our complete dockerfile:

FROM openjdk:17-ea-slim-buster

RUN apt update
RUN apt install -y curl wget


RUN curl -Ls https://sh.jbang.dev | bash -s - trust add https://repo1.maven.org/maven2/io/quarkus/quarkus-cli/
RUN curl -Ls https://sh.jbang.dev | bash -s - app install --fresh --force quarkus@quarkusio
ENV PATH="${PATH}:/root/.jbang/bin"

RUN mkdir -p /opt
RUN wget -q https://dlcdn.apache.org/maven/maven-3/3.9.4/binaries/apache-maven-3.9.4-bin.tar.gz -O /opt/apache-maven.tar.gz
RUN cd /opt && tar xf /opt/apache-maven.tar.gz

ENV PATH="${PATH}:/opt/apache-maven-3.9.4/bin"

WORKDIR /deployments/quarkus
COPY ./backend/pom.xml ./pom.xml

RUN mvn dependency:resolve

# previously CMD [ "quarkus", "dev" ]
CMD [ "sleep", "infinity" ] 

it works locally for @Bullfrog666 and the other dev who also do backend
(I don't even have the JDK on my machine I only ever used the docker image)

@iocanel
Copy link

iocanel commented Oct 9, 2023

@iocanel didbt you recently fix something around not having the cli fail when it cannot download the metadata?

The fix was merged 2 weeks ago and was specific to cli plug commands. If there are other areas of the cli that reach for the jbang catalog they might not be covered.

@maxandersen
Copy link
Member

so you you build that dockerfile, then do docker -it run and run quarkus dev inside and it fails?

@iocanel any idea why quarkus dev would ever want to fetch jbang catalog?

@maxandersen
Copy link
Member

Since the last update (3.4.2) we've been having an error when building our docker, which launches a back in quarkus :

[jbang] [ERROR] Could not download https://github.com/quarkusio/jbang-catalog/blob/HEAD/jbang-catalog.json

can you show the log that happens before that line?

@Bullfrog666 says it happens when installing quarkus cli but @vic1707 writes as it fails when running quarkus dev. those two are quite different so would be good to know exactly which scenario we are talking here.

In any cases if it works from some but fails for others I'm assuming it is some network setup differences. maybe one is running in a container that are not allowed to access github.com wiothut a proxy configuration or similar?

@vic1707
Copy link

vic1707 commented Oct 9, 2023

I talked with Bullfrog to clarify everything.

the jbang error happens when running any quarkus command in the container (as shown in my previous screenshot). the error only contains the two lines you can see on said screenshot.

Locally the quarkus app can be launched so you might be right about a proxy or network config.


I just went back the a few commits of our dockerfile and one of the last changes was

- FROM eclipse-temurin:17-jdk
+ FROM openjdk:17-ea-slim-buster

which once reverted seems to fix the jbang issue.
I have no clue as to why the base image could be the cause, if it's normal to fail on openjdk:17-ea-slim-buster, or even what is the difference between these (i'm not a java dev), I just now we had to change because CI was angry with the temurin image

@maxandersen
Copy link
Member

there could be some certificate issues/differences?

if you have a container that fails to execute quarkus commands - can you run jbang --verbose --fresh catalog list jbangdev ? then we can at least know if quarkus or jbang specific and jbang should tell print hints on what is the underlying cause to why it cannot download the github url.

curious to grok why temurin images are giving you a hard time but maybe that's a different issue.

@vic1707
Copy link

vic1707 commented Oct 9, 2023

here's the output

root@55d834aa1cde:/deployments/quarkus# jbang --verbose --fresh catalog list jbangdev
[jbang] [0:176] jbang version 0.111.0
[jbang] [0:183] Reading catalog from /root/.jbang/implicit-catalog.json
[jbang] [0:215] Reading catalog from classpath:/jbang-catalog.json
[jbang] [0:216] Local catalog 'jbangdev' not found, trying implicit catalogs...
[jbang] [0:262] Requesting HTTP GET https://www.jbang.dev/releases/latest/download/version.txt
[jbang] [0:262] Requesting HTTP GET https://raw.githubusercontent.com/jbangdev/jbang-catalog/HEAD/jbang-catalog.json
[jbang] [0:263] Headers {User-Agent=[JBang/0.111.0 (Linux/6.3.13-linuxkit/aarch64) Java/17-ea/Oracle Corporation]}
[jbang] [0:263] Headers {User-Agent=[JBang/0.111.0 (Linux/6.3.13-linuxkit/aarch64) Java/17-ea/Oracle Corporation]}
[jbang] [0:408] Redirected to: https://github.com/jbangdev/jbang/releases/latest/download/version.txt
[jbang] [0:566] Downloaded file https://raw.githubusercontent.com/jbangdev/jbang-catalog/HEAD/jbang-catalog.json
[jbang] [0:566] Reading catalog from https://github.com/jbangdev/jbang-catalog/blob/HEAD/jbang-catalog.json
[jbang] [0:568] Obtained catalog from https://github.com/jbangdev/jbang-catalog/blob/HEAD/jbang-catalog.json
[jbang] [0:568] Catalog found at https://github.com/jbangdev/jbang-catalog/blob/HEAD/jbang-catalog.json
[jbang] [0:570] Requesting HTTP GET https://raw.githubusercontent.com/jbangdev/jbang-catalog/HEAD/jbang-catalog.json
[jbang] [0:570] Headers {User-Agent=[JBang/0.111.0 (Linux/6.3.13-linuxkit/aarch64) Java/17-ea/Oracle Corporation]}
[jbang] [0:578] Downloaded file https://raw.githubusercontent.com/jbangdev/jbang-catalog/HEAD/jbang-catalog.json
[jbang] [0:578] Deleting folder /root/.jbang/cache/urls/0fcc09cf22b260b78984df78eab1b3a19a687c4e60c45d52307d8c73b75fa9a8.old
[jbang] [0:580] Deleting folder /root/.jbang/cache/urls/0fcc09cf22b260b78984df78eab1b3a19a687c4e60c45d52307d8c73b75fa9a8-meta.old
[jbang] [0:580] Reading catalog from https://github.com/jbangdev/jbang-catalog/blob/HEAD/jbang-catalog.json
[jbang] [0:582] Obtained catalog from https://github.com/jbangdev/jbang-catalog/blob/HEAD/jbang-catalog.json
[jbang] [0:582] Reading catalog from /root/.jbang/implicit-catalog.json
[jbang] [0:586] Requesting HTTP GET https://raw.githubusercontent.com/jbangdev/jbang-catalog/HEAD/jbang-catalog.json
[jbang] [0:586] Headers {User-Agent=[JBang/0.111.0 (Linux/6.3.13-linuxkit/aarch64) Java/17-ea/Oracle Corporation]}
[jbang] [0:594] Downloaded file https://raw.githubusercontent.com/jbangdev/jbang-catalog/HEAD/jbang-catalog.json
[jbang] [0:594] Deleting folder /root/.jbang/cache/urls/0fcc09cf22b260b78984df78eab1b3a19a687c4e60c45d52307d8c73b75fa9a8.old
[jbang] [0:595] Deleting folder /root/.jbang/cache/urls/0fcc09cf22b260b78984df78eab1b3a19a687c4e60c45d52307d8c73b75fa9a8-meta.old
[jbang] [0:595] Reading catalog from https://github.com/jbangdev/jbang-catalog/blob/HEAD/jbang-catalog.json
[jbang] [0:596] Obtained catalog from https://github.com/jbangdev/jbang-catalog/blob/HEAD/jbang-catalog.json
Aliases:
--------
bootstrap@jbangdev = Bootstrap a jbang script to make it self-contained.
                     (https://github.com/jbangdev/jbang-catalog/blob/HEAD/bootstrap.java)
bouncinglogo@jbangdev = https://github.com/jbangdev/jbang-catalog/blob/HEAD/bouncinglogo.java
catalog2readme@jbangdev = https://github.com/jbangdev/jbang-catalog/blob/HEAD/catalog2readme.java
dalle@jbangdev = https://github.com/jbangdev/jbang-catalog/blob/HEAD/dalle.java
ec@jbangdev = https://github.com/jbangdev/jbang-catalog/blob/HEAD/ec.jsh
env@jbangdev = Dump table of Environment Variables
               (https://github.com/jbangdev/jbang-catalog/blob/HEAD/env.java)
faker@jbangdev = https://github.com/jbangdev/jbang-catalog/blob/HEAD/faker.jsh
gavsearch@jbangdev = `gavsearch` lets you use search.maven.org from command line.
Example: `gavsearch hibernate` will search for artifacts with hibernate in its name.
You can use any of the search modifiers search.maven.org supports, i.e.:
`gavsearch c:QuarkusTest` will search for artifacts with class `QuarkusTest`
                     (https://github.com/jbangdev/jbang-catalog/blob/HEAD/gavsearch.java)
getjava@jbangdev = Experimental utility to download Java distributions using api.foojay.io.
                   (https://github.com/jbangdev/jbang-catalog/blob/HEAD/getjava.java)
git@jbangdev = Git command line tool implemented with jgit. Lets you do basic git features without installing git!
               (https://github.com/jbangdev/jbang-catalog/blob/HEAD/jgit.java)
h2@jbangdev = com.h2database:h2:1.4.200
              Arguments: -webAllowOthers -tcpAllowOthers
hello@jbangdev = Script that says hello back for each argument
                 (https://github.com/jbangdev/jbang-catalog/blob/HEAD/hello.java)
httpd@jbangdev = `httpd` runs a webserver serving out the content of a directory.
Example: `jbang httpd@jbangdev -d _site` will serve out the `_site` folder on localhost:8000.
                 (https://github.com/jbangdev/jbang-catalog/blob/HEAD/httpd.java)
properties@jbangdev = Dump table of System properties
                      (https://github.com/jbangdev/jbang-catalog/blob/HEAD/properties.java)
Templates:
----------
github@jbangdev = Simple cli to querying github
jitpack@jbangdev = Initializes a bare-bone jitpack.yml to enable publishing a jbang script as a maven artifact via jitpack.

Example: `jbang init -t jitpack@jbangdev myapp.java` and then commit this to github and visit jitpack.io to trigger its build.
junit@jbangdev = Basic template for JUnit tests
renovate@jbangdev = Initializes a renovate.json to enable automatic management of any .java file //DEPS section.

Example: `jbang init -t renovate@jbangdev .github/renovate.json` and then commit this to github and if you installed https://github.com/apps/renovate renovate will make issues and PR's for dependency updates.






root@55d834aa1cde:/deployments/quarkus# quarkus
[jbang] [ERROR] Could not download https://github.com/quarkusio/jbang-catalog/blob/HEAD/jbang-catalog.json
[jbang] Run with --verbose for more details

@maxandersen
Copy link
Member

damn - sorry i missed it was quarkusio it had issues with.

can you try jbang --verbose --fresh catalog list quarkusio ?

@maxandersen
Copy link
Member

you can also try run:

bash -x quarkus.

and you can also do "vi which quarkus" and edit that file to have --verbose in the call to jbang to get more details.

@vic1707
Copy link

vic1707 commented Oct 11, 2023

Sorry couldn't do it yesterday.

Bingo you were right !

root@c41822ecdde1:/deployments/quarkus# jbang --verbose --fresh catalog list quarkusio
[jbang] [0:187] jbang version 0.111.0
[jbang] [0:199] Reading catalog from /root/.jbang/implicit-catalog.json
[jbang] [0:229] Reading catalog from /root/.jbang/implicit-catalog.json
[jbang] [0:301] Requesting HTTP GET https://raw.githubusercontent.com/quarkusio/jbang-catalog/HEAD/jbang-catalog.json
[jbang] [0:301] Requesting HTTP GET https://www.jbang.dev/releases/latest/download/version.txt
[jbang] [0:301] Headers {User-Agent=[JBang/0.111.0 (Linux/6.3.13-linuxkit/aarch64) Java/17-ea/Oracle Corporation]}
[jbang] [0:301] Headers {User-Agent=[JBang/0.111.0 (Linux/6.3.13-linuxkit/aarch64) Java/17-ea/Oracle Corporation]}
[jbang] [0:472] Redirected to: https://github.com/jbangdev/jbang/releases/latest/download/version.txt
[jbang] [0:628] Downloaded file https://raw.githubusercontent.com/quarkusio/jbang-catalog/HEAD/jbang-catalog.json
[jbang] [0:629] Deleting folder /root/.jbang/cache/urls/18f27301d3b8b1c3bf70b027e02294385c940f21a743bc722c8f8f5bf569840c.tmp
[jbang] [0:632] Deleting folder /root/.jbang/cache/urls/18f27301d3b8b1c3bf70b027e02294385c940f21a743bc722c8f8f5bf569840c-meta.tmp
[jbang] [0:632] [ERROR] Could not download https://github.com/quarkusio/jbang-catalog/blob/HEAD/jbang-catalog.json
dev.jbang.cli.ExitException: Could not download https://github.com/quarkusio/jbang-catalog/blob/HEAD/jbang-catalog.json
        at dev.jbang.source.resolvers.RemoteResourceResolver.fetchFromURL(RemoteResourceResolver.java:118)
        at dev.jbang.source.resolvers.RemoteResourceResolver.resolve(RemoteResourceResolver.java:42)
        at dev.jbang.source.resolvers.CombinedResourceResolver.lambda$resolve$0(CombinedResourceResolver.java:27)
        at java.base/java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:197)
        at java.base/java.util.Spliterators$ArraySpliterator.tryAdvance(Spliterators.java:1002)
        at java.base/java.util.stream.ReferencePipeline.forEachWithCancel(ReferencePipeline.java:129)
        at java.base/java.util.stream.AbstractPipeline.copyIntoWithCancel(AbstractPipeline.java:527)
        at java.base/java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:513)
        at java.base/java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:499)
        at java.base/java.util.stream.FindOps$FindOp.evaluateSequential(FindOps.java:150)
        at java.base/java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
        at java.base/java.util.stream.ReferencePipeline.findFirst(ReferencePipeline.java:647)
        at dev.jbang.source.resolvers.CombinedResourceResolver.resolve(CombinedResourceResolver.java:29)
        at dev.jbang.source.ResourceResolver.resolve(ResourceResolver.java:52)
        at dev.jbang.source.ResourceRef.forResource(ResourceRef.java:131)
        at dev.jbang.catalog.Catalog.getByRef(Catalog.java:190)
        at dev.jbang.catalog.Catalog.getByName(Catalog.java:145)
        at dev.jbang.cli.CatalogList.doCall(Catalog.java:152)
        at dev.jbang.cli.BaseCommand.call(BaseCommand.java:145)
        at dev.jbang.cli.BaseCommand.call(BaseCommand.java:21)
        at picocli.CommandLine.executeUserObject(CommandLine.java:1953)
        at picocli.CommandLine.access$1300(CommandLine.java:145)
        at picocli.CommandLine$RunLast.executeUserObjectOfLastSubcommandWithSameParent(CommandLine.java:2358)
        at picocli.CommandLine$RunLast.handle(CommandLine.java:2352)
        at dev.jbang.cli.JBang$3.handle(JBang.java:148)
        at dev.jbang.cli.JBang$3.handle(JBang.java:143)
        at picocli.CommandLine$AbstractParseResultHandler.execute(CommandLine.java:2179)
        at picocli.CommandLine$RunLast.execute(CommandLine.java:2316)
        at picocli.CommandLine.execute(CommandLine.java:2078)
        at dev.jbang.Main.main(Main.java:14)
Caused by: java.nio.file.DirectoryNotEmptyException: /root/.jbang/cache/urls/18f27301d3b8b1c3bf70b027e02294385c940f21a743bc722c8f8f5bf569840c
        at java.base/sun.nio.fs.UnixCopyFile.ensureEmptyDir(UnixCopyFile.java:382)
        at java.base/sun.nio.fs.UnixCopyFile.move(UnixCopyFile.java:483)
        at java.base/sun.nio.fs.UnixFileSystemProvider.move(UnixFileSystemProvider.java:267)
        at java.base/java.nio.file.Files.move(Files.java:1432)
        at dev.jbang.util.Util$ResultHandler.lambda$downloadToTempDir$3(Util.java:982)
        at dev.jbang.util.Util$ResultHandler.lambda$throwOnError$1(Util.java:939)
        at dev.jbang.util.Util$ResultHandler.lambda$handleUnmodified$4(Util.java:1043)
        at dev.jbang.util.Util$ResultHandler.lambda$redirects$0(Util.java:892)
        at dev.jbang.util.Util.connect(Util.java:812)
        at dev.jbang.util.Util.downloadFileAndCache(Util.java:741)
        at dev.jbang.util.Util.downloadAndCacheFile(Util.java:692)
        at dev.jbang.source.resolvers.RemoteResourceResolver.fetchFromURL(RemoteResourceResolver.java:115)
        ... 29 more
[jbang] [0:634] If you believe this a bug in jbang, open an issue at https://github.com/jbangdev/jbang/issues

As for the edition of quarkus

root@c41822ecdde1:/deployments/quarkus# cat $(which quarkus)
#!/bin/sh
exec jbang --verbose run quarkus@quarkusio "$@"


root@c41822ecdde1:/deployments/quarkus# quarkus
[jbang] [0:143] jbang version 0.111.0
[jbang] [0:150] Resolving resource ref: quarkus@quarkusio
[jbang] [0:154] Reading catalog from /root/.jbang/implicit-catalog.json
[jbang] [0:207] Requesting HTTP GET https://www.jbang.dev/releases/latest/download/version.txt
[jbang] [0:207] Requesting HTTP GET https://raw.githubusercontent.com/quarkusio/jbang-catalog/HEAD/jbang-catalog.json
[jbang] [0:207] Headers {User-Agent=[JBang/0.111.0 (Linux/6.3.13-linuxkit/aarch64) Java/17-ea/Oracle Corporation]}
[jbang] [0:207] Headers {If-None-Match=["73fa634698f8c81e01bf4412be3fa68280faf796ffca59ddacac742f5f7ca127"], User-Agent=[JBang/0.111.0 (Linux/6.3.13-linuxkit/aarch64) Java/17-ea/Oracle Corporation], If-Modified-Since=[Tue, 3 Oct 2023 08:28:50 GMT]}
[jbang] [0:319] Downloaded file https://raw.githubusercontent.com/quarkusio/jbang-catalog/HEAD/jbang-catalog.json
[jbang] [0:319] Deleting folder /root/.jbang/cache/urls/18f27301d3b8b1c3bf70b027e02294385c940f21a743bc722c8f8f5bf569840c.tmp
[jbang] [0:321] Deleting folder /root/.jbang/cache/urls/18f27301d3b8b1c3bf70b027e02294385c940f21a743bc722c8f8f5bf569840c-meta.tmp
[jbang] [0:321] Retry using cache-evict: quarkus@quarkusio
[jbang] [0:322] Requesting HTTP GET https://raw.githubusercontent.com/quarkusio/jbang-catalog/HEAD/jbang-catalog.json
[jbang] [0:323] Headers {If-None-Match=["73fa634698f8c81e01bf4412be3fa68280faf796ffca59ddacac742f5f7ca127"], User-Agent=[JBang/0.111.0 (Linux/6.3.13-linuxkit/aarch64) Java/17-ea/Oracle Corporation], If-Modified-Since=[Tue, 3 Oct 2023 08:28:50 GMT]}
[jbang] [0:328] Redirected to: https://github.com/jbangdev/jbang/releases/latest/download/version.txt
[jbang] [0:332] Downloaded file https://raw.githubusercontent.com/quarkusio/jbang-catalog/HEAD/jbang-catalog.json
[jbang] [0:332] Deleting folder /root/.jbang/cache/urls/18f27301d3b8b1c3bf70b027e02294385c940f21a743bc722c8f8f5bf569840c.tmp
[jbang] [0:332] Deleting folder /root/.jbang/cache/urls/18f27301d3b8b1c3bf70b027e02294385c940f21a743bc722c8f8f5bf569840c-meta.tmp
[jbang] [0:333] [ERROR] Could not download https://github.com/quarkusio/jbang-catalog/blob/HEAD/jbang-catalog.json
dev.jbang.cli.ExitException: Could not download https://github.com/quarkusio/jbang-catalog/blob/HEAD/jbang-catalog.json
        at dev.jbang.source.resolvers.RemoteResourceResolver.fetchFromURL(RemoteResourceResolver.java:118)
        at dev.jbang.source.resolvers.RemoteResourceResolver.resolve(RemoteResourceResolver.java:42)
        at dev.jbang.source.resolvers.CombinedResourceResolver.lambda$resolve$0(CombinedResourceResolver.java:27)
        at java.base/java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:197)
        at java.base/java.util.Spliterators$ArraySpliterator.tryAdvance(Spliterators.java:1002)
        at java.base/java.util.stream.ReferencePipeline.forEachWithCancel(ReferencePipeline.java:129)
        at java.base/java.util.stream.AbstractPipeline.copyIntoWithCancel(AbstractPipeline.java:527)
        at java.base/java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:513)
        at java.base/java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:499)
        at java.base/java.util.stream.FindOps$FindOp.evaluateSequential(FindOps.java:150)
        at java.base/java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
        at java.base/java.util.stream.ReferencePipeline.findFirst(ReferencePipeline.java:647)
        at dev.jbang.source.resolvers.CombinedResourceResolver.resolve(CombinedResourceResolver.java:29)
        at dev.jbang.source.ResourceResolver.resolve(ResourceResolver.java:52)
        at dev.jbang.source.ResourceRef.forResource(ResourceRef.java:131)
        at dev.jbang.catalog.Catalog.getByRef(Catalog.java:190)
        at dev.jbang.catalog.Catalog.getByName(Catalog.java:145)
        at dev.jbang.catalog.Alias.fromCatalog(Alias.java:274)
        at dev.jbang.catalog.Alias.merge(Alias.java:200)
        at dev.jbang.catalog.Alias.get(Alias.java:165)
        at dev.jbang.source.resolvers.AliasResourceResolver.resolve(AliasResourceResolver.java:39)
        at dev.jbang.source.ResourceResolver.resolve(ResourceResolver.java:52)
        at dev.jbang.source.ProjectBuilder.lambda$resolveChecked$1(ProjectBuilder.java:241)
        at dev.jbang.util.Util.withConfig(Util.java:221)
        at dev.jbang.util.Util.withConfig(Util.java:214)
        at dev.jbang.util.Util.withCacheEvict(Util.java:208)
        at dev.jbang.util.Util.withCacheEvict(Util.java:204)
        at dev.jbang.source.ProjectBuilder.resolveChecked(ProjectBuilder.java:241)
        at dev.jbang.source.ProjectBuilder.build(ProjectBuilder.java:218)
        at dev.jbang.cli.Run.doCall(Run.java:73)
        at dev.jbang.cli.BaseCommand.call(BaseCommand.java:145)
        at dev.jbang.cli.BaseCommand.call(BaseCommand.java:21)
        at picocli.CommandLine.executeUserObject(CommandLine.java:1953)
        at picocli.CommandLine.access$1300(CommandLine.java:145)
        at picocli.CommandLine$RunLast.executeUserObjectOfLastSubcommandWithSameParent(CommandLine.java:2358)
        at picocli.CommandLine$RunLast.handle(CommandLine.java:2352)
        at dev.jbang.cli.JBang$3.handle(JBang.java:148)
        at dev.jbang.cli.JBang$3.handle(JBang.java:143)
        at picocli.CommandLine$AbstractParseResultHandler.execute(CommandLine.java:2179)
        at picocli.CommandLine$RunLast.execute(CommandLine.java:2316)
        at picocli.CommandLine.execute(CommandLine.java:2078)
        at dev.jbang.Main.main(Main.java:14)
Caused by: java.nio.file.DirectoryNotEmptyException: /root/.jbang/cache/urls/18f27301d3b8b1c3bf70b027e02294385c940f21a743bc722c8f8f5bf569840c
        at java.base/sun.nio.fs.UnixCopyFile.ensureEmptyDir(UnixCopyFile.java:382)
        at java.base/sun.nio.fs.UnixCopyFile.move(UnixCopyFile.java:483)
        at java.base/sun.nio.fs.UnixFileSystemProvider.move(UnixFileSystemProvider.java:267)
        at java.base/java.nio.file.Files.move(Files.java:1432)
        at dev.jbang.util.Util$ResultHandler.lambda$downloadToTempDir$3(Util.java:982)
        at dev.jbang.util.Util$ResultHandler.lambda$throwOnError$1(Util.java:939)
        at dev.jbang.util.Util$ResultHandler.lambda$handleUnmodified$4(Util.java:1043)
        at dev.jbang.util.Util$ResultHandler.lambda$redirects$0(Util.java:892)
        at dev.jbang.util.Util.connect(Util.java:812)
        at dev.jbang.util.Util.downloadFileAndCache(Util.java:741)
        at dev.jbang.util.Util.downloadAndCacheFile(Util.java:692)
        at dev.jbang.source.resolvers.RemoteResourceResolver.fetchFromURL(RemoteResourceResolver.java:115)
        ... 41 more
[jbang] [0:334] If you believe this a bug in jbang, open an issue at https://github.com/jbangdev/jbang/issues

@maxandersen
Copy link
Member

Sorry i missed the Update.

The error is Weird. Do You have it accessing a read only dir somehow ? Like. This shouldnt happen.

Where can You reproduce it ? Did You find a way around it ?

@maxandersen
Copy link
Member

Any chance you are having concurrent executions of jbang on the same filesystem?

I could possibly see a race condition here but I'm not fully convinced so really wanna be able to reproduce this.

@vic1707
Copy link

vic1707 commented Nov 8, 2023

Hello, no problem.

We found a way around it by calling quarkus through mvn quarkus:dev instead and changing base image for maven:3.9.4-ibm-semeru-17-focal. So we technically got rid of jbang (or at least we don't install it ourselves anymore).

I doubt there was concurrent executions of jbang since it was the only running container and I don't have jbang installed locally + seeing that quarkus --help also failed I don't think it was the issue.

I tried to reproduce it in a standalone container without success.
I restored our old Dockerfile which was failing and the issue seems to be gone, no idea why tho.
It seems that it was fixed by "accident" (by your team or mine in our project) so somewhat a good news even if we don't know why

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

5 participants