dub fetch broken on ubuntu:focal ?

[andre2007]

For this dockerfile

FROM ubuntu:focal

RUN apt-get update && apt-get upgrade -y \
    && apt-get install --no-install-recommends -y dub ldc

RUN dub fetch vibe-d --verbose

I get following docker build result:

Using dub registry url 'https://code.dlang.org/'
Refreshing local packages (refresh existing: true)...
Looking for local package map at /var/lib/dub/packages/local-packages.json
Looking for local package map at /root/.dub/packages/local-packages.json
Looking for local package map at /.dub/packages/local-packages.json
No valid package found in current working directory: No package file found in /, expected one of dub.json/dub.sdl/package.json
Package supplier registry at https://code.dlang.org/ failed with 'HTTP request returned status code 0 ()', trying fallbacks.
Fallback package supplier registry at https://code-mirror.dlang.io/ failed with 'Failed to download https://code-mirror.dlang.io/api/packages/infos?packages=%5B%22vibe-d%22%5D&include_dependencies=true&minimize=true'.
Fallback package supplier registry at https://dub-registry.herokuapp.com/ failed with 'HTTP request returned status code 503 (Service Unavailable)'.
Package vibe-d not found for registry at https://code.dlang.org/ (fallbacks registry at https://code-mirror.dlang.io/, registry at https://dub-registry.herokuapp.com/): HTTP request returned status code 0 ()
Getting a release version failed: No package vibe-d was found matching the dependency >=0.0.0
Retry with ~master...
Package supplier registry at https://code.dlang.org/ failed with 'HTTP request returned status code 0 ()', trying fallbacks.
Package vibe-d not found for registry at https://code.dlang.org/ (fallbacks registry at https://code-mirror.dlang.io/, registry at https://dub-registry.herokuapp.com/): HTTP request returned status code 0 ()
No package vibe-d was found matching the dependency ~master
The command '/bin/sh -c dub fetch vibe-d --verbose' returned a non-zero code: 2

DUB version 1.19.0-1build2, built on Mar 24 2020

[andre2007]

In very rare cases it succeeds only for heroku fallback server.

[andre2007]

Hi @ximion
I need your help here. The dub built for the debian repository has network issues.
It cannot connect to default dub registry server https://code.dlang.org/ but fails
with error code 0.

If I use the dub from e.g. https://github.com/ldc-developers/ldc/releases/download/v1.20.0/ldc2-1.20.0-linux-x86_64.tar.xz it works without any issue.

RUN curl -LO https://github.com/ldc-developers/ldc/releases/download/v1.20.0/ldc2-1.20.0-linux-x86_64.tar.xz \
    && tar -xf /ldc2-1.20.0-linux-x86_64.tar.xz \
    && ln -s /ldc2-1.20.0-linux-x86_64/bin/dub /bin/dub

[wilzbach]

Maybe you need to open a bug in the respective Ubuntu/Debian bug tracker then?
I'm not sure whether the maintainers check this repo.

[andre2007]

I also notified the dlang debian team via email. Not really sure how to officially file an issue for a debian package. But I think @ximion got the notification.

[andre2007]

https://bugs.launchpad.net/ubuntu/+source/dub/+bug/1876296

[ximion]

I also notified the dlang debian team via email. Not really sure how to officially file an issue for a debian package. But I think @ximion got the notification.

Hi! Yeah, I don't really know what exactly broke this, but it's really annoying that this could just suddenly break. We even had that in the past and it fixed itself just as randomly as it appeared. At the moment I don't have time to debug this, so in case someone wants to give it a try.... ;-)

We have a CI test for Debian for exactly this issue, since the last time DUB broke:
https://ci.debian.net/packages/d/dub/unstable/amd64/

When it broke, unfortunately both Curl-GnuTLS and LLVM were updated - I would guess the former update may have something to do with the issue. Since the dub package itself wasn't touched, I wonder whether something else is to blame. Or it's one of the classic issues of D ABI instability and dub just needs a rebuild.

[andre2007]

@ximion The ubuntu bug report mentions a curl bug in Phobos which needs to be integrated into the debian GDC package.

[ximion]

That would indeed explain the issue. In Debian, there will likely be a transition to GDC 10 soon, which (presumably) doesn't have this issue, Ubuntu will need that patch though.

[Geod24]

I suppose this is the PR needed : dlang/phobos#6752 ?
It's part of v2.084.0, and AFAIK it's not part of GCC10.
I just looked at the GCC sources as of yesterday (* e7ae6d32c7d - (upstream/master, origin/master, origin/HEAD, master) Daily bump. (34 hours ago) ) and the bug is still here.

[Geod24]

I'm seeing the same thing on Alpine Linux BTW, build with GCC 9.3:

Error querying versions for vibe-core, registry at https://code.dlang.org/ (fallbacks registry at https://codemirror.dlang.org/, registry at https://dub.bytecraft.nl/, registry at https://code-mirror.dlang.io/): HTTP request returned status code 0 ()
Selected package vibe-core 1.8.1 doesn't exist. Using latest matching version instead.
Error querying versions for botan-math, registry at https://code.dlang.org/ (fallbacks registry at https://codemirror.dlang.org/, registry at https://dub.bytecraft.nl/, registry at https://code-mirror.dlang.io/): HTTP request returned status code 0 ()
Selected package botan-math 1.0.3 doesn't exist. Using latest matching version instead.
Error querying versions for memutils, registry at https://code.dlang.org/ (fallbacks registry at https://codemirror.dlang.org/, registry at https://dub.bytecraft.nl/, registry at https://code-mirror.dlang.io/): HTTP request returned status code 0 ()
Selected package memutils 0.4.13 doesn't exist. Using latest matching version instead.
Error querying versions for taggedalgebraic, registry at https://code.dlang.org/ (fallbacks registry at https://codemirror.dlang.org/, registry at https://dub.bytecraft.nl/, registry at https://code-mirror.dlang.io/): HTTP request returned status code 0 ()
Selected package taggedalgebraic 0.11.8 doesn't exist. Using latest matching version instead.
Error querying versions for vibe-d, registry at https://code.dlang.org/ (fallbacks registry at https://codemirror.dlang.org/, registry at https://dub.bytecraft.nl/, registry at https://code-mirror.dlang.io/): HTTP request returned status code 0 ()
Selected package vibe-d 0.8.6 doesn't exist. Using latest matching version instead.
Error querying versions for stdx-allocator, registry at https://code.dlang.org/ (fallbacks registry at https://codemirror.dlang.org/, registry at https://dub.bytecraft.nl/, registry at https://code-mirror.dlang.io/): HTTP request returned status code 0 ()
Selected package stdx-allocator 2.77.5 doesn't exist. Using latest matching version instead.
Error querying versions for botan, registry at https://code.dlang.org/ (fallbacks registry at https://codemirror.dlang.org/, registry at https://dub.bytecraft.nl/, registry at https://code-mirror.dlang.io/): HTTP request returned status code 0 ()
Selected package botan 1.12.10 doesn't exist. Using latest matching version instead.
Error querying versions for diet-ng, registry at https://code.dlang.org/ (fallbacks registry at https://codemirror.dlang.org/, registry at https://dub.bytecraft.nl/, registry at https://code-mirror.dlang.io/): HTTP request returned status code 0 ()
Selected package diet-ng 1.6.0 doesn't exist. Using latest matching version instead.
Error querying versions for openssl, registry at https://code.dlang.org/ (fallbacks registry at https://codemirror.dlang.org/, registry at https://dub.bytecraft.nl/, registry at https://code-mirror.dlang.io/): HTTP request returned status code 0 ()
Selected package openssl 1.1.6+1.0.1g doesn't exist. Using latest matching version instead.
Error querying versions for libev, registry at https://code.dlang.org/ (fallbacks registry at https://codemirror.dlang.org/, registry at https://dub.bytecraft.nl/, registry at https://code-mirror.dlang.io/): HTTP request returned status code 0 ()
Selected package libev 5.0.0+4.04 doesn't exist. Using latest matching version instead.
Error querying versions for eventcore, registry at https://code.dlang.org/ (fallbacks registry at https://codemirror.dlang.org/, registry at https://dub.bytecraft.nl/, registry at https://code-mirror.dlang.io/): HTTP request returned status code 0 ()
Selected package eventcore 0.8.48 doesn't exist. Using latest matching version instead.
Error querying versions for mir-linux-kernel, registry at https://code.dlang.org/ (fallbacks registry at https://codemirror.dlang.org/, registry at https://dub.bytecraft.nl/, registry at https://code-mirror.dlang.io/): HTTP request returned status code 0 ()
Selected package mir-linux-kernel 1.0.1 doesn't exist. Using latest matching version instead.
Error querying versions for libevent, registry at https://code.dlang.org/ (fallbacks registry at https://codemirror.dlang.org/, registry at https://dub.bytecraft.nl/, registry at https://code-mirror.dlang.io/): HTTP request returned status code 0 ()
Selected package libevent 2.0.2+2.0.16 doesn't exist. Using latest matching version instead.
Error querying versions for libasync, registry at https://code.dlang.org/ (fallbacks registry at https://codemirror.dlang.org/, registry at https://dub.bytecraft.nl/, registry at https://code-mirror.dlang.io/): HTTP request returned status code 0 ()
Selected package libasync 0.8.4 doesn't exist. Using latest matching version instead.
Package vibe-d not found in registry at https://code.dlang.org/ (fallbacks registry at https://codemirror.dlang.org/, registry at https://dub.bytecraft.nl/, registry at https://code-mirror.dlang.io/): HTTP request returned status code 0 ()
Generating test runner configuration 'dub-test-library' for 'library' (library).
Excluding package.d file from test due to https://issues.dlang.org/show_bug.cgi?id=11847
Excluding package.d file from test due to https://issues.dlang.org/show_bug.cgi?id=11847
Performing "unittest" build using dmd for x86_64.

[ximion]

See https://forum.dlang.org/thread/[email protected]
So, this issue is fixed in our GCC 10.1 packaging in Debian, so I will be able to fix this by forcing GDC 10 for the dub package (usually this would get switched automatically when we transition to a new default compiler stack, but in this case it's probably just easier to accelerate this process a bit)

[andre2007]

What do you think about a quick fix for this issue: change dub registry server (code.dlang.org) to not use http/2 until were are in a good shape with the linux packages?

[Geod24]

See https://forum.dlang.org/thread/[email protected]

Thanks. I guess I'll have to backport this to GCC9 for Alpine then...

What do you think about a quick fix for this issue: change registry to not use http/2 until were are in a good shape with the linux packages?

That'd be great. CC @s-ludwig

[s-ludwig]

What do you think about a quick fix for this issue: change registry to not use http/2 until were are in a good shape with the linux packages?

That'd be great. CC @s-ludwig

Done.

[andre2007]

Fantastic, thanks a lot

[andre2007]

Validated, it works fine now.