Provide a `zig cc`-based image. #860

Alexhuszagh · 2022-06-25T11:38:03Z

Checklist

I've looked through the issues and pull requests for similar request
This feature could be solved with a custom docker image (optional)

Describe your request

Inspired by the suggestion on reddit, which would then allow users to favor using zig cc or similar, which would allow use of custom glibc versions, etc.

Effectively, we'd have a config option like:

[build]
use-zig = true

And then a composite image for all targets zig cc supports, and it would ensure we set the appropriate CC_$target and CXX_$target environment variables to zig cc and zig c++ appropriately.

Describe why this would be a good inclusion for `cross`

It would allow passing in custom glibc versions for *-linux-gnu targets, if done correctly, and could reduce image sizes by a lot. The only caveat would be is issues with zig respecting all C, C++ and link flags, which I know have been issues in the past. Making it opt-in, however, solves all of these.

The text was updated successfully, but these errors were encountered:

Emilgardis · 2022-06-25T11:50:59Z

This was previously mentioned in #764.

zig cc is rather small, I think it's fine to include it in the images.

We'd need to include one more flag for setting the glibc version though.

Alexhuszagh · 2022-06-25T12:03:54Z

zig cc is rather small, I think it's fine to include it in the images.

It might be worth having a single zig cc image, since to my recollection it comes bundled with all the relevant standard libraries by default and comes as its own compiler for C and C++. So it's probably worth creating a single image for zig, and use this if use-zig is provided, since it will be much more compact than using other images separately. After this weekend, I can work on this.

Emilgardis · 2022-06-25T12:23:38Z

I'm not fully sure of what you mean, but I think below would work nicely. We provide zero zig images, and instead use-zig = true will do the following custom dockerfile

ARG CROSS_BASE_IMAGE
FROM ubuntu:16.04 as zig

ARG $ZIG_DOWNLOAD_URL
RUN curl --retry 3 "${ZIG_DOWNLOAD_URL}" -O && /* untar & whatever else zig needs to setup */

FROM $CROSS_BASE_IMAGE
ARG CC_VER
# if prebuild, do it here

COPY --from=zig zig /usr/bin/zig

ENV CC_<triplet>="zig cc --target <zig-triplet>.${CC_VER}"

mntns · 2022-06-25T14:03:50Z

I like the feature idea a lot and would try my hand at implementing this. @Emilgardis, could you give me a pointer on where such a custom Dockerfile would be created in cross?

Emilgardis · 2022-06-25T15:09:00Z

@mntns definitely!

Here's all the pre-build/dockerfile logic.

cross/src/docker/shared.rs

Lines 404 to 474 in 48d2b7b

    
           pub fn needs_custom_image(target: &Target, config: &Config) -> bool { 
        
               config.dockerfile(target).unwrap_or_default().is_some() 
        
                   || !config 
        
                       .pre_build(target) 
        
                       .unwrap_or_default() 
        
                       .unwrap_or_default() 
        
                       .is_empty() 
        
           } 
        
           pub(crate) fn custom_image_build( 
        
               target: &Target, 
        
               config: &Config, 
        
               metadata: &CargoMetadata, 
        
               Directories { host_root, .. }: Directories, 
        
               engine: &Engine, 
        
               verbose: bool, 
        
           ) -> Result<String> { 
        
               let mut image = image_name(config, target)?; 
        
               if let Some(path) = config.dockerfile(target)? { 
        
                   let context = config.dockerfile_context(target)?; 
        
                   let name = config.image(target)?; 
        
                   let build = Dockerfile::File { 
        
                       path: &path, 
        
                       context: context.as_deref(), 
        
                       name: name.as_deref(), 
        
                   }; 
        
                   image = build 
        
                       .build( 
        
                           config, 
        
                           metadata, 
        
                           engine, 
        
                           &host_root, 
        
                           config.dockerfile_build_args(target)?.unwrap_or_default(), 
        
                           target, 
        
                           verbose, 
        
                       ) 
        
                       .wrap_err("when building dockerfile")?; 
        
               } 
        
               let pre_build = config.pre_build(target)?; 
        
               if let Some(pre_build) = pre_build { 
        
                   if !pre_build.is_empty() { 
        
                       let custom = Dockerfile::Custom { 
        
                           content: format!( 
        
                               r#" 
        
               FROM {image} 
        
               ARG CROSS_DEB_ARCH= 
        
               ARG CROSS_CMD 
        
               RUN eval "${{CROSS_CMD}}""# 
        
                           ), 
        
                       }; 
        
                       custom 
        
                           .build( 
        
                               config, 
        
                               metadata, 
        
                               engine, 
        
                               &host_root, 
        
                               Some(("CROSS_CMD", pre_build.join("\n"))), 
        
                               target, 
        
                               verbose, 
        
                           ) 
        
                           .wrap_err("when pre-building") 
        
                           .with_note(|| format!("CROSS_CMD={}", pre_build.join("\n")))?; 
        
                       image = custom.image_name(target, metadata)?; 
        
                   } 
        
               } 
        
               Ok(image) 
        
           }

Alexhuszagh · 2022-06-25T23:09:07Z

I think a better idea is to only have zig-cc for the image, and we can export every single environment variable. Something like:

FROM ubuntu:16.04

ARG $ZIG_DOWNLOAD_URL
RUN curl --retry 3 "${ZIG_DOWNLOAD_URL}" -O && /* untar & whatever else zig needs to setup */

# add a full qemu installation

CARGO_TARGET_AARCH64_UNKNOWN_LINUX_GNU_LINKER=aarch64-linux-gnu-gcc \
    CARGO_TARGET_AARCH64_UNKNOWN_LINUX_GNU_RUNNER="/linux-runner aarch64" \
    CARGO_TARGET_ARM_UNKNOWN_LINUX_GNUEABIHF_LINKER=arm-unknown-linux-gnueabihf-gcc \
    CARGO_TARGET_ARM_UNKNOWN_LINUX_GNUEABIHF_RUNNER=qemu-arm \
    BINDGEN_EXTRA_CLANG_ARGS_arm_unknown_linux_gnueabihf="--sysroot=/x-tools/arm-unknown-linux-gnueabihf/arm-unknown-linux-gnueabihf/sysroot/" \
    ZIG_CC="zig cc" \
    ZIG_CX="zig c++"

And then we provide the GLIBC_VER (or other libc version) when splitting the triple at the start, and our entrypoint reads that and defines the remaining environment variables:

entrypoint.sh

libc=""
if [[ -n "${GLIBC_VER}" ]]; then
    libc=".${GLIBC_VER}}
fi
export CC_aarch64_unknown_linux_gnu="${ZIG_CC} --target aarch64-unknown-linux-gnu${libc}"
export CXX_aarch64_unknown_linux_gnu="${ZIG_CXX} --target aarch64-unknown-linux-gnu${libc}"
export CC_arm_unknown_linux_gnueabihf="${ZIG_CC} --target arm-unknown-linux-gnueabihf${libc}"
export CXX_arm_unknown_linux_gnueabihf="${ZIG_CXX} --target arm-unknown-linux-gnueabihf{$libc}"

exec "${@}"

And do this for all targets. This way, we have a single, shared image for everything, which will be very compact, and it will support all targets out-of-the-box.

This would mean if use-zig is provided, we then only use this specific image (although it should conflict with a custom image definition, not sure which way to resolve that), and everything else should just work.

FYI: I'm planning on doing just this. We should obviously also export CROSS_BUILD_TARGET or CARGO_BUILD_TARGET, so any logic if people want to extend the image can do done on a target-by-target basis.

Emilgardis · 2022-06-25T23:33:26Z

Oh yeah, didn't think about that zig cc gives everything needed 😄

Alexhuszagh · 2022-06-25T23:36:28Z

Oh yeah, didn't think about that zig cc gives everything needed smile

Yeah we might want a way to deduplicate some of our code because then we have to ensure the definitions are identical between our other images and our zig cc one, but other than that we're good.

Alexhuszagh · 2022-06-28T23:58:16Z

I'm currently working on this, although there's a few hiccups so far. Currently when compiling, Rust expects libgcc to be passed, which zig cc doesn't seem to accept. I've tried overriding LDFLAGS for this, and likely will need to update some other values. I'm confused about the Access Denied which seems to be the actual error...

$ cross build --target arm-unknown-linux-gnueabi                                                                                                                           
   Compiling hello v0.1.0 (/project)                                                                                                                                                                
error: linking with `zig-cc` failed: exit status: 1                                                                                                                                                 
  |                                                                                                                                                                                                 
  = note: "zig-cc" "/tmp/rustcXSVBDR/symbols.o" "/target/arm-unknown-linux-gnueabi/debug/deps/hello-7ddbc5fb91e19054.1mm29ubbwuz8s7tw.rcgu.o" "/target/arm-unknown-linux-gnueabi/debug/deps/hello-7d
dbc5fb91e19054.39yjj3n05ap8v7wi.rcgu.o" "/target/arm-unknown-linux-gnueabi/debug/deps/hello-7ddbc5fb91e19054.3c3sh9qmv9xaoi8k.rcgu.o" "/target/arm-unknown-linux-gnueabi/debug/deps/hello-7ddbc5fb91
e19054.3f27pkp96oxdmtyc.rcgu.o" "/target/arm-unknown-linux-gnueabi/debug/deps/hello-7ddbc5fb91e19054.40m3wrv8wu3s80hs.rcgu.o" "/target/arm-unknown-linux-gnueabi/debug/deps/hello-7ddbc5fb91e19054.5
armrqmsk1t4s2mo.rcgu.o" "/target/arm-unknown-linux-gnueabi/debug/deps/hello-7ddbc5fb91e19054.5c023hewojurnpqw.rcgu.o" "/target/arm-unknown-linux-gnueabi/debug/deps/hello-7ddbc5fb91e19054.w24kmmgpv
wuuy6m.rcgu.o" "/target/arm-unknown-linux-gnueabi/debug/deps/hello-7ddbc5fb91e19054.1ctupkzk3ymebr7n.rcgu.o" "-Wl,--as-needed" "-L" "/target/arm-unknown-linux-gnueabi/debug/deps" "-L" "/target/deb
ug/deps" "-L" "/rust/lib/rustlib/arm-unknown-linux-gnueabi/lib" "-Wl,--start-group" "-Wl,-Bstatic" "/rust/lib/rustlib/arm-unknown-linux-gnueabi/lib/libstd-c44f223b81cbc265.rlib" "/rust/lib/rustlib
/arm-unknown-linux-gnueabi/lib/libpanic_unwind-e337a541de70384a.rlib" "/rust/lib/rustlib/arm-unknown-linux-gnueabi/lib/libobject-63d0354c15ba72ed.rlib" "/rust/lib/rustlib/arm-unknown-linux-gnueabi
/lib/libmemchr-75bff3fc76c53b72.rlib" "/rust/lib/rustlib/arm-unknown-linux-gnueabi/lib/libaddr2line-6aa5dfeb670effb4.rlib" "/rust/lib/rustlib/arm-unknown-linux-gnueabi/lib/libgimli-662b8c32ab27763
2.rlib" "/rust/lib/rustlib/arm-unknown-linux-gnueabi/lib/librustc_demangle-c71ddf2caceab29f.rlib" "/rust/lib/rustlib/arm-unknown-linux-gnueabi/lib/libstd_detect-c72b7765d9081eda.rlib" "/rust/lib/r
ustlib/arm-unknown-linux-gnueabi/lib/libhashbrown-5fd9196c3c9ef8ef.rlib" "/rust/lib/rustlib/arm-unknown-linux-gnueabi/lib/libminiz_oxide-94b5c12daf6ca5ad.rlib" "/rust/lib/rustlib/arm-unknown-linux
-gnueabi/lib/libadler-f57abdca5f5d1fb9.rlib" "/rust/lib/rustlib/arm-unknown-linux-gnueabi/lib/librustc_std_workspace_alloc-daa9b2520afd4f30.rlib" "/rust/lib/rustlib/arm-unknown-linux-gnueabi/lib/l
ibunwind-cea6091b3d8becdc.rlib" "/rust/lib/rustlib/arm-unknown-linux-gnueabi/lib/libcfg_if-9bb90634d22884cd.rlib" "/rust/lib/rustlib/arm-unknown-linux-gnueabi/lib/liblibc-cc64347f456a2e42.rlib" "/
rust/lib/rustlib/arm-unknown-linux-gnueabi/lib/liballoc-07b4d0493ef243fc.rlib" "/rust/lib/rustlib/arm-unknown-linux-gnueabi/lib/librustc_std_workspace_core-16c9d9a485713471.rlib" "/rust/lib/rustli
b/arm-unknown-linux-gnueabi/lib/libcore-21d632ec5a739f36.rlib" "-Wl,--end-group" "/rust/lib/rustlib/arm-unknown-linux-gnueabi/lib/libcompiler_builtins-867068f5f547f8aa.rlib" "-Wl,-Bdynamic" "-lgcc
_s" "-lutil" "-lrt" "-lpthread" "-lm" "-ldl" "-lc" "-Wl,--eh-frame-hdr" "-Wl,-znoexecstack" "-L" "/rust/lib/rustlib/arm-unknown-linux-gnueabi/lib" "-o" "/target/arm-unknown-linux-gnueabi/debug/dep
s/hello-7ddbc5fb91e19054" "-Wl,--gc-sections" "-pie" "-Wl,-zrelro,-znow" "-nodefaultlibs"                                                                                                           
  = note: warning: unsupported linker arg: -znoexecstack                                                                                                                                            
          warning: unsupported linker arg: -zrelro                                                                                                                                                  
          warning: unsupported linker arg: -znow                                                                                                                                                    
          warning: ignoring superfluous library 'gcc_s': this dependency is fulfilled instead by compiler-rt which zig unconditionally provides                                                     
          error: AccessDenied                                                                                                                                                                       
                                                                                                                                                                                                    
                                                                                                                                                                                                    
error: could not compile `hello` due to previous error

There current work can be seen at zigcc. It's probably required to remove some packages like gcc and libgcc_s (which I think we can do easily) before this will work.

Alexhuszagh · 2022-06-29T00:07:41Z

Oh this was a silly mistake on my part: it's the zig cache trying to write as a normal user to root directories.
ziglang/zig#7342

Uses cargo-zigbuild as a backend, and adds configuration options for zig under `[build.zig]` and `[target.(...).zig]`. If enabled, and an image override is not provided, `cross` will always use the `zig` image. It supports custom `glibc` versions by parsing the libc portion of the target, and extracting a the libc version if present. The target, if built-in, is then the triple/libc pair, otherwise, it's just the triple. The image does not provide runners, `bindgen` Clang args, or `pkg-config` paths, since `zig cc` does not provide the dynamic library loader (`ld-linux*.so`) required, meaning none of the binaries can be run. For `bindgen`, `zig cc` has an unusual directory structure, so there is no traditional sysroot with `usr`, `lib`, and `include` subdirectories. Finally, since we don't have system packages we can work with, exporting a `pkg-config` path makes little sense. Closes cross-rs#860.

Uses cargo-zigbuild as a backend, and adds configuration options for zig under `[build.zig]` and `[target.(...).zig]`. If enabled, and an image override is not provided, `cross` will always use the `zig` image. The feature can be enabled by providing `zig` as a table, bool, or string. It supports custom glibc versions by passing the `zig.version` key, and `zig` can be separately enabled or disabled by providing `zig.enable`. ``` [target.x86_64-unknown-linux-gnu.zig] enable = true # enable use of the zig image version = "2.17" # glibc version to use image = "ghcr.io/cross-rs/zig:local" # custom image to use ``` If provided as a bool, it will use the default glibc version: ``` [target.x86_64-unknown-linux-gnu] zig = true ``` If provided as a string, `zig` will be automatically enabled: ``` [target.x86_64-unknown-linux-gnu] zig = "2.17" ``` The image does not provide runners, `bindgen` Clang args, or `pkg-config` paths, since `zig cc` does not provide the dynamic library loader (`ld-linux*.so`) required, meaning none of the binaries can be run. For `bindgen`, `zig cc` has an unusual directory structure, so there is no traditional sysroot with `usr`, `lib`, and `include` subdirectories. Finally, since we don't have system packages we can work with, exporting a `pkg-config` path makes little sense. Closes cross-rs#860.

Uses cargo-zigbuild as a backend, and adds configuration options for zig under `[build.zig]` and `[target.(...).zig]`. If enabled, and an image override is not provided, `cross` will always use the `zig` image. The feature can be enabled by providing `zig` as a table, bool, or string. It supports custom glibc versions by passing the `zig.version` key, and `zig` can be separately enabled or disabled by providing `zig.enable`. ``` [target.x86_64-unknown-linux-gnu.zig] enable = true # enable use of the zig image version = "2.17" # glibc version to use image = "ghcr.io/cross-rs/zig:local" # custom image to use ``` If provided as a bool, it will use the default glibc version: ``` [target.x86_64-unknown-linux-gnu] \# equivalent to { enable = true } zig = true ``` If provided as a string, `zig` will be automatically enabled: ``` [target.x86_64-unknown-linux-gnu] \# equivalent to { enable = true, version = "2.17" } zig = "2.17" ``` The image does not provide runners, `bindgen` Clang args, or `pkg-config` paths, since `zig cc` does not provide the dynamic library loader (`ld-linux*.so`) required, meaning none of the binaries can be run. For `bindgen`, `zig cc` has an unusual directory structure, so there is no traditional sysroot with `usr`, `lib`, and `include` subdirectories. Finally, since we don't have system packages we can work with, exporting a `pkg-config` path makes little sense. Closes cross-rs#860.

Alexhuszagh added enhancement container-images feature-request labels Jun 25, 2022

Alexhuszagh modified the milestones: v0.3.0, v0.2.3 Jun 26, 2022

Alexhuszagh self-assigned this Jun 28, 2022

Alexhuszagh mentioned this issue Jun 29, 2022

Add a zig cc-based image. #880

Merged

Emilgardis modified the milestones: v0.2.3, v0.3.0 Jul 4, 2022

NobodyXu mentioned this issue Jul 6, 2022

Problems with dynamically linked binaries (e.g. no OpenSSL 3 compatibilty) cargo-bins/cargo-quickinstall#84

Closed

NobodyXu mentioned this issue Jul 14, 2022

Speedup CI: Build cargo-binstall in debug mode for PR cargo-bins/cargo-binstall#212

Merged

bors bot closed this as completed in 3f60cc1 Jul 17, 2022

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Provide a `zig cc`-based image. #860

Provide a `zig cc`-based image. #860

Alexhuszagh commented Jun 25, 2022 •

edited

Loading

Emilgardis commented Jun 25, 2022

Alexhuszagh commented Jun 25, 2022 •

edited

Loading

Emilgardis commented Jun 25, 2022 •

edited

Loading

mntns commented Jun 25, 2022

Emilgardis commented Jun 25, 2022

Alexhuszagh commented Jun 25, 2022 •

edited

Loading

Emilgardis commented Jun 25, 2022

Alexhuszagh commented Jun 25, 2022

Alexhuszagh commented Jun 28, 2022 •

edited

Loading

Alexhuszagh commented Jun 29, 2022

Provide a zig cc-based image. #860

Provide a zig cc-based image. #860

Comments

Alexhuszagh commented Jun 25, 2022 • edited Loading

Checklist

Describe your request

Describe why this would be a good inclusion for cross

Emilgardis commented Jun 25, 2022

Alexhuszagh commented Jun 25, 2022 • edited Loading

Emilgardis commented Jun 25, 2022 • edited Loading

mntns commented Jun 25, 2022

Emilgardis commented Jun 25, 2022

Alexhuszagh commented Jun 25, 2022 • edited Loading

Emilgardis commented Jun 25, 2022

Alexhuszagh commented Jun 25, 2022

Alexhuszagh commented Jun 28, 2022 • edited Loading

Alexhuszagh commented Jun 29, 2022

Provide a `zig cc`-based image. #860

Provide a `zig cc`-based image. #860

Alexhuszagh commented Jun 25, 2022 •

edited

Loading

Describe why this would be a good inclusion for `cross`

Alexhuszagh commented Jun 25, 2022 •

edited

Loading

Emilgardis commented Jun 25, 2022 •

edited

Loading

Alexhuszagh commented Jun 25, 2022 •

edited

Loading

Alexhuszagh commented Jun 28, 2022 •

edited

Loading