The perl binaries included in the toolchains are not static binaries

[malt3]

The perl binaries included in the toolchains are not static binaries. Instead, they link against a number of libraries:

ldd perl
        linux-vdso.so.1 (0x00007ffc8a09a000)
        libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x00007f525fd82000)
        libdl.so.2 => /lib/x86_64-linux-gnu/libdl.so.2 (0x00007f525fd7d000)
        libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x00007f525fc96000)
        libcrypt.so.1 => /lib/x86_64-linux-gnu/libcrypt.so.1 (0x00007f525fc5c000)
        libutil.so.1 => /lib/x86_64-linux-gnu/libutil.so.1 (0x00007f525fc57000)
        libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007f525fa2f000)
        /lib64/ld-linux-x86-64.so.2 (0x00007f525fda0000)

This makes the rules non-hermetic and prevents usage in clean environments like nixOS.
The issue also has been raised with the source of the perl binaries some time ago: skaji/relocatable-perl#13

Some background:

I want to compile openssl hermetically within bazel.

[malt3]

I would also be happy to help out with improving the situation but I would need some indication if this is an issue that rules_perl wants to solve.

[skeletonkey]

@malt3 - I normally review these on the weekend, however, was away from all electronics this last one.

This is one of Perl's deficits - wanting to be installed on the whole system. Openssl also surfers from this. Trying to compile these in a hermetical way would bring in most of the Linux kernel. At least that is my first impression - this is why we have Docker.

I'm open to any ideas that furthers Perl's first world status in Bazel! How would you proceed?

[malt3]

I think there are two ways to solve this:

• build a fully static version of perl with something like musl libc
• ship all or most of the dynamic libraries with the perl binary and make it work using runfiles in bazel

Once we have a fully static binary, using the first approach would be way simpler imo.
In any case this could be an opt-in for now (an additional toolchain that has to be selected by rules_perl users) so we don't break anyone's usage.

[GregBowyer]

Hi fellow nix user!

I wonder if we make it more obvious how perls toolchains is configured and let nix people just give it a nixos attribute instead

[malt3]

I wonder if we make it more obvious how perls toolchains is configured and let nix people just give it a nixos attribute instead

I'm afraid I don't understand what a nixos attribute is and how it would make the perl binaries embedded in the toolchain of rules_perl hermetic.
If at all possible, I would like to avoid dependencies on system package managers like nix for my bazel project.

[GregBowyer]

I wonder if we make it more obvious how perls toolchains is configured and let nix people just give it a nixos attribute instead

I'm afraid I don't understand what a nixos attribute is and how it would make the perl binaries embedded in the toolchain of rules_perl hermetic. If at all possible, I would like to avoid dependencies on system package managers like nix for my bazel project.

So my statement is based on your original quote.

This makes the rules non-hermetic and prevents usage in clean environments like nixOS.

Take this reply with a dismissal if I misunderstand your original statement, I assume here you mean nixOs as in nixOs reproducible builds and deployments

If you are (like me) on NixOS you have /nix around, I would suggest that you have the capability to be hermetic. One solution for you might be to use rules_nixpkgs to bring in a fixed known version of Perl and its runtime, which would look something like so:

# Use a fixed version of the nix tree (could also be done more in a default.nix)
nixpkgs_git_repository(
    name = "nixpkgs",
    revision = "22.05",
    sha256 = "0f8c25433a6611fa5664797cd049c80faefec91575718794c701f3b033f2db01",
)

# Grab perl
nixpkgs_package(
    name = "perl",
    attribute_path = "perl536",
    repositories = { "nixpkgs": "@nixpkgs//:default.nix" }
)

For what it's worth, we use this in some places at $DAYJOB to bring in complex software we still want to be known fixed versions, but we want to avoid building the entire tool chain from ground zero. Examples of our usage are shellcheck, ffmpeg, manim, wine and some very strange cryptographic prover tools, all of which do not readily exist in "just download this tarball and run" form and would be pretty hard to make work in that way.

This wouldn't be a system dep that would be fixed in time and highly hermetic, you would get a nix derivation specific to the build rather than anything you installed in home-manager or nix-env iA perl.

At that point, it might be possible (I have not tested) to use @perl//:bin/perl (and similar) as the Perl interpreter

I genuinely understand if you would rather not force nix onto your consumers, but you might consider this for your environment.

• If you have control over your machines and dev env to have NixOS in your developer orbit you could encourage this, and we could see if the rules could be tweaked to cope
• If you don't, and you are solving for your local problems, it could be something that can be configuration gated in .bazelrc
• It might be possible to pull the toolchains from a docker image like some other Bazel rules do, although that's going to require docker and be akin to the NixOS solution.

I would suggest that a full musl-libc static Perl build is possible, but might be beyond the scope of these rules and the support herein. If there is a known static build / tarball laying about, it might be possible to switch up to that.

[malt3]

@GregBowyer thanks for this detailed explanation. If at all possible I would like to make the Bazel workflow not depend on nix / rules_nixpkgs, although it does seem like a very ergonomic way to bring in complex build tools.
Gating this behind a config in .bazelrc also sounds nice. I may try this out.

Can you maybe elaborate what you mean by this sentence?

It might be possible to pull the toolchains from a docker image

This sounds like a possible workaround but I am not sure on what level this will be applied. Do you mean the docker sandbox?

I would suggest that a full musl-libc static Perl build is possible, but might be beyond the scope of these rules and the support herein. If there is a known static build / tarball laying about, it might be possible to switch up to that.

Yes this would be my preferred solution but from searching around, it seems this may not exist yet. I understand that this is beyond the scope.

[GregBowyer]

Yes the docker sandbox

[malt3]

I think I'll support my own fork of relocatable_perl and rules_perl for now. I also created release artifacts that offer a statically linked perl toolchain. I only need perl to configure OpenSSL so I don't strictly need a full perl (with extensions).

If you have ideas how a statically linked toolchain could co-exist, please feel free to reach out.

[andrewkatson]

@malt3 sorry this is so random but do you have a link to the way you configured openssl with bazel? I have been trying for several days and can't seem to get around various issues: #61. I looked through your repos and couldn't find the way you did it so it must be on a private repo.

[malt3]

@andrewkatson i was simply using the example from rules_foreign_cc. I since stopped doing that.
I can recommend using boringssl as a drop-in replacement instead. It comes with Bazel BUILD files out of the box and works well.
The rules_foreign_cc solution is not used by me anymore and I could not find my code from back then.

[andrewkatson]

No problem! Yeah I would use it but its missing some functions that my legacy application uses. So I am going to figure out how to migrate to boringssl or finish my current non rules_foreign_cc approach.