Cannot use out to set file name in pkg_zip

[criemen]

The docs of pkg_zip say

Use out or package_file_name to specify the output file name

However, using out results in

ERROR: Traceback (most recent call last):
        File "/workspaces/semmle-code/extractor-python/BUILD.bazel", line 73, column 5, in <toplevel>
                dist(
        File "/workspaces/semmle-code/common.bzl", line 12, column 12, in dist
                pkg_zip(name = name + ".zip", out = name, srcs = [name])
        File "/home/codespace/.cache/bazel/_bazel_codespace/3c41b00dfcd8c4fb714360e042371c98/external/rules_pkg/pkg.bzl", line 723, column 17, in pkg_zip
                pkg_zip_impl(
Error in pkg_zip_impl: rule(...) got multiple values for parameter 'out'

Presumably this is because out is unconditionally set. Instead, it should do something ala

if not kwargs.get("out"):
  kwargs["out"] = archive_name + "." + extension

instead.

[aiuto]

out is deprecated. Use package_file_name

• https://github.com/bazelbuild/rules_pkg/blob/main/pkg/docs/reference.md#common-attributes
• https://github.com/bazelbuild/rules_pkg/tree/main/examples/naming_package_files

I should fix the documentation, and have the rule stop using it.

[criemen]

I have a rule ala

pkg_zip(name = "bla.zip", package_file_name = "bla.zip", srcs = [...])

Because the name of the rule is bla.zip, the rule creates an out file with name bla.zip.zip. Due to package_file_name, this file gets renamed to bla.zip (great), but I also get a symlink from bla.zip.zip to bla.zip. Without overwriting out, it is not possible to get rid of this symlink.
It's not a big issue, as I can just ignore the symlink, but it is not as pretty as it could be, either.

[aiuto]

Right. The extra zip in bla.zip.zip is currently a bug. What is not a bug is getting a symlink of name + '.zip' != package_file_name. In that case the physical file is defined by package_file_name and you will get a symlink to that. This allows you do create real output files with dynamically created named (e.g. containing the --cpu, or date), while also having a predictable file name if you want to write a script that is a little lazy and just wants to grab bla.zip without having to look at the OutputGroup provider.

…

On Mon, Aug 23, 2021 at 6:26 AM Cornelius Riemenschneider < ***@***.***> wrote: I have a rule ala pkg_zip(name = "bla.zip", package_file_name = "bla.zip", srcs = [...]) Because the name of the rule is bla.zip, the rule creates an out file with name bla.zip.zip. Due to package_file_name, this file gets renamed to bla.zip (great), but I *also* get a symlink from bla.zip.zip to bla.zip. Without overwriting out, it is not possible to get rid of this symlink. It's not a big issue, as I can just ignore the symlink, but it is not as pretty as it could be, either. — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#414 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAXHHHHG423RCVZSYIYMST3T6IO4PANCNFSM5COBLB2Q> . Triage notifications on the go with GitHub Mobile for iOS <https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675> or Android <https://play.google.com/store/apps/details?id=com.github.android&utm_campaign=notification-email> .

[ghost]

out is deprecated. Use package_file_name

• https://github.com/bazelbuild/rules_pkg/blob/main/pkg/docs/reference.md#common-attributes
• https://github.com/bazelbuild/rules_pkg/tree/main/examples/naming_package_files

I should fix the documentation, and have the rule stop using it.

So, I ran into an small perf problem related to this. I have a 2+ GiB .zip created by pkg_zip. By not allowing me to use out, I'm forced into setup_output_files's ctx.symlink path, which then incurs a 15s critical path tax. One wouldn't expect "Creating symlink foo.zip" to take 15s, but I can only assume that Bazel is (re-)checksumming the symlink target or similar. (See profile snippet below.) Until this is addressed (in Bazel?), I'd still like to have an escape hatch to avoid (in our case unused) symlinks. (We're patching this out internally for the time being.)

Creating symlink bazel-out/k8-dbg-obj/bin/foo.zip
Category | action dependency checking
Wall Duration	15,173.772 ms

(Edit: Filed bazelbuild/bazel#14125 to discuss underlying ctx.actions.symlink perf.)

[aiuto]

I'm floored. That seems so wrong.
Are you willing to make a reproduction to file against bazel?

[ghost]

Are you willing to make a reproduction to file against bazel?

Absolutely. :)

bazelbuild/bazel#14125

[aiuto]

Maybe we could make the symlink "as-needed". The wrapper macro would look at package_file_name and if it had no expansions in it, would pass it as the output file name. The rule, seeing output and package_file_name were identical, would only return the one, not making the symlink. This would only impact people who were scripting picking up the output file based on name+'.zip', but that's not a precise approach anyway.