Add support for generating debuginfo RPMs (#842)

* Enable creation and capture of debuginfo RPMs

This change enables the creation and capture of debuginfo RPMs on
Fedora40 and CentOS7.

See:
https://docs.fedoraproject.org/en-US/packaging-guidelines/Debuginfo/

Fedora 40 expects the RPM contents to be located in a subdirectory
which is specified using the `buildsubdir` variable.  In order to
account for this, we need to tweak some of the location details if
debuginfo is enabled.

CentOS expects `buildsubdir` to have a value like `.` instead.

In both cases, we disable debugsource packages by ensuring that we
undefine `_debugsource_packages`, otherwise we'll try to generate them
alongside the debuginfo packages and will fail.

We only want this method of producing debuginfo to apply when we're
using the system `rpmbuild` because the underlying behaviour is
controlled by a combination of the rpmbuild version, macro
definitions, find-debuginfo.sh, and debugedit.  If we were to expand
this to use a hermetic debuginfo then a different approach might be
desirable.

* Add an RPM example that generates debuginfo

This provides a basic example that generates a debuginfo RPM
configured to run on CentOS7.

* Upgrade rules_python to 0.31.0

rules_python seems to fail us when we're generating debuginfo RPMs
unless we upgrade to a version more recent than 0.24.0.

* Only generate debuginfo RPM when pkg_rpm() asks for it

In lieu of enabling this behaviour by default on the supported
platforms, we add an additional argument to the pkg_rpm() rule that
will allow us to enable it for pkg_rpm() targets.  This prevents us
from enabling it in cases where it's not desired.

* Add test for building debuginfo RPM

This test is modelled on the subrpm test.  In lieu of using a simple
text file as an input it instead generates a binary that includes
debug symbosl from a C source file and includes that in the RPM.

The baseline comparison strips out the `.build-id` paths because the
hashes that are generated may not be stable.x

* Remove architecture and size from debuginfo test output

These values may vary depending on the platform that this is being run
on and we don't really care about them.

* Add period to docstring

* Enable debuginfo support for CentOS Stream 9

CentOS Stream 9 appears to work more or less the same for debuginfo
generation as CentOS 7.  `os-release` describes it as os == `centos`
and version == `9`.  This change creates an extra token for `centos9`
and sticks it in the places where we currently have controls for
`centos7`.

MODULE.bazel

@@ -7,7 +7,7 @@ module(
 
 # Do not update to newer versions until you need a specific new feature.
 bazel_dep(name = "rules_license", version = "0.0.4")
-bazel_dep(name = "rules_python", version = "0.24.0")
+bazel_dep(name = "rules_python", version = "0.31.0")
 bazel_dep(name = "bazel_skylib", version = "1.2.0")
 
 # Only for development

examples/rpm/debuginfo/BUILD

@@ -0,0 +1,55 @@
+# Copyright 2020 The Bazel Authors. All rights reserved.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#    http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+# -*- coding: utf-8 -*-
+
+
+load("@rules_pkg//pkg:mappings.bzl", "pkg_files")
+load("@rules_pkg//pkg:rpm.bzl", "pkg_rpm")
+
+cc_binary(
+    name = "test",
+    copts = ["-g"],
+    srcs = [
+        "test.c",
+    ],
+)
+
+pkg_files(
+    name = "rpm_files",
+    srcs = [
+        ":test",
+    ],
+)
+
+pkg_rpm(
+    name = "test-rpm",
+    srcs = [
+        ":rpm_files",
+    ],
+    release = "0",
+    version = "1",
+    license = "Some license",
+    summary = "Summary",
+    description = "Description",
+    debuginfo = True,
+)
+
+# If you have rpmbuild, you probably have rpm2cpio too.
+# Feature idea: Add rpm2cpio and cpio to the rpmbuild toolchain
+genrule(
+    name = "inspect_content",
+    srcs = [":test-rpm"],
+    outs = ["content.txt"],
+    cmd = "rpm2cpio $(locations :test-rpm) | cpio -ivt >$@",
+)

examples/rpm/debuginfo/MODULE.bazel

@@ -0,0 +1,32 @@
+# Copyright 2024 The Bazel Authors. All rights reserved.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#    http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+module(name = "rules_pkg_example_rpm_system_rpmbuild_bzlmod")
+
+bazel_dep(name = "rules_pkg")
+
+local_path_override(
+    module_name = "rules_pkg",
+    path = "../../..",
+)
+
+find_rpmbuild = use_extension(
+    "@rules_pkg//toolchains/rpm:rpmbuild_configure.bzl",
+    "find_system_rpmbuild_bzlmod",
+)
+use_repo(find_rpmbuild, "rules_pkg_rpmbuild")
+
+register_toolchains(
+    "@rules_pkg_rpmbuild//:all",
+)

examples/rpm/debuginfo/README.md

@@ -0,0 +1,17 @@
+# Using system rpmbuild with bzlmod and generating debuginfo
+
+## Summary
+
+This example uses the `find_system_rpmbuild_bzlmod` module extension to help
+us register the system rpmbuild as a toolchain in a bzlmod environment.
+
+It configures the system toolchain to be aware of which debuginfo configuration
+to use (defaults to "none", the example uses "centos7").
+
+## To use
+
+```
+bazel build :*
+rpm2cpio bazel-bin/test-rpm.rpm | cpio -ivt
+cat bazel-bin/content.txt
+```

examples/rpm/debuginfo/test.c

@@ -0,0 +1,3 @@
+int main() {
+  return 0;
+}

pkg/make_rpm.py

@@ -172,6 +172,7 @@ class RpmBuilder(object):
 
   SOURCE_DIR = 'SOURCES'
   BUILD_DIR = 'BUILD'
+  BUILD_SUBDIR = 'BUILD_SUB'
   BUILDROOT_DIR = 'BUILDROOT'
   TEMP_DIR = 'TMP'
   RPMS_DIR = 'RPMS'
@@ -345,7 +346,7 @@ def SetupWorkdir(self,
       shutil.copy(os.path.join(original_dir, file_list_path), RpmBuilder.BUILD_DIR)
       self.file_list_path = os.path.join(RpmBuilder.BUILD_DIR, os.path.basename(file_list_path))
 
-  def CallRpmBuild(self, dirname, rpmbuild_args):
+  def CallRpmBuild(self, dirname, rpmbuild_args, debuginfo_type):
     """Call rpmbuild with the correct arguments."""
 
     buildroot = os.path.join(dirname, RpmBuilder.BUILDROOT_DIR)
@@ -361,16 +362,31 @@ def CallRpmBuild(self, dirname, rpmbuild_args):
     if self.debug:
       args.append('-vv')
 
+    if debuginfo_type == "fedora40":
+      os.makedirs(f'{dirname}/{RpmBuilder.BUILD_DIR}/{RpmBuilder.BUILD_SUBDIR}')
+
     # Common options
     # NOTE: There may be a need to add '--define', 'buildsubdir .' for some
     # rpmbuild versions. But that breaks other rpmbuild versions, so before
     # adding it back in, add extensive tests.
     args += [
-        '--define', '_topdir %s' % dirname,
-        '--define', '_tmppath %s/TMP' % dirname,
-        '--define', '_builddir %s/BUILD' % dirname,
-        '--bb',
-        '--buildroot=%s' % buildroot,
+      '--define', '_topdir %s' % dirname,
+      '--define', '_tmppath %s/TMP' % dirname,
+      '--define', '_builddir %s/BUILD' % dirname,
+    ]
+
+    if debuginfo_type in ["fedora40", "centos7", "centos9"]:
+      args += ['--undefine', '_debugsource_packages']
+
+    if debuginfo_type in ["centos7", "centos9"]:
+      args += ['--define', 'buildsubdir .']
+
+    if debuginfo_type == "fedora40":
+      args += ['--define', f'buildsubdir {RpmBuilder.BUILD_SUBDIR}']
+
+    args += [
+      '--bb',
+      '--buildroot=%s' % buildroot,
     ]  # yapf: disable
 
     # Macro-based RPM parameter substitution, if necessary inputs provided.
@@ -382,7 +398,11 @@ def CallRpmBuild(self, dirname, rpmbuild_args):
       args += ['--define', 'build_rpm_install %s' % self.install_script_file]
     if self.file_list_path:
       # %files -f is taken relative to the package root
-      args += ['--define', 'build_rpm_files %s' % os.path.basename(self.file_list_path)]
+      base_path = os.path.basename(self.file_list_path)
+      if debuginfo_type == "fedora40":
+        base_path = os.path.join("..", base_path)
+
+      args += ['--define', 'build_rpm_files %s' % base_path]
 
     args.extend(rpmbuild_args)
 
@@ -459,7 +479,8 @@ def Build(self, spec_file, out_file, subrpm_out_files=None,
             posttrans_scriptlet_path=None,
             file_list_path=None,
             changelog_file=None,
-            rpmbuild_args=None):
+            rpmbuild_args=None,
+            debuginfo_type=None):
     """Build the RPM described by the spec_file, with other metadata in keyword arguments"""
 
     if self.debug:
@@ -490,7 +511,7 @@ def Build(self, spec_file, out_file, subrpm_out_files=None,
                         postun_scriptlet_path=postun_scriptlet_path,
                         posttrans_scriptlet_path=posttrans_scriptlet_path,
                         changelog_file=changelog_file)
-      status = self.CallRpmBuild(dirname, rpmbuild_args or [])
+      status = self.CallRpmBuild(dirname, rpmbuild_args or [], debuginfo_type)
       self.SaveResult(out_file, subrpm_out_files)
 
     return status
@@ -550,6 +571,8 @@ def main(argv):
 
   parser.add_argument('--rpmbuild_arg', dest='rpmbuild_args', action='append',
                       help='Any additional arguments to pass to rpmbuild')
+  parser.add_argument('--debuginfo_type', dest='debuginfo_type', default='none',
+                      help='debuginfo type to use (centos7, fedora40, or none)')
   parser.add_argument('files', nargs='*')
 
   options = parser.parse_args(argv or ())
@@ -574,7 +597,8 @@ def main(argv):
                          postun_scriptlet_path=options.postun_scriptlet,
                          posttrans_scriptlet_path=options.posttrans_scriptlet,
                          changelog_file=options.changelog,
-                         rpmbuild_args=options.rpmbuild_args)
+                         rpmbuild_args=options.rpmbuild_args,
+                         debuginfo_type=options.debuginfo_type)
   except NoRpmbuildFoundError:
     print('ERROR: rpmbuild is required but is not present in PATH')
     return 1

pkg/rpm_pfg.bzl

@@ -67,6 +67,13 @@ DEFAULT_FILE_MODE = "%defattr(-,root,root)"
 _INSTALL_FILE_STANZA_FMT = """
 install -d "%{{buildroot}}/$(dirname '{1}')"
 cp '{0}' '%{{buildroot}}/{1}'
+chmod +w '%{{buildroot}}/{1}'
+""".strip()
+
+_INSTALL_FILE_STANZA_FMT_FEDORA40_DEBUGINFO = """
+install -d "%{{buildroot}}/$(dirname '{1}')"
+cp '../{0}' '%{{buildroot}}/{1}'
+chmod +w '%{{buildroot}}/{1}'
 """.strip()
 
 # TODO(nacl): __install
@@ -172,7 +179,7 @@ def _make_absolute_if_not_already_or_is_macro(path):
 # TODO(nacl, #459): These are redundant with functions and structures in
 # pkg/private/pkg_files.bzl.  We should really use the infrastructure provided
 # there, but as of writing, it's not quite ready.
-def _process_files(pfi, origin_label, grouping_label, file_base, rpm_ctx):
+def _process_files(pfi, origin_label, grouping_label, file_base, rpm_ctx, debuginfo_type):
     for dest, src in pfi.dest_src_map.items():
         metadata = _package_contents_metadata(origin_label, grouping_label)
         if dest in rpm_ctx.dest_check_map:
@@ -196,7 +203,12 @@ def _process_files(pfi, origin_label, grouping_label, file_base, rpm_ctx):
         else:
             # Files are well-known.  Take care of them right here.
             rpm_ctx.rpm_files_list.append(_FILE_MODE_STANZA_FMT.format(file_base, abs_dest))
-            rpm_ctx.install_script_pieces.append(_INSTALL_FILE_STANZA_FMT.format(
+
+            install_stanza_fmt = _INSTALL_FILE_STANZA_FMT
+            if debuginfo_type == "fedora40":
+                install_stanza_fmt = _INSTALL_FILE_STANZA_FMT_FEDORA40_DEBUGINFO
+
+            rpm_ctx.install_script_pieces.append(install_stanza_fmt.format(
                 src.path,
                 abs_dest,
             ))
@@ -231,7 +243,7 @@ def _process_symlink(psi, origin_label, grouping_label, file_base, rpm_ctx):
         psi.attributes["mode"],
     ))
 
-def _process_dep(dep, rpm_ctx):
+def _process_dep(dep, rpm_ctx, debuginfo_type):
     # NOTE: This does not detect cases where directories are not named
     # consistently.  For example, all of these may collide in reality, but
     # won't be detected by the below:
@@ -255,6 +267,7 @@ def _process_dep(dep, rpm_ctx):
             None,  # group label
             _make_filetags(dep[PackageFilesInfo].attributes),  # file_base
             rpm_ctx,
+            debuginfo_type,
         )
 
     if PackageDirsInfo in dep:
@@ -285,6 +298,7 @@ def _process_dep(dep, rpm_ctx):
                 dep.label,
                 file_base,
                 rpm_ctx,
+                debuginfo_type
             )
         for entry, origin in pfg_info.pkg_dirs:
             file_base = _make_filetags(entry.attributes, "%dir")
@@ -306,7 +320,7 @@ def _process_dep(dep, rpm_ctx):
                 rpm_ctx,
             )
 
-def _process_subrpm(ctx, rpm_name, rpm_info, rpm_ctx):
+def _process_subrpm(ctx, rpm_name, rpm_info, rpm_ctx, debuginfo_type):
     sub_rpm_ctx = struct(
         dest_check_map = {},
         install_script_pieces = [],
@@ -356,7 +370,7 @@ def _process_subrpm(ctx, rpm_name, rpm_info, rpm_ctx):
         ]
 
         for dep in rpm_info.srcs:
-            _process_dep(dep, sub_rpm_ctx)
+            _process_dep(dep, sub_rpm_ctx, debuginfo_type)
 
         # rpmbuild will be unhappy if we have no files so we stick
         # default file mode in for that scenario
@@ -424,6 +438,7 @@ def _pkg_rpm_impl(ctx):
 
     files = []
     tools = []
+    debuginfo_type = "none"
     name = ctx.attr.package_name if ctx.attr.package_name else ctx.label.name
     rpm_ctx.make_rpm_args.append("--name=" + name)
 
@@ -448,6 +463,10 @@ def _pkg_rpm_impl(ctx):
             tools.append(executable_files)
             rpm_ctx.make_rpm_args.append("--rpmbuild=%s" % executable_files.executable.path)
 
+        if ctx.attr.debuginfo:
+            debuginfo_type = toolchain.debuginfo_type
+            rpm_ctx.make_rpm_args.append("--debuginfo_type=%s" % debuginfo_type)
+
     #### Calculate output file name
     # rpm_name takes precedence over name if provided
     if ctx.attr.package_name:
@@ -678,13 +697,14 @@ def _pkg_rpm_impl(ctx):
     # they aren't unnecessarily recreated.
 
     for dep in ctx.attr.srcs:
-        _process_dep(dep, rpm_ctx)
+        _process_dep(dep, rpm_ctx, debuginfo_type)
 
     #### subrpms
     if ctx.attr.subrpms:
         subrpm_lines = []
         for s in ctx.attr.subrpms:
-            subrpm_lines.extend(_process_subrpm(ctx, rpm_name, s[PackageSubRPMInfo], rpm_ctx))
+            subrpm_lines.extend(_process_subrpm(
+                ctx, rpm_name, s[PackageSubRPMInfo], rpm_ctx, debuginfo_type))
 
         subrpm_file = ctx.actions.declare_file(
             "{}.spec.subrpms".format(rpm_name),
@@ -696,6 +716,27 @@ def _pkg_rpm_impl(ctx):
         files.append(subrpm_file)
         rpm_ctx.make_rpm_args.append("--subrpms=" + subrpm_file.path)
 
+    if debuginfo_type != "none":
+        debuginfo_default_file = ctx.actions.declare_file(
+            "{}-debuginfo.rpm".format(rpm_name))
+        debuginfo_package_file_name = "%s-%s-%s-%s.%s.rpm" % (
+            rpm_name,
+            "debuginfo",
+            ctx.attr.version,
+            ctx.attr.release,
+            ctx.attr.architecture,
+        )
+
+        _, debuginfo_output_file, _ = setup_output_files(
+            ctx,
+            debuginfo_package_file_name,
+            default_output_file = debuginfo_default_file,
+        )
+
+        rpm_ctx.output_rpm_files.append(debuginfo_output_file)
+        rpm_ctx.make_rpm_args.append(
+            "--subrpm_out_file=debuginfo:%s" % debuginfo_output_file.path )
+
     #### Procedurally-generated scripts/lists (%install, %files)
 
     # We need to write these out regardless of whether we are using
@@ -1177,6 +1218,15 @@ pkg_rpm = rule(
                 [PackageSubRPMInfo],
             ],
         ),
+        "debuginfo": attr.bool(
+            doc = """Enable generation of debuginfo RPMs
+
+            For supported platforms this will enable the generation of debuginfo RPMs adjacent
+            to the regular RPMs.  Currently this is supported by Fedora 40, CentOS7 and
+            CentOS Stream 9.
+            """,
+            default = False,
+        ),
         "rpmbuild_path": attr.string(
             doc = """Path to a `rpmbuild` binary.  Deprecated in favor of the rpmbuild toolchain""",
         ),