for discussion: pkgfilgroup as a way of structuring package inputs

[aiuto]

The current packaging rules are not the most user friendly. This issue describes the key parts of what Google uses internally. I intend this to prompt discussion.

The basic idea is

• we have a pkgfilegroup rule that gathers sources and information about where they belong in the distributon
• the debian, rpm or tar builder depends on a list of pkgfilegroup rules to gather the content and build the package
• for the rpm case, you do not use classic .spec files to specify file modes. The pkg_rpm rule builds one for you and you include it from your .spec file.

So, it is a fairly heavy buy-in. You get more fine grained control over packaging, at the expense of doing things differently than you might have in the past. This is one of the motivators for not releasing Google's code originally. It is crufty in ways that other's might not like.

pkgfilegroup

pkgfilegroup(name, srcs, attr, prefix, section, strip_prefix)

srcs

List of labels; optional

A list of rules that are dependencies of this rule. All output of each of the rules mentioned in the srcs attribute will be included in the packages that depend on this pkgfilegroup.

attr

List of strings; optional

Permissions and ownership of the installed files. This list has 3 elements. The first element is what permissions the installed files should have, specified in octal. The second element is the username or UID that the files should be user-owned by. The third element is the username or GID that the files should be group-owned by. Each element may instead be "-" which specifies that package shouldn't care about that particular element and leave it at the default.

prefix

String; required

Where the package should install these files. All output of each of the rules mentioned in the srcs attribute to this rule will be installed on the end-users machine relative to this prefix.

section

String; optional

Type of file being installed. The string can be blank, doc, or config. Additionally, config can take two optional arguments: missingok and noreplace. These are all syntactically valid values for section:
doc
config
config(missingok)
config(noreplace)
config(missingok,noreplace)
genrpm passes the value of section through to the %files section of the RPM's .spec file. RPM handles documentation and configuration files slightly differently from normal files. For example, RPM will not just overwrite configuration files. It will make sure the older configuration file, if installed, is preserved. missingok means that an error is not reported if rpm --verify is run and the file is missing. noreplace means that, when the RPM is being upgraded, it should not move the existing file out of the way. Instead, it should create the new file with an .rpmnew extension file next to the existing file.

gendeb populates the conffiles control with the srcs of dependency pkgfilegroups with section set to config. gendeb ignores doc and the config optional arguments.

strip_prefix

String; optional; default is "."

Which part of the file name should be stripped. If it is ".", all subdirectories are stripped, leaving only the file name. If it starts with "/", it is interpreted relative to the root of the repository. Otherwise, it is resolved relative to the current package. This path is stripped from the beginning of the file path before appending the remaining part to prefix. For example, if prefix is "/etc", strip_prefix is "foo", and the file name relative to the package is "foo/bar/baz", the file will be installed as "/etc/bar/baz".If strip_prefix is an empty string, the full package-relative path is used.

An example

pkgfilegroup(
    name = "hello_bin",
    srcs = [
        "linux/bin/hello.sh",
        ":hello",
    ],
    attr = [
        "0755",
        "root",
        "root",
    ],
    prefix = "/usr/bin",
    # Note strip_prefix default == "."
)

cc_binary(name="hello", ...)

genrpm(
    name = "hello-rpm",
    release = RELEASE,
    spec = "hello.spec",
    strip = 1,
    version = VERSION,
    deps = [
        ":hello_bin",
       ...
    ],
)

This would create a spec file containing
%attr(0755,root,root) /usr/bin/hello.sh
%attr(0755,root,root) /usr/bin/hello

And that gets include in hello.spec as build_rpm_files

%include %build_rpm_options

Summary: Hello World Sample Application
License: Google
Group: devtools

%description
A hello world app to package as a test case for the genrpm rule.

%install

%include %build_rpm_install

%files -f %build_rpm_files
%defattr(755,nobody,bin)

Obvious problems.

• Since we depend on a list of pkgfilegroup rules rather than a tree, there is no way to rebase a set of things. That is, I can't take a rule which would produce /usr/bin/hello and re-wrap it to make those files appear as /distro1/usr/bin/hello.
• The strip_prefix behavior is too hard to understand. Trust me, it gets a lot of questions.
• It is very *nix centric. I should be able to describe things that can be installed on other OSes.

Good points

• fine grained file attributes
• the source layout does not have to mirror the distribution layout.

[aiuto]

This is an ancient design discussion that was mostly implemented (in concept). Closing it to reduce bug clutter.