proposal: instead of build.zig.ini, use build.zig.zon

[andrewrk]

Extracted from #14265.

Zig Object Notation

Instead of...

[package]
name=libffmpeg
version=5.1.2

[dependency]
name=libz
url=https://github.com/andrewrk/libz/archive/f0e53cc2391741034b144a2c2076ed8a9937b29b.tar.gz
hash=c9b30cffc40999d2c078ff350cbcee642970a224fe123c756d0892f876cf1aae

[dependency]
name=libmp3lame
url=https://github.com/andrewrk/libmp3lame/archive/497568e670bfeb14ab6ef47fb6459a2251358e43.tar.gz
hash=9ba4f49895b174a3f918d489238acbc146bd393575062b2e3be33488b688e36f

It would look like this:

.{
    .name = "libffmpeg",
    .version = "5.1.2",
    .dependencies = .{
        .libz = .{
            .url = "https://github.com/andrewrk/libz/archive/f0e53cc2391741034b144a2c2076ed8a9937b29b.tar.gz",
            .hash = "c9b30cffc40999d2c078ff350cbcee642970a224fe123c756d0892f876cf1aae",
        },
        .libmp3lame = .{
            .url = "https://github.com/andrewrk/libmp3lame/archive/497568e670bfeb14ab6ef47fb6459a2251358e43.tar.gz",
            .hash = "9ba4f49895b174a3f918d489238acbc146bd393575062b2e3be33488b688e36f",
        },
    },
}

It also should be renamed with the .zon extension, and furthermore, [package] should be renamed to [project] to match the correct terminology.

Perhaps the . in Zig language itself should be removed, and then it could be like this:

{
    .name = "libffmpeg",
    .version = "5.1.2",
    .dependencies = {
        .libz = {
            .url = "https://github.com/andrewrk/libz/archive/f0e53cc2391741034b144a2c2076ed8a9937b29b.tar.gz",
            .hash = "c9b30cffc40999d2c078ff350cbcee642970a224fe123c756d0892f876cf1aae",
        },
        .libmp3lame = {
            .url = "https://github.com/andrewrk/libmp3lame/archive/497568e670bfeb14ab6ef47fb6459a2251358e43.tar.gz",
            .hash = "9ba4f49895b174a3f918d489238acbc146bd393575062b2e3be33488b688e36f",
        },
    },
}

which is pretty unoffensive if you ask me. Lots of people hate that dot anyway. I don't know if the parsing would be problematic or not though, would need to investigate.

See competing proposal:

• proposal: make the build.zig.ini file a strict subset of TOML #14289

[ghost]

Ok, this isn't quite as bad as I was expecting. Though of course we'd have to provide a reference parser to be taken seriously, and if we write that in Zig, we'd have to make sure it itself has no dependencies.

Also I think removing the dot from dotbrace really would cause parsing problems. Is {} an empty tuple or a void constant? Is { key = "value" } a struct-like tuple or an assignment?

[deflock]

Maybe I missed something but what is the main profit of using this own DSL (not supported by any tool)?

In yaml:

name: libffmpeg
version: 5.1.2
dependencies:
  libz:
    url: https://github.com/andrewrk/libz/archive/f0e53cc2391741034b144a2c2076ed8a9937b29b.tar.gz
    hash: c9b30cffc40999d2c078ff350cbcee642970a224fe123c756d0892f876cf1aae
  libmp3lame:
    url: https://github.com/andrewrk/libmp3lame/archive/497568e670bfeb14ab6ef47fb6459a2251358e43.tar.gz
    hash: 9ba4f49895b174a3f918d489238acbc146bd393575062b2e3be33488b688e36f

[ghost]

@deflock In short, no existing tool satisfies all our requirements. For more detail, see the linked PR.

[andrewrk]

YAML is too complicated and has unacceptable footguns.

[deflock]

@EleanorNB I've scrolled one more time and I see why json/csv do not satisfy, s-expressions in imperative language is a bit weird :) So remaining candidates are toml, yaml, own format. I see only one profit of using own format - if it will be used directly within build.zig so you don't need additional file for declaring dependencies, but it's not possible bc of declarative reasons.

[batiati]

[andrewrk]

@deflock that's a nice point in favor of this proposal that I hadn't considered - similar to how in node.js you can require("foo.json"), it could make sense to support @import("foo.zon") in Zig which would give you comptime access to the data.

[andrewrk]

Another point in favor of this proposal over the TOML one: #14291. Note how line diffs against mirrors field (an array) would be annoying in the TOML case due to both elements being on the same line.

[kuon]

I think this is much better than TOML subset.

Upsides:

• can be @import as it is valid zig
• no new format that is TOML but not really
• allow for reuse of editor plugins
• could itself support @import, like this

.{
            .url = "https://github.com/andrewrk/libz/archive/f0e53cc2391741034b144a2c2076ed8a9937b29b.tar.gz",
            .hash = "c9b30cffc40999d2c078ff350cbcee642970a224fe123c756d0892f876cf1aae",
        }

.{
    .name = "libffmpeg",
    .version = "5.1.2",
    .dependencies = .{
        .libz = @import("libz.lulz"),
        .libmp3lame = .{
            .url = "https://github.com/andrewrk/libmp3lame/archive/497568e670bfeb14ab6ef47fb6459a2251358e43.tar.gz",
            .hash = "9ba4f49895b174a3f918d489238acbc146bd393575062b2e3be33488b688e36f",
        },
    },
}

• perhaps a few functions could be whitelisted (but this needs to be investigated)
• can be extended without having to make the format or the parser more complicated
• build.zig could have a package function which would return a struct like this, this can be used to generate the .lulz file with zig code, which leads to the next point
• this would put some effort into an object notation, which is really a nice supporting tool for a language, it can provide tooling for things like configurations, object format (game state...), rust has ron. I think this would be better than working on a pseudo TOML parser. To say it more plainly, work put into that would be directly put into zig.

Downsides:

• a bit harder to write
• further from the left margin
• people don't like .{ but maybe that dot could be removed, even if I personally like it

Please, don't use YAML, every time I work with that, I lose my sanity over some weird thing

[thejoshwolfe]

why do we have this

    .dependencies = .{
        .libz = .{
            .url = "https://github.com/andrewrk/libz/archive/f0e53cc2391741034b144a2c2076ed8a9937b29b.tar.gz",
            .hash = "c9b30cffc40999d2c078ff350cbcee642970a224fe123c756d0892f876cf1aae",
        },

instead of this?

    .dependencies = .{
        .{
            .name = "libz",
            .url = "https://github.com/andrewrk/libz/archive/f0e53cc2391741034b144a2c2076ed8a9937b29b.tar.gz",
            .hash = "c9b30cffc40999d2c078ff350cbcee642970a224fe123c756d0892f876cf1aae",
        },

does the former actually work in Zig if there's no struct with a libz: Dependency field defined in it? We don't have hashmap literals in Zig, right?

I would also like to vote against the name "lulz", as that word is often associated with mob harassment and racism.

[andrewrk]

I would also like to vote against the name "lulz", as that word is often associated with mob harassment and racism.

Alright fine, I changed the proposal to the much more conservative and obvious choice of .zon.

[andrewrk]

Alright, enough deliberation. Moving forward with this one; rejecting #14289.

[suhr]

This reminds me of https://github.com/ron-rs/ron.

[ikskuh]

I don't know if the parsing would be problematic or not though, would need to investigate.

That wouldn't be a problem as ZON is a data format, and we only need .{} (a struct/tuple) to be distinguished from a {} (block) in Zig, as we can execute code that might break data from the block.

We don't need to execute that here. But imho, for the sake of consistency, i'd prefer .{ over {, so i can copy-paste the code into Zig.

@thejoshwolfe: I actually prefer .libz = .{ over .{ .name = "libz", as the key syntax implies that the names have to be unique, due to fields have to be unique. The second syntax doesn't convey that information.

[likern]

I proposed #14265 (comment), this proposal alignes to my needs, more or less idea I envisioned. I upvote for them.

The only thing I dislike is build.zig.zon. If we already have custom format called Zig Object Notation just build.zon is better.

build.zig and build.zon ❤️ perfect match.

It looks and feels very much like improved JSON format. Very structered and understandable, but with comments.

[cryptocode]

Zon of a gun, this is brilliant ❤️

Will comments be // ?

[17dec]

Another point in favor of this proposal over the TOML one: #14291. Note how line diffs against mirrors field (an array) would be annoying in the TOML case due to both elements being on the same line.

With the ini format you could have duplicate url keys though, which would achieve the same.

@thejoshwolfe: I actually prefer .libz = .{ over .{ .name = "libz", as the key syntax implies that the names have to be unique, due to fields have to be unique. The second syntax doesn't convey that information.

While I agree that the syntax is nicer, I also believe that if a Zig data format is chosen, it makes sense that the schema can be described by a Zig type and that the data can be loaded directly into said type. Using the key syntax to describe a dynamic map kinda loses both of those properties

[wooster0]

At first I thought this was a new format with a subset of features of normal Zig but as far as I understand it it's just a regular Zig file with a single struct expression that can be imported, which I like very much.
But why do we have to call it build.zig.zon or build.zon? It looks kind of convoluted when we could just call it packages.zig? This could be the standard name for the file containing a struct expression with the packages listed.
build.zig.zon or more recently build.zon indicates to me it is a new separate format (Zig Object Notation) where I can't expect all features from Zig to work, even though that's not the case because it is in fact a normal Zig file (or did I get that wrong?).

build.zon sounds like build.zig; a file that builds my project, but written in "Zig Object Notation"...? Seems rather confusing.

IMO build.zig and packages.zig would be a much clearer and simpler naming.

---

If however this is actually supposed to be a new format with only a subset of features of normal Zig, why? Why do we need to create a new format when a normal Zig file could also contain this struct expression listing the packages?
It seems like this way there will always be new requests to port features from normal Zig over to this format, duplicating a lot of work.

[kuon]

@silversquirl Yes you are right. But I mentioned this for completeness. I am also in favor of zon but we have to be sure that it is justified.

[wooster0]

Now that I think we have found a good reason for introducing this new file format Zig Object Notation and not using the compiler for this, we can probably talk more about the file naming.
As for the current zig.build.zon, I agree we should shorten it to just .zon. Currently to me it means it's a complement to build.zig.zon that is written in ZON, which makes sense, but I think we can improve this and make it clear what actually is in this file.

I suggest packages.zon which just makes more sense to me as it is not the Zig Build Object Notation or such but simply ZON meaning it can be used for multiple purposes.
So packages.zon means it lists our packages in an object notation called ZON.

I think that's better than status quo build.zig.zon which can be ambiguous. Does it mean it's build.zig but written in ZON? No. Is it a complement to build.zig? Yes, but what it contains is still not clear.
I think this stack-up of file extensions is prone to confusion.

So I think packages.zon would be the most obvious naming and I think it matches the new build system terminology proposed in #14307, too.
Also, it's one character shorter.

[tato]

An advantage of build.zig.zon is that the file would always show up next to build.zig in listings sorted by file name.

[likern]

package.zon for defining package and direct (top-level) dependencies, package.lock.zon for fixed transitive (non-direct) dependencies, generated and maintained by package manager only.

Let's reuse knowledge of other package managers (npm as most advanced) which already faced with all kind of subtle bugs and errors.

[silversquirl]

@likern we shouldn't blindly copy other package managers. Looking at their decisions and why they were made is a good idea, but package managers like cargo, npm, etc. were not designed for Zig and differ quite a lot from Zig's package manager.

[michaelbartnett]

Possibly cursed question:

If .zon has the leading .{, should that imply that it could support a leading type annotation? Maybe in a .zton file? (Zig Typed Object Notation). Nice implications for if you need to do polymorphic serialization (thinking about use cases totally unrelated to packaging).

[silversquirl]

@michaelbartnett I think for that you'd use a union-style .{ .foo = .{ ... } } or .{ .bar = .{ ... } }

[tpmoney]

@likern we shouldn't blindly copy other package managers. Looking at their decisions and why they were made is a good idea, but package managers like cargo, npm, etc. were not designed for Zig and differ quite a lot from Zig's package manager.

Probably most similar in design to what Zig attempts to do (both with the build.zig file and the proposal for .zon to be a zig-ish (if not actually valid zig) is the Swift Package Manager might be some interesting ideas in there.

[lin72h]

@tpmoney I feel the same way about the Swift Package Manager format, and Erlang's configuration file also has very similar design, which is a subset of erlang syntax

[likern]

With all that functionality sooner or later we might need some if else branches, different sources for packages (with priority) and more. Seems sooner or later we will need full language support and .zon will not be enough.

I think like for 95-99% of packages .zon is Ok, but for that 1% we need ad-hoc solution with full language support (running code in 100% isolated light and fast VM).

[silversquirl]

@likern The whole point of zon is to not execute arbitrary code. See #14290 (comment) and #14290 (comment)

[expikr]

What about “Zig Attribute Grammar”?