Expanding on --import-paths for other BuildSettings fields

[Abscissa]

I'd like to expand on the recent "dub describe --[string-]import-paths" to support some of the other fields in BuildSettings (such as libs and versions), and also optionally allow these to be output pre-formatted for a particular compiler (such as DMD-style: "-Ipath1 -Ipath2").

But before getting started, I wanted to get feedback regarding the CLI interface for it. I'm wondering whether following the example set by "--[string-]import-paths" by adding "--versions", "--debug-versions", "--libs", "--pre-generate-commands", etc might be a bit cluttered. Should it be changed to something like "--build-setting=import-paths"? Or is current approach fine?

[w0rp]

I think I'd be happy with just about any approach which works and lets you get all of the values out in the right formats. When I added the path commands, I just wanted something so I could get the paths out without having to rely on some other program being installed, to make it easy to add the import paths to any syntax checker which uses dub already.

[MartinNowak]

Mmh, what's wrong with processing the json output? After all you don't want 10 invocations of dub anyhow. What might make sense is something akin to pkg-config --cflags --libs, e.g. dub describe --dflags --libs, but I'm not sure how well that works with dependencies.

[Abscissa]

Mmh, what's wrong with processing the json output?

It's a fairly fundamental package manager feature that (realistically) is necessary for using dub's package management functionality while opting out of its build system. Because of that, an external tool (while probably technically feasible) is really a bit too unwieldy and inconvenient.

After all you don't want 10 invocations of dub anyhow. What might make sense is something akin to pkg-config --cflags --libs, e.g. dub describe --dflags --libs,

I'm not all that familiar with pkg-config (I really should be though...), but I assume you're talking about outputting multiple types of flags (compiler flags and library filepaths) with the same invocation. If so, yes, I was thinking of allowing the same thing, like you say, to avoid extra invocations.

For outputting the values in a compiler-formatted style that shouldn't be too much of an issue. I'm not entirely certain what to do about the current formatting of "list each value, compiler-independent format, on a separate line". Either just dump everything and let the user fend for themself at sorting them out. Or separate each section with a blank line. Or just ONLY allow dub describe --dflags --libs combinations of data when outputting in a compiler-specific command line format.

but I'm not sure how well that works with dependencies.

I'm not sure what you mean by this. What potential issues do you see?

[Abscissa]

but I'm not sure how well that works with dependencies.

I'm not sure what you mean by this. What potential issues do you see?

If you mean deciding whether to include in the output the info for dependencies versus only displaying for root project only, then the more I think about this whole idea, the more convinced I am the user should be explicitly choosing "recursive / with dependencies" vs "non-recursive / root project only" each time for each individual setting.

So something like this (I'm not sure what to actually call the flags though):

# Get the args DUB would normally pass to DMD
dub describe --format=dmd --project-data=dflags  --project-data=version \
    --recursive-data=import-paths --recursive-data=import-files ...etc...

# List all packages involved in this project
dub describe --format=list --recursive-data=target-name

# Execute this project's pre-build step (--format=list is default)
`dub describe --project-data=pre-build-commands`

# Build project with RDMD, using optional "short" style:
rdmd --build-only `dub describe --format=dmd -P=lflags -P=version \
    -R=import-paths -R=libs -P=main-source-file`

# A few things would have to be an error:
dub describe --format=dmd --project-data=pre-build-commands
ERROR: Format "dmd" doesn't support the data "pre-build-commands"

Naturally some more common or potentially-fiddly uses would have a shortcut, like dub describe --rdmd-flags

[Abscissa]

Actually, I think this way would be better, because then (I think) the requested ordering could be preserved (and "package" is probably more accurate at implying non-recursive than "project"):

# Get (some of) the args DUB would normally pass to DMD
dub describe --format=dmd --data=package-dflags --data=package-versions \
    --data=recursive-import-paths --data=recursive-import-files ...etc...

[MartinNowak]

You can already specify the compiler using dub describe --compiler=ldc, so the format stuff shouldn't be needed.
Note that describe basically outputs the configurations of a project and it's dependencies, not how that project is build using a specific compiler.

[MartinNowak]

Mmh, what's wrong with processing the json output?

It's a fairly fundamental package manager feature that (realistically) is necessary for using dub's package management functionality while opting out of its build system. Because of that, an external tool (while probably technically feasible) is really a bit too unwieldy and inconvenient.

See, if you want to write an alternative build tool for dub, than you can simply parse the json output of dub describe. That's by no means more complicated than passing 20 arguments and parsing the output of that.

[Abscissa]

You can already specify the compiler using dub describe --compiler=ldc, so the format stuff shouldn't be needed.

If there's a way to determine "the user did not specify a --compiler=" as opposed to simply getting dub's default choice of compiler (I'm not sure offhand if there is. Is there?), then yes, I suppose that could work.

The downsides are that it would preclude the ability to explicitly request the default of --import-paths's current list-style format (ie --format=list), or the ability to say something like --format=default-detected-compiler, but I don't know how important such things would actually be. I think a separate --format= would be cleaner, but if reusing --compiler is preferred then that's fine too.

See, if you want to write an alternative build tool for dub, than you can simply parse the json output of dub describe. That's by no means more complicated than passing 20 arguments and parsing the output of that.

Yea it really actually would be more involved. Besides, I'm working towards some sort of dub describe --rdmd-flags (or some such) shortcuts. The --data=... switch would be more of a manual way to get whatever specific data you need for whatever reason (example: w0rp's use-case for the import paths) without having go parsing anything at all, or looking up dub describe's output schema and constructing an appropriate query then fixing relative paths and converting whatever to an actual compiler switches, etc...No, I really am very glad we have dub describe's JSON format, but it's a very, very low-level thing, and absolutely no real substitute for what I'm talking about here.

Additionally, this is realistically needed just to even use rdmd (let alone anything else) in a dub-distributed lib, and rdmd can hardly be considered an "alternative" build tool.

[Abscissa]

Solved via the new dub describe --data= feature, therefore closing.