Rule which will always rebuild

[timbertson]

I'm new to jbuilder so apologies if I've missed something obvious.

I've got a generated version.ml which I want to include in my library. I've added:

	(modules (:standard version))

to my (library ...) stanza to get the dependency, and then:

(rule (
	(targets (version.ml))
	(action (run gup -u version.ml))
))

However, this only runs the action if the file is missing - it won't know when to re-run it. I assume the intention is to specify (deps), but I can't do that (the dependencies aren't statically known), nor would I want to duplicate them since they're already specifid in the gup build rule.

Is it possible to specify that this rule should always run?

I understand that specifying deps accurately in jbuilder might make things faster, but in this case gup itself tracks dependencies accurately and finishes in a tenth of a second when there's nothing to do, so I'd be quite happy to pay that cost. I guess the other issue might be that it causes "everything depending on version.ml" to get rebuilt, even if the contents of version.ml don't change.

[lpw25]

I understand that specifying deps accurately in jbuilder might make things faster

Note that this kind of thing doesn't just make things slower, it bakes in a notion of "always". This is a problem if jbuilder went on to support proper reactive builds -- i.e. using things like inotify or polling to rebuild whenever something changes. In that situation it is not clear what "always rebuild" means.

Really you want to accurately specify all dependencies. Is there no way you can convince this gup tool to tell you what it wants to read to make version.ml -- or even what it already read when making version.ml?

I guess the other issue might be that it causes "everything depending on version.ml" to get rebuilt, even if the contents of version.ml don't change.

This shouldn't happen since jbuilder will take a digest of version.ml and use that to decide if things need rebuilding.

[timbertson]

Ah, that's a fair point.

Really you want to accurately specify all dependencies. Is there no way you can convince this gup tool to tell you what it wants to read to make version.ml

In this case the dependencies are dynamic. Before it's been built, gup knows it must need building. When it first builds it, the build script tracks the dependencies that it used. And then if they change, the builder is run again (at which point it may record different metadata).

The build script is here:
https://github.com/timbertson/passe/blob/30e7992aee9f399d2ebd8bdcdaf54432c718372d/src/common/version.ml.gup

The dependency logic is:

• depend on .git/HEAD
• if the contents of that file are a sha, we're done (at least in terms of deps)
• if the contents of that file match refs/heads/*, then depend on that file too

It's not a big set of dependencies, but it is dynamic, so I was curious what my options were with jbuilder. I thought I'd have to recursively depend on all of .git (which is a bit lame) until I reread the script and realised I could get away with depending on HEAD and refs/heads/*. It's inaccurate, but it'll do.

-- or even what it already read when making version.ml?

That sounds like exactly what I want! If I can get a list of the files that my (now-built target) last depended on, how could I get jbuilder to use it? Can I use :include for deps?

[jberdine]

From the perspective of fully reactive builds, it may be that a missing concept is that of a rule to generate a target which is executed only if there is some dependency on that target and that other dependent target needs to be rebuilt for some reason. That is, a rule that does not itself trigger rebuilds, but is executed if a rebuild is triggered for some other reason.

…

On Sep 14, 2017, at 3:47 PM, Leo White ***@***.***> wrote: I understand that specifying deps accurately in jbuilder might make things faster Note that this kind of thing doesn't just make things slower, it bakes in a notion of "always". This is a problem if jbuilder went on to support proper reactive builds -- i.e. using things like inotify or polling to rebuild whenever something changes. In that situation it is not clear what "always rebuild" means. Really you want to accurately specify all dependencies. Is there no way you can convince this gup tool to tell you what it wants to read to make version.ml -- or even what it already read when making version.ml? I guess the other issue might be that it causes "everything depending on version.ml" to get rebuilt, even if the contents of version.ml don't change. This shouldn't happen since jbuilder will take a digest of version.ml and use that to decide if things need rebuilding. — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#255 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ABZhztu6IMHwGmpOkrjWZLOHNZr3n7Xpks5siTyZgaJpZM4PXgNE>.

[timbertson]

I'd be quite happy with that suggestion, @jberdine.

For what it's worth I found another issue with this same use case - my version.ml.gup script (linked above) can be executed from outside a git repo (e.g. a downloaded tarball), in which case it sets commit to None in the generated module.

My jbuilder rule which depends on .git/HEAD and .git/refs/heads/* doesn't hold up in this scenario, as jbuilder doesn't appear to support optional dependencies and fails with "File unavailable: .git/HEAD"

I tried to trick it into depending on a possibly-empty-set via (glob_files ${SCOPE_ROOT}/../../.git/HEAD*), but then it failed with "Description: Build.get_glob_result_exn: got unevaluated"

I've created an issue for this in #298, but I thought I'd mention it here since it breaks my current workaround for this issue.

[timbertson]

A couple of thoughts about how this could be implemented. If you could depend on a magic token or singleton stanza (perhaps :world), where the behaviour is either:

1 - always dirty:

• :world is always considered dirty when considering what to build (i.e. on first run, and whenever a build is triggered for any other reason)
• it's impossible to trigger :world becoming dirty via file modification etc, because it doesn't map to a real file

2 - dirty on first run:

• :world is considered dirty when getting the initial set of stuff to build
• after that (on polling builds), :world has no effect (never dirty)

Honestly I'd be fine with either of these options.

[ghost]

One idea for polling builds would be to allow to connect arbitrary external sources of events to jbuilder. For instance jbuilder could start a command that would stay alive and send updates through stdout. This could be used to watch arbitrary resources.

In the meantime, in #602 I added a variable ${build-number} that expands to the number of builds started. When used in an action, this will cause the rule to be re-executed on every run. If you can't add the build number to the command line, you can go through a file as follow:

(rule (with-stdout-to build-number (echo ${build-number})))

(rule
 ((deps (build-number))
  (action (...))))