Unable to use external dependency with recursive links

[lazdmx]

Original issue - bazel-contrib/rules_go#424

I've found that I couldn't use a (non-Bazel) dependency with recursive links in codebase because of error:

ERROR: infinite symlink expansion detected
[start of symlink chain]

There is a lot of CLI tools which do some useful work with files and they usually are part of build process. And authors of course write tests on cases with recursive links which are part of testdata stored in repo.

For instance there is a tool https://github.com/jteeuwen/go-bindata

So, if I have a WORKSPACE like:

...
new_go_repository(
    name = "com_github_jteeuwen_go_bindata",
    commit = "a0ff2567cfb70903282db057e799fd826784d41d",
    importpath = "github.com/jteeuwen/go-bindata",
)

bind(
    name = "go-bindata",
    actual = "@com_github_jteeuwen_go_bindata//go-bindata",
)
...

and a BUILD file

...
genrule(
    name = "assets",
    srcs = [
		"//<patterns>",
    ],
    outs = ["assets.go"],
    cmd = "$(location //external:go-bindata) -o $@ -nocompress -pkg main -prefix $(@D) $(SRCS)",
    tools = [
        "//external:go-bindata"
    ]
)
...

try to build :assets will ends up with an error like

ERROR: /private/var/tmp/_bazel_dimka/360acecf2580728c6a28e523b52ec564/external/com_github_jteeuwen_go_bindata/BUILD:25:12: error globbing [testdata/**]: /private/var/tmp/_bazel_dimka/360acecf2580728c6a28e523b52ec564/external/com_github_jteeuwen_go_bindata/testdata/symlinkRecursiveParent/symlinkTarget/symlinkTarget/symlinkTarget/symlinkTarget/symlinkTarget/symlinkTarget/symlinkTarget/symlinkTarget/symlinkTarget/symlinkTarget/symlinkTarget/symlinkTarget/symlinkTarget/symlinkTarget/symlinkTarget/symlinkTarget/symlinkTarget/symlinkTarget/symlinkTarget/symlinkTarget/symlinkTarget/symlinkTarget/symlinkTarget/symlinkTarget/symlinkTarget/symlinkTarget/symlinkTarget/symlinkTarget/symlinkTarget/symlinkTarget/symlinkTarget/symlinkTarget/symlinkTarget (Too many levels of symbolic links).

Real problem comes generated BUILD file (attempt to glob files in testdata glob(["testdata/**"]))

go_test(
    name = "go_default_test",
    srcs = ["convert_test.go"],
    data = glob(["testdata/**"]),
    library = ":go_default_library",
)

[damienmg]

Pulling in @haxorz who knows everything about why is that IIRC.

[ghost]

Another situation where this ruins things a bit: https://github.com/buildroot/buildroot uses recursive symlink to root dir to provide /usr in host toolchain it builds. As a result Bazel can't use Buildroot toolchains.

[kerrick-lyft]

I'm hitting this issue trying to use an external Buildroot repository. Is it possible to fix Bazel so that it can handle self-referencing symlinks (e.g. foo/bar/baz points to foo)?

[mboes]

The question for me is - what was the initial motivation for Bazel detecting recursive symlinks? They seem benign to me. Not sure why Bazel is defensively excluding them.

[dslomov]

Bazel does symlink expansion so that it can accurately track changes in files; therefore recusive symlinks pose a problem.
What is blocked by this? Why do codebases use external symlinks?

[mboes]

See https://github.com/tweag/rules_haskell/blob/master/WORKSPACE#L248-L268. In Bazel, @nixpkgs_nodejs is shorthand for @nixpkgs_nodejs//:nixpkgs_nodejs, which is unnecessarily verbose and redundant. If we have a nixpkgs_nodejs -> . symlink, then we can expose a directory full of content under the short name. You might argue, "well why don't you define a target called @nixpkgs_nodejs//:stuffin a newBUILD` file, that exposes the entire content of the nodejs binary distribution. I could, but AFAIU this breaks this line:

node_repositories(vendored_node = "@nixpkgs_nodejs")

The reason is that node_repositories() really wants the vendored_node argument to be a physical path for some reason, not the name of a target defined in a BUILD file.

[katre]

Closed for lack of activity. Please ping the reviewers to reopen if this is still a problem.