Fix out-of-tree build of docs, fix #29757. #29760

fredrikekre · 2018-10-22T10:04:09Z

No description provided.

musm · 2018-10-22T12:22:30Z

Great, thanks

musm · 2018-10-22T15:39:13Z

Hmm, I checked out this PR, and this is the output:

Building HTML documentation.
  Updating registry at `C:\Users\Mus\work\julia\doc\deps\registries\General`
  Updating git-repo `https://github.com/JuliaRegistries/General.git`
Documenter: setting up build directory.
Documenter: expanding markdown templates.
Documenter: building cross-references.
Documenter: running document checks.
 !! Skipped doctesting.
 > checking footnote links.
Documenter: populating indices.
Documenter: rendering document.
Build finished. The HTML pages are in _build/html.
make[1]: Leaving directory '/cygdrive/c/Users/Mus/work/julia/build-win64-cyg/doc'
make: Leaving directory '/cygdrive/c/Users/Mus/work/julia/build-win64-cyg'

yet no _build/html directory exists in the path '/cygdrive/c/Users/Mus/work/julia/build-win64-cyg'

vchuravy · 2018-10-22T15:51:12Z

It should be in doc/_build/html/. On an OOT build under Linux this PR works.

Can you check /cygdrive/c/Users/Mus/work/julia/build-win64-cyg/doc/_build/html?

musm · 2018-10-22T15:54:35Z

Yes it exists in that directory.

The (primary and related) issue is that an OOT binary-dist build fails since it searches the OOT directory for _build/html, which does not exist since it was not copied over during the build process.

Please see the error message below

# Copy documentation
cp -R -L /cygdrive/c/Users/Mus/work/julia/build-win64-cyg/doc/_build/html /cygdrive/c/Users/Mus/work/julia/build-win64-cyg/julia-2d28c88deb/share/doc/julia/
cp: cannot stat '/cygdrive/c/Users/Mus/work/julia/build-win64-cyg/doc/_build/html': No such file or directory

vchuravy · 2018-10-22T16:07:15Z

Wait this confuses me. After this PR you should have a directory /cygdrive/c/Users/Mus/work/julia/build-win64-cyg/doc/_build/html that contains the build documentation. So the cp -R -L command should succeed now. Since it looks exactly at that path.

is that an OOT binary-dist build fails since it searches the OOT directory for _build/html, which does not exist since it was not copied over during the build process.

Right it should be looking for doc/_build/html, which it seems to do?

musm · 2018-10-22T16:07:41Z

I think I have a fix inbound

musm · 2018-10-22T16:12:05Z

Wait this confuses me. After this PR you should have a directory /cygdrive/c/Users/Mus/work/julia/build-win64-cyg/doc/_build/html that contains the build documentation. So the cp -R -L command should succeed now. Since it looks exactly at that path.

No build-win64-cyg/doc/_build/html does not exist, only /doc/_build/html (root directory) exists, which is why the cp -R -L command fails

musm · 2018-10-22T17:37:40Z

@fredrikekre BTW I confirmed that make.jl does pick up the correct buildroot directory, yet the make.jl script fails to copy over the files

fredrikekre · 2018-10-23T11:39:36Z

We apparently get the same error on AV with this PR:

# Copy documentation
cp -R -L /cygdrive/c/projects/julia/doc/_build/html /cygdrive/c/projects/julia/julia-19a0f204e8/share/doc/julia/
cp: cannot stat '/cygdrive/c/projects/julia/doc/_build/html': No such file or directory
make: *** [Makefile:333: install] Error 1

fredrikekre · 2018-10-23T13:13:21Z

So from the output:

[00:20:58] buildroot = "/cygdrive/c/projects/julia"

[...]

[00:22:49] readdir(joinpath(buildroot, "doc", "_build")) = ["html"] # after doc-build

[...]

[00:33:40] # Copy documentation
[00:33:40] cp -R -L /cygdrive/c/projects/julia/doc/_build/html /cygdrive/c/projects/julia/julia-55396b96b7/share/doc/julia/
[00:33:40] cp: cannot stat '/cygdrive/c/projects/julia/doc/_build/html': No such file or directory

so is the directory getting wiped somewhere?

musm · 2018-10-23T14:18:53Z

@fredrikekre I ran a clean build and monitored the output of the julia/builddir/doc folder and I saw no folders nor file being created.

mortenpi · 2018-10-25T01:17:30Z

I was able to reproduce this.

The problem is that BUILDROOT is an absolute path (e.g. BUILDROOT=/home/user/julia/build-win32). In Cygwin the root (/), from the Windows file system point of view, is something like C:\cygwin64\, and the BUILDROOT actually points to C:\cygwin64\home\user\julia\build-win32.

However, Julia interprets BUILDROOT to be an absolute path relative to C:\, i.e. C:\home\user\julia\build-win32, and that's where the docs get written. I suspect this is because it is a cross-compilation build -- Julia doesn't know it's being run in Cygwin? It's using the Windows, not Cygwin filesystem APIs?

So @musm, you should also have a C:\home\ directory, and that's where the docs went 😆

Don't know how to fix this though. Should we modify the BUILDROOT we pass to julia when cross-compiling? (cygpath -aw $BUILDROOT would give the absolute Windows path.)

vchuravy · 2018-10-25T01:22:49Z

I believe there is cygpath_w or something similar to do this conversion

mortenpi · 2018-10-25T01:26:20Z

But we'd need to check it's a cross-compilation build, right? Because otherwise we would break normal Cygwin builds?

FWIW, it seems that pwd() is set to $BUILDROOT/doc, so we could use that maybe, without having to interact with the BUILDROOT environment variable at all.

musm · 2018-10-25T18:46:11Z

However, Julia interprets BUILDROOT to be an absolute path relative to C:, i.e. C:\home\user\julia\build-win32, and that's where the docs get written. I suspect this is because it is a cross-compilation build -- Julia doesn't know it's being run in Cygwin? It's using the Windows, not Cygwin filesystem APIs?

This is the crux of the problem

musm · 2018-10-25T19:41:18Z

I updated my PR #29764 with a different fix

mortenpi

I think the keyword approach is the best here.

musm · 2018-10-26T15:07:23Z

Much better with now without the env variable. Can we get this backported since it prevents Cygwin builds for OOT builds?

(cherry picked from commit ffaf849)

fredrikekre added the docsystem The documentation building system label Oct 22, 2018

vchuravy approved these changes Oct 22, 2018

View reviewed changes

KristofferC mentioned this pull request Oct 22, 2018

Fix copying of documentation #29764

Closed

fredrikekre force-pushed the fe/doc-out-of-tree branch from 2d28c88 to 7622c55 Compare October 23, 2018 07:30

fredrikekre force-pushed the fe/doc-out-of-tree branch from fffbcf9 to d243295 Compare October 23, 2018 12:35

fredrikekre force-pushed the fe/doc-out-of-tree branch from d243295 to f58e461 Compare October 25, 2018 11:58

fredrikekre force-pushed the fe/doc-out-of-tree branch from f58e461 to fd1a419 Compare October 25, 2018 20:43

Fix out-of-tree build of docs, fix #29757.

e17e4d3

fredrikekre force-pushed the fe/doc-out-of-tree branch from fd1a419 to e17e4d3 Compare October 25, 2018 20:44

mortenpi approved these changes Oct 25, 2018

View reviewed changes

fredrikekre merged commit ffaf849 into master Oct 25, 2018

fredrikekre deleted the fe/doc-out-of-tree branch October 25, 2018 23:03

fredrikekre added the backport pending 1.0 label Oct 26, 2018

musm mentioned this pull request Oct 28, 2018

make binary-dist fails for out of tree builds (Cygwin) #29750

Closed

KristofferC pushed a commit that referenced this pull request Oct 29, 2018

Fix out-of-tree build of docs, fix #29757. (#29760)

c207be0

(cherry picked from commit ffaf849)

KristofferC mentioned this pull request Oct 29, 2018

WIP: Backports for 1.0.2 #29444

Merged

KristofferC added the bugfix This change fixes an existing bug label Oct 29, 2018

KristofferC pushed a commit that referenced this pull request Nov 2, 2018

Fix out-of-tree build of docs, fix #29757. (#29760)

1e298f7

(cherry picked from commit ffaf849)

KristofferC removed the backport pending 1.0 label Nov 12, 2018

KristofferC pushed a commit that referenced this pull request Feb 11, 2019

Fix out-of-tree build of docs, fix #29757. (#29760)

49af631

(cherry picked from commit ffaf849)

KristofferC pushed a commit that referenced this pull request Feb 20, 2020

Fix out-of-tree build of docs, fix #29757. (#29760)

5b4d0d3

(cherry picked from commit ffaf849)

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Fix out-of-tree build of docs, fix #29757. #29760

Fix out-of-tree build of docs, fix #29757. #29760

fredrikekre commented Oct 22, 2018

musm commented Oct 22, 2018

musm commented Oct 22, 2018

vchuravy commented Oct 22, 2018

musm commented Oct 22, 2018

vchuravy commented Oct 22, 2018

musm commented Oct 22, 2018

musm commented Oct 22, 2018 •

edited

Loading

musm commented Oct 22, 2018

fredrikekre commented Oct 23, 2018

fredrikekre commented Oct 23, 2018

musm commented Oct 23, 2018 •

edited

Loading

mortenpi commented Oct 25, 2018 •

edited

Loading

vchuravy commented Oct 25, 2018

mortenpi commented Oct 25, 2018 •

edited

Loading

musm commented Oct 25, 2018

musm commented Oct 25, 2018

mortenpi left a comment

musm commented Oct 26, 2018

Fix out-of-tree build of docs, fix #29757. #29760

Fix out-of-tree build of docs, fix #29757. #29760

Conversation

fredrikekre commented Oct 22, 2018

musm commented Oct 22, 2018

musm commented Oct 22, 2018

vchuravy commented Oct 22, 2018

musm commented Oct 22, 2018

vchuravy commented Oct 22, 2018

musm commented Oct 22, 2018

musm commented Oct 22, 2018 • edited Loading

musm commented Oct 22, 2018

fredrikekre commented Oct 23, 2018

fredrikekre commented Oct 23, 2018

musm commented Oct 23, 2018 • edited Loading

mortenpi commented Oct 25, 2018 • edited Loading

vchuravy commented Oct 25, 2018

mortenpi commented Oct 25, 2018 • edited Loading

musm commented Oct 25, 2018

musm commented Oct 25, 2018

mortenpi left a comment

Choose a reason for hiding this comment

musm commented Oct 26, 2018

musm commented Oct 22, 2018 •

edited

Loading

musm commented Oct 23, 2018 •

edited

Loading

mortenpi commented Oct 25, 2018 •

edited

Loading

mortenpi commented Oct 25, 2018 •

edited

Loading