Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[win] Fix source build without pre-built deps #45712

Draft
wants to merge 17 commits into
base: master
Choose a base branch
from

Conversation

inkydragon
Copy link
Member

@inkydragon inkydragon commented Jun 16, 2022

fix #45645, #45745

@inkydragon inkydragon changed the title Fix source build without pre-built deps [win] Fix source build without pre-built deps Jun 16, 2022
@gbaraldi gbaraldi added building Build system, or building Julia or its dependencies system:windows Affects only Windows external dependencies Involves LLVM, OpenBLAS, or other linked libraries labels Jun 16, 2022
deps/blastrampoline.mk Show resolved Hide resolved
deps/libgit2.mk Outdated Show resolved Hide resolved
deps/libssh2.mk Outdated Show resolved Hide resolved
@@ -5,6 +5,12 @@ ifneq ($(USE_BINARYBUILDER_P7ZIP),1)
$(SRCCACHE)/p7zip-$(P7ZIP_VER).tar.gz: | $(SRCCACHE)
$(JLDOWNLOAD) $@ https://github.com/jinfeihan57/p7zip/archive/refs/tags/v$(P7ZIP_VER).tar.gz

P7ZIP_BUILD_OPTS := $(MAKE_COMMON)
# Build with cygwin, not use cross build.
ifeq (,$(findstring CYGWIN,$(BUILD_OS)))
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Seems unlikely this should care about the BUILD_OS

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, it's a workaround for cygwin.
I'm not sure if there is a better solution.

It's incredibly frustrating to build p7zip on mingw, so instead we just redistribute 7z
https://github.com/JuliaPackaging/Yggdrasil/blob/b2e0c2c5851b71230fb7170f74d773393ce37f80/P/p7zip/build_tarballs.jl#L21

In my case, on Windows, build p7zip with gcc on cygwin, works well.
But when cross build p7zip with x86_64-w64-mingw32-gcc it fails.

@metab0t
Copy link
Contributor

metab0t commented Jun 17, 2022

Some suggestions:

  1. According to

    julia/Make.inc

    Lines 545 to 550 in 3df39c0

    CC_BASE := $(shell echo $(CC) | cut -d' ' -f1)
    CC_ARG := $(shell echo $(CC) | cut -s -d' ' -f2-)
    CXX_BASE := $(shell echo $(CXX) | cut -d' ' -f1)
    CXX_ARG := $(shell echo $(CXX) | cut -s -d' ' -f2-)
    FC_BASE := $(shell echo $(FC) | cut -d' ' -f1)
    FC_ARG := $(shell echo $(FC) | cut -s -d' ' -f2-)

    Maybe we should use
CC=$(CC_BASE) CFLAGS=$(CC_ARG) CXX=$(CXX_BASE) CXXFLAGS=$(CXX_ARG) FC=$(FC_BASE) FCFLAGS=$(FC_ARG)

for Makefile-based dep.

  1. For p7zip, we can set USE_BINARYBUILDER_P7ZIP=1 when cross compiling to Windows. It is an external tool.

  2. It seems that LLVM_CMAKE in llvm.mk should be initialized with $(CMAKE_COMMON)?

  3. We should add CC_FOR_BUILD=$(HOSTCC) before GMP configure. It is crucial for Linux cross compiler.
    Ref:
    https://gmplib.org/manual/Build-Options
    https://aur.archlinux.org/cgit/aur.git/tree/PKGBUILD?h=mingw-w64-gmp#n22

@inkydragon
Copy link
Member Author

inkydragon commented Jun 17, 2022

  1. Maybe we should use ... for Makefile-based dep.

MAKE_COMMON := DESTDIR="" prefix=$(build_prefix) bindir=$(build_depsbindir) libdir=$(build_libdir) shlibdir=$(build_shlibdir) libexecdir=$(build_libexecdir) datarootdir=$(build_datarootdir) includedir=$(build_includedir) sysconfdir=$(build_sysconfdir) O=

Yes, we can add those flags to MAKE_COMMON

  1. For p7zip, we can set USE_BINARYBUILDER_P7ZIP=1 when cross compiling to Windows. It is an external tool.

But it will break offline build steps.
It might be worth creating a new issue to discuss the 7z build.

  1. It seems that LLVM_CMAKE in llvm.mk should be initialized with $(CMAKE_COMMON)?

$(CMAKE) $(SRCCACHE)/$(LLVM_SRC_DIR)/llvm $(CMAKE_GENERATOR_COMMAND) $(CMAKE_COMMON) $(LLVM_CMAKE) \

Well hidden

  1. We should add CC_FOR_BUILD=$(HOSTCC) before GMP configure. It is crucial for Linux cross compiler.

Maybe in another pr?

OK, done. And I'm testing cross build on WSL2.

@inkydragon
Copy link
Member Author

I would like to split this pr into a series of smaller pr's for easier review:

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
building Build system, or building Julia or its dependencies external dependencies Involves LLVM, OpenBLAS, or other linked libraries system:windows Affects only Windows
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Compile from source without pre-built deps failed
4 participants