Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[PETSc] Add builder #845

Merged
merged 55 commits into from
Aug 20, 2020
Merged

[PETSc] Add builder #845

merged 55 commits into from
Aug 20, 2020

Conversation

jd-lara
Copy link
Contributor

@jd-lara jd-lara commented Apr 12, 2020

@ViralBShah this is the first attempt. It worked locally for Linux 64 bit. Probably will fail in windows and require some iterations to get it right.

Windows instructions
https://www.mcs.anl.gov/petsc/documentation/installation.html#windows

P/PETSc/build_tarballs.jl Outdated Show resolved Hide resolved
P/PETSc/build_tarballs.jl Show resolved Hide resolved
P/PETSc/build_tarballs.jl Outdated Show resolved Hide resolved
@giordano
Copy link
Member

Just by looking to the list of platforms that succeeded, my guess is that the build script is trying to execute a program that has built with the cross-compiler

@jd-lara
Copy link
Contributor Author

jd-lara commented Apr 12, 2020

Just by looking to the list of platforms that succeeded, I guess that the build script is trying to execute a program that has built with the cross-compiler

I am trying to figure out why they are failing, but the details are stored in a log file after running
./config locally the wizard is having some trouble with windows, and I can't get to the same place as the CI.

Do you have any recommendations to display the log file in the terminal of the different platforms if the config run doesn't succeed?

@jd-lara
Copy link
Contributor Author

jd-lara commented Apr 12, 2020

Just by looking to the list of platforms that succeeded, my guess is that the build script is trying to execute a program that has built with the cross-compiler

The issue is the use of SOWING which is mandatory apparently.

@ViralBShah
Copy link
Member

Do you have any recommendations to display the log file in the terminal of the different platforms if the config run doesn't succeed?

You can echo things, or cat files. One possibility is after the configure step to tail -n 200 config.log or some such thing so that it leaves that in the logs. If the configure itself fails, I am not sure if the next command will run.

There may be a way to not make the entire bash script exit on failure of an individual command. Perhaps the opposite of set -e.

@ViralBShah
Copy link
Member

ViralBShah commented Apr 12, 2020

It looks like SOWING produces the man pages. Is there a configure option to disable it or the building of documentation altogether, since we do not do anything with the man pages. It's odd that they make it mandatory.

@jd-lara
Copy link
Contributor Author

jd-lara commented Apr 12, 2020

It looks like SOWING produces the man pages. Is there a configure option to disable it or the building of documentation altogether, since we do not do anything with the man pages. It's odd that they make it mandatory.

It doesn't like it.

*******************************************************************************
         UNABLE to CONFIGURE with GIVEN OPTIONS    (see configure.log for details):
-------------------------------------------------------------------------------
Cannot use --with-sowing=0 if using Fortran (bindings) and git repository for PETSc
*******************************************************************************

@ViralBShah
Copy link
Member

What if we build with ArchiveSource instead of GitSource?

@jd-lara
Copy link
Contributor Author

jd-lara commented Apr 12, 2020

Some progress, in windows now is complaining because of this

-lopenblas64_ -lstdc++ -lgfortran -lgcc_s -lquadmath -lm
cannot be used with a shared library
Either run ./configure with --with-shared-libraries=0 or use a different BLAS/LAPACK library

@giordano Is there a fix or shall we remove the shared libraries in windows?

@jd-lara
Copy link
Contributor Author

jd-lara commented Apr 13, 2020

What if we build with ArchiveSource instead of GitSource?

This worked for me locally 🤞 this will work on the CI

P/PETSc/build_tarballs.jl Outdated Show resolved Hide resolved
P/PETSc/build_tarballs.jl Outdated Show resolved Hide resolved
@jd-lara
Copy link
Contributor Author

jd-lara commented Apr 13, 2020

@ViralBShah neither me or @giordano can reproduce the windows error locally to check at the logs. Any ideas?

@jd-lara
Copy link
Contributor Author

jd-lara commented Aug 8, 2020

@giordano and @simonbyrne ready to merge.

@jd-lara
Copy link
Contributor Author

jd-lara commented Aug 8, 2020

I can't reproduce the issue locally I get

image

@simonbyrne
Copy link
Contributor

The error on MacOS is:

ld: unknown option: -soname
clang-9: error: linker command failed with exit code 1 (use -v to see invocation)
make[2]: *** [SRC/CMakeFiles/superlu.dir/build.make:2787: SRC/libsuperlu.so.5.2.1] Error 1
make[1]: *** [CMakeFiles/Makefile2:990: SRC/CMakeFiles/superlu.dir/all] Error 2
make: *** [Makefile:144: all] Error 2

any ideas?

@simonbyrne
Copy link
Contributor

From what I can tell, the problem appears to be that the default linker function:
https://github.com/petsc/petsc/blob/v3.13.4/gmakefile#L16
should instead be set by the file generated by this:
https://github.com/petsc/petsc/blob/v3.13.4/config/PETSc/options/sharedLibraries.py#L73
but it seems like that file isn't being generated?

@giordano
Copy link
Member

It wouldn't be the first bug we find and fix in their build system...

@jd-lara
Copy link
Contributor Author

jd-lara commented Aug 12, 2020

I got this in i686-linux-musl-libgfortran5

[100%] Linking C shared library libsuperlu.so
cd /workspace/srcdir/petsc-3.13.4/i686-linux-musl/externalpackages/git.superlu/petsc-build/SRC && /usr/bin/cmake -E cmake_link_script CMakeFiles/superlu.dir/link.txt --verbose=1
/opt/bin/cc -fPIC -DUSE_VENDOR_BLAS -DPRNTlevel=0 -DDEBUGlevel=0 -fstack-protector -O3 -DAdd_ -fstack-protector -O3 -DAdd_  -shared -Wl,-soname,libsuperlu.so.5 -o libsuperlu.so.5.2.1 CMakeFiles/superlu.dir/superlu_timer.c.o CMakeFiles/superlu.dir/util.c.o CMakeFiles/superlu.dir/memory.c.o CMakeFiles/superlu.dir/get_perm_c.c.o CMakeFiles/superlu.dir/mmd.c.o CMakeFiles/superlu.dir/sp_coletree.c.o CMakeFiles/superlu.dir/sp_preorder.c.o CMakeFiles/superlu.dir/sp_ienv.c.o CMakeFiles/superlu.dir/relax_snode.c.o CMakeFiles/superlu.dir/heap_relax_snode.c.o CMakeFiles/superlu.dir/colamd.c.o CMakeFiles/superlu.dir/ilu_relax_snode.c.o CMakeFiles/superlu.dir/ilu_heap_relax_snode.c.o CMakeFiles/superlu.dir/mark_relax.c.o CMakeFiles/superlu.dir/mc64ad.c.o CMakeFiles/superlu.dir/qselect.c.o CMakeFiles/superlu.dir/input_error.c.o CMakeFiles/superlu.dir/dmach.c.o CMakeFiles/superlu.dir/smach.c.o CMakeFiles/superlu.dir/slacon2.c.o CMakeFiles/superlu.dir/sgssv.c.o CMakeFiles/superlu.dir/sgssvx.c.o CMakeFiles/superlu.dir/ssp_blas2.c.o CMakeFiles/superlu.dir/ssp_blas3.c.o CMakeFiles/superlu.dir/sgscon.c.o CMakeFiles/superlu.dir/slangs.c.o CMakeFiles/superlu.dir/sgsequ.c.o CMakeFiles/superlu.dir/slaqgs.c.o CMakeFiles/superlu.dir/spivotgrowth.c.o CMakeFiles/superlu.dir/sgsrfs.c.o CMakeFiles/superlu.dir/sgstrf.c.o CMakeFiles/superlu.dir/sgstrs.c.o CMakeFiles/superlu.dir/scopy_to_ucol.c.o CMakeFiles/superlu.dir/ssnode_dfs.c.o CMakeFiles/superlu.dir/ssnode_bmod.c.o CMakeFiles/superlu.dir/spanel_dfs.c.o CMakeFiles/superlu.dir/spanel_bmod.c.o CMakeFiles/superlu.dir/sreadhb.c.o CMakeFiles/superlu.dir/sreadrb.c.o CMakeFiles/superlu.dir/sreadtriple.c.o CMakeFiles/superlu.dir/sreadMM.c.o CMakeFiles/superlu.dir/scolumn_dfs.c.o CMakeFiles/superlu.dir/scolumn_bmod.c.o CMakeFiles/superlu.dir/spivotL.c.o CMakeFiles/superlu.dir/spruneL.c.o CMakeFiles/superlu.dir/smemory.c.o CMakeFiles/superlu.dir/sutil.c.o CMakeFiles/superlu.dir/smyblas2.c.o CMakeFiles/superlu.dir/sgsisx.c.o CMakeFiles/superlu.dir/sgsitrf.c.o CMakeFiles/superlu.dir/sldperm.c.o CMakeFiles/superlu.dir/ilu_sdrop_row.c.o CMakeFiles/superlu.dir/ilu_ssnode_dfs.c.o CMakeFiles/superlu.dir/ilu_scolumn_dfs.c.o CMakeFiles/superlu.dir/ilu_spanel_dfs.c.o CMakeFiles/superlu.dir/ilu_scopy_to_ucol.c.o CMakeFiles/superlu.dir/ilu_spivotL.c.o CMakeFiles/superlu.dir/sdiagonal.c.o CMakeFiles/superlu.dir/dlacon2.c.o CMakeFiles/superlu.dir/dgssv.c.o CMakeFiles/superlu.dir/dgssvx.c.o CMakeFiles/superlu.dir/dsp_blas2.c.o CMakeFiles/superlu.dir/dsp_blas3.c.o CMakeFiles/superlu.dir/dgscon.c.o CMakeFiles/superlu.dir/dlangs.c.o CMakeFiles/superlu.dir/dgsequ.c.o CMakeFiles/superlu.dir/dlaqgs.c.o CMakeFiles/superlu.dir/dpivotgrowth.c.o CMakeFiles/superlu.dir/dgsrfs.c.o CMakeFiles/superlu.dir/dgstrf.c.o CMakeFiles/superlu.dir/dgstrs.c.o CMakeFiles/superlu.dir/dcopy_to_ucol.c.o CMakeFiles/superlu.dir/dsnode_dfs.c.o CMakeFiles/superlu.dir/dsnode_bmod.c.o CMakeFiles/superlu.dir/dpanel_dfs.c.o CMakeFiles/superlu.dir/dpanel_bmod.c.o CMakeFiles/superlu.dir/dreadhb.c.o CMakeFiles/superlu.dir/dreadrb.c.o CMakeFiles/superlu.dir/dreadtriple.c.o CMakeFiles/superlu.dir/dreadMM.c.o CMakeFiles/superlu.dir/dcolumn_dfs.c.o CMakeFiles/superlu.dir/dcolumn_bmod.c.o CMakeFiles/superlu.dir/dpivotL.c.o CMakeFiles/superlu.dir/dpruneL.c.o CMakeFiles/superlu.dir/dmemory.c.o CMakeFiles/superlu.dir/dutil.c.o CMakeFiles/superlu.dir/dmyblas2.c.o CMakeFiles/superlu.dir/dgsisx.c.o CMakeFiles/superlu.dir/dgsitrf.c.o CMakeFiles/superlu.dir/dldperm.c.o CMakeFiles/superlu.dir/ilu_ddrop_row.c.o CMakeFiles/superlu.dir/ilu_dsnode_dfs.c.o CMakeFiles/superlu.dir/ilu_dcolumn_dfs.c.o CMakeFiles/superlu.dir/ilu_dpanel_dfs.c.o CMakeFiles/superlu.dir/ilu_dcopy_to_ucol.c.o CMakeFiles/superlu.dir/ilu_dpivotL.c.o CMakeFiles/superlu.dir/ddiagonal.c.o CMakeFiles/superlu.dir/clacon2.c.o CMakeFiles/superlu.dir/scsum1.c.o CMakeFiles/superlu.dir/icmax1.c.o CMakeFiles/superlu.dir/scomplex.c.o CMakeFiles/superlu.dir/cgssv.c.o CMakeFiles/superlu.dir/cgssvx.c.o CMakeFiles/superlu.dir/csp_blas2.c.o CMakeFiles/superlu.dir/csp_blas3.c.o CMakeFiles/superlu.dir/cgscon.c.o CMakeFiles/superlu.dir/clangs.c.o CMakeFiles/superlu.dir/cgsequ.c.o CMakeFiles/superlu.dir/claqgs.c.o CMakeFiles/superlu.dir/cpivotgrowth.c.o CMakeFiles/superlu.dir/cgsrfs.c.o CMakeFiles/superlu.dir/cgstrf.c.o CMakeFiles/superlu.dir/cgstrs.c.o CMakeFiles/superlu.dir/ccopy_to_ucol.c.o CMakeFiles/superlu.dir/csnode_dfs.c.o CMakeFiles/superlu.dir/csnode_bmod.c.o CMakeFiles/superlu.dir/cpanel_dfs.c.o CMakeFiles/superlu.dir/cpanel_bmod.c.o CMakeFiles/superlu.dir/creadhb.c.o CMakeFiles/superlu.dir/creadrb.c.o CMakeFiles/superlu.dir/creadtriple.c.o CMakeFiles/superlu.dir/creadMM.c.o CMakeFiles/superlu.dir/ccolumn_dfs.c.o CMakeFiles/superlu.dir/ccolumn_bmod.c.o CMakeFiles/superlu.dir/cpivotL.c.o CMakeFiles/superlu.dir/cpruneL.c.o CMakeFiles/superlu.dir/cmemory.c.o CMakeFiles/superlu.dir/cutil.c.o CMakeFiles/superlu.dir/cmyblas2.c.o CMakeFiles/superlu.dir/cgsisx.c.o CMakeFiles/superlu.dir/cgsitrf.c.o CMakeFiles/superlu.dir/cldperm.c.o CMakeFiles/superlu.dir/ilu_cdrop_row.c.o CMakeFiles/superlu.dir/ilu_csnode_dfs.c.o CMakeFiles/superlu.dir/ilu_ccolumn_dfs.c.o CMakeFiles/superlu.dir/ilu_cpanel_dfs.c.o CMakeFiles/superlu.dir/ilu_ccopy_to_ucol.c.o CMakeFiles/superlu.dir/ilu_cpivotL.c.o CMakeFiles/superlu.dir/cdiagonal.c.o CMakeFiles/superlu.dir/zlacon2.c.o CMakeFiles/superlu.dir/dzsum1.c.o CMakeFiles/superlu.dir/izmax1.c.o CMakeFiles/superlu.dir/dcomplex.c.o CMakeFiles/superlu.dir/zgssv.c.o CMakeFiles/superlu.dir/zgssvx.c.o CMakeFiles/superlu.dir/zsp_blas2.c.o CMakeFiles/superlu.dir/zsp_blas3.c.o CMakeFiles/superlu.dir/zgscon.c.o CMakeFiles/superlu.dir/zlangs.c.o CMakeFiles/superlu.dir/zgsequ.c.o CMakeFiles/superlu.dir/zlaqgs.c.o CMakeFiles/superlu.dir/zpivotgrowth.c.o CMakeFiles/superlu.dir/zgsrfs.c.o CMakeFiles/superlu.dir/zgstrf.c.o CMakeFiles/superlu.dir/zgstrs.c.o CMakeFiles/superlu.dir/zcopy_to_ucol.c.o CMakeFiles/superlu.dir/zsnode_dfs.c.o CMakeFiles/superlu.dir/zsnode_bmod.c.o CMakeFiles/superlu.dir/zpanel_dfs.c.o CMakeFiles/superlu.dir/zpanel_bmod.c.o CMakeFiles/superlu.dir/zreadhb.c.o CMakeFiles/superlu.dir/zreadrb.c.o CMakeFiles/superlu.dir/zreadtriple.c.o CMakeFiles/superlu.dir/zreadMM.c.o CMakeFiles/superlu.dir/zcolumn_dfs.c.o CMakeFiles/superlu.dir/zcolumn_bmod.c.o CMakeFiles/superlu.dir/zpivotL.c.o CMakeFiles/superlu.dir/zpruneL.c.o CMakeFiles/superlu.dir/zmemory.c.o CMakeFiles/superlu.dir/zutil.c.o CMakeFiles/superlu.dir/zmyblas2.c.o CMakeFiles/superlu.dir/zgsisx.c.o CMakeFiles/superlu.dir/zgsitrf.c.o CMakeFiles/superlu.dir/zldperm.c.o CMakeFiles/superlu.dir/ilu_zdrop_row.c.o CMakeFiles/superlu.dir/ilu_zsnode_dfs.c.o CMakeFiles/superlu.dir/ilu_zcolumn_dfs.c.o CMakeFiles/superlu.dir/ilu_zpanel_dfs.c.o CMakeFiles/superlu.dir/ilu_zcopy_to_ucol.c.o CMakeFiles/superlu.dir/ilu_zpivotL.c.o CMakeFiles/superlu.dir/zdiagonal.c.o -Wl,-rpath,:::::::::::::::::::::: -Wl,-rpath,/workspace/destdir/lib -L/workspace/destdir/lib -lopenblas -lstdc++ -ldl -lgfortran -lm -Wl,-rpath,/opt/i686-linux-musl/lib/gcc/i686-linux-musl/8.1.0 -L/opt/i686-linux-musl/lib/gcc/i686-linux-musl/8.1.0 -Wl,-rpath,/opt/i686-linux-musl/lib/gcc -L/opt/i686-linux-musl/lib/gcc -Wl,-rpath,/opt/i686-linux-musl/i686-linux-musl/lib -L/opt/i686-linux-musl/i686-linux-musl/lib -Wl,-rpath,/opt/i686-linux-musl/i686-linux-musl/sys-root/lib -L/opt/i686-linux-musl/i686-linux-musl/sys-root/lib -Wl,-rpath,/opt/i686-linux-musl/i686-linux-musl/sys-root/usr/lib -L/opt/i686-linux-musl/i686-linux-musl/sys-root/usr/lib -lgfortran -lm -lgcc_s -lquadmath -lm 
make[2]: Leaving directory '/workspace/srcdir/petsc-3.13.4/i686-linux-musl/externalpackages/git.superlu/petsc-build'
make[1]: Leaving directory '/workspace/srcdir/petsc-3.13.4/i686-linux-musl/externalpackages/git.superlu/petsc-build'/opt/i686-linux-musl/bin/../lib/gcc/i686-linux-musl/8.1.0/../../../../i686-linux-musl/bin/ld: CMakeFiles/superlu.dir/util.c.o: in function `StatInit':
util.c:(.text+0x8d6): undefined reference to `__stack_chk_fail_local'
/opt/i686-linux-musl/bin/../lib/gcc/i686-linux-musl/8.1.0/../../../../i686-linux-musl/bin/ld: CMakeFiles/superlu.dir/util.c.o: in function `check_repfnz':
util.c:(.text+0x11e4): undefined reference to `__stack_chk_fail_local'
/opt/i686-linux-musl/bin/../lib/gcc/i686-linux-musl/8.1.0/../../../../i686-linux-musl/bin/ld: CMakeFiles/superlu.dir/memory.c.o: in function `intMalloc':
memory.c:(.text+0x256): undefined reference to `__stack_chk_fail_local'
/opt/i686-linux-musl/bin/../lib/gcc/i686-linux-musl/8.1.0/../../../../i686-linux-musl/bin/ld: CMakeFiles/superlu.dir/memory.c.o: in function `intCalloc':
memory.c:(.text+0x316): undefined reference to `__stack_chk_fail_local'
/opt/i686-linux-musl/bin/../lib/gcc/i686-linux-musl/8.1.0/../../../../i686-linux-musl/bin/ld: CMakeFiles/superlu.dir/get_perm_c.c.o: in function `get_colamd':
get_perm_c.c:(.text+0x3d9): undefined reference to `__stack_chk_fail_local'
/opt/i686-linux-musl/bin/../lib/gcc/i686-linux-musl/8.1.0/../../../../i686-linux-musl/bin/ld: CMakeFiles/superlu.dir/get_perm_c.c.o:get_perm_c.c:(.text+0xa08): more undefined references to `__stack_chk_fail_local' follow
collect2: error: ld returned 1 exit status
make[2]: *** [SRC/CMakeFiles/superlu.dir/build.make:2787: SRC/libsuperlu.so.5.2.1] Error 1
make[1]: *** [CMakeFiles/Makefile2:990: SRC/CMakeFiles/superlu.dir/all] Error 2
make: *** [Makefile:144: all] Error 2

@simonbyrne
Copy link
Contributor

It's really weird, since the configure.log does say that

            Defined make macro "SL_LINKER_FUNCTION" to "-dynamiclib -install_name $(call SONAME_FUNCTION,$(1),$(2)) -compatibility_version $(2) -current_version $(3) -single_module -multiply_defined suppress -undefined dynamic_lookup"

but I don't see any file with that

@jd-lara
Copy link
Contributor Author

jd-lara commented Aug 16, 2020

@simonbyrne @giordano I removed adding superlu. After researching more it doesn't look like there are significant advantages without Superlu Dist and at least from searching online it is not supported out-of-the-box to integrate superluMT. We can merge this so it can be used in Sundials

@ChrisRackauckas
Copy link

Does that effect the ability to use SuperLUMT in Sundials?

@jd-lara
Copy link
Contributor Author

jd-lara commented Aug 16, 2020

Does that effect the ability to use SuperLUMT in Sundials?

No, PETSc in Sundials enables another Non-linear solver. SuperLUMT in Sundials is used in the linear solver. I want to enable this workflow:

https://github.com/LLNL/sundials/blob/master/examples/ida/petsc/idaHeat2D_petsc_snes.c

@jd-lara
Copy link
Contributor Author

jd-lara commented Aug 17, 2020

@ViralBShah this PR is passing so we can incorporate it with #995

@simonbyrne
Copy link
Contributor

Woohoo! is this ready to go?

@giordano
Copy link
Member

Good to go?

@jd-lara
Copy link
Contributor Author

jd-lara commented Aug 20, 2020

@giordano yes. Please see the note above about SuperLU.

@giordano
Copy link
Member

Let's merge and see what happens?

@giordano giordano merged commit d16f945 into JuliaPackaging:master Aug 20, 2020
@jd-lara
Copy link
Contributor Author

jd-lara commented Aug 20, 2020

Let's go ahead, I built this with the objective of integrating it into Sundials so we will learn how did it go.

@simonbyrne
Copy link
Contributor

Thanks @jd-lara!

@ViralBShah
Copy link
Member

Fingers crossed!

@simonbyrne
Copy link
Contributor

getting a failure on windows:
https://github.com/JuliaParallel/PETSc.jl/pull/99/checks?check_run_id=1023966725

A simple function to replicate this is:

using PETSc_jll

function dtype(name::AbstractString)
    dtype_ref = Ref{Cint}()
    found_ref = Ref{Cint}()
    ccall((:PetscDataTypeFromString, libpetsc), Cint,
               (Cstring, Ptr{Cint}, Ptr{Cint}),
               name, dtype_ref, found_ref)
    return found_ref[] == 1 ? dtype_ref[] : nothing
end

dtype("Real")

@giordano
Copy link
Member

Does it fail only on v1.5+?

@simonbyrne
Copy link
Contributor

Passes on 1.3, fails on 1.4+:

https://github.com/JuliaParallel/PETSc.jl/runs/1024945378

@giordano
Copy link
Member

Oops, I missed the 1.4 job, but yeah, I noticed it works with v1.3

@simonbyrne
Copy link
Contributor

(I added 1.4 to the build matrix to check)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Windows 🙄 Doesn't build for Windows
Projects
None yet
Development

Successfully merging this pull request may close these issues.

6 participants