Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Issue 1324 netcdf parallel #1388

Merged
merged 49 commits into from
Jun 25, 2015
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
49 commits
Select commit Hold shift + click to select a range
c203c67
working on build first, added pnetcdf
doutriaux1 May 15, 2015
6836aa7
setting up build to enable parallel NetCDF
doutriaux1 May 18, 2015
9de63c8
new conf file to use mpi
doutriaux1 May 18, 2015
558d921
had to upgrade our netcdf distribution
doutriaux1 May 18, 2015
5591fff
pyopenssl issues
doutriaux1 May 18, 2015
30ddfdb
pycparser files
doutriaux1 May 18, 2015
9fe77b4
first pass at splitting between pnetcdf and regular netcdf4 api calls…
doutriaux1 May 20, 2015
607e641
correct md5 for cython
doutriaux1 May 21, 2015
0b888b5
adding begining of a test
doutriaux1 May 21, 2015
3aa7b54
when file was already here but not readable (like a bad write before)
doutriaux1 May 21, 2015
fa55207
can't using mpi if shuffle/deflate so turn them off
doutriaux1 May 21, 2015
2852538
we cannot write mpi if defalte on/off
doutriaux1 May 21, 2015
afb597a
looks like sometimes processors would remove
doutriaux1 May 21, 2015
bb2f037
starting to split write bit over all mpi available
doutriaux1 May 21, 2015
b471baf
getting there but still not getting starts ok
doutriaux1 May 21, 2015
7e35912
actually it was a bad idea to try to split write over ranks
doutriaux1 May 21, 2015
99060f7
new test/example files
doutriaux1 May 21, 2015
15a82c9
getting there with switch to pnetcdf api
doutriaux1 May 22, 2015
f3fd3e0
got mpi to use indep data mode
doutriaux1 May 22, 2015
8bd0ea7
got it to compile again when no PARALLEL macro (i.e. like before)
doutriaux1 May 26, 2015
e33f05a
made sure (???) if not PARALLEL then use same calls as before
doutriaux1 May 27, 2015
13ba0bb
removed useless extra layer of call cdsm2_nc funcs
doutriaux1 May 27, 2015
98ae516
restored setup.py to use PARALLEL
doutriaux1 May 27, 2015
ec223c8
one last set of ncmpi func that was left
doutriaux1 May 27, 2015
f02a993
cleaned up fprintf statements
doutriaux1 May 28, 2015
20ecb58
unlimited dimension is not compatible with //
doutriaux1 May 29, 2015
7b4bfcf
correct arguments for nc_open_par
doutriaux1 May 30, 2015
467e6c6
following recommendation of NetCDF support group
doutriaux1 Jun 3, 2015
2d30664
removed pnetcdf
doutriaux1 Jun 9, 2015
3b4fcb8
will not create missing value attribute if no missing value
doutriaux1 Jun 10, 2015
71a7d41
removing ncredef ncendef
doutriaux1 Jun 10, 2015
5b6c9b3
now you can write 0D variables
doutriaux1 Jun 11, 2015
7fe229e
tweaks to get pyopenssl going on rhea
doutriaux1 Jun 11, 2015
2efad59
removed debug print statements
doutriaux1 Jun 12, 2015
1cee7d3
cleaning up a bit CMakeList.txt
doutriaux1 Jun 12, 2015
7387ea0
newer scipy
doutriaux1 Jun 12, 2015
0077894
@painter1 noticed we couldn't write unlimited dim anymore
doutriaux1 Jun 12, 2015
6438eab
test1 was failing because wrong test on use of //
doutriaux1 Jun 14, 2015
77fa625
switched NetCDF build to CMake
doutriaux1 Jun 15, 2015
7fed9ef
let cdms2 write attribute starting with _
doutriaux1 Jun 15, 2015
19bf6e9
copy/pasting that I thought had been fixed before
doutriaux1 Jun 16, 2015
ac1819d
another attribute starting with _ that need to be in reserved list
doutriaux1 Jun 16, 2015
8d40166
removed print statements
doutriaux1 Jun 16, 2015
166a598
created keyword for switching define mode nc on/off
doutriaux1 Jun 16, 2015
bf5368d
Merge remote branch 'me/cdms2_define_mode_switch' into issue_1324_net…
doutriaux1 Jun 17, 2015
c83f5e4
use define mode can be turned off
doutriaux1 Jun 17, 2015
b7f1a56
Produces netcdf4 files by default
doutriaux1 Jun 17, 2015
40cf745
Revert "switched NetCDF build to CMake"
doutriaux1 Jun 18, 2015
4465053
should fix vcs errors (does on rhea except for pix diff)
doutriaux1 Jun 23, 2015
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion CMake/cdat_modules/cffi_deps.cmake
Original file line number Diff line number Diff line change
@@ -1 +1 @@
set(cffi_deps ${pip_pkg} ${six_pkg} ${ffi_pkg})
set(CFFI_deps ${pip_pkg} ${six_pkg} ${ffi_pkg} ${pycparser_pkg} )
8 changes: 4 additions & 4 deletions CMake/cdat_modules/cython_pkg.cmake
Original file line number Diff line number Diff line change
@@ -1,10 +1,10 @@
set(CYTHON_MAJOR 0)
set(CYTHON_MINOR 21)
set(CYTHON_PATCH 2)
set(CYTHON_VERSION ${CYTHON_MAJOR}.${CYTHON_MINOR}.${CYTHON_PATCH})
set(CYTHON_MINOR 22)
set(CYTHON_PATCH )
set(CYTHON_VERSION ${CYTHON_MAJOR}.${CYTHON_MINOR})
set(CYTHON_URL ${LLNL_URL} )
set(CYTHON_GZ Cython-${CYTHON_VERSION}.tar.gz)
set(CYTHON_MD5 d21adb870c75680dc857cd05d41046a4)
set(CYTHON_MD5 e67b03e8b3667c8e4e7c774ef2e2b638)
set(CYTHON_SOURCE ${CYTHON_URL}/${CYTHON_GZ})

add_cdat_package_dependent(Cython "" "" OFF "CDAT_BUILD_LEAN" OFF)
3 changes: 3 additions & 0 deletions CMake/cdat_modules/hdf5_deps.cmake
Original file line number Diff line number Diff line change
@@ -1 +1,4 @@
set(HDF5_deps ${pkgconfig_pkg} ${libxml2_pkg} ${libxslt_pkg} ${zlib_pkg})
if (CDAT_BUILD_PARALLEL)
list(APPEND HDF5_deps ${mpi_pkg})
endif()
14 changes: 12 additions & 2 deletions CMake/cdat_modules/hdf5_external.cmake
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,15 @@ set(HDF5_install "${cdat_EXTERNALS}")
configure_file(${cdat_CMAKE_SOURCE_DIR}/cdat_modules_extra/hdf5_patch_step.cmake.in
${cdat_CMAKE_BINARY_DIR}/hdf5_patch_step.cmake
@ONLY)

if (CDAT_BUILD_PARALLEL)
set(hdf5_configure_args "--enable-parallel")
set(hdf5_additional_cflags "-w -fPIC")
set(configure_file "cdatmpi_configure_step.cmake")
else()
set(hdf5_configure_args "")
set(hdf5_additional_cflags "-w")
set(configure_file "cdat_configure_step.cmake")
endif()
# we disable HDF5 warnings because it has way too many of them.
ExternalProject_Add(HDF5
DOWNLOAD_DIR ${CDAT_PACKAGE_CACHE_DIR}
Expand All @@ -15,7 +23,9 @@ ExternalProject_Add(HDF5
URL_MD5 ${HDF5_MD5}
BUILD_IN_SOURCE 1
#PATCH_COMMAND ${CMAKE_COMMAND} -DWORKING_DIR=<SOURCE_DIR> -P ${cdat_CMAKE_BINARY_DIR}/hdf5_patch_step.cmake
CONFIGURE_COMMAND ${CMAKE_COMMAND} -DADDITIONAL_CFLAGS=-w -DADDITIONAL_CPPFPAGS=-w -DINSTALL_DIR=<INSTALL_DIR> -DWORKING_DIR=<SOURCE_DIR> -P ${cdat_CMAKE_BINARY_DIR}/cdat_configure_step.cmake
CONFIGURE_COMMAND ${CMAKE_COMMAND} -DCONFIGURE_ARGS=${hdf5_configure_args} -DADDITIONAL_CFLAGS=${hdf5_additional_cflags} -DADDITIONAL_CPPFPAGS=-w -DINSTALL_DIR=<INSTALL_DIR> -DWORKING_DIR=<SOURCE_DIR> -P ${cdat_CMAKE_BINARY_DIR}/${configure_file}
BUILD_COMMAND ${CMAKE_COMMAND} -Dmake=$(MAKE) -DWORKING_DIR=<SOURCE_DIR> -P ${cdat_CMAKE_BINARY_DIR}/cdat_make_step.cmake
INSTALL_COMMAND ${CMAKE_COMMAND} -DWORKING_DIR=<SOURCE_DIR> -P ${cdat_CMAKE_BINARY_DIR}/cdat_install_step.cmake
LOG_BUILD 1
DEPENDS ${HDF5_deps}
${ep_log_options}
Expand Down
4 changes: 2 additions & 2 deletions CMake/cdat_modules/hdf5_pkg.cmake
Original file line number Diff line number Diff line change
@@ -1,9 +1,9 @@
set(HDF5_MAJOR_SRC 1)
set(HDF5_MINOR_SRC 8)
set(HDF5_PATCH_SRC 13)
set(HDF5_PATCH_SRC 15)
set(HDF5_URL ${LLNL_URL})
set(HDF5_GZ hdf5-${HDF5_MAJOR_SRC}.${HDF5_MINOR_SRC}.${HDF5_PATCH_SRC}.tar.gz)
set(HDF5_MD5 c03426e9e77d7766944654280b467289)
set(HDF5_MD5 03cccb5b33dbe975fdcd8ae9dc021f24 )

set (nm HDF5)
string(TOUPPER ${nm} uc_nm)
Expand Down
3 changes: 3 additions & 0 deletions CMake/cdat_modules/netcdf_deps.cmake
Original file line number Diff line number Diff line change
@@ -1 +1,4 @@
set(NetCDF_deps ${pkgconfig_pkg} ${hdf5_pkg} ${curl_pkg} ${zlib_pkg} ${jpeg_pkg} )
if (CDAT_BUILD_PARALLEL)
list(APPEND NetCDF_deps ${mpi_pkg} )
endif()
9 changes: 8 additions & 1 deletion CMake/cdat_modules/netcdf_external.cmake
Original file line number Diff line number Diff line change
@@ -1,6 +1,11 @@
set(netcdf_source "${CMAKE_CURRENT_BINARY_DIR}/build/netcdf")
set(netcdf_install "${cdat_EXTERNALS}")
set(netcdf_configure_args "--enable-netcdf-4")
if (CDAT_BUILD_PARALLEL)
set(configure_file "cdatmpi_configure_step.cmake")
else()
set(configure_file "cdat_configure_step.cmake")
endif()

configure_file(${cdat_CMAKE_SOURCE_DIR}/cdat_modules_extra/netcdf_patch_step.cmake.in
${cdat_CMAKE_BINARY_DIR}/netcdf_patch_step.cmake
Expand All @@ -17,7 +22,9 @@ ExternalProject_Add(NetCDF
URL_MD5 ${NC4_MD5}
BUILD_IN_SOURCE 1
PATCH_COMMAND ${netcdf_PATCH_COMMAND}
CONFIGURE_COMMAND ${CMAKE_COMMAND} -DINSTALL_DIR=<INSTALL_DIR> -DWORKING_DIR=<SOURCE_DIR> -D CONFIGURE_ARGS=${netcdf_configure_args} -P ${cdat_CMAKE_BINARY_DIR}/cdat_configure_step.cmake
CONFIGURE_COMMAND ${CMAKE_COMMAND} -DINSTALL_DIR=<INSTALL_DIR> -DWORKING_DIR=<SOURCE_DIR> -DCONFIGURE_ARGS=${netcdf_configure_args} -P ${cdat_CMAKE_BINARY_DIR}/${configure_file}
BUILD_COMMAND ${CMAKE_COMMAND} -Dmake=$(MAKE) -DWORKING_DIR=<SOURCE_DIR> -P ${cdat_CMAKE_BINARY_DIR}/cdat_make_step.cmake
INSTALL_COMMAND ${CMAKE_COMMAND} -DWORKING_DIR=<SOURCE_DIR> -P ${cdat_CMAKE_BINARY_DIR}/cdat_install_step.cmake
DEPENDS ${NetCDF_deps}
${ep_log_options}
)
Expand Down
4 changes: 2 additions & 2 deletions CMake/cdat_modules/netcdf_pkg.cmake
Original file line number Diff line number Diff line change
@@ -1,9 +1,9 @@
set(NC4_MAJOR_SRC 4)
set(NC4_MINOR_SRC 3)
set(NC4_PATCH_SRC 2)
set(NC4_PATCH_SRC 3.1)
set(NC4_URL ${LLNL_URL})
set(NC4_GZ netcdf-${NC4_MAJOR_SRC}.${NC4_MINOR_SRC}.${NC4_PATCH_SRC}.tar.gz)
set(NC4_MD5 2fd2365e1fe9685368cd6ab0ada532a0)
set(NC4_MD5 5c9dad3705a3408d27f696e5b31fb88c )

set (nm NC4)
string(TOUPPER ${nm} uc_nm)
Expand Down
1 change: 1 addition & 0 deletions CMake/cdat_modules/pnetcdf_deps.cmake
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
set(PNETCDF_deps ${pkgconfig_pkg} ${mpi_pkg})
17 changes: 17 additions & 0 deletions CMake/cdat_modules/pnetcdf_external.cmake
Original file line number Diff line number Diff line change
@@ -0,0 +1,17 @@
set(pnetcdf_source "${CMAKE_CURRENT_BINARY_DIR}/build/pnetcdf")
set(pnetcdf_install "${cdat_EXTERNALS}")
set(pnetcdf_configure_args "--with-mpi=${cdat_EXTERNALS}")
set(pnetcdf_additional_cflags "-fPIC")

ExternalProject_Add(PNETCDF
LIST_SEPARATOR ^^
DOWNLOAD_DIR ${CDAT_PACKAGE_CACHE_DIR}
SOURCE_DIR ${pnetcdf_source}
INSTALL_DIR ${pnetcdf_install}
URL ${PNETCDF_URL}/${PNETCDF_GZ}
URL_MD5 ${PNETCDF_MD5}
BUILD_IN_SOURCE 1
CONFIGURE_COMMAND ${CMAKE_COMMAND} -DADDITIONAL_CFLAGS=${pnetcdf_additional_cflags} -DINSTALL_DIR=<INSTALL_DIR> -DWORKING_DIR=<SOURCE_DIR> -D CONFIGURE_ARGS=${pnetcdf_configure_args} -P ${cdat_CMAKE_BINARY_DIR}/cdat_configure_step.cmake
DEPENDS ${PNETCDF_deps}
${ep_log_options}
)
13 changes: 13 additions & 0 deletions CMake/cdat_modules/pnetcdf_pkg.cmake
Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@
set(PNETCDF_MAJOR_SRC 1)
set(PNETCDF_MINOR_SRC 6)
set(PNETCDF_PATCH_SRC 0)
set(PNETCDF_URL ${LLNL_URL})
set(PNETCDF_GZ parallel-netcdf-${PNETCDF_MAJOR_SRC}.${PNETCDF_MINOR_SRC}.${PNETCDF_PATCH_SRC}.tar.gz)
set(PNETCDF_MD5 4893a50ddcd487a312c64383bdeb2631)

set (nm PNETCDF)
string(TOUPPER ${nm} uc_nm)
set(${uc_nm}_VERSION ${${nm}_MAJOR_SRC}.${${nm}_MINOR_SRC}.${${nm}_PATCH_SRC})
set(PNETCDF_SOURCE ${PNETCDF_URL}/${PNETCDF_GZ})

add_cdat_package(PNETCDF "" "" OFF)
1 change: 1 addition & 0 deletions CMake/cdat_modules/pycparser_deps.cmake
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
set(PYCPARSER_deps ${python_pkg} ${pip_pkg})
6 changes: 6 additions & 0 deletions CMake/cdat_modules/pycparser_external.cmake
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@

# create an external project to install MyProxyClient,
# and configure and build it
set(nm PYCPARSER)

include(pipinstaller)
10 changes: 10 additions & 0 deletions CMake/cdat_modules/pycparser_pkg.cmake
Original file line number Diff line number Diff line change
@@ -0,0 +1,10 @@
set(PYCPARSER_MAJOR_SRC 2)
set(PYCPARSER_MINOR_SRC 13)
set(PYCPARSER_PATCH_SRC )

set(PYCPARSER_VERSION ${PYCPARSER_MAJOR_SRC}.${PYCPARSER_MINOR_SRC})
set(PYCPARSER_GZ pycparser-${PYCPARSER_VERSION}.tar.gz)
set(PYCPARSER_SOURCE ${LLNL_URL}/${PYCPARSER_GZ})
set(PYCPARSER_MD5 e4fe1a2d341b22e25da0d22f034ef32f )

add_cdat_package_dependent(PYCPARSER "" "" ON "CDAT_BUILD_LEAN" OFF)
2 changes: 1 addition & 1 deletion CMake/cdat_modules/pyopenssl_deps.cmake
Original file line number Diff line number Diff line change
@@ -1 +1 @@
set(PYOPENSSL_deps ${python_pkg} ${pip_pkg})
set(PYOPENSSL_deps ${python_pkg} ${pip_pkg} ${six_pkg} ${cryptography_pkg} ${cffi_pkg} ${ffi_pkg} ${pycparser_pkg} )
12 changes: 6 additions & 6 deletions CMake/cdat_modules/scipy_pkg.cmake
Original file line number Diff line number Diff line change
@@ -1,12 +1,12 @@
set(SCIPY_MAJOR 0)
set(SCIPY_MINOR 10)
set(SCIPY_MINOR 16)
set(SCIPY_PATCH 0)
set(SCIPY_MAJOR_SRC dde07a64407ffaa9442b3d8298c6c26ff91fb384)
set(SCIPY_MINOR_SRC)
set(SCIPY_PATCH_SRC)
set(SCIPY_MAJOR_SRC 0)
set(SCIPY_MINOR_SRC 16)
set(SCIPY_PATCH_SRC 0b2)
set(SCIPY_URL ${LLNL_URL})
set(SCIPY_GZ scipy-dde07a64407ffaa9442b3d8298c6c26ff91fb384.zip)
set(SCIPY_MD5 03fed4942984c0220615fac2d8d675c7)
set(SCIPY_GZ scipy-${SCIPY_MAJOR_SRC}.${SCIPY_MINOR_SRC}.${SCIPY_PATCH_SRC}.tar.gz)
set(SCIPY_MD5 cf2ca99b54a3951ecafb5c95730aa17f )
set(SCIPY_SOURCE ${SCIPY_URL}/${SCIPY_GZ})

set (nm SCIPY)
Expand Down
10 changes: 5 additions & 5 deletions CMake/cdat_modules_extra/cdat_common_environment.cmake.in
Original file line number Diff line number Diff line change
@@ -1,12 +1,12 @@
message("[INFO] ADDITIONAL CFLAGS ${ADDITIONAL_CFLAGS}")
set(ENV{PATH} "@cdat_EXTERNALS@/bin:$ENV{PATH}")
set(ENV{@LIBRARY_PATH@} "@CMAKE_INSTALL_PREFIX@/lib:@cdat_EXTERNALS@/lib:@cdat_EXTERNALS@/lib/paraview-@PARAVIEW_MAJOR@.@PARAVIEW_MINOR@:$ENV{@LIBRARY_PATH@}")
set(ENV{@LIBRARY_PATH@} "@CMAKE_INSTALL_PREFIX@/lib:@cdat_EXTERNALS@/lib:@cdat_EXTERNALS@/lib64:@cdat_EXTERNALS@/lib/paraview-@PARAVIEW_MAJOR@.@PARAVIEW_MINOR@:$ENV{@LIBRARY_PATH@}")
if (NOT DEFINED SKIP_LDFLAGS)
set(ENV{LDFLAGS} "-L@cdat_EXTERNALS@/lib @cdat_external_link_directories@ @cdat_rpath_flag@@CMAKE_INSTALL_PREFIX@/lib @cdat_rpath_flag@@cdat_EXTERNALS@/lib")
set(ENV{LDFLAGS} "-L@cdat_EXTERNALS@/lib -L@cdat_EXTERNALS@/lib64 @cdat_external_link_directories@ -Wl,-rpath,@cdat_EXTERNALS@/lib64 @cdat_rpath_flag@@CMAKE_INSTALL_PREFIX@/lib @cdat_rpath_flag@@cdat_EXTERNALS@/lib")
endif()
set(ENV{CFLAGS} "-I@cdat_EXTERNALS@/include @cdat_osx_flags@ @cdat_external_include_directories@ ${ADDITIONAL_CFLAGS}")
set(ENV{CPPFLAGS} "-I@cdat_EXTERNALS@/include @cdat_osx_cppflags@ @cdat_external_include_directories@ ${ADDITIONAL_CPPFLAGS}")
set(ENV{CXXFLAGS} "-I@cdat_EXTERNALS@/include @cdat_osx_cxxflags@ ${ADDITIONAL_CXXFLAGS}")
set(ENV{CFLAGS} "-I@cdat_EXTERNALS@/include -I@cdat_EXTERNALS@/lib/libffi-3.1/include @cdat_osx_flags@ @cdat_external_include_directories@ ${ADDITIONAL_CFLAGS}")
set(ENV{CPPFLAGS} "-I@cdat_EXTERNALS@/include -I@cdat_EXTERNALS@/lib/libffi-3.1/include @cdat_osx_cppflags@ @cdat_external_include_directories@ ${ADDITIONAL_CPPFLAGS}")
set(ENV{CXXFLAGS} "-I@cdat_EXTERNALS@/include -I@cdat_EXTERNALS@/lib/libffi-3.1/include @cdat_osx_cxxflags@ ${ADDITIONAL_CXXFLAGS}")
set(ENV{PKG_CONFIG_PATH} "@cdat_EXTERNALS@/lib/pkgconfig:/usr/lib64/pkgconfig:/usr/lib/pkgconfig:/usr/lib/x86_64-linux-gnu/pkgconfig:/usr/share/pkgconfig:$ENV{PKG_CONFIG_PATH}")
set(ENV{PKG_CONFIG} "@cdat_PKG_CONFIG_EXECUTABLE@")
set(ENV{FC} "")
Expand Down
21 changes: 21 additions & 0 deletions CMake/cdat_modules_extra/cdatmpi_configure_step.cmake.in
Original file line number Diff line number Diff line change
@@ -0,0 +1,21 @@
set(ENV{PKG_CONFIG_PATH} "@cdat_EXTERNALS@/lib/pkgconfig:/usr/lib/x86_64-linux-gnu/pkgconfig:/usr/lib64/pkgconfig:/usr/lib/pkgconfig:/usr/share/pkgconfig:$ENV{PKG_CONFIG_PATH}")
set(ENV{PKG_CONFIG} "@cdat_PKG_CONFIG_EXECUTABLE@")

include(@cdat_CMAKE_BINARY_DIR@/cdat_common_environment.cmake)

set(ENV{CC} mpicc)

message("CONFIGURE_ARGS IS ${CONFIGURE_ARGS}")
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please take out this debug messages

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍 thanks for spotting and reviewing!

message("LD_ARGS IS $ENV{@LIBRARY_PATH@}")
message("CFLAGS : $ENV{CFLAGS}")

execute_process(
COMMAND env CC=$ENV{CC} CFLAGS=$ENV{CFLAGS} LD_LIBRARY_PATH=$ENV{@LIBRARY_PATH@} DYLD_FALLBACK_LIBRARY_PATH=$ENV{@LIBRARY_PATH@} @LIBRARY_PATH@=$ENV{@LIBRARY_PATH@} PKG_CONFIG=$ENV{PKG_CONFIG} PKG_CONFIG_PATH=$ENV{PKG_CONFIG_PATH} sh configure --prefix=${INSTALL_DIR} ${CONFIGURE_ARGS}
WORKING_DIRECTORY "${WORKING_DIR}"
RESULT_VARIABLE res)

if(NOT ${res} EQUAL 0)
message("Config Errors detected: \n${CDAT_OUT}\n${CDAT_ERR}")
message(FATAL_ERROR "Error in config")
endif()
message("Config succeeded.")
4 changes: 2 additions & 2 deletions CMake/cmake_modules/pipinstaller.cmake
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
# create an external project to install MyProxyClient,
# and configure and build it

include(@cdat_CMAKE_BINARY_DIR@/cdat_common_environment.cmake)
include(${cdat_CMAKE_BINARY_DIR}/cdat_common_environment.cmake)
string(TOUPPER ${nm} uc_nm)

if(NOPREFIXOPT )
Expand Down Expand Up @@ -33,7 +33,7 @@ ExternalProject_Add(${nm}
BUILD_IN_SOURCE 1
CONFIGURE_COMMAND ""
BUILD_COMMAND ""
INSTALL_COMMAND env PYTHONPATH=${PYTHONPATH} ${USR_ENVS} ${PYTHON_EXECUTABLE} setup.py install ${PRFX}
INSTALL_COMMAND env LD_LIBRARY_PATH=$ENV{LD_LIBRARY_PATH} CFLAGS=$ENV{CFLAGS} LDFLAGS=$ENV{LDFLAGS} PKG_CONFIG_PATH=$ENV{PKG_CONFIG_PATH} PYTHONPATH=${PYTHONPATH} ${USR_ENVS} ${PYTHON_EXECUTABLE} setup.py install ${PRFX}
DEPENDS ${${nm}_deps}
${ep_log_options}
)
Expand Down
14 changes: 11 additions & 3 deletions CMakeLists.txt
Original file line number Diff line number Diff line change
Expand Up @@ -295,6 +295,7 @@ if (CDAT_BUILD_ESGF)
endif()
set(CDAT_BUILD_MODE "DEFAULT" CACHE STRING "Build mode for CDAT <ALL, LEAN, DEFAULT>")
set_property(CACHE CDAT_BUILD_MODE PROPERTY STRINGS "DEFAULT" "ALL" "LEAN")
message([INFO] BUILD MODE} ${CDAT_BUILD_MODE})

# Set the state of LEAN all based on the MODE
if (CDAT_BUILD_MODE STREQUAL "LEAN")
Expand All @@ -303,9 +304,11 @@ if (CDAT_BUILD_MODE STREQUAL "LEAN")
elseif (CDAT_BUILD_MODE STREQUAL "ALL")
set(CDAT_BUILD_LEAN OFF)
set(CDAT_BUILD_ALL ON)
else()
elseif (CDAT_BUILD_MODE STREQUAL "DEFAULT")
set(CDAT_BUILD_LEAN OFF)
set(CDAT_BUILD_ALL OFF)
else()
message(FATAL_ERROR "[ERROR] Unknown CDAT_BUILD_MODE \"${CDAT_BUILD_MODE}\" VALID MODES ARE \"DEFAULT\" \"ALL\" \"LEAN\"")
endif()

# First of all if LEAN then turn OFF GRAPHICS and PARALLEL
Expand Down Expand Up @@ -484,8 +487,12 @@ include(libcf_pkg)
include(libdrs_pkg)
include(matplotlib_pkg)
include(six_pkg)
include(cryptography_pkg)
include(cffi_pkg)
include(ffi_pkg)
include(dateutils_pkg)
include(pyparsing_pkg)
include(pycparser_pkg)
include(md5_pkg)
include(mpi4py_pkg)
include(pep8_pkg)
Expand Down Expand Up @@ -600,8 +607,8 @@ configure_file(${cdat_CMAKE_SOURCE_DIR}/cdat_modules_extra/cleanenv_configure_st
@ONLY
)

configure_file(${cdat_CMAKE_SOURCE_DIR}/cdat_modules_extra/cdat_configure_step.cmake.in
${cdat_CMAKE_BINARY_DIR}/cdat_configure_step.cmake
configure_file(${cdat_CMAKE_SOURCE_DIR}/cdat_modules_extra/cdatmpi_configure_step.cmake.in
${cdat_CMAKE_BINARY_DIR}/cdatmpi_configure_step.cmake
@ONLY
)

Expand Down Expand Up @@ -674,6 +681,7 @@ set(SETUP_LIBRARY_PATHS
"Externals/lib/paraview-${PARAVIEW_MAJOR}.${PARAVIEW_MINOR} "
"Externals/lib/R/lib "
"Externals/lib "
"Externals/lib64 "
"lib "
)
string(REPLACE ";" " " SETUP_LIBRARY_PATHS ${SETUP_LIBRARY_PATHS})
Expand Down
9 changes: 8 additions & 1 deletion Packages/cdms2/Lib/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -27,7 +27,14 @@
from grid import createGenericGrid, createGlobalMeanGrid, createRectGrid, createUniformGrid, createZonalGrid, setClassifyGrids, createGaussianGrid, writeScripGrid, isGrid

# Dataset functions
from dataset import createDataset, openDataset, useNetcdf3, getNetcdfClassicFlag, getNetcdfShuffleFlag, getNetcdfDeflateFlag, getNetcdfDeflateLevelFlag, setNetcdfClassicFlag, setNetcdfShuffleFlag, setNetcdfDeflateFlag, setNetcdfDeflateLevelFlag, setCompressionWarnings
from dataset import createDataset, openDataset, useNetcdf3, \
getNetcdfClassicFlag, getNetcdfShuffleFlag, getNetcdfDeflateFlag, getNetcdfDeflateLevelFlag,\
setNetcdfClassicFlag, setNetcdfShuffleFlag, setNetcdfDeflateFlag, setNetcdfDeflateLevelFlag,\
setNetcdfUseNCSwitchModeFlag,getNetcdfUseNCSwitchModeFlag,\
setCompressionWarnings,\
setNetcdf4Flag, getNetcdf4Flag,\
setNetcdfUseParallelFlag, getNetcdfUseParallelFlag

open = openDataset

# Database functions
Expand Down
6 changes: 5 additions & 1 deletion Packages/cdms2/Lib/cdmsobj.py
Original file line number Diff line number Diff line change
Expand Up @@ -464,7 +464,11 @@ def _setatts(self,value):

def __init__(self, node = None):
if not hasattr(self,'___cdms_internals__'):
self.__dict__['___cdms_internals__']=['__cdms_internals__','___cdms_internals__','_node_','parent','attributes','shape','autoApiInfo']
self.__dict__['___cdms_internals__']=[
'__cdms_internals__','___cdms_internals__',
'_node_','_obj_',
'_numericType_','_grid_','_bounds_',
'parent','attributes','shape','autoApiInfo']
self.attributes={}
self._node_ = node
if node is not None:
Expand Down
Loading