904 Add nvcc (cuda) to CI #907

lifflander · 2020-07-03T08:40:58Z

Fixes #904

TODO:

See if we can make the container smaller... 7 GB currently!
Fix all the compile-time bugs (see issue Failures when compiling with nvcc on develop #908)
See if we can speedup compilation if its taking too long in the container to run correctly once the bugs are solved

lifflander · 2020-07-03T08:44:16Z

I worked around one nvcc bug on develop. But we have several more. And it's really slow to compile...

codecov · 2020-07-03T09:24:07Z

Codecov Report

Merging #907 into develop will not change coverage.
The diff coverage is 25.00%.

@@           Coverage Diff            @@
##           develop     #907   +/-   ##
========================================
  Coverage    77.21%   77.21%           
========================================
  Files          648      648           
  Lines        24837    24837           
========================================
  Hits         19179    19179           
  Misses        5658     5658

Impacted Files	Coverage Δ
src/vt/messaging/active.impl.h	`97.79% <ø> (ø)`
src/vt/messaging/message/message_serialize.h	`100.00% <ø> (ø)`
src/vt/rdmahandle/handle.index.impl.h	`0.00% <0.00%> (ø)`
src/vt/rdmahandle/handle.node.impl.h	`0.00% <0.00%> (ø)`
src/vt/rdmahandle/manager.impl.h	`0.00% <ø> (ø)`
tests/perf/ping_pong.cc	`95.65% <100.00%> (ø)`
tests/unit/pool/test_pool_message_sizes.cc	`100.00% <100.00%> (ø)`

lifflander · 2020-07-03T21:02:58Z

I think we should start moving away from Travis CI. We are hitting their time limit occasionally causing spurious failures. To do this we need a GitHub Action that does the code coverage and uploads it from the container

pnstickne · 2020-07-12T20:37:48Z

After fixing void return.

2020-07-12T20:14:57.4110898Z /build/checkpoint/install/include/checkpoint/dispatch/dispatch_serializer_nonbyte.h: In instantiation of 'void checkpoint::dispatch::SerializerDispatchNonByte<SerializerT, T, Dispatcher>::apply(SerializerT&, T*, checkpoint::SerialSizeType, checkpoint::dispatch::SerializerDispatchNonByte<SerializerT, T, Dispatcher>::hasNotVirtualSerialize<U>*) [with U = vt::rdma::impl::HandleData; SerializerT = checkpoint::Sizer; T = vt::rdma::impl::HandleData; Dispatcher = checkpoint::dispatch::BasicDispatcher<checkpoint::Sizer, vt::rdma::impl::HandleData>; checkpoint::SerialSizeType = long unsigned int; checkpoint::dispatch::SerializerDispatchNonByte<SerializerT, T, Dispatcher>::hasNotVirtualSerialize<U> = vt::rdma::impl::HandleData]':
2020-07-12T20:14:57.4112358Z /build/checkpoint/install/include/checkpoint/dispatch/dispatch_serializer_nonbyte.h:99:6:   required from 'void checkpoint::dispatch::SerializerDispatchNonByte<SerializerT, T, Dispatcher>::operator()(SerializerT&, T*, checkpoint::SerialSizeType) [with U = vt::rdma::impl::HandleData; SerializerT = checkpoint::Sizer; T = vt::rdma::impl::HandleData; Dispatcher = checkpoint::dispatch::BasicDispatcher<checkpoint::Sizer, vt::rdma::impl::HandleData>; checkpoint::SerialSizeType = long unsigned int]'
2020-07-12T20:14:57.4113949Z /build/checkpoint/install/include/checkpoint/dispatch/dispatch_serializer_byte.h:107:9:   required from 'void checkpoint::dispatch::SerializerDispatchByte<SerializerT, T, Dispatcher>::operator()(SerializerT&, T*, checkpoint::SerialSizeType, checkpoint::dispatch::SerializerDispatchByte<SerializerT, T, Dispatcher>::isNotByteCopyType<U>*) [with U = vt::rdma::impl::HandleData; SerializerT = checkpoint::Sizer; T = vt::rdma::impl::HandleData; Dispatcher = checkpoint::dispatch::BasicDispatcher<checkpoint::Sizer, vt::rdma::impl::HandleData>; checkpoint::SerialSizeType = long unsigned int; checkpoint::dispatch::SerializerDispatchByte<SerializerT, T, Dispatcher>::isNotByteCopyType<U> = vt::rdma::impl::HandleData]'
2020-07-12T20:14:57.4115055Z /build/checkpoint/install/include/checkpoint/dispatch/dispatch.impl.h:71:3:   required from 'static TraverserT& checkpoint::dispatch::Traverse::with(T&, TraverserT&, checkpoint::SerialSizeType) [with T = vt::rdma::impl::HandleData; TraverserT = checkpoint::Sizer; checkpoint::SerialSizeType = long unsigned int]'
2020-07-12T20:14:57.4115911Z /build/checkpoint/install/include/checkpoint/dispatch/dispatch.impl.h:55:32:   required from 'Serializer& checkpoint::operator|(Serializer&, T&) [with Serializer = checkpoint::Sizer; T = vt::rdma::impl::HandleData]'
2020-07-12T20:14:57.4116823Z /vt/src/vt/collective/reduce/operators/default_msg.h:95:3:   required from 'void vt::collective::reduce::operators::ReduceDataMsg<DataType>::serialize(SerializeT&) [with SerializeT = checkpoint::Sizer; DataType = vt::rdma::impl::HandleData]'
2020-07-12T20:14:57.4117330Z /vt/src/vt/messaging/message/message_serialize.h:401:16:   [ skipping 10 instantiation contexts, use -ftemplate-backtrace-limit=0 to disable ]
2020-07-12T20:14:57.4118644Z /vt/src/vt/collective/reduce/operators/default_op.impl.h:64:1:   required from 'static void vt::collective::reduce::operators::ReduceCombine<T>::msgHandler(MsgT*) [with MsgT = vt::rdma::impl::ConstructMsg<short unsigned int, (vt::rdma::HandleEnum)1, vt::objgroup::proxy::Proxy<vt::tests::unit::TestObjGroup> >; Op = vt::collective::reduce::operators::PlusOp<vt::rdma::impl::HandleData>; ActOp = vt::collective::reduce::operators::ReduceCallback<vt::rdma::impl::ConstructMsg<short unsigned int, (vt::rdma::HandleEnum)1, vt::objgroup::proxy::Proxy<vt::tests::unit::TestObjGroup> > >; T = void]'
2020-07-12T20:14:57.4121075Z /vt/src/vt/objgroup/proxy/proxy_objgroup.h:134:21:   required by substitution of 'template<class OpT, class MsgPtrT, class MsgT, void (* f)(MsgT*)> void vt::objgroup::proxy::Proxy<vt::rdma::Manager>::reduce<OpT, MsgPtrT, MsgT, f>(MsgPtrT, vt::Callback<MsgT>, vt::objgroup::proxy::Proxy<vt::rdma::Manager>::ReduceStamp) const [with OpT = vt::collective::reduce::operators::PlusOp<vt::rdma::impl::HandleData>; MsgPtrT = vt::rdma::impl::ConstructMsg<short unsigned int, (vt::rdma::HandleEnum)1, vt::objgroup::proxy::Proxy<vt::tests::unit::TestObjGroup> >*; MsgT = vt::rdma::impl::ConstructMsg<short unsigned int, (vt::rdma::HandleEnum)1, vt::objgroup::proxy::Proxy<vt::tests::unit::TestObjGroup> >; void (* f)(MsgT*) = vt::collective::reduce::operators::ReduceCombine<>::msgHandler<vt::rdma::impl::ConstructMsg<short unsigned int, (vt::rdma::HandleEnum)1, vt::objgroup::proxy::Proxy<vt::tests::unit::TestObjGroup> >, vt::collective::reduce::operators::PlusOp<vt::rdma::impl::HandleData>, vt::collective::reduce::operators::ReduceCallback<vt::rdma::impl::ConstructMsg<short unsigned int, (vt::rdma::HandleEnum)1, vt::objgroup::proxy::Proxy<vt::tests::unit::TestObjGroup> > > >]'
2020-07-12T20:14:57.4122335Z /vt/src/vt/rdmahandle/manager.impl.h:139:1:   required from 'vt::rdma::Handle<T, E> vt::rdma::Manager::makeHandleCollectiveObjGroup(ProxyT, std::size_t, bool) [with T = short unsigned int; vt::rdma::HandleEnum E = (vt::rdma::HandleEnum)1; ProxyT = vt::objgroup::proxy::Proxy<vt::tests::unit::TestObjGroup>; std::size_t = long unsigned int]'
2020-07-12T20:14:57.4123234Z /vt/src/vt/objgroup/proxy/proxy_objgroup.impl.h:169:128:   required from 'vt::rdma::Handle<T> vt::objgroup::proxy::Proxy<ObjT>::makeHandleRDMA(std::size_t, bool) const [with T = short unsigned int; ObjT = vt::tests::unit::TestObjGroup; std::size_t = long unsigned int]'
2020-07-12T20:14:57.4124138Z /vt/tests/unit/rdma/test_rdma_handle.cc:64:61:   required from 'vt::HandleRDMA<T> vt::tests::unit::TestObjGroup::makeHandle(std::size_t, bool) [with T = short unsigned int; vt::HandleRDMA<T> = vt::rdma::Handle<short unsigned int, (vt::rdma::HandleEnum)1, short int, void>; std::size_t = long unsigned int]'
2020-07-12T20:14:57.4124958Z /vt/tests/unit/rdma/test_rdma_handle.cc:245:29:   required from 'void vt::tests::unit::gtest_suite_TestRDMAHandle_::test_rdma_handle_5<gtest_TypeParam_>::TestBody() [with gtest_TypeParam_ = short unsigned int]'
2020-07-12T20:14:57.4125180Z /vt/src/vt/group/region/group_shallow_list.h:55:8:   required from here
2020-07-12T20:14:57.4126241Z /build/checkpoint/install/include/checkpoint/dispatch/dispatch_serializer_nonbyte.h:218:12: error: no matching function for call to 'checkpoint::dispatch::SerializerDispatchNonByte<checkpoint::Sizer, vt::rdma::impl::HandleData, checkpoint::dispatch::BasicDispatcher<checkpoint::Sizer, vt::rdma::impl::HandleData> >::applyStatic(checkpoint::Sizer&, vt::rdma::impl::HandleData*&, checkpoint::SerialSizeType&)'
2020-07-12T20:14:57.4126586Z      applyStatic(s, val, num);
2020-07-12T20:14:57.4126734Z      ~~~~~~~^~~~~~~~~~~~~
2020-07-12T20:14:57.4127059Z /build/checkpoint/install/include/checkpoint/dispatch/dispatch_serializer_nonbyte.h:163:1: note: candidate: template<class U> void checkpoint::dispatch::SerializerDispatchNonByte<SerializerT, T, Dispatcher>::applyStatic(SerializerT&, T*, checkpoint::SerialSizeType, checkpoint::dispatch::SerializerDispatchNonByte<SerializerT, T, Dispatcher>::hasInSerialize<U>*) [with U = U; SerializerT = checkpoint::Sizer; T = vt::rdma::impl::HandleData; Dispatcher = checkpoint::dispatch::BasicDispatcher<checkpoint::Sizer, vt::rdma::impl::HandleData>]
2020-07-12T20:14:57.4127344Z    void applyStatic(
2020-07-12T20:14:57.4127486Z  ^ ~~~~~~~~~
2020-07-12T20:14:57.4127681Z /build/checkpoint/install/include/checkpoint/dispatch/dispatch_serializer_nonbyte.h:163:1: note:   template argument deduction/substitution failed:
2020-07-12T20:14:57.4128047Z /build/checkpoint/install/include/checkpoint/dispatch/dispatch_serializer_nonbyte.h:177:1: note: candidate: template<class U> void checkpoint::dispatch::SerializerDispatchNonByte<SerializerT, T, Dispatcher>::applyStatic(SerializerT&, T*, checkpoint::SerialSizeType, checkpoint::dispatch::SerializerDispatchNonByte<SerializerT, T, Dispatcher>::hasNoninSerialize<U>*) [with U = U; SerializerT = checkpoint::Sizer; T = vt::rdma::impl::HandleData; Dispatcher = checkpoint::dispatch::BasicDispatcher<checkpoint::Sizer, vt::rdma::impl::HandleData>]
2020-07-12T20:14:57.4128324Z    void applyStatic(
2020-07-12T20:14:57.4128466Z  ^ ~~~~~~~~~
2020-07-12T20:14:57.4128651Z /build/checkpoint/install/include/checkpoint/dispatch/dispatch_serializer_nonbyte.h:177:1: note:   template argument deduction/substitution failed:
2020-07-12T20:14:57.4129050Z /build/checkpoint/install/include/checkpoint/dispatch/dispatch_serializer_nonbyte.h:190:1: note: candidate: template<class U> void checkpoint::dispatch::SerializerDispatchNonByte<SerializerT, T, Dispatcher>::applyStatic(SerializerT&, T*, checkpoint::SerialSizeType, checkpoint::dispatch::SerializerDispatchNonByte<SerializerT, T, Dispatcher>::isEnum<U>*) [with U = U; SerializerT = checkpoint::Sizer; T = vt::rdma::impl::HandleData; Dispatcher = checkpoint::dispatch::BasicDispatcher<checkpoint::Sizer, vt::rdma::impl::HandleData>]
2020-07-12T20:14:57.4129355Z    void applyStatic(
2020-07-12T20:14:57.4129497Z  ^ ~~~~~~~~~
2020-07-12T20:14:57.4129679Z /build/checkpoint/install/include/checkpoint/dispatch/dispatch_serializer_nonbyte.h:190:1: note:   template argument deduction/substitution failed:
2020-07-12T20:15:26.2342461Z [251/441] Building CXX object tests/CMakeFiles/test_mpi_access_guards.dir/unit/main.cc.o

pnstickne · 2020-07-28T07:52:52Z

src/vt/rdmahandle/handle.index.impl.h

- * \brief Convenience using for when U is a vt::NodeType
- */
-template <typename U>
-using isIdx = typename std::enable_if_t<not std::is_same<U,vt::NodeType>::value>;


D:

Seems unfortunate having to move this in line.

ci/docker/ubuntu-18.04-nvidia-cpp.dockerfile

docs/md/building.md

PhilMiller · 2020-07-29T00:24:57Z

Even with this being "out-of-date", I'd suggest waiting for the CI actions to run to completion before pushing the rebase, so that the ccache bits all get digested first

- NVCC warning on constexpr assignment of -1 to unsigned type.

- Usage of decltype for a deleted member in ~some~ SFINAE contexts is failing with an error instead of failing the substitution. Pulling out a pre member check AND using conjunction instead of 'and' appears to appears the compiler. Neither by themselves is sufficient, nor is moving the conjunction inside the has_own_member template sufficient.

- Makes it easier to identify/change.

- NVCC is not able to infer these usages. 10.1 infers more/better than 11.. the ping-pong example did not requires changes in 10.1.

- RdmaHandle is used in serialization. However it is neither a primitive, nor is it marked as byte-copyable. This type is appears to be forcing serialization in VT with NVCC, and the rules for 'is byte copyable' in VT might need to be unified back with checkpoint.

lifflander · 2020-07-29T15:55:32Z

Here is an overview of what got changed by this pull request:

Clones added
============
- src/vt/rdmahandle/handle.node.impl.h  16
- src/vt/rdmahandle/handle.index.impl.h  14

See the complete overview on Codacy

lifflander requested review from PhilMiller, pnstickne, nlslatt and uhetmaniuk July 3, 2020 08:40

lifflander mentioned this pull request Jul 3, 2020

Failures when compiling with nvcc on develop #908

Closed

2 tasks

pnstickne force-pushed the 904-add-cuda-to-ci branch from f697bc0 to 8d741dd Compare July 28, 2020 07:45

pnstickne reviewed Jul 28, 2020

View reviewed changes

pnstickne approved these changes Jul 28, 2020

View reviewed changes

lifflander force-pushed the 904-add-cuda-to-ci branch from 8d741dd to b5b9c5b Compare July 28, 2020 21:37

lifflander commented Jul 29, 2020

View reviewed changes

ci/docker/ubuntu-18.04-nvidia-cpp.dockerfile Outdated Show resolved Hide resolved

PhilMiller reviewed Jul 29, 2020

View reviewed changes

docs/md/building.md Show resolved Hide resolved

PhilMiller approved these changes Jul 29, 2020

View reviewed changes

lifflander and others added 13 commits July 29, 2020 08:52

#904: CI: add cuda to docker compose file

58ebaa2

#904: docs: update to reflect CUDA build mode

95b453f

#904: CI: rewrite cuda container from base image w/nvcc_wrapper

8c37485

#904: CI: implement a new github workflow for CUDA 10.1

c9c4a42

#904: CI: fix small bug in container

924de5c

#904: rdma: work around a nvcc SFINAE bug

3a03f17

#904: CI: add nvcc-11; make base CUDA images much smaller

06afa7c

#904: CI: github workflow for nvcc 11.0

3a60c33

#904: CI: generate the nvcc-10 workflow

d97b19f

#904: CI: generate the nvcc-11 workflow

e30caf3

#904: CI: add extended tests with label trigger for nvcc

deba102

#908 message- add static cast for -1 conversion

f099ebd

- NVCC warning on constexpr assignment of -1 to unsigned type.

pnstickne and others added 9 commits July 29, 2020 08:52

#908 build - pull up revisions in build_cpp.sh

a3c62d0

- Makes it easier to identify/change.

#908 nvcc- add explicit types to templates used has sendMsg handlers

8f27e08

- NVCC is not able to infer these usages. 10.1 infers more/better than 11.. the ping-pong example did not requires changes in 10.1.

#908 cmake- increase compiler template backtrace limit

20b0d68

#904: CI: put all the slower nvcc/intel tests in an optional category

4127366

#904: CI: use the new script for fetching MPI

9f469b1

#904: CI: fix capitalization

6062304

#904: CI: update docs on docker build

d36bd34

#904: docs: update readme with new nvcc badges

ce9b48f

lifflander force-pushed the 904-add-cuda-to-ci branch from 7a1e36b to ce9b48f Compare July 29, 2020 15:53

lifflander merged commit 9ec8d37 into develop Jul 29, 2020

cz4rs mentioned this pull request Jun 15, 2021

move vt version number into a separate file #1467

Closed

7 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

904 Add nvcc (cuda) to CI #907

904 Add nvcc (cuda) to CI #907

lifflander commented Jul 3, 2020 •

edited

Loading

lifflander commented Jul 3, 2020

codecov bot commented Jul 3, 2020 •

edited

Loading

lifflander commented Jul 3, 2020

pnstickne commented Jul 12, 2020 •

edited

Loading

pnstickne Jul 28, 2020 •

edited

Loading

PhilMiller commented Jul 29, 2020

lifflander commented Jul 29, 2020

904 Add nvcc (cuda) to CI #907

904 Add nvcc (cuda) to CI #907

Conversation

lifflander commented Jul 3, 2020 • edited Loading

lifflander commented Jul 3, 2020

codecov bot commented Jul 3, 2020 • edited Loading

Codecov Report

lifflander commented Jul 3, 2020

pnstickne commented Jul 12, 2020 • edited Loading

pnstickne Jul 28, 2020 • edited Loading

Choose a reason for hiding this comment

PhilMiller commented Jul 29, 2020

lifflander commented Jul 29, 2020

lifflander commented Jul 3, 2020 •

edited

Loading

codecov bot commented Jul 3, 2020 •

edited

Loading

pnstickne commented Jul 12, 2020 •

edited

Loading

pnstickne Jul 28, 2020 •

edited

Loading