Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add cmake for majel #2118

Merged
merged 7 commits into from
May 12, 2017
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
9 changes: 0 additions & 9 deletions Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -61,15 +61,6 @@ RUN git clone https://github.com/woboq/woboq_codebrowser /woboq && \
-DCMAKE_BUILD_TYPE=Release . \
make)

# Install gtest.
#
# NOTE: This is added for quick hack of the development work of
# majel-in-paddle.
RUN git clone https://github.com/google/googletest /gtest && \
cd /gtest && \
git checkout -b release-1.8.0 && \
cmake . && make install

# Configure OpenSSH server. c.f. https://docs.docker.com/engine/examples/running_ssh_service
RUN mkdir /var/run/sshd
RUN echo 'root:root' | chpasswd
Expand Down
10 changes: 0 additions & 10 deletions majel/Makefile

This file was deleted.

61 changes: 0 additions & 61 deletions majel/place.cu

This file was deleted.

8 changes: 8 additions & 0 deletions paddle/CMakeLists.txt
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,14 @@ add_subdirectory(pserver)
add_subdirectory(trainer)
add_subdirectory(scripts)

find_package(boost QUIET)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

我记得design doc里说用轻量级的boost替代品,这里仍然是用的boost吗?

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

这里是boost,不是替代品。

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Explained in #2122


if(Boost_FOUND)
include_directories(${Boost_INCLUDE_DIRS})
include_directories(${CMAKE_CURRENT_SOURCE_DIR})
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

我对cmake不是很熟悉,为何这里要把当前目录给include了?repo/paddle下面貌似没有source code呀。

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Explained in #2122

add_subdirectory(majel)
endif()

if(WITH_C_API)
add_subdirectory(capi)
endif()
Expand Down
2 changes: 2 additions & 0 deletions paddle/majel/.gitignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
build
third-party
34 changes: 34 additions & 0 deletions paddle/majel/CMakeLists.txt
Original file line number Diff line number Diff line change
@@ -0,0 +1,34 @@
cmake_minimum_required(VERSION 3.0)

if(GTEST_INCLUDE_DIR AND GTEST_LIBRARIES)
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Given that we are building Docker container, we can assume in CMakeLists.txt that we have boost and gtest already installed and don't need conditions like this, isn't it?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Explained in #2122

message("-- Found gtest (include: ${GTEST_INCLUDE_DIR}, library: ${GTEST_LIBRARIES})")
else()
# find #include <majel/xx.h>
get_filename_component(PARENT_DIR ${CMAKE_CURRENT_SOURCE_DIR} DIRECTORY)
include_directories(${PARENT_DIR})

# find cmake directory modules
get_filename_component(PARENT_DIR ${PARENT_DIR} DIRECTORY)
set(CMAKE_MODULE_PATH ${CMAKE_MODULE_PATH} "${PARENT_DIR}/cmake")

# enable c++11
set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} -std=c++11")

# enable gtest
set(THIRD_PARTY_PATH ${CMAKE_CURRENT_SOURCE_DIR}/third_party)
Copy link
Contributor

@helinwang helinwang May 12, 2017

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

third_party要不要放到运行cmake的目录(比如./build/)里面,而不是${CMAKE_CURRENT_SOURCE_DIR}里面?这样用户如果删了./build/就肯定是一个clean build。

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Explained in #2122

set(WITH_TESTING ON)
include(external/gtest)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could you explain where does "external" directory come from (have not found any directory named "external" after building majel)? This line works, but I don't understand how it works...

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Explained in #2122

endif()

########################### Build Majel #############################
set(MAJEL_CXX_FILES place.cpp)
Copy link
Contributor Author

@gangliao gangliao May 12, 2017

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

PADDLE includes both CPU and GPU versions. Majel is built with GPU only if it equipped one.

If CMake detects the host machine equipped GPUs, it will invoke cuda_add_library to generate the library, vice visa.

NOTE: The suffix of unit test files should strictly limit to *.cpp, because it doesn't contain any CUDA syntaxes and semantics.

Copy link
Collaborator

@wangkuiyi wangkuiyi May 12, 2017

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If CMake detects the host machine equipped GPUs, it will invoke cuda_add_library to generate the library, vice visa.

If we are going to port Majel to Paddle, I think we will have to enable GPU and CPU versions. Also, we'll have to be able to specify which one to build, as we do with Paddle. Also, I think we shouldn't assume that the development computer having a GPU means that the runtime environment would have GPUs.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

NOTE: The suffix of unit test files should strictly limit to .cpp, because it doesn't contain any CUDA syntaxes and semantics.

I agree that if a source file doesn't include CUDA code, the suffix shouldn't be .cu. But I personally prefer .cc over .cpp following Google C++ Style Guide.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Explained in #2122

set(MAJEL_CUDA_FILES "")

if(CUDA_FOUND)
cuda_add_library(majel ${MAJEL_CUDA_FILES} ${MAJEL_CXX_FILES})
else()
add_library(majel ${MAJEL_CXX_FILES})
endif()
#####################################################################

add_subdirectory(test)
49 changes: 49 additions & 0 deletions paddle/majel/place.cpp
Original file line number Diff line number Diff line change
@@ -0,0 +1,49 @@
#include <majel/place.h>

namespace majel {

namespace detail {

class PlacePrinter : public boost::static_visitor<> {
private:
std::ostream& os_;

public:
PlacePrinter(std::ostream& os) : os_(os) {}

void operator()(const CpuPlace&) { os_ << "CpuPlace"; }

void operator()(const GpuPlace& p) { os_ << "GpuPlace(" << p.device << ")"; }
};

} // namespace majel

static Place the_default_place;

void set_place(const Place& place) { the_default_place = place; }

const Place& get_place() { return the_default_place; }

const GpuPlace default_gpu() { return GpuPlace(0); }

const CpuPlace default_cpu() { return CpuPlace(); }

bool is_gpu_place(const Place& p) {
return boost::apply_visitor(IsGpuPlace(), p);
}

bool is_cpu_place(const Place& p) {
return !boost::apply_visitor(IsGpuPlace(), p);
}

bool places_are_same_class(const Place& p1, const Place& p2) {
return is_gpu_place(p1) == is_gpu_place(p2);
}

std::ostream& operator<<(std::ostream& os, const majel::Place& p) {
majel::detail::PlacePrinter printer(os);
boost::apply_visitor(printer, p);
return os;
}

} // namespace majel
File renamed without changes.
10 changes: 10 additions & 0 deletions paddle/majel/test/CMakeLists.txt
Original file line number Diff line number Diff line change
@@ -0,0 +1,10 @@
file(GLOB_RECURSE ALL_TEST_FILES RELATIVE "${CMAKE_CURRENT_SOURCE_DIR}" "*.cpp" "*.cc")

add_executable(majel_tests ${ALL_TEST_FILES})
add_dependencies(majel_tests majel)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

请问为何既然有了

add_dependencies(majel_tests majel)

还需要

target_link_libraries(majel_tests     
                       ...
                       majel
                      )

感觉不是重复地指定依赖了?
我试了一下删掉add_dependencies,貌似也能编译。

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Explained in #2122

target_link_libraries(majel_tests
${Boost_LIBRARIES}
${GTEST_LIBRARIES}
majel
)
add_test(majel_tests majel_tests)
16 changes: 8 additions & 8 deletions majel/place_test.cu → paddle/majel/test/place_test.cpp
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
#include "gtest/gtest.h"
#include "majel/place.h"
#include <sstream>
#include "gtest/gtest.h"
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry that I forgot to reorder the include files following https://google.github.io/styleguide/cppguide.html#Names_and_Order_of_Includes. You can do that in this PR.

I think the right order is:

#include "majel/place.h"

#include <sstream>

#include "gtest/gtest.h"

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Explained in #2122


TEST(Place, Equality) {
majel::CpuPlace cpu;
Expand All @@ -18,12 +18,12 @@ TEST(Place, Equality) {
}

TEST(Place, Default) {
EXPECT_TRUE(majel::is_gpu_place( majel::get_place()));
EXPECT_TRUE(majel::is_gpu_place( majel::default_gpu()));
EXPECT_TRUE(majel::is_cpu_place( majel::default_cpu()));
EXPECT_TRUE(majel::is_gpu_place(majel::get_place()));
EXPECT_TRUE(majel::is_gpu_place(majel::default_gpu()));
EXPECT_TRUE(majel::is_cpu_place(majel::default_cpu()));

majel::set_place(majel::CpuPlace());
EXPECT_TRUE(majel::is_cpu_place( majel::get_place()));
EXPECT_TRUE(majel::is_cpu_place(majel::get_place()));
}

TEST(Place, Print) {
Expand All @@ -33,8 +33,8 @@ TEST(Place, Print) {
EXPECT_EQ("GpuPlace(1)", ss.str());
}
{
std::stringstream ss;
ss << majel::CpuPlace();
EXPECT_EQ("CpuPlace", ss.str());
std::stringstream ss;
ss << majel::CpuPlace();
EXPECT_EQ("CpuPlace", ss.str());
}
}
6 changes: 6 additions & 0 deletions paddle/majel/test/test_framework.cpp
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
#include "gtest/gtest.h"
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why we need this file? I think we can just link -lgtest_main to have the main function?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Explained in #2122


int main(int argc, char** argv) {
::testing::InitGoogleTest(&argc, argv);
return RUN_ALL_TESTS();
}