Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[AD-794] Enable code coverage for all platforms #88

Closed
wants to merge 71 commits into from
Closed
Show file tree
Hide file tree
Changes from 61 commits
Commits
Show all changes
71 commits
Select commit Hold shift + click to select a range
c286ad0
[AD-794] sync mac github actions
alinaliBQ Jun 15, 2022
491e5b8
[AD-794] install llvm in mac debug build
alinaliBQ Jun 15, 2022
ca0b188
[AD-794] update mac build script to generate code coverage
alinaliBQ Jun 15, 2022
37bf3dd
[AD-794] define env RUN_CODE_COVERAGE
alinaliBQ Jun 15, 2022
01c195f
[AD-794] fix mac debug build script
alinaliBQ Jun 15, 2022
12a7b0f
[AD-794] attempt to install llvm-cov
alinaliBQ Jun 15, 2022
b8a38e7
[AD-794] upload code coverage MacOS
alinaliBQ Jun 15, 2022
d834867
[AD-794] attempt to fix mac build error
alinaliBQ Jun 15, 2022
27326df
[AD-794] enable Linux debug build on GitHub Actions
alinaliBQ Jun 15, 2022
b59eb9d
[AD-794] update linux debug build script
alinaliBQ Jun 15, 2022
75d18f5
[AD-794] set path of CommandLineTools
alinaliBQ Jun 15, 2022
0f0b414
[AD-794] enable run code coverage on linux
alinaliBQ Jun 15, 2022
62b7ed3
[AD-794] set GITHUB_PATH in Github action
alinaliBQ Jun 15, 2022
a81e389
[AD-794] change build_linux_debug64 to use var BUILD_TYPE
alinaliBQ Jun 15, 2022
5efd190
[AD-794] attempt to fix linux build error
alinaliBQ Jun 15, 2022
d796a1a
[AD-794] attempt to install CommandLineTools
alinaliBQ Jun 15, 2022
602a905
[AD-794] remove install command for CommandLineTools
alinaliBQ Jun 15, 2022
65dd702
[AD-794] append path to command line tools to GITHUB_PATH
alinaliBQ Jun 15, 2022
a31b5c7
[AD-794] update mac and linux debug build script
alinaliBQ Jun 15, 2022
e13d552
[AD-794] update GITHUB_PATH
alinaliBQ Jun 16, 2022
e322601
[AD-794] update mac and linux bash script to run ccov-all on demand
alinaliBQ Jun 16, 2022
75d935c
[AD-784] set BOOST_TEST_CATCH_SYSTEM_ERRORS in Mac and Linux GitHub A…
alinaliBQ Jun 16, 2022
e1757e6
Merge branch 'develop' into alinaliBQ/AD-794/enable-code-cov-mac-linux
alinaliBQ Jun 16, 2022
1e0a57c
[AD-794] rename mac debug build to accurately indicate itself
alinaliBQ Jun 16, 2022
dbb5361
[AD-794] append PATH to GITHUB_PATH
alinaliBQ Jun 16, 2022
add9cd0
[AD-794] fix windows build
alinaliBQ Jun 16, 2022
61b3002
[AD-794] temporarily upload test files for mac and linux build
alinaliBQ Jun 16, 2022
791d4c0
[AD-794] change names for odbc test results
alinaliBQ Jun 16, 2022
93770d5
[AD-794] move env var
alinaliBQ Jun 16, 2022
0bd7048
[AD-794] move `target_code_coverage` before `target_link_libraries` i…
alinaliBQ Jun 16, 2022
cdadebf
[AD-794] update path to odbc test result file from running ccov-all
alinaliBQ Jun 16, 2022
08ece85
[AD-794] update set-PATH
alinaliBQ Jun 16, 2022
ab409ae
[AD-794] create script for generating code coverage on Unix systems
alinaliBQ Jun 16, 2022
dfdaf29
[AD-794] debug mac build
alinaliBQ Jun 16, 2022
5aa5c24
[AD-794] run mac debug script for build-mac-debug action
alinaliBQ Jun 16, 2022
2ed1064
[AD-794] fix mac debug build error
alinaliBQ Jun 16, 2022
39c1914
[AD-794] fix post test result error on Mac debug build
alinaliBQ Jun 16, 2022
47fcfde
[AD-794] prepend paths individually
alinaliBQ Jun 16, 2022
ea4166e
[AD-794] attempt to fix Linux build
alinaliBQ Jun 16, 2022
4be0768
[AD-794] run on Mac OS 12 for debug build
alinaliBQ Jun 16, 2022
45bdcac
[AD-794] test to see if mac release fails with prepending path
alinaliBQ Jun 16, 2022
8d501c6
updating code-coverage module and fixing linux debug build
affonsov Jun 17, 2022
da19184
[AD-794] test Mac Debug build on GitHub Actions
alinaliBQ Jun 17, 2022
384f631
Merge branch 'alinaliBQ/AD-794/enable-code-cov-mac-linux' of https://…
alinaliBQ Jun 17, 2022
367dc39
[AD-794] add log messages
alinaliBQ Jun 17, 2022
6ce5339
[AD-794] add log headers to files
alinaliBQ Jun 17, 2022
5d69fef
[AD-794] add log msg
alinaliBQ Jun 17, 2022
9cef585
[AD-794] add more logs and fix old logs
alinaliBQ Jun 17, 2022
f6b1904
[AD-794] Correct call in SQLError with regards to buffer length inter…
Jun 17, 2022
604087d
[AD-794] enable uploading code coverage from mac builds
alinaliBQ Jun 17, 2022
54317fb
[AD-794] disable upload-test-file in Linux debug build
alinaliBQ Jun 17, 2022
e087310
[AD-794] make Linux release build only run on push to develop
alinaliBQ Jun 17, 2022
5e47abd
[AD-794] remove logs
alinaliBQ Jun 17, 2022
43ccbdb
[AD-794] turn back code coverage option for MacOS
alinaliBQ Jun 17, 2022
b7ef5a8
[AD-794] remove unnecessary log msgs
alinaliBQ Jun 17, 2022
75a3347
Revert "[AD-794] add log headers to files"
alinaliBQ Jun 17, 2022
5749fd5
[AD-794] make Mac release build only run if pushing to develop
alinaliBQ Jun 17, 2022
3f336b9
[AD-794] trouble shoot macos debug build
alinaliBQ Jun 17, 2022
035f022
[AD-794] trouble shoot macos debug build
alinaliBQ Jun 17, 2022
670a041
[AD-794] temporarily enable Linux release build
alinaliBQ Jun 17, 2022
f9b9731
[AD-794] call ssh tunnel for Linux and mac debug build
alinaliBQ Jun 17, 2022
19917e5
[AD-794] remove unnecessary step on mac build yaml
alinaliBQ Jun 18, 2022
1e731ba
[AD-794] comment out debug msg
alinaliBQ Jun 18, 2022
ff4bfbc
[AD-794] only enable Linux release on push to develop
alinaliBQ Jun 18, 2022
a1b5317
[AD-794] delete unneeded action `upload-test-file-code-cov`
alinaliBQ Jun 18, 2022
c116276
[AD-794] small format changes
alinaliBQ Jun 18, 2022
6c68cb8
[AD-794] attempt to make Linux code coverage not include files under …
alinaliBQ Jun 18, 2022
0e285d1
Revert "[AD-794] attempt to make Linux code coverage not include file…
alinaliBQ Jun 20, 2022
2538c56
Revert "Revert "[AD-794] attempt to make Linux code coverage not incl…
alinaliBQ Jun 20, 2022
cc4dab8
[AD-794] attempt to exclude 3rd party dependencies on Linux build
alinaliBQ Jun 20, 2022
83511e7
[AD-794] attempt to exclude 3rd party dependencies on Linux build
alinaliBQ Jun 20, 2022
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
162 changes: 162 additions & 0 deletions .github/workflows/linux-build.yml
Original file line number Diff line number Diff line change
Expand Up @@ -24,6 +24,7 @@ env:
DOC_DB_LOCAL_PORT: 27019
DOC_DB_REMOTE_PORT: 27017
DOC_DB_PRIV_KEY_FILE: ~/certs/docdb-sshtunnel.pem
DOC_DB_ODBC_INTEGRATION_TEST: 1
DOC_DB_LOG_PATH: "${{github.workspace}}/build/odbc/logs"
DOC_DB_LOG_LEVEL: "debug"
JDBC_DRIVER_VERSION: "1.2.4"
Expand All @@ -32,6 +33,8 @@ env:
jobs:
build-linux64:
runs-on: ubuntu-20.04
# TODO remove comment out
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

a todo

# if: ${{ github.event_name == 'push' && github.ref == 'refs/heads/develop' }}
steps:
- uses: actions/checkout@v2
- name: Get Java distribution
Expand Down Expand Up @@ -161,6 +164,7 @@ jobs:
check_name: "Ubuntu 20.04 Build Unit Test Results Check"
comment_title: "Ubuntu 20.04 Build Unit Test Results"
files: ./odbc_test_result.xml

- name: upload-test-file
if: always()
uses: actions/upload-artifact@v2
Expand All @@ -169,7 +173,165 @@ jobs:
path: |
./odbc_test_result.xml
./build/odbc/logs/docdb_odbc_*.log

build-linux64-debug:
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

build-linux64-debug is built upon build-linux64

runs-on: ubuntu-20.04
steps:
- uses: actions/checkout@v2
- name: Get Java distribution
run: |
sudo apt-get -y install software-properties-common
wget -O- https://apt.corretto.aws/corretto.key | sudo apt-key add -
sudo apt-add-repository 'deb https://apt.corretto.aws stable main'
sudo apt-get update
sudo apt-get install -y java-17-amazon-corretto-jdk

- name: Get latest version of CMake
uses: lukka/get-cmake@latest

- name: run-cppcheck
run: |
sudo apt install cppcheck
sh run_cppcheck.sh

- name: upload-cppcheck-results
if: failure()
uses: actions/upload-artifact@v2
with:
name: cppcheck-results
path: cppcheck-results.log

- name: Extract key-pair into file
run: |
mkdir ~/certs
echo "${{env.DOC_DB_KEYPAIR}}" > ${{env.DOC_DB_PRIV_KEY_FILE}}
chmod 400 ${{env.DOC_DB_PRIV_KEY_FILE}}

- name: get-dependencies
if: success()
run: |
sudo apt update
sudo apt install libcurl4-openssl-dev libssl-dev uuid-dev zlib1g-dev libpulse-dev linux-headers-$(uname -r) gcc gcc-multilib g++ g++-multilib linux-headers-$(uname -r) build-essential valgrind libboost-all-dev libbson-dev libsasl2-dev lcov

- name: Cache DocumentDB JDBC JAR
id: cache-documentdb-jdbc-jar
uses: actions/cache@v3
with:
path: |
cache/jar
key: cache-documentdb-jdbc-jar-${{env.JDBC_DRIVER_VERSION}}

- name: Download DocumentDB JDBC JAR
if: steps.cache-documentdb-jdbc-jar.outputs.cache-hit != 'true'
run: |
mkdir -p cache/jar
cd cache/jar
wget https://github.com/aws/amazon-documentdb-jdbc-driver/releases/download/v${{env.JDBC_DRIVER_VERSION}}/documentdb-jdbc-${{env.JDBC_DRIVER_VERSION}}-all.jar

- name: Install DocumentDB JDBC JAR
run: |
mkdir -p ${{env.ODBC_BIN_PATH}}/libs
cp cache/jar/documentdb-jdbc-${{env.JDBC_DRIVER_VERSION}}-all.jar ${{env.ODBC_BIN_PATH}}/libs

- name: build-and-install-mongocxx
run: |
wget https://github.com/mongodb/mongo-c-driver/releases/download/1.21.1/mongo-c-driver-1.21.1.tar.gz
tar xzf mongo-c-driver-1.21.1.tar.gz
cd mongo-c-driver-1.21.1
mkdir -p cmake-build
cd cmake-build
cmake -DENABLE_AUTOMATIC_INIT_AND_CLEANUP=OFF ..
sudo make install
cd ../..
git clone https://github.com/mongodb/mongo-cxx-driver.git --branch releases/stable --depth 1
cd mongo-cxx-driver/build
cmake .. -DCMAKE_BUILD_TYPE=Release -DBSONCXX_POLY_USE_MNMLSTC=1 -DCMAKE_INSTALL_PREFIX=/usr/local
sudo make install

- name: install-mongocxx
run: |
cd mongo-c-driver-1.21.1/cmake-build
cmake -DENABLE_AUTOMATIC_INIT_AND_CLEANUP=OFF ..
sudo make
sudo make install
cd ../../
cd mongo-cxx-driver/build
cmake .. -DCMAKE_BUILD_TYPE=Release -DBSONCXX_POLY_USE_MNMLSTC=1 -DCMAKE_INSTALL_PREFIX=/usr/local
sudo make
sudo make install

# TODO enable clang-tidy-check
# https://bitquill.atlassian.net/browse/AD-726

# generate compile_commands.json file for clang-tidy-check, requires mongocxx and boost dependencies
# - name: generate-compile-commands-file
# run: |
# cmake "${{github.workspace}}/src" -DCMAKE_EXPORT_COMPILE_COMMANDS=on -DCMAKE_BUILD_TYPE=Release -DCODE_COVERAGE="OFF" -DBUILD_SHARED_LIBS="OFF" -DWITH_TESTS="ON" -DWITH_CORE="OFF" -DWITH_ODBC="ON"

# - name: clang-tidy-check
# uses: ZedThree/[email protected]
# id: review
# with:
# If there are any comments, fail the check
# - if: steps.review.outputs.total_comments > 0
# run: exit 1

- name: configure-and-build-driver
run: |
./build_linux_debug64.sh

- name: register-odbc-driver
run: |
chmod +r -R ${{env.ODBC_LIB_PATH}}
chmod +x scripts/register_driver_unix.sh
sudo bash scripts/register_driver_unix.sh

- name: set-up-local-mongodb
run: |
docker run --name mongo -e MONGO_INITDB_ROOT_USERNAME=${{env.DOC_DB_USER_NAME}} -e MONGO_INITDB_ROOT_PASSWORD=${{env.DOC_DB_PASSWORD}} -d -p 27017:27017 mongo:latest
chmod +x ./src/odbc-test/scripts/import_test_data.sh
./src/odbc-test/scripts/import_test_data.sh

- name: generate-code-coverage-report
run: |
ssh -f -N -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no -i ${{env.DOC_DB_PRIV_KEY_FILE}} -L${{env.DOC_DB_LOCAL_PORT}}:${{secrets.DOC_DB_HOST}}:${{env.DOC_DB_REMOTE_PORT}} ${{secrets.DOC_DB_USER}}
chmod +x generate_code_cov.sh
./generate_code_cov.sh
env:
RUN_CODE_COVERAGE: ${{ true }}
BOOST_TEST_CATCH_SYSTEM_ERRORS: no

- name: run-tests
run: |
mkdir -p "${{env.DOC_DB_LOG_PATH}}"
ssh -f -N -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no -i ${{env.DOC_DB_PRIV_KEY_FILE}} -L${{env.DOC_DB_LOCAL_PORT}}:${{secrets.DOC_DB_HOST}}:${{env.DOC_DB_REMOTE_PORT}} ${{secrets.DOC_DB_USER}}
./build/odbc/bin/ignite-odbc-tests --catch_system_errors=false

- name: upload-test-report
if: always()
uses: EnricoMi/publish-unit-test-result-action/[email protected]
with:
check_name: "Ubuntu 20.04 Debug Build Unit Test Results Check"
comment_title: "Ubuntu 20.04 Debug Build Unit Test Results"
files: ./odbc_test_result.xml

# TODO Do not create debug log artifacts if build successful.
# https://bitquill.atlassian.net/browse/AD-803
# - name: upload-test-file
# if: always()
# uses: actions/upload-artifact@v3
# with:
# name: odbc-test-results-linux-debug
# path: |
# ./odbc_test_result.xml
# ./build/odbc/logs/docdb_odbc_*.log

- name: upload-coverage
if: always()
uses: actions/upload-artifact@v3
with:
name: code-coverage-Linux
path: cmake-build64/ccov/all-merged
build-linux-docker-image:
runs-on: ubuntu-20.04
steps:
Expand Down
10 changes: 8 additions & 2 deletions .github/workflows/mac-build.yml
Original file line number Diff line number Diff line change
Expand Up @@ -32,6 +32,7 @@ env:
jobs:
build-mac:
runs-on: macos-11
if: ${{ github.event_name == 'push' && github.ref == 'refs/heads/develop' }}
steps:
- uses: actions/checkout@v2
- name: Get Java distribution
Expand Down Expand Up @@ -64,7 +65,12 @@ jobs:
brew install openssl
brew install boost
brew install mongo-cxx-driver


- name: set-PATH
run: |
echo "/Library/Developer/CommandLineTools/usr/bin" >> $GITHUB_PATH
echo "/usr/local/bin/" >> $GITHUB_PATH
echo "$PATH"
- name: Cache DocumentDB JDBC JAR
id: cache-documentdb-jdbc-jar
uses: actions/cache@v3
Expand Down Expand Up @@ -178,4 +184,4 @@ jobs:
# uses: actions/upload-artifact@v2
# with:
# name: mac-test-results
# path: test-output
# path: test-output
Loading