Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update ext build system #103

Merged
merged 9 commits into from
Aug 3, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
122 changes: 122 additions & 0 deletions .github/workflows/Linux.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,122 @@
name: Linux
on: [push, pull_request,repository_dispatch]
concurrency:
group: ${{ github.workflow }}-${{ github.ref }}-${{ github.head_ref || '' }}-${{ github.base_ref || '' }}-${{ github.ref != 'refs/heads/main' || github.sha }}
cancel-in-progress: true
defaults:
run:
shell: bash

jobs:
linux:
name: Linux Release
runs-on: ubuntu-latest
container: ${{ matrix.container }}
strategy:
matrix:
# Add commits/tags to build against other DuckDB versions
duckdb_version: [ '<submodule_version>' ]
arch: ['linux_amd64', 'linux_arm64', 'linux_amd64_gcc4']
vcpkg_version: [ '2023.04.15' ]
include:
- arch: 'linux_amd64_gcc4'
container: 'quay.io/pypa/manylinux2014_x86_64'
vcpkg_triplet: 'x64-linux'
- arch: 'linux_amd64'
container: 'ubuntu:18.04'
vcpkg_triplet: 'x64-linux'
- arch: 'linux_arm64'
container: 'ubuntu:18.04'
vcpkg_triplet: 'arm64-linux'
env:
VCPKG_TARGET_TRIPLET: ${{ matrix.vcpkg_triplet }}
GEN: Ninja
VCPKG_TOOLCHAIN_PATH: ${{ github.workspace }}/vcpkg/scripts/buildsystems/vcpkg.cmake

steps:
- name: Install required ubuntu packages
if: ${{ matrix.arch == 'linux_amd64' || matrix.arch == 'linux_arm64' }}
run: |
apt-get update -y -qq
apt-get install -y -qq software-properties-common
add-apt-repository ppa:git-core/ppa
apt-get update -y -qq
apt-get install -y -qq ninja-build make gcc-multilib g++-multilib libssl-dev wget openjdk-8-jdk zip maven unixodbc-dev libc6-dev-i386 lib32readline6-dev libssl-dev libcurl4-gnutls-dev libexpat1-dev gettext unzip build-essential checkinstall libffi-dev curl libz-dev openssh-client
apt-get install -y -qq tar pkg-config

- name: Install Git 2.18.5
if: ${{ matrix.arch == 'linux_amd64' || matrix.arch == 'linux_arm64' }}
run: |
wget https://github.com/git/git/archive/refs/tags/v2.18.5.tar.gz
tar xvf v2.18.5.tar.gz
cd git-2.18.5
make
make prefix=/usr install
git --version

- uses: actions/checkout@v3
with:
fetch-depth: 0
submodules: 'true'

- name: Checkout DuckDB to version
if: ${{ matrix.duckdb_version != '<submodule_version>'}}
run: |
cd duckdb
git checkout ${{ matrix.duckdb_version }}

- name: Setup ManyLinux2014
if: ${{ matrix.arch == 'linux_amd64_gcc4' }}
run: |
./duckdb/scripts/setup_manylinux2014.sh general aws-cli ccache ssh openssl python_alias

- name: Setup Ubuntu
if: ${{ matrix.arch == 'linux_amd64' || matrix.arch == 'linux_arm64' }}
uses: ./duckdb/.github/actions/ubuntu_18_setup
with:
aarch64_cross_compile: 1

- name: Setup vcpkg
uses: lukka/run-vcpkg@v11
with:
vcpkgGitCommitId: 501db0f17ef6df184fcdbfbe0f87cde2313b6ab1

- name: Install postgres (amd64_gcc4)
if: ${{ matrix.arch == 'linux_amd64_gcc4' }}
run: |
yum install -y sudo
yum install -y postgresql-server

- name: Setup Postgres
if: ${{ matrix.arch == 'linux_amd64_gcc4' }}
run: |
mkdir pgdata
chown postgres pgdata
sudo -u postgres initdb -D pgdata
sudo -u postgres pg_ctl -D pgdata start
sleep 5
sudo -u postgres createuser root -s
createdb
psql -c "SELECT 42"

- name: Build extension
env:
GEN: ninja
STATIC_LIBCPP: 1
CC: ${{ matrix.arch == 'linux_arm64' && 'aarch64-linux-gnu-gcc' || '' }}
CXX: ${{ matrix.arch == 'linux_arm64' && 'aarch64-linux-gnu-g++' || '' }}
run: |
make release

- name: Test extension
if: ${{ matrix.arch == 'linux_amd64_gcc4' }}
run: |
psql -c "SELECT 43"
source ./create-postgres-tables.sh
make test_release

- uses: actions/upload-artifact@v2
with:
name: ${{matrix.arch}}-extensions
path: |
build/release/extension/postgres_scanner/postgres_scanner.duckdb_extension
73 changes: 73 additions & 0 deletions .github/workflows/MacOS.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,73 @@
name: MacOS
on: [push, pull_request,repository_dispatch]
concurrency:
group: ${{ github.workflow }}-${{ github.ref }}-${{ github.head_ref || '' }}-${{ github.base_ref || '' }}-${{ github.ref != 'refs/heads/main' || github.sha }}
cancel-in-progress: true
defaults:
run:
shell: bash

jobs:
macos:
name: MacOS Release (${{ matrix.osx_build_arch }})
runs-on: macos-latest
strategy:
matrix:
# Add commits/tags to build against other DuckDB versions
duckdb_version: [ '<submodule_version>' ]
vcpkg_version: [ '2023.04.15' ]
vcpkg_triplet: [ 'x64-osx', 'arm64-osx' ]
include:
- vcpkg_triplet: 'x64-osx'
osx_build_arch: 'x86_64'
duckdb_arch: 'osx_amd64'
- vcpkg_triplet: 'arm64-osx'
osx_build_arch: 'arm64'
duckdb_arch: 'osx_arm64'

env:
VCPKG_TARGET_TRIPLET: ${{ matrix.vcpkg_triplet }}
OSX_BUILD_ARCH: ${{ matrix.osx_build_arch }}
GEN: Ninja
VCPKG_TOOLCHAIN_PATH: ${{ github.workspace }}/vcpkg/scripts/buildsystems/vcpkg.cmake

steps:
- uses: actions/checkout@v3
with:
fetch-depth: 0
submodules: 'true'

- name: Install Ninja
run: brew install ninja

- name: Setup Ccache
uses: hendrikmuhs/ccache-action@main
with:
key: ${{ github.job }}-${{ matrix.duckdb_version }}
save: ${{ github.ref == 'refs/heads/master' || github.repository != 'duckdb/duckdb' }}

- uses: actions/setup-python@v2
with:
python-version: '3.7'

- name: Setup vcpkg
uses: lukka/run-vcpkg@v11
with:
vcpkgGitCommitId: 501db0f17ef6df184fcdbfbe0f87cde2313b6ab1

- name: Build extension
shell: bash
run: |
make release

- name: Test Extension
if: ${{ matrix.osx_build_arch == 'x86_64'}}
shell: bash
run: |
./build/release/test/unittest "*test/postgres_scanner/aws-rds.test"

- uses: actions/upload-artifact@v2
with:
name: osx-${{ matrix.osx_build_arch }}-extension
path: |
build/release/extension/postgres_scanner/postgres_scanner.duckdb_extension
61 changes: 61 additions & 0 deletions .github/workflows/Windows.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,61 @@
name: Windows
on: [push, pull_request,repository_dispatch]
concurrency:
group: ${{ github.workflow }}-${{ github.ref }}-${{ github.head_ref || '' }}-${{ github.base_ref || '' }}-${{ github.ref != 'refs/heads/main' || github.sha }}
cancel-in-progress: true
defaults:
run:
shell: bash

jobs:
windows:
name: Release
runs-on: windows-latest
strategy:
matrix:
# Add commits/tags to build against other DuckDB versions
duckdb_version: [ '<submodule_version>' ]
vcpkg_version: [ '2023.04.15' ]
vcpkg_triplet: ['x64-windows-static-md']
env:
VCPKG_TARGET_TRIPLET: ${{ matrix.vcpkg_triplet }}
GEN: Ninja
VCPKG_ROOT: ${{ github.workspace }}\vcpkg
VCPKG_TOOLCHAIN_PATH: ${{ github.workspace }}\vcpkg\scripts\buildsystems\vcpkg.cmake

steps:
- uses: actions/checkout@v3
with:
fetch-depth: 0
submodules: 'true'

- name: Setup vcpkg
uses: lukka/run-vcpkg@v11
with:
vcpkgGitCommitId: 501db0f17ef6df184fcdbfbe0f87cde2313b6ab1

- uses: actions/setup-python@v2
with:
python-version: '3.7'

- name: Checkout DuckDB to version
# Add commits/tags to build against other DuckDB versions
if: ${{ matrix.duckdb_version != '<submodule_version>'}}
run: |
cd duckdb
git checkout ${{ matrix.duckdb_version }}

- name: Build extension
run: |
make release

- name: Test Extension
shell: bash
run: |
./build/release/test/Release/unittest "*test/postgres_scanner/aws-rds.test"

- uses: actions/upload-artifact@v2
with:
name: windows-extension
path: |
build/release/extension/postgres_scanner/postgres_scanner.duckdb_extension
109 changes: 0 additions & 109 deletions .github/workflows/main.yml

This file was deleted.

1 change: 0 additions & 1 deletion CMakeLists.txt
Original file line number Diff line number Diff line change
Expand Up @@ -112,7 +112,6 @@ endfunction(PREPEND)

prepend(LIBPG_SOURCES_FULLPATH ${CMAKE_CURRENT_SOURCE_DIR} ${LIBPG_SOURCES})

message(${CMAKE_SOURCE_DIR})
add_custom_command(
OUTPUT ${LIBPG_SOURCES_FULLPATH}
WORKING_DIRECTORY ${CMAKE_CURRENT_SOURCE_DIR}
Expand Down
Loading