Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

CRYPTO-172: Add support for Linux-riscv64 #264

Merged
merged 23 commits into from
Nov 4, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
23 commits
Select commit Hold shift + click to select a range
b2785f4
Add support for Linux-riscv64
luhenry Oct 30, 2023
2a68c3e
Revert maven version bump
luhenry Oct 31, 2023
e827a92
Add linux-riscv64 docker-based testing
luhenry Oct 31, 2023
2fc4e2e
It's been tested on linux-riscv64
luhenry Oct 31, 2023
736f5f6
Add testing on aarch64 and riscv64 on GitHub Actions
luhenry Oct 31, 2023
58ea881
Merge branch 'master' of github.com:apache/commons-crypto into port-r…
luhenry Oct 31, 2023
294977f
Reduce verbosity
luhenry Oct 31, 2023
c2bdc6f
Reduce verbosity even further
luhenry Oct 31, 2023
930a836
Reduce verbosity some more (and speed up build as well)
luhenry Oct 31, 2023
332fcac
Downgrade JNA version to 5.12
luhenry Oct 31, 2023
28c4884
Fix bad package name
luhenry Oct 31, 2023
616e12b
review
luhenry Nov 1, 2023
d0be99f
review
luhenry Nov 1, 2023
7c146c5
Remove JNA version downgrade
luhenry Nov 2, 2023
f087df9
Merge branch 'master' of github.com:apache/commons-crypto into port-r…
luhenry Nov 2, 2023
54bc9a8
Downgrade on linux-riscv64 on CI
luhenry Nov 2, 2023
27b1197
JNA version override only needed when _running_ the test on linux-ris…
luhenry Nov 2, 2023
57b2c97
Merge branch 'master' of github.com:apache/commons-crypto into port-r…
luhenry Nov 2, 2023
9de357b
review
luhenry Nov 2, 2023
7db0236
Fix merge conflict resolution mistake
luhenry Nov 2, 2023
19644b6
Fix indentation
sebbASF Nov 3, 2023
5e6ad7f
Try again to insert a tab
sebbASF Nov 3, 2023
4c69119
Trim down aarch64 and riscv64 docker images to speed up CI
luhenry Nov 3, 2023
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
22 changes: 22 additions & 0 deletions .github/workflows/maven.yml
Original file line number Diff line number Diff line change
Expand Up @@ -122,3 +122,25 @@ jobs:
OPENSSL_HOME: "C:\\Miniconda\\Library"
run: |
mvn -V -B -ntp clean test-compile -Pbenchmark

build-cross:
strategy:
matrix:
include:
- platform: aarch64
- platform: riscv64
jna_override: "-Djna.version=5.12.0" # See https://github.com/java-native-access/jna/issues/1557
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@b4ffde65f46336ab88eb53be808477a3936bae11 # v4.1.1
with:
persist-credentials: false
- uses: docker/setup-qemu-action@2b82ce82d56a2a04d2637cd93a637ae1b359c0a7 # v2
with:
platforms: ${{ matrix.platform }}
- run: |
# Build package
docker compose -f src/docker/docker-compose.yaml run crypto src/docker/build.sh
# Run on platform
docker compose -f src/docker/docker-compose.yaml run crypto-${{ matrix.platform }} \
mvn -V -B -ntp surefire:test ${{ matrix.jna_override }}
4 changes: 4 additions & 0 deletions Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -125,6 +125,10 @@ linux-armhf:
linux-aarch64:
$(MAKE) native CROSS_PREFIX=aarch64-linux-gnu- OS_NAME=Linux OS_ARCH=aarch64

# for cross-compilation on Ubuntu, install the g++-riscv64-linux-gnu
linux-riscv64:
$(MAKE) native CROSS_PREFIX=riscv64-linux-gnu- OS_NAME=Linux OS_ARCH=riscv64

linux-ppc: # TODO: Untested; may need additional CROSS_PREFIX define
$(MAKE) native OS_NAME=Linux OS_ARCH=ppc

Expand Down
12 changes: 11 additions & 1 deletion Makefile.common
Original file line number Diff line number Diff line change
Expand Up @@ -55,7 +55,7 @@ endif
# The following list must include all OS entries below (apart from Default)
# Also there should be a target in the makefile for each of the combinations
# For example, 'Linux-x86' is invoked by the target 'linux32'
known_os_archs := Linux-x86 Linux-x86_64 Linux-aarch64 Linux-arm Linux-armhf Linux-ppc Linux-ppc64 \
known_os_archs := Linux-x86 Linux-x86_64 Linux-aarch64 Linux-riscv64 Linux-arm Linux-armhf Linux-ppc Linux-ppc64 \
Mac-x86 Mac-x86_64 Mac-arm64 Mac-aarch64 \
FreeBSD-x86_64 \
Windows-x86 Windows-x86_64 \
Expand Down Expand Up @@ -197,6 +197,16 @@ Linux-aarch64_LIBNAME := libcommons-crypto.so
Linux-aarch64_LIBNAME_OSSL3 := libcommons-crypto-ossl3.so
Linux-aarch64_COMMONS_CRYPTO_FLAGS :=

Linux-riscv64_CC := $(CROSS_PREFIX)gcc
Linux-riscv64_CXX := $(CROSS_PREFIX)g++
Linux-riscv64_STRIP := $(CROSS_PREFIX)strip
Linux-riscv64_CXXFLAGS := -Ilib/inc_linux -I"$(JAVA_HOME)/include" -Ilib/inc_mac -O2 -fPIC -fvisibility=hidden -Wall -Werror
Linux-riscv64_CFLAGS := -Ilib/inc_linux -I"$(JAVA_HOME)/include" -Ilib/inc_mac -O2 -fPIC -fvisibility=hidden -Wall -Werror
Linux-riscv64_LINKFLAGS := -shared -static-libgcc
Linux-riscv64_LIBNAME := libcommons-crypto.so
Linux-riscv64_LIBNAME_OSSL3 := libcommons-crypto-ossl3.so
Linux-riscv64_COMMONS_CRYPTO_FLAGS :=

ifndef Mac_INC_OPENSSL
Mac_INC_OPENSSL := /usr/local/opt/openssl/include
endif
Expand Down
1 change: 1 addition & 0 deletions RELEASE-NOTES.txt
Original file line number Diff line number Diff line change
Expand Up @@ -197,6 +197,7 @@ Testing:
- Linux x86_64; OpenSSL 1.1.1; thanks to Alex Remily.
- Windows 64 (mingw64); OpenSSL 1.1.1d; thanks to Alex Remily.
- linux-aarch64; OpenSSL 1.0.2k-fips; thanks to Geoffrey Blake.
- linux-riscv64; OpenSSL 1.1.1f; thanks to Ludovic Henry.
- debian-arm64; OpenSSL 1.1.1f; thanks to Geoffrey Blake.

Historical list of changes: https://commons.apache.org/proper/commons-crypto/changes-report.html
Expand Down
6 changes: 6 additions & 0 deletions pom.xml
Original file line number Diff line number Diff line change
Expand Up @@ -205,6 +205,12 @@ The following provides more details on the included cryptographic software:
<target.name>linux-aarch64</target.name>
</properties>
</profile>
<profile>
<id>linux-riscv64</id>
<properties>
<target.name>linux-riscv64</target.name>
</properties>
</profile>
sebbASF marked this conversation as resolved.
Show resolved Hide resolved
<profile>
<id>jacoco</id>
<activation>
Expand Down
2 changes: 2 additions & 0 deletions src/docker/Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -30,6 +30,8 @@ RUN apt-get update && apt-get --assume-yes install software-properties-common \
&& apt-get --assume-yes install libssl-dev \
&& apt-get --assume-yes install gcc-aarch64-linux-gnu \
&& apt-get --assume-yes install g++-aarch64-linux-gnu \
&& apt-get --assume-yes install gcc-riscv64-linux-gnu \
&& apt-get --assume-yes install g++-riscv64-linux-gnu \
sebbASF marked this conversation as resolved.
Show resolved Hide resolved
&& apt-get --assume-yes install mingw-w64 \
&& apt-get --assume-yes install wget \
&& apt-get --assume-yes install dos2unix \
Expand Down
38 changes: 38 additions & 0 deletions src/docker/Dockerfile.aarch64
Original file line number Diff line number Diff line change
@@ -0,0 +1,38 @@
# Licensed to the Apache Software Foundation (ASF) under one or more
# contributor license agreements. See the NOTICE file distributed with
# this work for additional information regarding copyright ownership.
# The ASF licenses this file to You under the Apache License, Version 2.0
# (the "License"); you may not use this file except in compliance with
# the License. You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.

# This file creates a Docker image for use in building linux-aarch64 on CI

FROM ubuntu:20.04

ENV JAVA_HOME=/usr/lib/jvm/java-17-openjdk-arm64
ENV MAVEN_HOME=/opt/maven
ENV PATH=${MAVEN_HOME}/bin:${PATH}
# Install 64-bit dependencies and tooling.
RUN apt-get update -qq && apt-get -y -qq install \
openjdk-17-jre-headless \
libssl-dev \
curl \
# Bug workaround see https://github.com/docker-library/openjdk/issues/19.
&& /var/lib/dpkg/info/ca-certificates-java.postinst configure

# Do this separately to make upgrades easier
RUN curl -L https://dlcdn.apache.org/maven/maven-3/3.8.8/binaries/apache-maven-3.8.8-bin.tar.gz \
| tar xzf - -C /opt && ln -s /opt/apache-maven-3.8.8 /opt/maven

# Ensure we are in the correct directory (this will be overlaid by the virtual mount)
WORKDIR /home/crypto

CMD /bin/bash
38 changes: 38 additions & 0 deletions src/docker/Dockerfile.riscv64
Original file line number Diff line number Diff line change
@@ -0,0 +1,38 @@
# Licensed to the Apache Software Foundation (ASF) under one or more
# contributor license agreements. See the NOTICE file distributed with
# this work for additional information regarding copyright ownership.
# The ASF licenses this file to You under the Apache License, Version 2.0
# (the "License"); you may not use this file except in compliance with
# the License. You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.

# This file creates a Docker image for use in building linux-riscv64 on CI

FROM riscv64/ubuntu:20.04

ENV JAVA_HOME=/usr/lib/jvm/java-17-openjdk-riscv64
ENV MAVEN_HOME=/opt/maven
ENV PATH=${MAVEN_HOME}/bin:${PATH}
# Install 64-bit dependencies and tooling.
RUN apt-get update -qq && apt-get -y -qq install \
openjdk-17-jre-headless \
libssl-dev \
curl \
# Bug workaround see https://github.com/docker-library/openjdk/issues/19.
&& /var/lib/dpkg/info/ca-certificates-java.postinst configure

# Do this separately to make upgrades easier
RUN curl -L https://dlcdn.apache.org/maven/maven-3/3.8.8/binaries/apache-maven-3.8.8-bin.tar.gz \
| tar xzf - -C /opt && ln -s /opt/apache-maven-3.8.8 /opt/maven

# Ensure we are in the correct directory (this will be overlaid by the virtual mount)
WORKDIR /home/crypto

CMD /bin/bash
5 changes: 3 additions & 2 deletions src/docker/build.sh
Original file line number Diff line number Diff line change
Expand Up @@ -17,18 +17,19 @@

# Script to build native files under Docker

set -e
set -ex

cd /home/crypto # must agree with virtual mount in docker-compose.yaml

# Ensure the correct config file is installed
cp /usr/include/x86_64-linux-gnu/openssl/opensslconf.h /usr/include/openssl

# Run the 64-bit builds.
mvn -V package -Drat.skip
mvn -V -B -ntp clean package -Drat.skip

# use process-classes rather than package to speed up builds
mvn -DskipTests -Drat.skip process-classes -P linux-aarch64
mvn -DskipTests -Drat.skip process-classes -P linux-riscv64
mvn -DskipTests -Drat.skip process-classes -P win64
mvn -DskipTests -Drat.skip process-classes -P linux64

Expand Down
26 changes: 25 additions & 1 deletion src/docker/docker-compose.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,31 @@ version: '3'
services:
crypto:
image: commons-crypto
build: .
build:
context: .
dockerfile: Dockerfile
# mount the source and Maven repo
volumes:
- ../..:/home/crypto
- ~/.m2/repository:/root/.m2/repository

crypto-aarch64: &crypto-aarch64
image: commons-crypto:aarch64
platform: linux/arm64/v8
build:
context: .
dockerfile: Dockerfile.aarch64
# mount the source and Maven repo
volumes:
- ../..:/home/crypto
- ~/.m2/repository:/root/.m2/repository

crypto-riscv64: &crypto-riscv64
image: commons-crypto:riscv64
platform: linux/riscv64
build:
context: .
dockerfile: Dockerfile.riscv64
# mount the source and Maven repo
volumes:
- ../..:/home/crypto
Expand Down