Skip to content

Commit

Permalink
add the impl for get file location
Browse files Browse the repository at this point in the history
  • Loading branch information
xloya committed Sep 5, 2024
2 parents e99f7e0 + 939b419 commit 18427c1
Show file tree
Hide file tree
Showing 233 changed files with 4,623 additions and 1,209 deletions.
96 changes: 96 additions & 0 deletions .github/workflows/access-control-integration-test.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,96 @@
name: Access Control Integration Test

# Controls when the workflow will run
on:
# Triggers the workflow on push or pull request events but only for the "main" branch
push:
branches: [ "main", "branch-*" ]
pull_request:
branches: [ "main", "branch-*" ]

concurrency:
group: ${{ github.workflow }}-${{ github.event.pull_request.number || github.ref }}
cancel-in-progress: true

jobs:
changes:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v3
- uses: dorny/paths-filter@v2
id: filter
with:
filters: |
source_changes:
- api/**
- authorizations/**
- catalogs/**
- clients/client-java/**
- clients/client-java-runtime/**
- common/**
- core/**
- integration-test-common/**
- server/**
- server-common/**
outputs:
source_changes: ${{ steps.filter.outputs.source_changes }}

# Integration test for AMD64 architecture
test-amd64-arch:
needs: changes
if: needs.changes.outputs.source_changes == 'true'
runs-on: ubuntu-latest
timeout-minutes: 60
strategy:
matrix:
# Integration test for AMD64 architecture
architecture: [linux/amd64]
java-version: [ 17 ]
test-mode: [ embedded, deploy ]
include:
- test-mode: 'embedded'
backend: 'h2'
- test-mode: 'deploy'
backend: 'mysql'

env:
PLATFORM: ${{ matrix.architecture }}
steps:
- uses: actions/checkout@v3

- uses: actions/setup-java@v4
with:
java-version: ${{ matrix.java-version }}
distribution: 'temurin'
cache: 'gradle'

- name: Set up QEMU
uses: docker/setup-qemu-action@v2

- name: Check required command
run: |
dev/ci/check_commands.sh
- name: Package Gravitino
if : ${{ matrix.test-mode == 'deploy' }}
run: |
./gradlew compileDistribution -x test -PjdkVersion=${{ matrix.java-version }}
- name: Free up disk space
run: |
dev/ci/util_free_space.sh
- name: Authorization Integration Test (JDK${{ matrix.java-version }}-${{ matrix.test-mode }}-${{ matrix.backend }})
id: integrationTest
run: |
./gradlew -PskipTests -PtestMode=${{ matrix.test-mode }} -PjdbcBackend=${{ matrix.backend }} -PjdkVersion=${{ matrix.java-version }} -PskipDockerTests=false :authorizations:authorization-ranger:test --tests "org.apache.gravitino.authorization.ranger.integration.test.**"
- name: Upload integrate tests reports
uses: actions/upload-artifact@v3
if: ${{ (failure() && steps.integrationTest.outcome == 'failure') || contains(github.event.pull_request.labels.*.name, 'upload log') }}
with:
name: authorizations-integrate-test-reports-${{ matrix.java-version }}
path: |
build/reports
distribution/package/logs/gravitino-server.out
distribution/package/logs/gravitino-server.log
14 changes: 8 additions & 6 deletions .github/workflows/backend-integration-test.yml
Original file line number Diff line number Diff line change
Expand Up @@ -55,7 +55,7 @@ jobs:
needs: changes
if: (github.event_name == 'push' && needs.changes.outputs.source_changes == 'true')
runs-on: ubuntu-latest
timeout-minutes: 60
timeout-minutes: 90
strategy:
matrix:
# Integration test for AMD64 architecture
Expand Down Expand Up @@ -95,13 +95,14 @@ jobs:
run: |
dev/ci/util_free_space.sh
- name: Backend Integration Test(JDK${{ matrix.java-version }}-${{ matrix.test-mode }}-${{ matrix.backend }})
- name: Backend Integration Test (JDK${{ matrix.java-version }}-${{ matrix.test-mode }}-${{ matrix.backend }})
id: integrationTest
run: >
./gradlew test -PskipTests -PtestMode=${{ matrix.test-mode }} -PjdkVersion=${{ matrix.java-version }} -PjdbcBackend=${{ matrix.backend }} -PskipWebITs -PskipDockerTests=false
-x :web:test -x :clients:client-python:test -x :flink-connector:test -x :spark-connector:test -x :spark-connector:spark-common:test
-x :web:test -x :clients:client-python:test -x :flink-connector:flink:test -x :spark-connector:test -x :spark-connector:spark-common:test
-x :spark-connector:spark-3.3:test -x :spark-connector:spark-3.4:test -x :spark-connector:spark-3.5:test
-x :spark-connector:spark-runtime-3.3:test -x :spark-connector:spark-runtime-3.4:test -x :spark-connector:spark-runtime-3.5:test
-x :authorizations:authorization-ranger:test
- name: Upload integrate tests reports
uses: actions/upload-artifact@v3
Expand All @@ -124,7 +125,7 @@ jobs:
needs: changes
if: (github.event_name == 'pull_request' && needs.changes.outputs.source_changes == 'true')
runs-on: ubuntu-latest
timeout-minutes: 60
timeout-minutes: 90
strategy:
matrix:
# Integration test for AMD64 architecture
Expand Down Expand Up @@ -164,13 +165,14 @@ jobs:
run: |
dev/ci/util_free_space.sh
- name: Backend Integration Test(JDK${{ matrix.java-version }}-${{ matrix.test-mode }}-${{ matrix.backend }})
- name: Backend Integration Test (JDK${{ matrix.java-version }}-${{ matrix.test-mode }}-${{ matrix.backend }})
id: integrationTest
run: >
./gradlew test -PskipTests -PtestMode=${{ matrix.test-mode }} -PjdkVersion=${{ matrix.java-version }} -PjdbcBackend=${{ matrix.backend }} -PskipWebITs -PskipDockerTests=false
-x :web:test -x :clients:client-python:test -x :flink-connector:test -x :spark-connector:test -x :spark-connector:spark-common:test
-x :web:test -x :clients:client-python:test -x :flink-connector:flink:test -x :spark-connector:test -x :spark-connector:spark-common:test
-x :spark-connector:spark-3.3:test -x :spark-connector:spark-3.4:test -x :spark-connector:spark-3.5:test
-x :spark-connector:spark-runtime-3.3:test -x :spark-connector:spark-runtime-3.4:test -x :spark-connector:spark-runtime-3.5:test
-x :authorizations:authorization-ranger:test
- name: Upload integrate tests reports
uses: actions/upload-artifact@v3
Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/docker-image.yml
Original file line number Diff line number Diff line change
Expand Up @@ -115,7 +115,7 @@ jobs:
sudo rm -rf /usr/local/lib/android
sudo rm -rf /opt/hostedtoolcache/CodeQL
if [[ "${image_type}" == "gravitino" || "{image_type}" == "iceberg-rest-server" ]]; then
if [[ "${image_type}" == "gravitino" || "${image_type}" == "iceberg-rest-server" ]]; then
./dev/docker/build-docker.sh --platform all --type ${image_type} --image ${image_name} --tag ${{ github.event.inputs.version }} --latest
else
./dev/docker/build-docker.sh --platform all --type ${image_type} --image ${image_name} --tag "${tag_name}-${{ github.event.inputs.version }}"
Expand Down
10 changes: 5 additions & 5 deletions .github/workflows/flink-integration-test.yml
Original file line number Diff line number Diff line change
Expand Up @@ -86,8 +86,8 @@ jobs:
- name: Flink Integration Test
id: integrationTest
run: |
./gradlew -PskipTests -PtestMode=embedded -PjdkVersion=${{ matrix.java-version }} -PskipDockerTests=false :flink-connector:test --tests "org.apache.gravitino.flink.connector.integration.test.**"
./gradlew -PskipTests -PtestMode=deploy -PjdkVersion=${{ matrix.java-version }} -PskipDockerTests=false :flink-connector:test --tests "org.apache.gravitino.flink.connector.integration.test.**"
./gradlew -PskipTests -PtestMode=embedded -PjdkVersion=${{ matrix.java-version }} -PskipDockerTests=false :flink-connector:flink:test --tests "org.apache.gravitino.flink.connector.integration.test.**"
./gradlew -PskipTests -PtestMode=deploy -PjdkVersion=${{ matrix.java-version }} -PskipDockerTests=false :flink-connector:flink:test --tests "org.apache.gravitino.flink.connector.integration.test.**"
- name: Upload integrate tests reports
uses: actions/upload-artifact@v3
Expand All @@ -96,9 +96,9 @@ jobs:
name: flink-connector-integrate-test-reports-${{ matrix.java-version }}
path: |
build/reports
flink-connector/build/flink-connector-integration-test.log
flink-connector/build/*.tar
flink-connector/flink/build/*.log
flink-connector/flink/build/*.tar
distribution/package/logs/gravitino-server.out
distribution/package/logs/gravitino-server.log
catalogs/**/*.log
catalogs/**/*.tar
catalogs/**/*.tar
2 changes: 1 addition & 1 deletion DISCLAIMER_WIP.txt
Original file line number Diff line number Diff line change
Expand Up @@ -3,4 +3,4 @@ Apache Gravitino is an effort undergoing incubation at The Apache Software Found
Some of the incubating project’s releases may not be fully compliant with ASF policy and while we have documented the licensing of all code in detail, we know that currently our release would:
- Contains code that may not be compatible with the Apache License

If you are planning to incorporate this work into your product/project, please be aware that you will need to conduct a thorough licensing review to determine the overall implications of including this work. For the current status of this project through the Apache Incubator, visit: https://incubator.apache.org/projects/Apache Podling-Name.html
If you are planning to incorporate this work into your product/project, please be aware that you will need to conduct a thorough licensing review to determine the overall implications of including this work. For the current status of this project through the Apache Incubator, visit: https://incubator.apache.org/projects/gravitino.html
1 change: 1 addition & 0 deletions LICENSE
Original file line number Diff line number Diff line change
Expand Up @@ -266,6 +266,7 @@
./dev/docker/hive/hadoop-env.sh
./dev/docker/kerberos-hive/hadoop-env.sh
./dev/docker/kerberos-hive/yarn-env.sh
./common/src/main/java/org/apache/gravitino/audit/CallerContext.java
./common/src/main/java/org/apache/gravitino/auth/KerberosUtils.java
./server-common/src/main/java/org/apache/gravitino/server/authentication/KerberosAuthenticator.java
./server-common/src/test/java/org/apache/gravitino/server/authentication/KerberosTestUtils.java
Expand Down
13 changes: 12 additions & 1 deletion LICENSE.bin
Original file line number Diff line number Diff line change
Expand Up @@ -296,6 +296,11 @@
Apache HttpCore
Apache HttpClient
Apache Iceberg
Apache Iceberg Aliyun
Apache Iceberg api
Apache Iceberg AWS
Apache Iceberg core
Apache Iceberg Hive metastore
Apache Ivy
Apache Log4j 1.x Compatibility API
Apache Log4j API
Expand All @@ -310,6 +315,8 @@
Apache Yetus - Audience Annotations
Apache Kerby
Apache Kyuubi
Apache Ranger
Apache Ranger intg
Jackson JSON processor
DataNucleus
Modernizer Maven Plugin
Expand Down Expand Up @@ -356,6 +363,8 @@
WildFly
Confluent Kafka Streams Examples
Apache Arrow
Rome
Jettison

This product bundles various third-party components also under the
Apache Software Foundation License 1.1
Expand All @@ -364,7 +373,7 @@

This product bundles various third-party components also under the
BSD license

JSR305
LevelDB JNI
RocksDB JNI
Expand Down Expand Up @@ -402,6 +411,7 @@
Common Development and Distribution License 1.0

Javax Activation
Javax Mail
Stax API
Java Servlet API
JSR311 API
Expand Down Expand Up @@ -446,6 +456,7 @@
Jakarta RESTful WS API
Jakarta XML Binding API
JavaServer Pages(TM) API
Javax WS RS API
HK2 API Module
HK2 Service Locator
HK2 Utils
Expand Down
4 changes: 2 additions & 2 deletions NOTICE
Original file line number Diff line number Diff line change
Expand Up @@ -4,8 +4,8 @@ Copyright 2024 The Apache Software Foundation
This product includes software developed at
The Apache Software Foundation (http://www.apache.org/).

This product includes software developed at
Datastrato (https://datastrato.ai).
The initial code for the Gravitino project was donated
to the ASF by Datastrato (https://datastrato.ai/) copyright 2023-2024.

The Web UI also has a NOTICE file please see web/NOTICE
for it's contents.
Expand Down
29 changes: 16 additions & 13 deletions NOTICE.bin
Original file line number Diff line number Diff line change
Expand Up @@ -4,8 +4,8 @@ Copyright 2024 The Apache Software Foundation
This product includes software developed at
The Apache Software Foundation (http://www.apache.org/).

This product includes software developed at
Datastrato (https://datastrato.ai).
The initial code for the Gravitino project was donated
to the ASF by Datastrato (https://datastrato.ai/) copyright 2023-2024.

The Web UI also has a NOTICE file please see web/NOTICE
for it's contents.
Expand All @@ -22,6 +22,9 @@ Copyright 2008-2023 The Apache Software Foundation
Apache Zeppelin
Copyright 2016-2023 The Apache Software Foundation

Apache Ranger
Copyright 2014-2024 The Apache Software Foundation

Apache Hadoop
Copyright 2006 and onwards The Apache Software Foundation.

Expand Down Expand Up @@ -228,8 +231,8 @@ framework implementation, which can be obtained at:
* license/LICENSE.felix.txt (Apache License 2.0)
* HOMEPAGE:
* http://felix.apache.org/
This product optionally depends on 'Webbit', a Java event based

This product optionally depends on 'Webbit', a Java event based
WebSocket and HTTP server:

* LICENSE:
Expand All @@ -256,13 +259,13 @@ This product includes software from the Eigenbase project, licensed from
DynamoBI Corporation.

Copyright (C) 2005 Dynamo BI Corporation

The ExoLab Project (http://www.exolab.org/)

Sun Microsystems (http://www.sun.com/)

Codehaus (http://castor.codehaus.org)

Tatu Saloranta (http://wiki.fasterxml.com/TatuSaloranta)

Apache Ant
Expand Down Expand Up @@ -306,14 +309,14 @@ and are placed here unaltered.

(C) Copyright 1997,2004 International Business Machines Corporation. All rights reserved.

(C) Copyright IBM Corp. 2003.
(C) Copyright IBM Corp. 2003.


=========================================================================


The portion of the functionTests under 'nist' was originally
developed by the National Institute of Standards and Technology (NIST),
The portion of the functionTests under 'nist' was originally
developed by the National Institute of Standards and Technology (NIST),
an agency of the United States Department of Commerce, and adapted by
International Business Machines Corporation in accordance with the NIST
Software Acknowledgment and Redistribution document at
Expand Down Expand Up @@ -813,8 +816,8 @@ The Apache Software Foundation (http://www.apache.org/).

## Base64 implementation

JJWT's `io.jsonwebtoken.io.Base64` implementation is based on [MigBase64](https://github.com/brsanthu/migbase64) with
continued modifications for Base64 URL support and additional test cases. The MigBase64 copyright and license notice
JJWT's `io.jsonwebtoken.io.Base64` implementation is based on [MigBase64](https://github.com/brsanthu/migbase64) with
continued modifications for Base64 URL support and additional test cases. The MigBase64 copyright and license notice
have been retained and are repeated here per that code's requirements:

```
Expand Down Expand Up @@ -917,4 +920,4 @@ Third-Party Notices and Licenses:

- Hadoop: Apache Hadoop is used as a dependency
License: Apache License 2.0
Source/Reference: https://github.com/apache/hadoop/blob/trunk/NOTICE.txt
Source/Reference: https://github.com/apache/hadoop/blob/trunk/NOTICE.txt
Loading

0 comments on commit 18427c1

Please sign in to comment.