Skip to content

Commit

Permalink
Merge remote-tracking branch 'me/issue_2115' into issue_2115
Browse files Browse the repository at this point in the history
  • Loading branch information
yuqi1129 committed Apr 23, 2024
2 parents 98b544e + df94aa3 commit 35f4967
Show file tree
Hide file tree
Showing 651 changed files with 38,716 additions and 6,233 deletions.
6 changes: 4 additions & 2 deletions .github/workflows/backend-integration-test.yml
Original file line number Diff line number Diff line change
Expand Up @@ -95,7 +95,7 @@ jobs:
- name: Backend Integration Test
id: integrationTest
run: |
./gradlew test --rerun-tasks -PskipTests -PtestMode=${{ matrix.test-mode }} -PjdkVersion=${{ matrix.java-version }} -PskipWebITs -P${{ matrix.backend }}
./gradlew test --rerun-tasks -PskipTests -PtestMode=${{ matrix.test-mode }} -PjdkVersion=${{ matrix.java-version }} -PskipWebITs -P${{ matrix.backend }} -PskipPyClientITs
- name: Upload integrate tests reports
uses: actions/upload-artifact@v3
Expand All @@ -106,7 +106,9 @@ jobs:
build/reports
integration-test/build/*.log
integration-test/build/*.tar
integration-test/build/trino-ci-container-log/hive/*.*
integration-test/build/trino-ci-container-log/hdfs/*.*
distribution/package/logs/gravitino-server.out
distribution/package/logs/gravitino-server.log
catalogs/**/*.log
catalogs/**/*.tar
catalogs/**/*.tar
3 changes: 3 additions & 0 deletions .github/workflows/build.yml
Original file line number Diff line number Diff line change
Expand Up @@ -85,6 +85,9 @@ jobs:
java-version: ${{ matrix.java-version }}
distribution: 'temurin'

- name: Test publish to local
run: ./gradlew publishToMavenLocal -x test -PjdkVersion=${{ matrix.java-version }}

- name: Build with Gradle
run: ./gradlew build -PskipITs -PjdkVersion=${{ matrix.java-version }}

Expand Down
10 changes: 5 additions & 5 deletions .github/workflows/python-integration-test.yml
Original file line number Diff line number Diff line change
Expand Up @@ -57,17 +57,17 @@ jobs:
- name: Set up QEMU
uses: docker/setup-qemu-action@v2

- name: Free up disk space
run: |
dev/ci/util_free_space.sh
- name: Python Client Integration Test
id: integrationTest
run: |
./gradlew compileDistribution -x test -PjdkVersion=${{ matrix.java-version }}
for pythonVersion in "3.8" "3.9" "3.10" "3.11"
do
./gradlew -PjdkVersion=${{ matrix.java-version }} -PpythonVersion=${pythonVersion} :client:client-python:integrationTest
echo "Use Python version ${pythonVersion} to test the Python client."
./gradlew -PjdkVersion=${{ matrix.java-version }} -PpythonVersion=${pythonVersion} :client:client-python:test
# Clean Gravitino database to clean test data
rm -rf ./distribution/package/data
done
- name: Upload integrate tests reports
Expand Down
2 changes: 1 addition & 1 deletion .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -35,7 +35,7 @@ out/**
*.ipr

distribution
server/src/main/resources/project.properties
common/src/main/resources/project.properties

dev/docker/*/packages
docs/build
Expand Down
12 changes: 12 additions & 0 deletions LICENSE
Original file line number Diff line number Diff line change
Expand Up @@ -266,9 +266,21 @@
Apache Submarine
./bin/common.sh

Confluent Kafka Streams Examples
./catalogs/catalog-kafka/src/test/java/com/datastrato/gravitino/catalog/kafka/embedded/KafkaClusterEmbedded.java
./catalogs/catalog-kafka/src/test/java/com/datastrato/gravitino/catalog/kafka/embedded/KafkaEmbedded.java
./catalogs/catalog-kafka/src/test/java/com/datastrato/gravitino/catalog/kafka/embedded/ZooKeeperEmbedded.java

Trino
./integration-test-common/src/test/java/com/datastrato/gravitino/integration/test/util/CloseableGroup.java
./trino-connector/src/main/java/com/datastrato/gravitino/trino/connector/catalog/hive/SortingColumn.java

Apache Arrow
./dev/ci/util_free_space.sh

This product bundles a third-party component under the
MIT License.

Kyligence/kylinpy
./clients/client-python/gravitino/utils/exceptions.py
./clients/client-python/gravitino/utils/http_client.py
2 changes: 2 additions & 0 deletions LICENSE.bin
Original file line number Diff line number Diff line change
Expand Up @@ -354,6 +354,7 @@
Snappy Java
XNIO API
WildFly
Confluent Kafka Streams Examples

This product bundles various third-party components also under the
Apache Software Foundation License 1.1
Expand Down Expand Up @@ -391,6 +392,7 @@
Janino Common Compiler
Protocol Buffers
Treelayout
Kyligence/kylinpy

This product bundles various third-party components also under the
Common Development and Distribution License 1.0
Expand Down
62 changes: 62 additions & 0 deletions MAINTAINERS.md
Original file line number Diff line number Diff line change
Expand Up @@ -19,3 +19,65 @@ This document lists the maintainers of the Project. Maintainers may be added onc
| Clearvive | Clearvive | Datastrato |
| Cheyne | ch3yne | Datastrato |
| Jerry Shao | jerryshao | Datastrato |

## Review process

All contributions to the project must be reviewed by **AT LEAST** one maintainer before merging.
For larger changes, it is recommended to have more than one reviewer. In particular, if you are
working on an area you are not familiar with, it is recommended to ask for review from that area by
using `git log --format=full <filename>` to see who committed the most.

## When to commit/merge a pull request

PRs shall not be merged during active, ongoing discussions, unless they address issues such as
critical security fixes or public vulnerabilities. Time should be given prior to merging for
those involved with conversations to be resolved.

## How to merge a pull request

Changes pushed to the main branch on Gravitino cannot be **removed**, that is, we can't do
force-push to it. So please don't add any test commits or anything like that. If the PR is
merged by mistake, please using `git revert` to revert the commit, not `git reset` or anything
like that.

Please use the "Squash and merge" option when merging a PR. This will keep the commit history
clean and meaningful.

Gravitino uses "issue" to track the project progresses. So make sure each PR has a related issue, if
not, please ask the PR author to create one. Unless this PR is a trivial one, like fixing a typo or
something like that, all PRs should have related issues.

1. Before merging, make sure the PR is approved by **at least** one maintainer.
2. Check and modify the PR title and description if necessary to follow the project's guidelines.
3. Make sure the PR has a related issue, if not, please ask the PR author to create one; if wrong,
please correct the PR title and description.
4. Assign the "Assignees" to the PR author.
5. If this PR needs to be backported to other branches, please add the corresponding labels to the
PR, like "branch-0.5", "branch-0.6", etc. GitHub Actions will automatically create a backport
PR for you.
6. After PR is merged, please check the related issue:
- If the issue is not closed, please close it as fixed manually.
- Assign the issue "Assignees" to the PR author.
- Starting from 0.6.0, we will use the "labels" to manage the release versions, so please add
the corresponding labels to the issue. For example, if the issue is fixed in 0.6.0, please
add the label "0.6.0". If the issue is fixed both in 0.6.0 and 0.5.1, please add both labels.

## Policy on backporting bug fixes

Gravitino maintains several branches for different versions, so backporting bug fixes is a
common and necessary operation for maintenance releases. The decision point is when you have a
bug fix, and it's not clear whether it is worth backporting or not. Here is the general policy:

For those bug fixes we should backport:

1. Both the bug and fix are well understood and isolated, that is, the fix is not a large change
that could introduce new bugs. Code being modified is well tested.
2. The bug being addressed is high priority or critical to the community.
3. The backported fix does not vary widely from the main branch fix.

For those bug fixes we should not backport:

1. The bug or fix is not well understood. The code is not well tested. The fix is a large change
that could introduce new bugs.
2. The bug being addressed is low priority or not critical to the community.
3. The backported fix varies widely from the main branch fix.
File renamed without changes.
Original file line number Diff line number Diff line change
@@ -0,0 +1,119 @@
/*
* Copyright 2024 Datastrato Pvt Ltd.
* This software is licensed under the Apache License version 2.
*/
package com.datastrato.gravitino.authorization;

import com.datastrato.gravitino.annotation.Unstable;

/**
* The interface of a privilege. The privilege represents the ability to execute kinds of operations
* for kinds of entities
*/
@Unstable
public interface Privilege {

/** @return The generic name of the privilege. */
Name name();

/** @return A readable string representation for the privilege. */
String simpleString();

/** The name of this privilege. */
enum Name {
/** The privilege to create a catalog. */
CREATE_CATALOG(0L, 1L),
/** The privilege to drop a catalog. */
DROP_CATALOG(0L, 1L << 1),
/** The privilege to alter a catalog. */
ALTER_CATALOG(0L, 1L << 2),
/** The privilege to use a catalog. */
USE_CATALOG(0L, 1L << 3),
/** The privilege to create a schema. */
CREATE_SCHEMA(0L, 1L << 4),
/** The privilege to drop a schema. */
DROP_SCHEMA(0L, 1L << 5),
/** The privilege to alter a schema. */
ALTER_SCHEMA(0L, 1L << 6),
/** the privilege to use a schema. */
USE_SCHEMA(0L, 1L << 7),
/** The privilege to create a table. */
CREATE_TABLE(0L, 1L << 8),
/** The privilege to drop a table. */
DROP_TABLE(0L, 1L << 9),
/** The privilege to write a table. */
WRITE_TABLE(0L, 1L << 10),
/** The privilege to read a table. */
READ_TABLE(0L, 1L << 11),
/** The privilege to create a fileset. */
CREATE_FILESET(0L, 1L << 12),
/** The privilege to drop a fileset. */
DROP_FILESET(0L, 1L << 13),
/** The privilege to write a fileset. */
WRITE_FILESET(0L, 1L << 14),
/** The privilege to read a fileset. */
READ_FILESET(0L, 1L << 15),
/** The privilege to create a topic. */
CREATE_TOPIC(0L, 1L << 16),
/** The privilege to drop a topic. */
DROP_TOPIC(0L, 1L << 17),
/** The privilege to write a topic. */
WRITE_TOPIC(0L, 1L << 18),
/** The privilege to read a topic. */
READ_TOPIC(0L, 1L << 19),
/** The privilege to create a metalake. */
CREATE_METALAKE(0L, 1L << 20),
/** The privilege to manage a metalake, including drop and alter a metalake. */
MANAGE_METALAKE(0L, 1L << 21),
/** The privilege to use a metalake, the user can load the information of the metalake. */
USE_METALAKE(0L, 1L << 22),
/** The privilege to add a user */
ADD_USER(0L, 1L << 23),
/** The privilege to remove a user */
REMOVE_USER(0L, 1L << 24),
/** The privilege to get a user */
GET_USER(0L, 1L << 25),
/** The privilege to add a group */
ADD_GROUP(0L, 1L << 26),
/** The privilege to remove a group */
REMOVE_GROUP(0L, 1L << 27),
/** The privilege to get a group */
GET_GROUP(0L, 1L << 28),
/** The privilege to create a role */
CREATE_ROLE(0L, 1L << 29),
/** The privilege to delete a role */
DELETE_ROLE(0L, 1L << 30),
/** The privilege to grant a role to the user or the group. */
GRANT_ROLE(0L, 1L << 31),
/** The privilege to revoke a role from the user or the group. */
REVOKE_ROLE(0L, 1L << 32),
/** The privilege to get a role */
GET_ROLE(0L, 1L << 33);

private final long highBits;
private final long lowBits;

Name(long highBits, long lowBits) {
this.highBits = highBits;
this.lowBits = lowBits;
}

/**
* Return the low bits of Name
*
* @return The low bits of Name
*/
public long getLowBits() {
return lowBits;
}

/**
* Return the high bits of Name
*
* @return The high bits of Name
*/
public long getHighBits() {
return highBits;
}
}
}
Loading

0 comments on commit 35f4967

Please sign in to comment.