Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Feature][Resource Center] Add support for Alibaba Cloud OSS as storage of resource center #11708

Merged
merged 11 commits into from
Sep 20, 2022
Merged

[Feature][Resource Center] Add support for Alibaba Cloud OSS as storage of resource center #11708

merged 11 commits into from
Sep 20, 2022

Conversation

EricGao888
Copy link
Member

@EricGao888 EricGao888 commented Aug 31, 2022 •
edited
Loading

Purpose of the pull request

Brief change log

  • Add support for Alibaba Cloud OSS as storage of resource center.
  • Add UTs.
  • Update related docs.

Verify this pull request

  • Verified by manual tests and UTs.

@EricGao888 EricGao888 added feature new feature backend miss:docs missing documents in PR miss:tests missing unit tests in PR labels Aug 31, 2022
@EricGao888 EricGao888 added this to the 3.1.0 milestone Aug 31, 2022
@EricGao888 EricGao888 self-assigned this Aug 31, 2022
@EricGao888 EricGao888 removed miss:docs missing documents in PR miss:tests missing unit tests in PR labels Sep 2, 2022
@EricGao888 EricGao888 marked this pull request as ready for review September 2, 2022 06:45
@apache apache deleted a comment from sonarcloud bot Sep 2, 2022
@apache apache deleted a comment from codecov-commenter Sep 2, 2022
@EricGao888 EricGao888 added the improvement make more easy to user or prompt friendly label Sep 2, 2022
@EricGao888 EricGao888 mentioned this pull request Sep 2, 2022
Closed
ruanwenjun
ruanwenjun requested changes Sep 5, 2022
View reviewed changes
dolphinscheduler-common/pom.xml Outdated Show resolved Hide resolved
...duler-common/src/main/java/org/apache/dolphinscheduler/common/config/StoreConfiguration.java Show resolved Hide resolved
dolphinscheduler-common/src/main/java/org/apache/dolphinscheduler/common/utils/OssOperator.java Outdated Show resolved Hide resolved
dolphinscheduler-common/src/main/java/org/apache/dolphinscheduler/common/utils/OssOperator.java Outdated Show resolved Hide resolved
dolphinscheduler-common/src/main/java/org/apache/dolphinscheduler/common/utils/OssOperator.java Outdated Show resolved Hide resolved
...uler-common/src/main/java/org/apache/dolphinscheduler/common/utils/PropertyUtilsWrapper.java Outdated Show resolved Hide resolved
tools/dependencies/known-dependencies.txt Show resolved Hide resolved
@caishunfeng
Copy link
Contributor

caishunfeng commented Sep 19, 2022 •
edited
Loading

Hi @EricGao888 please solve the conflicts.
If it can't be merged this week, I will put into 3.2.0.

@EricGao888
Copy link
Member Author

Hi @EricGao888 please solve the conflicts. If it can't be merged this week, I will put into 3.2.0.

I just did some refactoring based on the comments from @ruanwenjun and resolved the conflicts. Thanks for the reminder @caishunfeng : )

@codecov-commenter
Copy link

codecov-commenter commented Sep 19, 2022 •
edited
Loading

Codecov Report

Merging #11708 (179feb2) into dev (ad683c3) will decrease coverage by 0.02%.
The diff coverage is 34.32%.

@@             Coverage Diff              @@
##                dev   #11708      +/-   ##
============================================
- Coverage     38.68%   38.65%   -0.03%     
- Complexity     4006     4031      +25     
============================================
  Files          1002     1005       +3     
  Lines         37213    37344     +131     
  Branches       4249     4250       +1     
============================================
+ Hits          14394    14435      +41     
- Misses        21186    21270      +84     
- Partials       1633     1639       +6
Impacted Files Coverage Δ
...heduler/api/service/impl/ResourcesServiceImpl.java 52.14% <ø> (ø)
.../org/apache/dolphinscheduler/common/Constants.java 78.26% <ø> (ø)
...hinscheduler/common/config/StoreConfiguration.java 0.00% <0.00%> (ø)
...e/dolphinscheduler/common/enums/ResUploadType.java 0.00% <0.00%> (ø)
...phinscheduler/common/factory/OssClientFactory.java 0.00% <0.00%> (ø)
...e/dolphinscheduler/common/utils/PropertyUtils.java 34.83% <ø> (ø)
...ache/dolphinscheduler/spi/enums/ResUploadType.java 0.00% <0.00%> (ø)
...olphinscheduler/plugin/task/api/TaskConstants.java 0.00% <0.00%> (ø)
...che/dolphinscheduler/common/utils/OssOperator.java 34.16% <34.16%> (ø)
...e/dolphinscheduler/common/model/OssConnection.java 100.00% <100.00%> (ø)
... and 3 more

📣 We’re building smart automated test selection to slash your CI/CD build times. Learn more

caishunfeng
caishunfeng previously approved these changes Sep 19, 2022
View reviewed changes
Copy link
Contributor

@caishunfeng caishunfeng left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

...scheduler-common/src/test/java/org/apache/dolphinscheduler/common/utils/OssOperatorTest.java
import com.aliyun.oss.OSS;

@RunWith(MockitoJUnitRunner.class)
public class OssOperatorTest {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍

@caishunfeng
Copy link
Contributor

PTAL @ruanwenjun

ruanwenjun
ruanwenjun approved these changes Sep 20, 2022
View reviewed changes
Copy link
Member

@ruanwenjun ruanwenjun left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@sonarcloud
Copy link

sonarcloud bot commented Sep 20, 2022

Kudos, SonarCloud Quality Gate passed!    Quality Gate passed

Bug A 0 Bugs
Vulnerability A 0 Vulnerabilities
Security Hotspot A 0 Security Hotspots
Code Smell A 10 Code Smells

35.3% 35.3% Coverage
4.8% 4.8% Duplication

@EricGao888 EricGao888 merged commit 66958b9 into apache:dev Sep 20, 2022
@caishunfeng
Copy link
Contributor

Hi @EricGao888 can you cherry-pick this pr to 3.1.0-prepare and do some test for it?

@EricGao888
Copy link
Member Author

Hi @EricGao888 can you cherry-pick this pr to 3.1.0-prepare and do some test for it?

Sure, I will do it later today, together with #12006

@EricGao888 EricGao888 mentioned this pull request Sep 21, 2022
Merged
@caishunfeng caishunfeng added the release cherry-pick Mark this issue/PR had cherry-pick for release version label Sep 23, 2022
xdu-chenrj pushed a commit to xdu-chenrj/dolphinscheduler that referenced this pull request Oct 13, 2022
@EricGao888 @xdu-chenrj
[Feature][Resource Center] Add support for Alibaba Cloud OSS as stora…
d55a8b6 
…ge of resource center (apache#11708)

* Add support for Alibaba Cloud OSS as storage of resource center

* Add dependency licenses

* Refactor code, add unit tests and related docs
@rickchengx rickchengx mentioned this pull request Jan 19, 2023
Closed
3 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@ruanwenjun ruanwenjun ruanwenjun approved these changes

@caishunfeng caishunfeng caishunfeng left review comments

@kezhenxu94 kezhenxu94 Awaiting requested review from kezhenxu94

@zhongjiajie zhongjiajie Awaiting requested review from zhongjiajie

@Tianqi-Dotes Tianqi-Dotes Awaiting requested review from Tianqi-Dotes

@SbloodyS SbloodyS Awaiting requested review from SbloodyS SbloodyS is a code owner

@zhuangchong zhuangchong Awaiting requested review from zhuangchong

Assignees

@EricGao888 EricGao888

Labels
backend document feature new feature improvement make more easy to user or prompt friendly release cherry-pick Mark this issue/PR had cherry-pick for release version
Projects
None yet
Milestone
3.1.0
Development

Successfully merging this pull request may close these issues.

[Feature][Resource Center] Add support for Alibaba Cloud OSS as storage for resource center
4 participants
@EricGao888 @caishunfeng @codecov-commenter @ruanwenjun