Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Feature][Resource Center] One-time migration script for unmanaged resources #13512

Merged
merged 4 commits into from
Feb 8, 2023

Conversation

Radeity
Copy link
Member

@Radeity Radeity commented Feb 5, 2023

Purpose of the pull request

Brief change log

  • Provide one-time migration script for managing out-dated resources, after refactoring, they can not be managed by new resource center.

  • Usage of the script: (example)

    • User assigns an existed target tenant abc.
    • Run script with: $DOLPHINSCHEDULER_HOME/tools/bin/migrate-resource.sh abc
    • All resources will migrate to specific target tenant's temporary migrate directory .migrate, e.g. resource a/b.sh will migrate to {TENANT_BASE_DIR}/resources/.migrate/a/b.sh, udf x/y.jar will migrate to {TENANT_BASE_DIR}/udf/.migrate/x/y.jar.
  • Update resourceName in t_ds_udfs table if there are udfs bind with migrated resources.

Verify this pull request

  • Manually tested.

@codecov-commenter
Copy link

codecov-commenter commented Feb 5, 2023

Codecov Report

Merging #13512 (661dd34) into dev (0ee82c5) will decrease coverage by 0.02%.
The diff coverage is n/a.

@@             Coverage Diff              @@
##                dev   #13512      +/-   ##
============================================
- Coverage     39.60%   39.59%   -0.02%     
+ Complexity     4356     4354       -2     
============================================
  Files          1097     1097              
  Lines         41156    41131      -25     
  Branches       4714     4696      -18     
============================================
- Hits          16299    16284      -15     
+ Misses        23047    23034      -13     
- Partials       1810     1813       +3     
Impacted Files Coverage Δ
...er/master/dispatch/host/assign/RandomSelector.java 77.77% <0.00%> (-5.56%) ⬇️
...erver/master/processor/queue/TaskEventService.java 75.00% <0.00%> (-5.36%) ⬇️
...dolphinscheduler/remote/future/ResponseFuture.java 81.96% <0.00%> (-1.64%) ⬇️
...hinscheduler/plugin/alert/script/ScriptSender.java 70.58% <0.00%> (-1.64%) ⬇️
...hinscheduler/plugin/task/flink/FlinkArgsUtils.java 63.90% <0.00%> (-1.21%) ⬇️
...cheduler/plugin/alert/dingtalk/DingTalkSender.java 34.13% <0.00%> (-0.78%) ⬇️
...rver/master/runner/task/BlockingTaskProcessor.java 75.86% <0.00%> (-0.55%) ⬇️
...inscheduler/plugin/registry/etcd/EtcdRegistry.java 50.69% <0.00%> (-0.35%) ⬇️
.../dolphinscheduler/plugin/task/datax/DataxTask.java 36.62% <0.00%> (-0.26%) ⬇️
.../dolphinscheduler/plugin/task/spark/SparkTask.java 72.56% <0.00%> (-0.25%) ⬇️
... and 17 more

📣 We’re building smart automated test selection to slash your CI/CD build times. Learn more

Copy link
Member

@SbloodyS SbloodyS left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please add some docs.

@SbloodyS SbloodyS added feature new feature 3.2.0 for 3.2.0 version labels Feb 6, 2023
@SbloodyS SbloodyS added this to the 3.2.0 milestone Feb 6, 2023
@Radeity
Copy link
Member Author

Radeity commented Feb 6, 2023

Please add some docs.

Done, PTAL. @SbloodyS

kezhenxu94
kezhenxu94 previously approved these changes Feb 6, 2023
SbloodyS
SbloodyS previously approved these changes Feb 6, 2023
Copy link
Member

@SbloodyS SbloodyS left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+1

@SbloodyS
Copy link
Member

SbloodyS commented Feb 7, 2023

@Radeity Please check the failed CI.

…er/tools/resource/MigrateResourceService.java

Co-authored-by: kezhenxu94 <[email protected]>
@Radeity Radeity dismissed stale reviews from SbloodyS and kezhenxu94 via 41eb171 February 7, 2023 15:17
@Radeity
Copy link
Member Author

Radeity commented Feb 7, 2023

@Radeity Please check the failed CI.

Which one do you mean?

@sonarcloud
Copy link

sonarcloud bot commented Feb 7, 2023

SonarCloud Quality Gate failed.    Quality Gate failed

Bug A 0 Bugs
Vulnerability A 0 Vulnerabilities
Security Hotspot A 0 Security Hotspots
Code Smell A 2 Code Smells

0.0% 0.0% Coverage
0.0% 0.0% Duplication

@kezhenxu94 kezhenxu94 merged commit f3134c7 into apache:dev Feb 8, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
3.2.0 for 3.2.0 version backend document feature new feature
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants