Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[BUG] master task log can't output repair #7029

Merged
merged 4 commits into from
Nov 30, 2021
Merged

[BUG] master task log can't output repair #7029

merged 4 commits into from
Nov 30, 2021

Conversation

boy-xiaozhang
Copy link
Contributor

Fix the problem of master task log can't output in version 2.x. The logger object is not initialized, causing some of the master task logs to fail to output

boy-xiaozhang and others added 4 commits November 23, 2021 12:34
update task_definition_log  field task_params type,modify text to longtext to be consistent with the main table
update t_ds_task_definition_log field task_params
@codecov-commenter
Copy link

Codecov Report

Merging #7029 (8d7adec) into dev (4c4ebb8) will decrease coverage by 0.08%.
The diff coverage is 0.00%.

Impacted file tree graph

@@             Coverage Diff              @@
##                dev    #7029      +/-   ##
============================================
- Coverage     32.66%   32.57%   -0.09%     
+ Complexity     1604     1603       -1     
============================================
  Files           431      431              
  Lines         14342    14347       +5     
  Branches       1437     1437              
============================================
- Hits           4685     4674      -11     
- Misses         9208     9221      +13     
- Partials        449      452       +3     
Impacted Files Coverage Δ
...r/server/master/runner/task/BaseTaskProcessor.java 4.96% <0.00%> (-0.26%) ⬇️
...server/master/runner/task/CommonTaskProcessor.java 4.83% <0.00%> (-1.62%) ⬇️
...ver/master/runner/task/ConditionTaskProcessor.java 0.00% <0.00%> (ø)
...ver/master/runner/task/DependentTaskProcessor.java 0.00% <0.00%> (ø)
...er/server/master/runner/task/SubTaskProcessor.java 0.00% <0.00%> (ø)
...server/master/runner/task/SwitchTaskProcessor.java 0.00% <0.00%> (ø)
...r/plugin/registry/zookeeper/ZookeeperRegistry.java 47.27% <0.00%> (-8.19%) ⬇️
...e/dolphinscheduler/remote/NettyRemotingClient.java 50.70% <0.00%> (-1.41%) ⬇️
...org/apache/dolphinscheduler/remote/utils/Host.java 40.00% <0.00%> (+2.22%) ⬆️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 4c4ebb8...8d7adec. Read the comment docs.

@sonarcloud
Copy link

sonarcloud bot commented Nov 28, 2021

SonarCloud Quality Gate failed.    Quality Gate failed

Bug A 0 Bugs
Vulnerability A 0 Vulnerabilities
Security Hotspot A 0 Security Hotspots
Code Smell A 4 Code Smells

0.0% 0.0% Coverage
0.0% 0.0% Duplication

@boy-xiaozhang boy-xiaozhang changed the title [FIX:211128] master task log can't output repair [BUG] master task log can't output repair Nov 28, 2021
@brave-lee
Copy link
Contributor

Please associate issue

@boy-xiaozhang
Copy link
Contributor Author

Please associate issue

#7040 This is the issue of the pr

Copy link
Contributor

@lenboo lenboo left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+1

@lenboo lenboo merged commit c2585a1 into apache:dev Nov 30, 2021
@lenboo lenboo added this to the 2.0.2-release milestone Nov 30, 2021
Lucaszlei pushed a commit to Lucaszlei/incubator-dolphinscheduler that referenced this pull request Dec 26, 2021
* update task_definition_log field typ

update task_definition_log  field task_params type,modify text to longtext to be consistent with the main table

* update t_ds_task_definition_log field task_params

update t_ds_task_definition_log field task_params

* [FIX:211128] master task log Can't output repair

Co-authored-by: crane <[email protected]>
lenboo added a commit that referenced this pull request Dec 27, 2021
…ervice (#7634)

* [DS-5229][fix] server port custom config
This closes #5229

* [DS-5229][feat] Implement server port custom config
This closes #5229

* [Bug] [readme] Error link to Docker and k8s in readme #6802 (#6862)

* [Bug] [readme] Error link to Docker and k8s in readme #6802
1、modify the error link

* [Bug] [readme] Error link to Docker and k8s in readme #6802
1、modify the error link in readme_zh_cn.md

* [DS-6829][WorkerServer] skip create log dir and print log in dryRun model (#6852)

Co-authored-by: caishunfeng <[email protected]>

* [DS-5229][feat] Implement server port custom config
Modify review suggestion
This closes #5229

* to #6957 implement interface deleteUpstreamRelation deleteDownstreamRelation  (#6966)

* to #6957

* to #6957

* to #6957

* to #6957: group by process definition code

* to #6957: fix sonar

* to #6957: fix sonar

* to #6957: add ut , add preTaskVersion

* to #6957: fix style

* to #6957: use org.apache.dolphinscheduler.spi.utils.StringUtils

Co-authored-by: honghuo.zw <[email protected]>

* Add a demo docker-compose for testing the meter module (#6938)

* update schdule api (#6977)

* [DS-6961][MasterServer] batch dispatch (#6962)

* [DS-6961][MasterServer] batch dispatch

* fix test

Co-authored-by: caishunfeng <[email protected]>

* Fix shell tools for database schema cannot run (#7003)

* [improvement] improve install.sh if then statement (#6782)

* [Fix] Executing install.sh failed when password and others contain @

* Update install.sh

Co-authored-by: Jiajie Zhong <[email protected]>

Co-authored-by: Kirs <[email protected]>
Co-authored-by: Jiajie Zhong <[email protected]>

* [Improvement-5785][common] Fix code format (#6864)

* [DS-5785][style] Fix common module code format

* [DS-5785][style] Fix common module code format

* to #7005: transform params to SPRING_DATASOURCE_XXX (#7007)

* to #7005: transform params to SPRING_DATASOURCE_XXX

* to #7005: exclude h2

Co-authored-by: honghuo.zw <[email protected]>

* fix(kerboros): set correct classloader for kerboros conf load in plugin tasks (#7027)

Co-authored-by: xudong.zhang <[email protected]>

* Refactor code, add class import. (#7028)

* [community] Add new release version to issue template (#7021)

* [Improvement][API][num-1 & 2] Implement createEmptyProcessDefinition & updateProcessDefinitionBasicInfo (#6984)

* [feature] finish interface createEmptyProcessDefinition

* [feature] finish interface updateProcessDefinitionBasicInfo

* [feature] use class Schedule replace ScheduleDto and delete it.

* [feature] delete chinese comment.

* [Improvement] change '.*' to single form of import

Co-authored-by: 熠然 <[email protected]>
Co-authored-by: edward-yang <[email protected]>

* [Upgrade][install] upgrade compatibility exception (#7030)

* upgrade compatibility exception

* upgrade compatibility exception

* to #7000: implement deleteTaskProcessRelation (#7043)

* to #7000: implement deleteTaskProcessRelation

* to #7000: ! CollectionUtils.isEmpty -> CollectionUtils.isNotEmpty

* to #7000: fix bug

* to #7000: add UT

Co-authored-by: honghuo.zw <[email protected]>

* add processDefinition releaseWorkflowAndSchedule (#7044)

* to #7001: delete certain edge in processDefinition (#7038)

* to #7001: delete certain edge in processDefinition

* to #7001: delete certain edge in processDefinition

* to #7001: delete certain edge in processDefinition

* to #7001: delete certain edge in processDefinition

* to #7001: fix checkstyle

* to #7001: delete edge by projectCode,processDefinitionCode,preTaskCode and postTaskCode

* to #7001: add UT

Co-authored-by: honghuo.zw <[email protected]>

* [Improvement][API][num-1] save schedule when workflow is offline in interface 'createEmptyProcessDefinition' (#7047)

* [Improvement] save schedule when workflow is offline

* [Improvement] rollback transactional when creating/updating schedule failed.

* [Improvement] resolve merge conflict

* [Improvement] do not set projectName and processDefinitionName when saving schedule

* [Improvement] do not set projectName and processDefinitionName when saving schedule

Co-authored-by: edward-yang <[email protected]>

* [DS-6963][MasterServer]add cache manager for tenant and user (#6974)

* cache manager for tenant and user

* add test

* fix checkstyle

* add dependencies

* fix sonar check

* use spring boot cache configuration

* add license

* check style

* check style

* add license

* change proxy name to processor

* add license header

* check style

* add license file

* add license file

* add license

Co-authored-by: caishunfeng <[email protected]>

* [Fix-7019][MasterServer] Fix the issue with failing to save and run the process that contains the switch task. (#7046)

* fix token delete request 404 (#6979)

* Fix view task status module on project homepage cannot read property label of undefined (#6956)

* [Improvement][API][num-9] add ProcessTaskRelationServiceImpl.moveTaskProcessRelation (#7053)

* add processDefinition releaseWorkflowAndSchedule

* add ProcessTaskRelationServiceImpl.moveTaskProcessRelation

* add dependency and subprocess judgement

* [DS-7004][MasterServer]fix master still work when it lose zk connection (#7045)

Co-authored-by: caishunfeng <[email protected]>

* [BUG] master task log can't output repair (#7029)

* update task_definition_log field typ

update task_definition_log  field task_params type,modify text to longtext to be consistent with the main table

* update t_ds_task_definition_log field task_params

update t_ds_task_definition_log field task_params

* [FIX:211128] master task log Can't output repair

Co-authored-by: crane <[email protected]>

* return domain after creating success (#7050)

Co-authored-by: ouyangyewei <[email protected]>

* Add E2E tests for some core features (#7025)

* to #7055: tenant-auto-create: true (#7056)

Co-authored-by: honghuo.zw <[email protected]>

* Recover unit tests that have not been run for some time (#7054)

* [Fix-6809][dolphinscheduler-task-sql] Fix sql task can't send alert (#7089)

* [Fix][Create Warning Instance Edit] Fix the problem of incomplete display of the edit form. (#7096)

* [Improvement-7092][UI] Optimize DAG to adapt to situations without locations (#7102)

* [Fix-7070][UI] Remedy the issue with no saving the timeout's strategy into the database. (#7107)

* [Improvement][API] fix 6772 new api bug (#7098)

* add processDefinition releaseWorkflowAndSchedule

* add ProcessTaskRelationServiceImpl.moveTaskProcessRelation

* add dependency and subprocess judgement

* fix api bug

* fix codestyle

* [Fix-7083] [UI] The switch task is missing the else branch (#7084)

* to #7108 (#7109)

* add TaskProcessorFactoryTest Ignore. (#7130)

* [Fix-7103] Fix the issue with the sqoop task missing the verification of props. (#7119)

* add the verification of parameters for the sqoop task

* [Feature][Workflow relationship] Improve English support. (#7129)

* [Feature][Workflow relationship] Format time. (#7123)

* [Feature-6930] [Python]Add workflow as code task type sub_process (#7022)

* [python] add subProcess task

* refactor python gateway server and task for subprocess

* add function comment for getProcessDefinition

* change process_definition usage of Subprocess task

* [Feature-#6422] task group queue (#6722)

* add task group

* modify task group

* pull dev

* add license header

* modify code style

* fix code style

* fix sql error

* fix error

* fix test

* fix test

* fix test

* fix test

* fix code style

* fix ut

* code style

* fix unit test

* test ut

* ut

* add unittest

* test ut

* modify back ut

* majorization code

* fix conflict

* fix ut

Co-authored-by: wangxj <wangxj31>

* [Feature][Workflow relationship] Format name. (#7143)

* to #7065: fix ExecutorService and schedulerService (#7072)

Co-authored-by: honghuo.zw <[email protected]>

* [Feature][dolphinscheduler-api] access control of processDefinition and processInstance in project #7073 (#7080)

* to #7073

* to #7073

* to #7073

Co-authored-by: honghuo.zw <[email protected]>

* Optimize the display of the pop-up window for creating worker groups (#7148)

* [Feature][dolphinscheduler-api] access control of taskDefinition and taskInstance in project to #7081  (#7082)

* to #7081

* fix #7081

* to #7081

Co-authored-by: honghuo.zw <[email protected]>

* fix this issue (#7120)

* [UT#7157]Fix Ut Run Error (#7158)

* to #7157

* to #7157

Co-authored-by: honghuo.zw <[email protected]>

* [Feature-6988][MasterServer] add cache manager for workflow (#7090)

* add cache manager for workflow

* [DS-6988][MasterServer] add cache manager for workflow

* cache evict code optimization

* test

Co-authored-by: caishunfeng <[email protected]>

* [Bug] [dolphinscheduler-api] The pages of project management, workflow instance and task instance are accessed slowly #7061 (#7139)

1、I found that the SQL behind this interface is very slow. The main table is large, but there is no index to use. By add the index, the all mode in the query plan can be changed into a more efficient ref mode. If this optimization still fails to meet the requirements, please contact me and I will continue to optimize。
2、modify index start_time_index in init script and upgrade scripts.

* [BUG][SQL] fix the process list page load slowly. (#7166)

* fix #7601 move ddl to 2.0.1

* fix #7601 move ddl to 2.0.1

* [Feature][MasterServer] add cache for workGroup and schedule #6987 (#7161)

* to #6987

* to #6987

* to #6987

* to #6987

* to #6987

* to #6987: @param -> @name

* to #6987: fix Sonar

* to #6987: fix Sonar

Co-authored-by: honghuo.zw <[email protected]>

* to feature #7169 (#7170)

Co-authored-by: ouyangyewei <[email protected]>

* Refactor code, add class import, keep ‘logger’ variable names uniform. (#7164)

* to feature #7110 (#7111)

Co-authored-by: ouyangyewei <[email protected]>

* [Feature][Workflow] Main package. (#7177)

* [Fix][Warning instance manage] Expand the support of `i18n`. (#7184)

* [Fix-7093][dolphincheduler-server] fix dependent node stuck in running (#7116)

* fix bug 7093

* remove redundant else

Co-authored-by: SbloodyS <[email protected]>

* [Fix-5551][common] Fix ExecutionStatus for hive-on-Tez application (#6289)

* add ENDED state

* Update HadoopUtils.java

* to feature #7180 (#7185)

Co-authored-by: ouyangyewei <[email protected]>

* [Fix-7037][dolphincheduler-server] process instance can not be killed when complement data (#7140)

* fix bug 7037

* update processComplementData

Co-authored-by: SbloodyS <[email protected]>

* [Feature][dolphinscheduler-api] add queryLog and downloadTaskLog with projectCode #7153 (#7192)

* [Feature][dolphinscheduler-api] add queryLog and downloadTaskLog with projectCode #7153

* [Feature][dolphinscheduler-api] add queryLog and downloadTaskLog with projectCode #7153

* [Feature][dolphinscheduler-api] add queryLog and downloadTaskLog with projectCode #7153

Co-authored-by: honghuo.zw <[email protected]>

* to feature #7191 (#7194)

Co-authored-by: ouyangyewei <[email protected]>

* [Fix-7154][API] Fix MultipartFile resource [file] cannot be resolved to absolute file… (#7155)

* fix MultipartFile resource [file] cannot be resolved to absolute file path

* update testCopyFile.

* [Implement][server]TaskProcessor code optimization (#7146)

* TaskProcessor code optimization

* fix code style

* update Collectors.toMap

* fix worker group display (#7208)

* [Fix-7195] The file path of  2.0.1-schema  used a wrong separator. (#7200)

* [Bug-7206] [MasterServer] fix process isntance always running when task timeout (#7207)

* fix timeout

* add task timeout map to avoid repeated timeout event

* split task check list for retry and timeout

Co-authored-by: caishunfeng <[email protected]>

* add two files (#7223)

* fix postgre upgrade (#7230)

Co-authored-by: caishunfeng <[email protected]>

* [Bug] [API] queryProcessDefinitionByCode bug (#7221)

* [Bug] [API] task definition release/delete bug (#7232)

* fix worker group display

* fix workflow query bug

* fix workflow query bug

* fix task definition release/delete bug

* [Feature-6963][MasterServer] unified cache manager (#7187)

* unified cache

* reduce db select

* checkstyle

Co-authored-by: caishunfeng <[email protected]>

* [Refactor]Rename class file, DatasourceUtil  to  DataSourceUtils. (#7181)

* Rename class file, DatasourceUtil  to  DataSourceUtils.

* Fix homepage user wrong statistics result for new user (#7242)

* Fix homepage user wrong statistics result for new user

For now, when we create a new user, it would see homepage
statistic info as admin user like, cause `projectCodeArray`
would not filter when we render SQL. So I add prefix conditions
to handle this situation.

Also, I do some minimal refactor here. including change function
name no quit fit

fix: #7182

* Fix UT

* to #7228 (#7249)

Co-authored-by: honghuo.zw <[email protected]>

* [Feature][dolphinscheduler-api] parse traceId in http header for Cross system delivery to #7237 (#7238)

* to #7237

* rerun test

Co-authored-by: honghuo.zw <[email protected]>

* [bug] fix task definition delete (#7244)

Co-authored-by: edward-yang <[email protected]>

* fix  Property or field 'all' cannot be found (#7246)

* [Fix-7235][datasource] unable to confige the multi-hosts (#7235) (#7253)

* [Bug][dolphinscheduler ui] When the edit time is repaired, the alarm group display. (#7262)

* [ISSUE 6444]Optimize task node helper document in workflow toolbar (#7226)

* [ISSUE 6444]Optimize task node helper document in workflow toolbar

* fix some review issue than submit by DS

* refact by add guard judgement when type is undefined

* move method helpUrlEnable and helpUrl nest in 'methods' body

* fix failover logic (#7275)

Co-authored-by: caishunfeng <[email protected]>

* [python] Refactor get object define communicate to gateway (#7272)

* Change class Base `to_dict` to `get_define` for more clearer
* Remove class TaskParams and sub class make code easy to
  understand and avoid task implement cycle dependence
* Remove class ObjectJsonBase in Task to reduce complexity

fix: #7271

* [Fix-7277][datasource] Support Kerberos auto renewal (#7277) (#7278)

* Split the components into individual package, refactor configurations to use Spring properties (#7160)

* Split the components into individual package

A follow-up PR will be made to build dedicated Docker images for each
component, so that every component Docker image has minimal jars, which
is easy to maintain and good for security fixes.

* Split the components into individual package

A follow-up PR will be made to build dedicated Docker images for each
component, so that every component Docker image has minimal jars, which
is easy to maintain and good for security fixes.

* Split the components into individual package

A follow-up PR will be made to build dedicated Docker images for each
component, so that every component Docker image has minimal jars, which
is easy to maintain and good for security fixes.

* fix in the process alarm message, projectid is 0 (#7298)

* to feature-7301 (#7308)

Co-authored-by: ouyangyewei <[email protected]>

* fix TaskResponseProcessor process NullPointerException (#7318)

* [Bug-7292][ApiServer] fix cache error when standalone (#7293)

* Fix the missing project information in the process instance timeout alarm (#7324)

* fix a few issues in the sql file (#7327)

* Fix some SQL issues and make docker-compose runnable (#7329)

* Fix typo in docker swarm (#7336)

* Adjust Helm Chart templates in dev branch (#7353)

* [python] Add task base database and procedure (#7279)

We add a new task procedure, and add parent class database
for both sql task and procedure task

fix: #6929

* Fix E2E workflow (#7377)

* [Fix-7375][datasource] remove excess hive jdbc principal (#7375) (#7381)

* [Fix-7375][datasource] remove excess hive jdbc principal (#7375)

* fix ut

* [Bug-7319][MasterServer] fix taskNode NPE when switch else branch is empty (#7320)

Co-authored-by: caishunfeng <[email protected]>

* [Fix-7291][datasource] Compatible with different versions of MySQL data source loading (#7291) (#7305)

* [Feature][UI-Next]: Build a front-end refactoring project. (#7387)

* Fix alert plugin instance filter (#7172) (#7210)

* Fix alert plugin instance filter (#7172)

alert plugin could not filter by given
pattern, cause api server do not handle
parameter searchVal, This patch add parameter
`searchVal` for alert plugin.

fix: #7209

* Add test

* Recover AlertPluginInstanceVO

* [Feature-7110][dolphinscheduler-api] support grant project by code (#7390)

Co-authored-by: ouyangyewei <[email protected]>

* [Fix][Project List UI] Fix the calling interface that keeps decreasing by one after manually setting the `pageNo` parameter. (#7399)

* to feature 7401 (#7404)

Co-authored-by: ouyangyewei <[email protected]>

* [Refactor] Rename class file, unified word spelling. (#7261)

* [python] Fix task procedure define error (#7407)

Cause procedure parameter name need `method` instead of `sql`
we change it's name. And remove parameter `sql` from parent
class `Database`

closes: #7386

* [Feature-7408][dolphinscheduler-api] Built GenerateToken into the UpdateToken API (#7411)

* to feature 7408

* trigger e2e test

Co-authored-by: ouyangyewei <[email protected]>

* [DS-7256][refactor] Optimization for task instances query (#7331)

- assemble a Map<Long, User> outside of the loop instead of the query inside

Co-authored-by: gq47193 <[email protected]>

* to feature 7418 (#7420)

Co-authored-by: ouyangyewei <[email protected]>

* [Fix-7392][dolphinscheduler-datasource] Add hive datasource failed (#7393)

* fix bug_7392

* fix bug_7392

* fix bug_7392

Co-authored-by: SbloodyS <[email protected]>

* [Feature][UI Next] Improve some basic configuration. (#7430)

* [Fix-#6854][dao] fix "ResourceMapper.xml -> queryResourcePaging" sql bug (#6907)

* fix complement data retry bug; issues: [Bug] [Master] A bug on task retry mechanism #6613

* [DS-#5293][chore]upgrade cron-utils version 5.0.5 -> 9.1.3

* [DS-#5293]update the licensing information of that dependency

* [DS-#5293]update the licensing information of that dependency

* update the licensing information of that dependency

* [DS-#5293]update the licensing information of that dependency

* [DS-#6854][fix] fix "ResourceMapper.xml -> queryResourcePaging" sql bug

* [DS-#6854][fix] fix "ResourceMapper.xml -> queryResourcePaging" sql bug

Co-authored-by: yangqiyu <[email protected]>

* [python] Add task dependent (#7405)

fix: #6926

* feat: Added the configuration of routes (#7433)

* add task-logger config (#7423)

Co-authored-by: caishunfeng <[email protected]>

* [Fix-7340] Improve the prompt content for deleting a project that didn't exist. (#7449)

* improve the prompt

* [Improvement-7213][MasterServer] execute thread pool code optimization (#7258)

* threadpool optimization

* threadpool params

* rebase dev

* ut check fix

* add return

* rebase dev

* event loop

Co-authored-by: caishunfeng <[email protected]>

* Build and publish snapshot Docker images for testing (#7454)

* Adjust logging levels for third party libraries (#7453)

* feat: Added login pages and functions (#7456)

* [Feature-7349] [Python]Add workflow as code task type datax (#7437)

* support datax task by python
* add task datax example

Co-authored-by: Jiajie Zhong <[email protected]>

* fix this bug (#7461)

* [Feature][UI Next] Add apis. (#7463)

* [Feature][UI Next] Interface debugging(login)

* [Feature-7460][Alert] Wechat alert add send to group chat (#7465)

* feature_7460

* feature_7460

Co-authored-by: SbloodyS <[email protected]>

* remove date+1 when the data is supplemented (#7452)

* Update ProcessDefinitionServiceImpl.java (#7470)

fix typos for logger.error

* [Feature][UI Next] Write part of the api method. (#7476)

* Remove redundant parameters [zookeeper.version] in dolphinscheduler/pom.xml(#7486)

* [Fix-7146][server]Fix the task processor thread is not safe. (#7488)

* Fix the task processor thread is not safe.

* fix code style

* [Feature][UI Next] Write part of the api method. (#7507)

* [DS-7227][refactor] optimize alert plugin instance list order (#7506)

[DS-7227][refactor] optimize alert plugin instance list order
query all alert plugin instance list order by update time desc
optimize `select *` sql in the AlterPluginInstanceMapper.xml

* Add new version 2.0.1 to issue bug template (#7458)

Add new version 2.0.1 for github issue template and remove versions not LTS

* [Fix-7485][datasource-plugin] Fix create hive datasource using ZooKeeper way in 2.0.1 (#7489)

* refactor: Refactor Login to better match the composition API implementation (#7519)

* fix http alarm msg ignored (#7523)

See #7522

* [Fix-7525][Feature][UI Next] Provide a common  layout for the module of dolphinscheduler-ui-next. (#7526)

* add layout

* add layout

* add layout

* add routes

* add routes

* modify layout

* modify layout

* modify layout

* modify layout

* improve a few issues with code specification

* [Feature][UI Next]: Rearrange the routing and format the code (#7533)

* fix: Rearrange the routing and format the code

* To fix the problem

To fix the problem

* [python] Add task condition (#7505)

* [python] Add task condition

close: #6927

* Add example and set downstream dep

* [python] Add task switch (#7531)

* [python] Add task switch

close: #6928

* Fix code style

* Unify the element selector rules for E2E (#7409)

* Unify the element selector rules for E2E

* Fix typo in docker swarm

* [Feature][UI Next] Add charts setting. (#7543)

* [Feature][UI Next] Add charts setting. (#7547)

* fix bug_7537 (#7551)

Co-authored-by: SbloodyS <[email protected]>

* [Bug-7333][MasterServer] fix duplicate key task definition (#7335)

Co-authored-by: caishunfeng <[email protected]>

* [Improvement-#7529][tools] Init DB schema from the full sql file. (#7530)

* [Fix-#7480] [Standalone] add logger server (#7481)

* Change build context to avoid cache

Co-authored-by: kezhenxu94 <[email protected]>

* Fix database lose data in standalone (#7556)

* Modify contribution document (#7552)

* [Improvement] Merge redundant codes (#7550)

* [Feature][UI Next]Refactoring layout  (#7558)

* refactor: Refactoring layout

* chore: Delete unnecessary packages

* finished issue#7527 (#7561)

Co-authored-by: 时光 <[email protected]>

* Update README.md (#7586)

* Update README.md

* Update README.md

Co-authored-by: Kirs <[email protected]>

* Revert "Fix database lose data in standalone (#7556)" (#7604)

This reverts commit 82075a4.

* [Fix-7534][Master] zookeeper failover error (#7562)

* fix bug_7534

* fix bug_7534

* fix bug_7534

* fix bug_7534

* fix bug_7534

* fix bug_7534

Co-authored-by: SbloodyS <[email protected]>

* [Feature][UI Next] Add charts i18n. (#7568)

* [Feature-#6422] [api-server] task group queue (#7491)

* add task group

* modify task group

* pull dev

* add license header

* modify code style

* fix code style

* fix sql error

* fix error

* fix test

* fix test

* fix test

* fix test

* fix code style

* fix ut

* code style

* fix unit test

* test ut

* ut

* add unittest

* test ut

* modify back ut

* majorization code

* fix conflict

* fix ut

* add task group api

* reset file

* fix ut

* fix lost column

* fix ut

* fix ut

* fix ut

* fix ut

* delete duplicate code

* fix code style 、name

* fix ut

* fix mapper

Co-authored-by: wangxj <wangxj31>

* [Feature][UI Next] Add card component. (#7606)

* [Feature][UI Next] Add card component.
* [Feature][UI Next] Update card component props.

* Required E2E to pass before PR can be merged (#7612)

* [python] Correct python import syntax in example (#7617)

fix: #7615

* Try (#7603)

* [Fix-7520][Master] Data too long for column 'task_params' at row 1 (#7521)

* fix bug_7520

* fix bug_7520

* fix bug_7520

* bug_7520

Co-authored-by: SbloodyS <[email protected]>

* [BUG-7554] Fix database lose data in standalone (#7620)

* Fix the wrong lifecycle phase of initialization

Co-authored-by: kezhenxu94 <[email protected]>

* [Feature-7576][Master] Optimize complement task's date (#7585)

* feature_7576

* feature_7576

* feature 7576

* feature 7576

* to #7609 (#7611)

Co-authored-by: honghuo.zw <[email protected]>

* [python] Fix task relation wrong hashable function (#7628)

* [python] Fix task relation wrong hashable function

Currently our :class:`TaskRelation` have wrong __hash__
function, would mistake a object as same object. It failed
our dependence and process definition. this patch fix it
by adding `_KEY_ATTR` to overwrite TaskRelation's __eq__
and correct __hash__ func

close: #7627

* Fix code format

* fix bug_7564 (#7625)

* fix bug_7629 (#7630)

* [DS-7259][fix] Modify the prompt for

This closes #7259

Co-authored-by: GaoTianDuo <[email protected]>
Co-authored-by: wind <[email protected]>
Co-authored-by: caishunfeng <[email protected]>
Co-authored-by: zwZjut <[email protected]>
Co-authored-by: honghuo.zw <[email protected]>
Co-authored-by: kezhenxu94 <[email protected]>
Co-authored-by: JinYong Li <[email protected]>
Co-authored-by: xuhhui <[email protected]>
Co-authored-by: Kirs <[email protected]>
Co-authored-by: Jiajie Zhong <[email protected]>
Co-authored-by: lvshaokang <[email protected]>
Co-authored-by: sparklezzz <[email protected]>
Co-authored-by: xudong.zhang <[email protected]>
Co-authored-by: Zonglei Dong <[email protected]>
Co-authored-by: EdwardYang <[email protected]>
Co-authored-by: 熠然 <[email protected]>
Co-authored-by: edward-yang <[email protected]>
Co-authored-by: Hua Jiang <[email protected]>
Co-authored-by: Kerwin <[email protected]>
Co-authored-by: zhang <[email protected]>
Co-authored-by: crane <[email protected]>
Co-authored-by: ouyangyewei <[email protected]>
Co-authored-by: ouyangyewei <[email protected]>
Co-authored-by: 张俊杰 <[email protected]>
Co-authored-by: songjianet <[email protected]>
Co-authored-by: wangyizhi <[email protected]>
Co-authored-by: Devosend <[email protected]>
Co-authored-by: wangxj3 <[email protected]>
Co-authored-by: BaoLiang <[email protected]>
Co-authored-by: SbloodyS <[email protected]>
Co-authored-by: SbloodyS <[email protected]>
Co-authored-by: Manhua <[email protected]>
Co-authored-by: mask <[email protected]>
Co-authored-by: 百岁 <[email protected]>
Co-authored-by: mgduoduo <[email protected]>
Co-authored-by: gq47193 <[email protected]>
Co-authored-by: KyoYang <[email protected]>
Co-authored-by: yangqiyu <[email protected]>
Co-authored-by: labbomb <[email protected]>
Co-authored-by: Jiajie Zhong <[email protected]>
Co-authored-by: Ivan0626 <[email protected]>
Co-authored-by: huangxiaohai <[email protected]>
Co-authored-by: Tom <[email protected]>
Co-authored-by: wangbowen <[email protected]>
Co-authored-by: Jiezhi.G <[email protected]>
Co-authored-by: lgcareer <[email protected]>
Co-authored-by: Wenjun Ruan <[email protected]>
Co-authored-by: J·Y <[email protected]>
Co-authored-by: 时光 <[email protected]>
Co-authored-by: 时光 <[email protected]>
Co-authored-by: lifeng <[email protected]>
Co-authored-by: Kirs <[email protected]>
aaronlinv pushed a commit to aaronlinv/dolphinscheduler that referenced this pull request Dec 28, 2021
…ervice (apache#7634)

* [DS-5229][fix] server port custom config
This closes apache#5229

* [DS-5229][feat] Implement server port custom config
This closes apache#5229

* [Bug] [readme] Error link to Docker and k8s in readme apache#6802 (apache#6862)

* [Bug] [readme] Error link to Docker and k8s in readme apache#6802
1、modify the error link

* [Bug] [readme] Error link to Docker and k8s in readme apache#6802
1、modify the error link in readme_zh_cn.md

* [DS-6829][WorkerServer] skip create log dir and print log in dryRun model (apache#6852)

Co-authored-by: caishunfeng <[email protected]>

* [DS-5229][feat] Implement server port custom config
Modify review suggestion
This closes apache#5229

* to apache#6957 implement interface deleteUpstreamRelation deleteDownstreamRelation  (apache#6966)

* to apache#6957

* to apache#6957

* to apache#6957

* to apache#6957: group by process definition code

* to apache#6957: fix sonar

* to apache#6957: fix sonar

* to apache#6957: add ut , add preTaskVersion

* to apache#6957: fix style

* to apache#6957: use org.apache.dolphinscheduler.spi.utils.StringUtils

Co-authored-by: honghuo.zw <[email protected]>

* Add a demo docker-compose for testing the meter module (apache#6938)

* update schdule api (apache#6977)

* [DS-6961][MasterServer] batch dispatch (apache#6962)

* [DS-6961][MasterServer] batch dispatch

* fix test

Co-authored-by: caishunfeng <[email protected]>

* Fix shell tools for database schema cannot run (apache#7003)

* [improvement] improve install.sh if then statement (apache#6782)

* [Fix] Executing install.sh failed when password and others contain @

* Update install.sh

Co-authored-by: Jiajie Zhong <[email protected]>

Co-authored-by: Kirs <[email protected]>
Co-authored-by: Jiajie Zhong <[email protected]>

* [Improvement-5785][common] Fix code format (apache#6864)

* [DS-5785][style] Fix common module code format

* [DS-5785][style] Fix common module code format

* to apache#7005: transform params to SPRING_DATASOURCE_XXX (apache#7007)

* to apache#7005: transform params to SPRING_DATASOURCE_XXX

* to apache#7005: exclude h2

Co-authored-by: honghuo.zw <[email protected]>

* fix(kerboros): set correct classloader for kerboros conf load in plugin tasks (apache#7027)

Co-authored-by: xudong.zhang <[email protected]>

* Refactor code, add class import. (apache#7028)

* [community] Add new release version to issue template (apache#7021)

* [Improvement][API][num-1 & 2] Implement createEmptyProcessDefinition & updateProcessDefinitionBasicInfo (apache#6984)

* [feature] finish interface createEmptyProcessDefinition

* [feature] finish interface updateProcessDefinitionBasicInfo

* [feature] use class Schedule replace ScheduleDto and delete it.

* [feature] delete chinese comment.

* [Improvement] change '.*' to single form of import

Co-authored-by: 熠然 <[email protected]>
Co-authored-by: edward-yang <[email protected]>

* [Upgrade][install] upgrade compatibility exception (apache#7030)

* upgrade compatibility exception

* upgrade compatibility exception

* to apache#7000: implement deleteTaskProcessRelation (apache#7043)

* to apache#7000: implement deleteTaskProcessRelation

* to apache#7000: ! CollectionUtils.isEmpty -> CollectionUtils.isNotEmpty

* to apache#7000: fix bug

* to apache#7000: add UT

Co-authored-by: honghuo.zw <[email protected]>

* add processDefinition releaseWorkflowAndSchedule (apache#7044)

* to apache#7001: delete certain edge in processDefinition (apache#7038)

* to apache#7001: delete certain edge in processDefinition

* to apache#7001: delete certain edge in processDefinition

* to apache#7001: delete certain edge in processDefinition

* to apache#7001: delete certain edge in processDefinition

* to apache#7001: fix checkstyle

* to apache#7001: delete edge by projectCode,processDefinitionCode,preTaskCode and postTaskCode

* to apache#7001: add UT

Co-authored-by: honghuo.zw <[email protected]>

* [Improvement][API][num-1] save schedule when workflow is offline in interface 'createEmptyProcessDefinition' (apache#7047)

* [Improvement] save schedule when workflow is offline

* [Improvement] rollback transactional when creating/updating schedule failed.

* [Improvement] resolve merge conflict

* [Improvement] do not set projectName and processDefinitionName when saving schedule

* [Improvement] do not set projectName and processDefinitionName when saving schedule

Co-authored-by: edward-yang <[email protected]>

* [DS-6963][MasterServer]add cache manager for tenant and user (apache#6974)

* cache manager for tenant and user

* add test

* fix checkstyle

* add dependencies

* fix sonar check

* use spring boot cache configuration

* add license

* check style

* check style

* add license

* change proxy name to processor

* add license header

* check style

* add license file

* add license file

* add license

Co-authored-by: caishunfeng <[email protected]>

* [Fix-7019][MasterServer] Fix the issue with failing to save and run the process that contains the switch task. (apache#7046)

* fix token delete request 404 (apache#6979)

* Fix view task status module on project homepage cannot read property label of undefined (apache#6956)

* [Improvement][API][num-9] add ProcessTaskRelationServiceImpl.moveTaskProcessRelation (apache#7053)

* add processDefinition releaseWorkflowAndSchedule

* add ProcessTaskRelationServiceImpl.moveTaskProcessRelation

* add dependency and subprocess judgement

* [DS-7004][MasterServer]fix master still work when it lose zk connection (apache#7045)

Co-authored-by: caishunfeng <[email protected]>

* [BUG] master task log can't output repair (apache#7029)

* update task_definition_log field typ

update task_definition_log  field task_params type,modify text to longtext to be consistent with the main table

* update t_ds_task_definition_log field task_params

update t_ds_task_definition_log field task_params

* [FIX:211128] master task log Can't output repair

Co-authored-by: crane <[email protected]>

* return domain after creating success (apache#7050)

Co-authored-by: ouyangyewei <[email protected]>

* Add E2E tests for some core features (apache#7025)

* to apache#7055: tenant-auto-create: true (apache#7056)

Co-authored-by: honghuo.zw <[email protected]>

* Recover unit tests that have not been run for some time (apache#7054)

* [Fix-6809][dolphinscheduler-task-sql] Fix sql task can't send alert (apache#7089)

* [Fix][Create Warning Instance Edit] Fix the problem of incomplete display of the edit form. (apache#7096)

* [Improvement-7092][UI] Optimize DAG to adapt to situations without locations (apache#7102)

* [Fix-7070][UI] Remedy the issue with no saving the timeout's strategy into the database. (apache#7107)

* [Improvement][API] fix 6772 new api bug (apache#7098)

* add processDefinition releaseWorkflowAndSchedule

* add ProcessTaskRelationServiceImpl.moveTaskProcessRelation

* add dependency and subprocess judgement

* fix api bug

* fix codestyle

* [Fix-7083] [UI] The switch task is missing the else branch (apache#7084)

* to apache#7108 (apache#7109)

* add TaskProcessorFactoryTest Ignore. (apache#7130)

* [Fix-7103] Fix the issue with the sqoop task missing the verification of props. (apache#7119)

* add the verification of parameters for the sqoop task

* [Feature][Workflow relationship] Improve English support. (apache#7129)

* [Feature][Workflow relationship] Format time. (apache#7123)

* [Feature-6930] [Python]Add workflow as code task type sub_process (apache#7022)

* [python] add subProcess task

* refactor python gateway server and task for subprocess

* add function comment for getProcessDefinition

* change process_definition usage of Subprocess task

* [Feature-apache#6422] task group queue (apache#6722)

* add task group

* modify task group

* pull dev

* add license header

* modify code style

* fix code style

* fix sql error

* fix error

* fix test

* fix test

* fix test

* fix test

* fix code style

* fix ut

* code style

* fix unit test

* test ut

* ut

* add unittest

* test ut

* modify back ut

* majorization code

* fix conflict

* fix ut

Co-authored-by: wangxj <wangxj31>

* [Feature][Workflow relationship] Format name. (apache#7143)

* to apache#7065: fix ExecutorService and schedulerService (apache#7072)

Co-authored-by: honghuo.zw <[email protected]>

* [Feature][dolphinscheduler-api] access control of processDefinition and processInstance in project apache#7073 (apache#7080)

* to apache#7073

* to apache#7073

* to apache#7073

Co-authored-by: honghuo.zw <[email protected]>

* Optimize the display of the pop-up window for creating worker groups (apache#7148)

* [Feature][dolphinscheduler-api] access control of taskDefinition and taskInstance in project to apache#7081  (apache#7082)

* to apache#7081

* fix apache#7081

* to apache#7081

Co-authored-by: honghuo.zw <[email protected]>

* fix this issue (apache#7120)

* [UT#7157]Fix Ut Run Error (apache#7158)

* to apache#7157

* to apache#7157

Co-authored-by: honghuo.zw <[email protected]>

* [Feature-6988][MasterServer] add cache manager for workflow (apache#7090)

* add cache manager for workflow

* [DS-6988][MasterServer] add cache manager for workflow

* cache evict code optimization

* test

Co-authored-by: caishunfeng <[email protected]>

* [Bug] [dolphinscheduler-api] The pages of project management, workflow instance and task instance are accessed slowly apache#7061 (apache#7139)

1、I found that the SQL behind this interface is very slow. The main table is large, but there is no index to use. By add the index, the all mode in the query plan can be changed into a more efficient ref mode. If this optimization still fails to meet the requirements, please contact me and I will continue to optimize。
2、modify index start_time_index in init script and upgrade scripts.

* [BUG][SQL] fix the process list page load slowly. (apache#7166)

* fix #7601 move ddl to 2.0.1

* fix #7601 move ddl to 2.0.1

* [Feature][MasterServer] add cache for workGroup and schedule apache#6987 (apache#7161)

* to apache#6987

* to apache#6987

* to apache#6987

* to apache#6987

* to apache#6987

* to apache#6987: @param -> @name

* to apache#6987: fix Sonar

* to apache#6987: fix Sonar

Co-authored-by: honghuo.zw <[email protected]>

* to feature apache#7169 (apache#7170)

Co-authored-by: ouyangyewei <[email protected]>

* Refactor code, add class import, keep ‘logger’ variable names uniform. (apache#7164)

* to feature apache#7110 (apache#7111)

Co-authored-by: ouyangyewei <[email protected]>

* [Feature][Workflow] Main package. (apache#7177)

* [Fix][Warning instance manage] Expand the support of `i18n`. (apache#7184)

* [Fix-7093][dolphincheduler-server] fix dependent node stuck in running (apache#7116)

* fix bug 7093

* remove redundant else

Co-authored-by: SbloodyS <[email protected]>

* [Fix-5551][common] Fix ExecutionStatus for hive-on-Tez application (apache#6289)

* add ENDED state

* Update HadoopUtils.java

* to feature apache#7180 (apache#7185)

Co-authored-by: ouyangyewei <[email protected]>

* [Fix-7037][dolphincheduler-server] process instance can not be killed when complement data (apache#7140)

* fix bug 7037

* update processComplementData

Co-authored-by: SbloodyS <[email protected]>

* [Feature][dolphinscheduler-api] add queryLog and downloadTaskLog with projectCode apache#7153 (apache#7192)

* [Feature][dolphinscheduler-api] add queryLog and downloadTaskLog with projectCode apache#7153

* [Feature][dolphinscheduler-api] add queryLog and downloadTaskLog with projectCode apache#7153

* [Feature][dolphinscheduler-api] add queryLog and downloadTaskLog with projectCode apache#7153

Co-authored-by: honghuo.zw <[email protected]>

* to feature apache#7191 (apache#7194)

Co-authored-by: ouyangyewei <[email protected]>

* [Fix-7154][API] Fix MultipartFile resource [file] cannot be resolved to absolute file… (apache#7155)

* fix MultipartFile resource [file] cannot be resolved to absolute file path

* update testCopyFile.

* [Implement][server]TaskProcessor code optimization (apache#7146)

* TaskProcessor code optimization

* fix code style

* update Collectors.toMap

* fix worker group display (apache#7208)

* [Fix-7195] The file path of  2.0.1-schema  used a wrong separator. (apache#7200)

* [Bug-7206] [MasterServer] fix process isntance always running when task timeout (apache#7207)

* fix timeout

* add task timeout map to avoid repeated timeout event

* split task check list for retry and timeout

Co-authored-by: caishunfeng <[email protected]>

* add two files (apache#7223)

* fix postgre upgrade (apache#7230)

Co-authored-by: caishunfeng <[email protected]>

* [Bug] [API] queryProcessDefinitionByCode bug (apache#7221)

* [Bug] [API] task definition release/delete bug (apache#7232)

* fix worker group display

* fix workflow query bug

* fix workflow query bug

* fix task definition release/delete bug

* [Feature-6963][MasterServer] unified cache manager (apache#7187)

* unified cache

* reduce db select

* checkstyle

Co-authored-by: caishunfeng <[email protected]>

* [Refactor]Rename class file, DatasourceUtil  to  DataSourceUtils. (apache#7181)

* Rename class file, DatasourceUtil  to  DataSourceUtils.

* Fix homepage user wrong statistics result for new user (apache#7242)

* Fix homepage user wrong statistics result for new user

For now, when we create a new user, it would see homepage
statistic info as admin user like, cause `projectCodeArray`
would not filter when we render SQL. So I add prefix conditions
to handle this situation.

Also, I do some minimal refactor here. including change function
name no quit fit

fix: apache#7182

* Fix UT

* to apache#7228 (apache#7249)

Co-authored-by: honghuo.zw <[email protected]>

* [Feature][dolphinscheduler-api] parse traceId in http header for Cross system delivery to apache#7237 (apache#7238)

* to apache#7237

* rerun test

Co-authored-by: honghuo.zw <[email protected]>

* [bug] fix task definition delete (apache#7244)

Co-authored-by: edward-yang <[email protected]>

* fix  Property or field 'all' cannot be found (apache#7246)

* [Fix-7235][datasource] unable to confige the multi-hosts (apache#7235) (apache#7253)

* [Bug][dolphinscheduler ui] When the edit time is repaired, the alarm group display. (apache#7262)

* [ISSUE 6444]Optimize task node helper document in workflow toolbar (apache#7226)

* [ISSUE 6444]Optimize task node helper document in workflow toolbar

* fix some review issue than submit by DS

* refact by add guard judgement when type is undefined

* move method helpUrlEnable and helpUrl nest in 'methods' body

* fix failover logic (apache#7275)

Co-authored-by: caishunfeng <[email protected]>

* [python] Refactor get object define communicate to gateway (apache#7272)

* Change class Base `to_dict` to `get_define` for more clearer
* Remove class TaskParams and sub class make code easy to
  understand and avoid task implement cycle dependence
* Remove class ObjectJsonBase in Task to reduce complexity

fix: apache#7271

* [Fix-7277][datasource] Support Kerberos auto renewal (apache#7277) (apache#7278)

* Split the components into individual package, refactor configurations to use Spring properties (apache#7160)

* Split the components into individual package

A follow-up PR will be made to build dedicated Docker images for each
component, so that every component Docker image has minimal jars, which
is easy to maintain and good for security fixes.

* Split the components into individual package

A follow-up PR will be made to build dedicated Docker images for each
component, so that every component Docker image has minimal jars, which
is easy to maintain and good for security fixes.

* Split the components into individual package

A follow-up PR will be made to build dedicated Docker images for each
component, so that every component Docker image has minimal jars, which
is easy to maintain and good for security fixes.

* fix in the process alarm message, projectid is 0 (apache#7298)

* to feature-7301 (apache#7308)

Co-authored-by: ouyangyewei <[email protected]>

* fix TaskResponseProcessor process NullPointerException (apache#7318)

* [Bug-7292][ApiServer] fix cache error when standalone (apache#7293)

* Fix the missing project information in the process instance timeout alarm (apache#7324)

* fix a few issues in the sql file (apache#7327)

* Fix some SQL issues and make docker-compose runnable (apache#7329)

* Fix typo in docker swarm (apache#7336)

* Adjust Helm Chart templates in dev branch (apache#7353)

* [python] Add task base database and procedure (apache#7279)

We add a new task procedure, and add parent class database
for both sql task and procedure task

fix: apache#6929

* Fix E2E workflow (apache#7377)

* [Fix-7375][datasource] remove excess hive jdbc principal (apache#7375) (apache#7381)

* [Fix-7375][datasource] remove excess hive jdbc principal (apache#7375)

* fix ut

* [Bug-7319][MasterServer] fix taskNode NPE when switch else branch is empty (apache#7320)

Co-authored-by: caishunfeng <[email protected]>

* [Fix-7291][datasource] Compatible with different versions of MySQL data source loading (apache#7291) (apache#7305)

* [Feature][UI-Next]: Build a front-end refactoring project. (apache#7387)

* Fix alert plugin instance filter (apache#7172) (apache#7210)

* Fix alert plugin instance filter (apache#7172)

alert plugin could not filter by given
pattern, cause api server do not handle
parameter searchVal, This patch add parameter
`searchVal` for alert plugin.

fix: apache#7209

* Add test

* Recover AlertPluginInstanceVO

* [Feature-7110][dolphinscheduler-api] support grant project by code (apache#7390)

Co-authored-by: ouyangyewei <[email protected]>

* [Fix][Project List UI] Fix the calling interface that keeps decreasing by one after manually setting the `pageNo` parameter. (apache#7399)

* to feature 7401 (apache#7404)

Co-authored-by: ouyangyewei <[email protected]>

* [Refactor] Rename class file, unified word spelling. (apache#7261)

* [python] Fix task procedure define error (apache#7407)

Cause procedure parameter name need `method` instead of `sql`
we change it's name. And remove parameter `sql` from parent
class `Database`

closes: apache#7386

* [Feature-7408][dolphinscheduler-api] Built GenerateToken into the UpdateToken API (apache#7411)

* to feature 7408

* trigger e2e test

Co-authored-by: ouyangyewei <[email protected]>

* [DS-7256][refactor] Optimization for task instances query (apache#7331)

- assemble a Map<Long, User> outside of the loop instead of the query inside

Co-authored-by: gq47193 <[email protected]>

* to feature 7418 (apache#7420)

Co-authored-by: ouyangyewei <[email protected]>

* [Fix-7392][dolphinscheduler-datasource] Add hive datasource failed (apache#7393)

* fix bug_7392

* fix bug_7392

* fix bug_7392

Co-authored-by: SbloodyS <[email protected]>

* [Feature][UI Next] Improve some basic configuration. (apache#7430)

* [Fix-apache#6854][dao] fix "ResourceMapper.xml -> queryResourcePaging" sql bug (apache#6907)

* fix complement data retry bug; issues: [Bug] [Master] A bug on task retry mechanism apache#6613

* [DS-apache#5293][chore]upgrade cron-utils version 5.0.5 -> 9.1.3

* [DS-apache#5293]update the licensing information of that dependency

* [DS-apache#5293]update the licensing information of that dependency

* update the licensing information of that dependency

* [DS-apache#5293]update the licensing information of that dependency

* [DS-apache#6854][fix] fix "ResourceMapper.xml -> queryResourcePaging" sql bug

* [DS-apache#6854][fix] fix "ResourceMapper.xml -> queryResourcePaging" sql bug

Co-authored-by: yangqiyu <[email protected]>

* [python] Add task dependent (apache#7405)

fix: apache#6926

* feat: Added the configuration of routes (apache#7433)

* add task-logger config (apache#7423)

Co-authored-by: caishunfeng <[email protected]>

* [Fix-7340] Improve the prompt content for deleting a project that didn't exist. (apache#7449)

* improve the prompt

* [Improvement-7213][MasterServer] execute thread pool code optimization (apache#7258)

* threadpool optimization

* threadpool params

* rebase dev

* ut check fix

* add return

* rebase dev

* event loop

Co-authored-by: caishunfeng <[email protected]>

* Build and publish snapshot Docker images for testing (apache#7454)

* Adjust logging levels for third party libraries (apache#7453)

* feat: Added login pages and functions (apache#7456)

* [Feature-7349] [Python]Add workflow as code task type datax (apache#7437)

* support datax task by python
* add task datax example

Co-authored-by: Jiajie Zhong <[email protected]>

* fix this bug (apache#7461)

* [Feature][UI Next] Add apis. (apache#7463)

* [Feature][UI Next] Interface debugging(login)

* [Feature-7460][Alert] Wechat alert add send to group chat (apache#7465)

* feature_7460

* feature_7460

Co-authored-by: SbloodyS <[email protected]>

* remove date+1 when the data is supplemented (apache#7452)

* Update ProcessDefinitionServiceImpl.java (apache#7470)

fix typos for logger.error

* [Feature][UI Next] Write part of the api method. (apache#7476)

* Remove redundant parameters [zookeeper.version] in dolphinscheduler/pom.xml(apache#7486)

* [Fix-7146][server]Fix the task processor thread is not safe. (apache#7488)

* Fix the task processor thread is not safe.

* fix code style

* [Feature][UI Next] Write part of the api method. (apache#7507)

* [DS-7227][refactor] optimize alert plugin instance list order (apache#7506)

[DS-7227][refactor] optimize alert plugin instance list order
query all alert plugin instance list order by update time desc
optimize `select *` sql in the AlterPluginInstanceMapper.xml

* Add new version 2.0.1 to issue bug template (apache#7458)

Add new version 2.0.1 for github issue template and remove versions not LTS

* [Fix-7485][datasource-plugin] Fix create hive datasource using ZooKeeper way in 2.0.1 (apache#7489)

* refactor: Refactor Login to better match the composition API implementation (apache#7519)

* fix http alarm msg ignored (apache#7523)

See apache#7522

* [Fix-7525][Feature][UI Next] Provide a common  layout for the module of dolphinscheduler-ui-next. (apache#7526)

* add layout

* add layout

* add layout

* add routes

* add routes

* modify layout

* modify layout

* modify layout

* modify layout

* improve a few issues with code specification

* [Feature][UI Next]: Rearrange the routing and format the code (apache#7533)

* fix: Rearrange the routing and format the code

* To fix the problem

To fix the problem

* [python] Add task condition (apache#7505)

* [python] Add task condition

close: apache#6927

* Add example and set downstream dep

* [python] Add task switch (apache#7531)

* [python] Add task switch

close: apache#6928

* Fix code style

* Unify the element selector rules for E2E (apache#7409)

* Unify the element selector rules for E2E

* Fix typo in docker swarm

* [Feature][UI Next] Add charts setting. (apache#7543)

* [Feature][UI Next] Add charts setting. (apache#7547)

* fix bug_7537 (apache#7551)

Co-authored-by: SbloodyS <[email protected]>

* [Bug-7333][MasterServer] fix duplicate key task definition (apache#7335)

Co-authored-by: caishunfeng <[email protected]>

* [Improvement-apache#7529][tools] Init DB schema from the full sql file. (apache#7530)

* [Fix-apache#7480] [Standalone] add logger server (apache#7481)

* Change build context to avoid cache

Co-authored-by: kezhenxu94 <[email protected]>

* Fix database lose data in standalone (apache#7556)

* Modify contribution document (apache#7552)

* [Improvement] Merge redundant codes (apache#7550)

* [Feature][UI Next]Refactoring layout  (apache#7558)

* refactor: Refactoring layout

* chore: Delete unnecessary packages

* finished issue#7527 (apache#7561)

Co-authored-by: 时光 <[email protected]>

* Update README.md (apache#7586)

* Update README.md

* Update README.md

Co-authored-by: Kirs <[email protected]>

* Revert "Fix database lose data in standalone (apache#7556)" (apache#7604)

This reverts commit 82075a4.

* [Fix-7534][Master] zookeeper failover error (apache#7562)

* fix bug_7534

* fix bug_7534

* fix bug_7534

* fix bug_7534

* fix bug_7534

* fix bug_7534

Co-authored-by: SbloodyS <[email protected]>

* [Feature][UI Next] Add charts i18n. (apache#7568)

* [Feature-apache#6422] [api-server] task group queue (apache#7491)

* add task group

* modify task group

* pull dev

* add license header

* modify code style

* fix code style

* fix sql error

* fix error

* fix test

* fix test

* fix test

* fix test

* fix code style

* fix ut

* code style

* fix unit test

* test ut

* ut

* add unittest

* test ut

* modify back ut

* majorization code

* fix conflict

* fix ut

* add task group api

* reset file

* fix ut

* fix lost column

* fix ut

* fix ut

* fix ut

* fix ut

* delete duplicate code

* fix code style 、name

* fix ut

* fix mapper

Co-authored-by: wangxj <wangxj31>

* [Feature][UI Next] Add card component. (apache#7606)

* [Feature][UI Next] Add card component.
* [Feature][UI Next] Update card component props.

* Required E2E to pass before PR can be merged (apache#7612)

* [python] Correct python import syntax in example (apache#7617)

fix: apache#7615

* Try (apache#7603)

* [Fix-7520][Master] Data too long for column 'task_params' at row 1 (apache#7521)

* fix bug_7520

* fix bug_7520

* fix bug_7520

* bug_7520

Co-authored-by: SbloodyS <[email protected]>

* [BUG-7554] Fix database lose data in standalone (apache#7620)

* Fix the wrong lifecycle phase of initialization

Co-authored-by: kezhenxu94 <[email protected]>

* [Feature-7576][Master] Optimize complement task's date (apache#7585)

* feature_7576

* feature_7576

* feature 7576

* feature 7576

* to apache#7609 (apache#7611)

Co-authored-by: honghuo.zw <[email protected]>

* [python] Fix task relation wrong hashable function (apache#7628)

* [python] Fix task relation wrong hashable function

Currently our :class:`TaskRelation` have wrong __hash__
function, would mistake a object as same object. It failed
our dependence and process definition. this patch fix it
by adding `_KEY_ATTR` to overwrite TaskRelation's __eq__
and correct __hash__ func

close: apache#7627

* Fix code format

* fix bug_7564 (apache#7625)

* fix bug_7629 (apache#7630)

* [DS-7259][fix] Modify the prompt for

This closes apache#7259

Co-authored-by: GaoTianDuo <[email protected]>
Co-authored-by: wind <[email protected]>
Co-authored-by: caishunfeng <[email protected]>
Co-authored-by: zwZjut <[email protected]>
Co-authored-by: honghuo.zw <[email protected]>
Co-authored-by: kezhenxu94 <[email protected]>
Co-authored-by: JinYong Li <[email protected]>
Co-authored-by: xuhhui <[email protected]>
Co-authored-by: Kirs <[email protected]>
Co-authored-by: Jiajie Zhong <[email protected]>
Co-authored-by: lvshaokang <[email protected]>
Co-authored-by: sparklezzz <[email protected]>
Co-authored-by: xudong.zhang <[email protected]>
Co-authored-by: Zonglei Dong <[email protected]>
Co-authored-by: EdwardYang <[email protected]>
Co-authored-by: 熠然 <[email protected]>
Co-authored-by: edward-yang <[email protected]>
Co-authored-by: Hua Jiang <[email protected]>
Co-authored-by: Kerwin <[email protected]>
Co-authored-by: zhang <[email protected]>
Co-authored-by: crane <[email protected]>
Co-authored-by: ouyangyewei <[email protected]>
Co-authored-by: ouyangyewei <[email protected]>
Co-authored-by: 张俊杰 <[email protected]>
Co-authored-by: songjianet <[email protected]>
Co-authored-by: wangyizhi <[email protected]>
Co-authored-by: Devosend <[email protected]>
Co-authored-by: wangxj3 <[email protected]>
Co-authored-by: BaoLiang <[email protected]>
Co-authored-by: SbloodyS <[email protected]>
Co-authored-by: SbloodyS <[email protected]>
Co-authored-by: Manhua <[email protected]>
Co-authored-by: mask <[email protected]>
Co-authored-by: 百岁 <[email protected]>
Co-authored-by: mgduoduo <[email protected]>
Co-authored-by: gq47193 <[email protected]>
Co-authored-by: KyoYang <[email protected]>
Co-authored-by: yangqiyu <[email protected]>
Co-authored-by: labbomb <[email protected]>
Co-authored-by: Jiajie Zhong <[email protected]>
Co-authored-by: Ivan0626 <[email protected]>
Co-authored-by: huangxiaohai <[email protected]>
Co-authored-by: Tom <[email protected]>
Co-authored-by: wangbowen <[email protected]>
Co-authored-by: Jiezhi.G <[email protected]>
Co-authored-by: lgcareer <[email protected]>
Co-authored-by: Wenjun Ruan <[email protected]>
Co-authored-by: J·Y <[email protected]>
Co-authored-by: 时光 <[email protected]>
Co-authored-by: 时光 <[email protected]>
Co-authored-by: lifeng <[email protected]>
Co-authored-by: Kirs <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
Status: Done
Development

Successfully merging this pull request may close these issues.

4 participants