Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Feature][Manager] Support Dolphinscheduler schedule engine #11401

Open
1 of 2 tasks
aloyszhang opened this issue Oct 23, 2024 · 2 comments
Open
1 of 2 tasks

[Feature][Manager] Support Dolphinscheduler schedule engine #11401

aloyszhang opened this issue Oct 23, 2024 · 2 comments
Assignees
Milestone

Comments

@aloyszhang
Copy link
Contributor

aloyszhang commented Oct 23, 2024

Description

Currently, InLong supports both real-time and offline data synchronization.
The offline data sync is scheduled repeatedly by the schedule engine.
#10412 introduced a built-in schedule engine and #10515 supports the built-in engine to trigger the Flink batch job.
So, InLong has the basic capability for offline data synchronization.

There are some mature and stable scheduling engines like Apache Airflow and Apache Dolphinscheduler. It will be helpful for Inlong to support these scheduling engines in offline data synchronization.

This issue is part of this which aims to support Dolphinscheduler Airflow.

Ref:
InLong offline synchronization
Offline data synchronization Example
Offline Sync Connector Extension

Use case

No response

Are you willing to submit PR?

  • Yes, I am willing to submit a PR!

Code of Conduct

@aloyszhang aloyszhang added this to the 2.1.0 milestone Oct 23, 2024
@emptyOVO
Copy link
Contributor

emptyOVO commented Oct 24, 2024

I'm interested in related implementation, could you plz assign this to me?

@aloyszhang
Copy link
Contributor Author

I'm interested in related implementation, could you plz assign this to me?

Sure, I've assigned it to you.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

2 participants