Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Umbrella][Connector-V2] SeaTunnel Connector V2 Tracking Table #3018

Open
2 of 3 tasks
EricJoy2048 opened this issue Oct 8, 2022 · 89 comments
Open
2 of 3 tasks

[Umbrella][Connector-V2] SeaTunnel Connector V2 Tracking Table #3018

EricJoy2048 opened this issue Oct 8, 2022 · 89 comments

Comments

@EricJoy2048
Copy link
Member

EricJoy2048 commented Oct 8, 2022

Search before asking

  • I had searched in the feature and found no similar feature requirement.

Description

Completed Connectors

No Connector priority difficulty Source/Sink Contributer Status Issue/PR
1 Console Sink @ruanwenjun Done #1864
2 Fake Source @ruanwenjun Done #1864
3 ElasticSearch Sink @iture123 Done #2330
4 ElasticSearch Source @iture123 Done #2810
5 ClickHouse Source @Hisoka-X Done #2051
6 ClickHouse Sink @Hisoka-X Done #2051
7 JDBC Source @ic4y Done #2048
8 JDBC Sink @ic4y Done #2048
9 File(Local) Source @TyrantLucifer Done
10 File(Local) Sink @EricJoy2048 Done #2117
11 File(HDFS) Source @TyrantLucifer Done
12 File(HDFS) Sink @EricJoy2048 Done
13 File(OSS) Source @TyrantLucifer Done
14 File(OSS) Sink @TyrantLucifer Done
15 Hudi Source @Emor-nj Done #2147
16 Iceberg Source @hailin0 Done #2615
17 Kafka Source @Hisoka-X Done
18 Kafka Sink @ruanwenjun Done
19 Kudu Source @2013650523 Done #2254
20 Kudu Sink @2013650523 Done #2254
21 MongoDB Source @wuchunfu Done
22 MongoDB Sink @wuchunfu Done
23 Neo4j Sink @getChan Done #2434
24 Phoenix Source @531651225 Done #2499
25 Phoenix Sink @531651225 Done #2499
26 Redis Source @TyrantLucifer Done
27 Redis Sink @TyrantLucifer Done
28 Socket Source @zhuangchong Done #1999
29 Socket Sink @531651225 Done #2549
30 Webhook Sink @TyrantLucifer Done #2348
31 Pulsar Source @ashulin Done #1984
32 Email Sink @2013650523 Done #2304
33 Assert Sink @lhyundeadsoul Done
34 Sentry Sink @Saintyang done #2244
35 DingTalk Sink @MRYOG Done #2257
36 IoTDB Source @CalvinKirs Done
37 IoTDB Sink @hailin0 Done
38 Hive Source @CalvinKirs Done
39 Hive Sink @EricJoy2048 Done
40 HTTP Source @zhuangchong Done #2012
41 Greenplum Source @hailin0 Done #2429
42 Greenplum Sink @hailin0 Done #2429
43 Enterprise WeChat Sink @531651225 Done #2412
44 [Jdbc]Sql-Server middle middle Sink @liugddx Done #2646
45 [Jdbc]Sql-Server middle middle Source @liugddx Done #2646
46 [JDBC]DM(达梦) Source @laglangyue Done #2377
47 [JDBC]DM(达梦) Sink @laglangyue Done #2377
48 [File]JSON Sink @hailin0 Done
49 [File]JSON Source @TyrantLucifer Done
50 Ftp Sink @chessplay Done #2774
51 Ftp Source @guanboo Done #2774
52 DataHub Sink @chessplay Done
53 [Jdbc]Oracle high low Sink @liugddx Done #2550
54 [Jdbc]Oracle high low Source @liugddx Done #2550
55 influxDB middle middle Source @531651225 Done #2697
56 [JDBC]StarRocks high low Sink @tonyDong-code Done #3060
57 [JDBC]StarRocks high low Source @531651225 Done #3060
58 Gbase 8a middle middle Source @liugddx Done #3026
59 File(S3) high middle Source @TyrantLucifer Done #3119
60 File(S3) high middle Sink @TyrantLucifer Done #3119
61 DB2 middle middle Source @laglangyue Done #2410
62 DB2 middle middle Sink @laglangyue Done #2410
63 SFTP middle middle Source @TyrantLucifer Done #3006
64 SFTP middle middle Sink @TyrantLucifer Done #3006
65 Neo4j middle middle Source @getChan Done #2777
66 FeiShu low low Sink @TyrantLucifer Done
67 Amazon dynamodb high middle Source @liugddx Done #3166
68 Amazon dynamodb high middle Sink @liugddx Done #3166
69 StarRocks high high Sink @531651225 Done #3164
70 Influxdb high middle Sink @531651225 Done #3174
71 Cassandra high middle Source @FWLamb Done #3229
72 Cassandra high middle Sink @FWLamb Done #3229
73 My Hours middle low Source @TaoZex Done #3228
74 Google Sheets middle middle Source @skyoct Done #3185
75 Aliyun Tablestore high middle Source @liugddx Done #3309
76 Aliyun Tablestore high middle Sink @liugddx Done #3309
77 Klaviyo middle middle Source @TaoZex Done #3443
78 lemlist middle middle Source @TaoZex Done #3346
79 Onesignal middle middle Source @TaoZex Done #3454
80 Slack middle low Sink @lianghuan-xatu Done #3226
81 Gitlab high low Source @liugddx Done #3408
82 rabbitmq middle middle Source @531651225 Done #3312
83 rabbitmq middle middle Sink @531651225 Done #3312
84 [JDBC]SqlLite middle low Sink @nutsjian Done #3089
85 [JDBC]SqlLite middle low Source @nutsjian Done #3089
86 Jira high middle Source @liugddx Done #3473
87 OpenMLDB middle high Source @TyrantLucifer Done #3313
88 Teradata middle middle Source @FWLamb Done #3362
89 Teradata middle middle Sink @FWLamb Done #3362
90 [JDBC]Doris high Source @TaoZex Done #3586
91 [JDBC]Doris high Sink @TaoZex Done #3586
92 OSS-HDFS high middle Source @TyrantLucifer Done #3456
93 OSS-HDFS high middle Sink @TyrantLucifer Done #3456
94 MaxCompute middle middle Source @stdnt-xiao Done #3640
95 MaxCompute middle middle Sink @stdnt-xiao Done #3640
96 Doris StreamLoad Sink high high Sink @TaoZex Done #3631
97 RedshiftS3 high low Sink @CalvinKirs Done #3736
98 Persistiq high low Source @TaoZex Done #3460
99 [JDBC]SAP HANA middle low Source @FlechazoW Done #3017
100 [JDBC]SAP HANA middle low Sink @FlechazoW Done #3017
101 [CDC] MySQL CDC Source Done
102 [CDC] SQL Server CDC Source Done
103 SelectDB Cloud Done #3958
104 TD-engine middle Source @lhyundeadsoul Done #2707
105 TD-engine middle Sink @lhyundeadsoul Done #2832
106 GitHub Done #4155

Usage Scenario

No response

Related issues

No response

Are you willing to submit a PR?

  • Yes I am willing to submit a PR!

Code of Conduct

@EricJoy2048
Copy link
Member Author

EricJoy2048 commented Oct 8, 2022

To do list for new Connectors

About Difficulty

Source Connector

high
  • Support parallelism and snapshot(like kafka source)
  • Or use Native API for better performance
  • Both batch and stream need implement
middle
  • Support parallelism
low
  • No parallelism and snapshot

Sink Connector

high
  • Support Excetly-once
  • Or use Native API for better performance
  • Or CDC support
middle
  • better performance
low
  • other
No Connector priority difficulty Source/Sink Contributer Status Issue/PR
1 Doris Api Source high Source @531651225
3 Druid high Source @guanboo #2937
4 Druid high Sink @guanboo #2937
6 FeiShu low Source @1553010490
9 Hudi high Sink @harveyyue
10 Iceberg high Sink @s7monk #2645
14 Pulsar high Sink @FlechazoW
16 Gitlab high Source
17 HubSpot middle Source
18 Instagram middle Source
19 Bing ADs middle Source
20 Google Analytics Data Api middle Source @MRYOG
21 Intercom middle Source
22 Zendesk middle Source
23 TikTok Marketing middle Source
24 Salesforce middle Source
25 LinkedIn Ads middle Source
26 Stripe middle Source
27 DingTalk low Source @MRYOG #2757
30 HBase high Source @zhuzhengjun01
31 HBase high Sink @zhuzhengjun01
34 ADB PostgreSQL high Sink @etcZYP
35 OceanBase middle Sink @silenceland #2617
36 OceanBase middle Source @silenceland #2617
39 BigSource middle Source
41 Slack middle Source @lianghuan-xatu
43 Snowflake middle Sink
44 Snowflake middle Source
45 StarRocks API high Source @531651225
47 [JDBC]Rds high Sink @s7monk #2829
48 [JDBC]Rds high Source @s7monk #2829
53 [File]excel middle Sink @Bingz2 #2585
54 [File]excel middle Source @MonsterChenzhuo
58 OpenMLDB middle Sink @dlimeng
59 GaussDB middle Source @Builder34
60 GaussDB middle Sink @Builder34
65 DataHub middle Source @selectbook
68 Flink Table Store high Sink @iture123
69 Flink Table Store high Source @zhaomin1423
70 Vertica middle Source @FlechazoW
71 Vertica middle Sink @FlechazoW
72 Kylin middle Source @531651225
74 Neocrm middle Source
75 TiDB middle Source @Xuxiaotuan #2830
76 TiDB middle Sink @Xuxiaotuan #2830
77 Sentry middle Source
78 PolarDB middle Source @liujingwen1225
79 PolarDB middle Sink @liujingwen1225
80 PolarDB-X middle Source
81 PolarDB-X middle Sink
82 AnalyticDB middle Sink
83 TDSQL middle Sink
84 SequoiaDB low Sink
85 TcaplusDB middle Source @FWLamb
86 TcaplusDB middle Sink @FWLamb
87 GoldenDB low Source
88 GoldenDB low Sink
89 AntDB low Source
90 AntDB low Sink
91 OushuDB middle Sink
92 SUNDB low Source
93 SUNDB low Sink
94 UXDB low Source
95 UXDB low Sink
96 DolphinDB low Source @silenceland
97 DolphinDB low Sink @silenceland
98 RapidsDB low Source
99 RapidsDB low Sink
100 GreatDB low Source
101 GreatDB low Sink
102 CirroData low Sink
105 KunlunDB low Sink
106 Percona low Source
107 Percona low Sink
108 Splunk low Source
109 Splunk low Sink
112 Microsoft Azure SQL Database high Source @FWLamb
113 Microsoft Azure SQL Database high Sink @FWLamb
114 Neo5j high Source
115 Neo5j high Sink
116 Solr high Sink @skyoct
117 BigQuery high Source
118 BigQuery high Sink
119 SAP Adaptive Server high Source
120 SAP Adaptive Server high Sink
121 Microsoft Azure Cosmos DB high Source @TyrantLucifer
122 Microsoft Azure Cosmos DB high Sink @TyrantLucifer
123 PostGIS middle Source @cry2133
124 PostGIS middle Sink @cry2133
125 Couchbase middle Sink
126 Vika middle Sink
129 aws-datalake high Sink
130 azure-blob-storage high Sink @tsaiggo
131 bigquery-denormalized middle Sink
132 firebolt middle Source @TaoZex
133 firebolt middle Sink @TaoZex
134 Google firestore high Sink
135 Google Cloud Storage high Sink @dengd1937
137 Google Sheets high Sink @skyoct
138 keen middle Sink
139 kinesis middle Sink
140 kvdb middle Sink
141 Mariadb ColumnStore middle Sink
142 meilisearch middle Sink @haibo-duan
143 mqtt middle Sink @DarkAssassinator
146 Google PubSub high Sink
147 R2 middle Sink
149 Scylla middle Sink
150 Adjust middle Source
151 airtable middle Source
152 alloydb middle Source
155 Amplitude middle Source
156 Apify Dataset middle Source
157 Databricks Lakehouse middle Sink
158 appfollow middle Source
159 appsflyer middle Source
160 Appstore Singer middle Source
161 asana middle Source
162 aws-cloudtrail high Source
163 azure-table high Source
164 Bamboo Hr middle Source
165 bigcommerce middle Source
166 braintree middle Source
167 Cart.com middle Source
168 Chargebee middle Source
169 chargify middle Source
170 chartmogul middle Source
171 Close.com middle Source
172 cockroachdb middle Source
173 Commercetools middle Source
174 confluence high Source @liugddx
175 delighted middle Source
176 dixa middle Source
177 dockerhub middle Source
178 drift middle Source
179 DISPLAY & VIDEO 360 middle Source
180 Exchange Rates middle Source
181 Facebook Marketing high Source
182 Facebook pages high Source
183 Faker low Source
184 fauna low Source
185 Flexport low Source
186 Freshcaller low Source
187 Freshdesk low Source
188 Freshsales low Source
189 Freshservice low Source
190 Glassfrog low Source
191 Google Analytics 4 (GA4) high Source
192 Google Directory high Source
193 Google Search Console high Source
194 Google Workspace Admin Reports high Source
195 Greenhouse low Source
196 Harvest low Source
197 Hellobaton low Source
198 Hubplanner low Source
199 HubSpot low Source
200 Instagram low Source
201 Intercom low Source
202 Iterable high Source
205 Source Kustomer Singer high Source
206 Kyriba high Source
208 Lever Hiring high Source
209 Linkedin Ads high Source
210 Linkedin Pages high Source
211 Linnworks high Source
212 Looker high Source
213 Mailchimp high Source
214 Mailgun high Source
215 Marketo high Source
216 Metabase high Source
217 Microsoft Teams high Source
218 Mixpanel high Source
220 Netsuite high Source
221 Notion high Source @TaoZex
222 Okta high Source
224 Open Weather high Source
225 Orb high Source
226 Orbit high Source
227 Outreach] high Source
228 Pardot high Source
229 Paypal Transaction high Source
230 Paystack high Source
232 Pinterest high Source
233 Pipedrive high Source
234 Pivotal Tracker high Source
235 Plaid high Source
236 PostHog high Source
237 PrestaShop high Source
238 Primetric high Source
239 Qualaroo high Source
240 QuickBooks Singer high Source
241 Recharge high Source
242 Recurly high Source
243 Relational Database high Source
244 Retently high Source
245 RKI Covid high Source
246 Salesloft high Source
247 Scaffold Java Jdbc high Source
248 Scaffold Http Source high Source
249 Search Metrics high Source
250 Sendgrid high Source
251 Shopify high Source
252 Shortio high Source
253 Smartsheets high Source
254 Snapchat Marketing high Source
255 Square high Source
256 Strava high Source
257 Stripe high Source
258 Surveymonkey high Source
259 Talkdesk-Explore high Source
260 Tempo high Source
261 Tiktok Marketing high Source
262 3PL Central high Source
263 Trello high Source
264 Twilio high Source
265 Typeform high Source
266 Us Census high Source
267 Webflow high Source
268 Woocommerce high Source
269 Wrike high Source
270 Yahoo Finance high Source
271 Yandex Metrica high Source
272 Youtube Analytics high Source
273 Zendesk Chat high Source
274 Zendesk Sunshine high Source
275 Zendesk Support high Source
276 Zendesk Talk high Source
277 Zenefits high Source
278 Zenloop high Source
279 Zoho Crm high Source
280 Zoom Singer high Source
281 Zuora high Source
286 Amazon SQS high Source @TaoZex
287 Amazon SQS high Sink @TaoZex
288 Amazon Aurora high Source
289 Amazon Aurora high Sink
290 Redshift high Source
292 Amazon Ads high Source
293 Amazon Seller-Partner high Source
294 Amazon MemoryDB for Redis high Source @liugddx
295 Amazon MemoryDB for Redis high Sink @liugddx
296 Amazon ElastiCache high Source
297 Amazon ElastiCache high Sink
298 Amazon DocumentDB high Source @liugddx
299 Amazon DocumentDB high Sink @liugddx
300 Amazon Keyspaces (for Apache Cassandra) high Source
301 Amazon Keyspaces (for Apache Cassandra) high Sink
302 Amazon Timestream high Source
303 Amazon Timestream high Sink
304 Amazon Quantum Ledger Database (QLDB) high Source
305 Amazon Quantum Ledger Database (QLDB) high Sink
306 Amazon Elastic File System high Source
307 Amazon Elastic File System high Sink
308 Amazon FSx high Source
309 Amazon FSx high Sink
310 Amazon Elastic Block Store (EBS) high Source
311 Amazon Elastic Block Store (EBS) high Sink
312 NebulaGraph high Source @Carl-Zhou-CN
313 NebulaGraph high Sink @Carl-Zhou-CN
314 [ADLS2]azure dalta lake storage2 high Source
315 [ADLS2]azure dalta lake storage2 high Sink
316 RocketMQ high Source @sunxiaojian
317 RocketMQ high Sink @sunxiaojian
318 ADLS Gen2 high Source @sudeephazra
319 ADLS Gen2 high Sink @sudeephazra

@Hisoka-X Hisoka-X changed the title [Feature][Module Name] Feature title [Feature][Connector-V2] SeaTunnel Connector V2 Tracking Table Oct 8, 2022
@FlechazoW
Copy link
Contributor

SAP HANA done, the pr #3017 .

@531651225
Copy link
Contributor

i can work for StarRocks Source connector, thanks

@EricJoy2048
Copy link
Member Author

EricJoy2048 commented Oct 8, 2022

i can work for StarRocks Source connector, thanks

Ok, StarRocks Source connector assign to you.

@liugddx
Copy link
Member

liugddx commented Oct 9, 2022

Oracle connector already done.

@EricJoy2048
Copy link
Member Author

Oracle connector already done.

Get it. Thanks for you contribution.

@liugddx
Copy link
Member

liugddx commented Oct 10, 2022

Assign hudi to me

@nutsjian
Copy link
Contributor

Assign [JDBC]SqlLite Source, [JDBC]SqlLite Sink to me. thanks.

@TyrantLucifer
Copy link
Member

Microsoft Azure Cosmos DB sink & source thx.

@FWLamb
Copy link
Contributor

FWLamb commented Oct 12, 2022

Assign Cassandra sink & source to me,thx

@DarkAssassinator
Copy link
Contributor

mqtt, thx

@531651225
Copy link
Contributor

i can work for InfluxDB sink connector,thx

@TaoZex
Copy link
Contributor

TaoZex commented Oct 14, 2022

Kylin sink connector done.Because kylin's sink connector is required to be imported into hive and cannot be imported into kylin, we don't need to implement it.

@TaoZex
Copy link
Contributor

TaoZex commented Oct 14, 2022

I want to work for My Hours source connector,thx.

@EricJoy2048
Copy link
Member Author

Kylin sink connector done.Because kylin's sink connector is required to be imported into hive and cannot be imported into kylin, we don't need to implement it.

Thanks, I will delete kylin sink connector.

@1553010490
Copy link

i can work for FeiShu Source connector, thanks

@liugddx
Copy link
Member

liugddx commented Oct 17, 2022

Assgin MsSQL (SQL Server) sink & source to me ,thanks

@liugddx
Copy link
Member

liugddx commented Oct 17, 2022

Assgin MsSQL (SQL Server) sink & source to me ,thanks
SQL Server already done,and MsSQL is SQL Server.This is a task for Sybase?

@EricJoy2048
Copy link
Member Author

Sorry, I will delete MsSQL. Thanks for you report.

Sorry, I will delete MsSQL. Thanks for you report.

@531651225
Copy link
Contributor

Whether there is a rabbitmq source connector that supports eos semantics, I can work for it.

@EricJoy2048
Copy link
Member Author

EricJoy2048 commented Oct 17, 2022

Whether there is a rabbitmq source connector that supports eos semantics, I can work for it.

I will add it to this list. You can do it.

@531651225
Copy link
Contributor

Whether there is a rabbitmq source connector that supports eos semantics, I can work for it.

I will add it to this list. You can do it.

I can do for rabbitmq source and sink connector,thanks

@stdnt-xiao
Copy link
Contributor

Assign Maxcompute sink & source to me,thx

@TaoZex
Copy link
Contributor

TaoZex commented Dec 10, 2022

Please assign Amazon SQS source & sink connector and Firebolt source & sink connector me. Thanks.

@cry2133
Copy link

cry2133 commented Jan 9, 2023

I can work for POSTGIS Source and Sink connector, thanks

@sunxiaojian
Copy link
Contributor

I can work for RocketMQ Source and Sink connector, thanks

@cdmikechen
Copy link

cdmikechen commented Jan 11, 2023

I can work for POSTGIS Source and Sink connector, thanks

@cry2133
Hi~ I’m also using postgis in my company, if you have create a PR or progress update could you also @ me when you do so, please?

There are many commercial databases (e.g. oracle) that also support spatial fields, and here I feel that there may be a need to add some types of separate spatial fields to work together.

@2000liux
Copy link

There is a new connection and I will check to see if I can support it

@Hisoka-X
Copy link
Member

There is a new connection and I will check to see if I can support it

@2000liux Hi, you can pick up any not finished connector which you are interested.

@2000liux
Copy link

@Hisoka-X ok。I am willing to try

@Al-assad
Copy link
Member

@EricJoy2048 Hi, please assign Github, AWS Redshift, AWS ElasticCache, AWS Keyspaces to me, I'm willing to contribute to these.

@EricJoy2048
Copy link
Member Author

@EricJoy2048 Hi, please assign Github, AWS Redshift, AWS ElasticCache, AWS Keyspaces to me, I'm willing to contribute to these.

AWS Redshift is finished.
AWS ElasticCache you mean AWS OpenSearch? Elasticsearch connector can read and write data to AWS OpenSearch now.
I think you can contribute to AWS Keyspaces.

@davidzollo
Copy link
Contributor

@FlechazoW , vertica has assigned to you, looking forward to your PR ^-^

@HaoXuAI
Copy link
Contributor

HaoXuAI commented Apr 1, 2023

Initiated a PR for snowflake #4470

@silenceland
Copy link
Contributor

hi, I can work for DolphinDB source & sink, please assign to me, thanks!

@q7208783
Copy link

hi. how can I use aws redshift on source side ?

@EricJoy2048 EricJoy2048 unpinned this issue Oct 26, 2023
@sudeephazra
Copy link

I would like to work on ADLS Gen2 source and sink

@Hisoka-X
Copy link
Member

Thanks @sudeephazra , I already assgin to you.

@wenchaoh997
Copy link

I would like to try on the Shopify source, thanks.

@W-dragan
Copy link

mqtt, thx
@DarkAssassinator
I would like to ask if there is any code implementation or PR, and I now need mqtt as the source side

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
Status: Doing
Development

No branches or pull requests