-
Notifications
You must be signed in to change notification settings - Fork 1.8k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[api-draft][connector] apache pulsar source #1984
Conversation
Please add license to LICENSE file in seatunnel-dist |
Also add pulsar connector into pom.xml in |
@benjfan I have added the licenses. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Some Notices also need to be added. https://github.com/apache/incubator-seatunnel/blob/dev/seatunnel-dist/release-docs/NOTICE
@CalvinKirs Do I need to add any licenses or notices? |
f8bc938
to
71560d9
Compare
# Conflicts: # seatunnel-connectors/seatunnel-connectors-seatunnel/pom.xml
# Conflicts: # seatunnel-connectors/seatunnel-connectors-seatunnel-dist/pom.xml
I run this command with the jar "connector-pulsar-2.3.3.jar" ./bin/seatunnel.sh --config ./config/v2.streaming.pulsar2console.template -e local the connection to admin and client is ok, but it can not consume the messages . the connection configuration below like this: env {
"job.mode"=STREAMING
"job.name"="seatunnel_schema2console_test"
}
source {
Pulsar {
"subscription.name"="seatunnel_schema2console_test.sub"
"cursor.startup.mode"=LATEST
"cursor.reset.mode"=null
"cursor.stop.mode"=NEVER
"topic-discovery.interval"=-1
"poll.timeout"=100
"poll.interval"=50
"poll.batch.size"=500
parallelism=1
"result_table_name"=Table11621381770080
topic="persistent://public/default/seatunnel.schema.test"
schema {
fields {
id=string
}
}
operationTimeoutMs="30000"
useTls="false"
connectionMaxIdleSeconds="30000"
useTcpNoDelay="true"
"client.service-url"="pulsar://xxx.xxx.xxx.xxx:6650"
"admin.service-url"="http://xxx.xxx.xxx.xxx:8080"
topic-pattern="false"
lookupTimeoutMs="30000"
tlsHostnameVerificationEnable="true"
}
}
transform {
}
sink {
Console {
}
} is there some demo for pulsar? |
Purpose of this pull request
apache pulsar source
Check list
New License Guide