You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
I have a scenario where I need to constantly write data to Doris via Flink. The scenario is not particularly concerned with data consistency, but needs to maximize throughput as much as possible.
I briefly looked at flink-doris-connector's code and found 3 ways to import:
StreamLoad.
Batch
Copy
In particular, Copy, which was not introduced at all before viewing the code. I would like to ask the difference between these modes. Which one is more recommended for my scenario?
Also I found that three formats are supported for import.
csv
json
arrow
I tried to use Arrrow, but the import fails and I can't see why (accessing the error reporting page changes to downloading the binary file. And I can't open it).
reacted with thumbs up emoji reacted with thumbs down emoji reacted with laugh emoji reacted with hooray emoji reacted with confused emoji reacted with heart emoji reacted with rocket emoji reacted with eyes emoji
-
I have a scenario where I need to constantly write data to Doris via Flink. The scenario is not particularly concerned with data consistency, but needs to maximize throughput as much as possible.
I briefly looked at
flink-doris-connector
's code and found 3 ways to import:In particular, Copy, which was not introduced at all before viewing the code. I would like to ask the difference between these modes. Which one is more recommended for my scenario?
Also I found that three formats are supported for import.
I tried to use Arrrow, but the import fails and I can't see why (accessing the error reporting page changes to downloading the binary file. And I can't open it).
Would Arrow be more efficient?
Beta Was this translation helpful? Give feedback.
All reactions