Skip to content

Latest commit

 

History

History
23 lines (17 loc) · 581 Bytes

README.md

File metadata and controls

23 lines (17 loc) · 581 Bytes
Raw

Flink iceberg integration tests

Testing flink jobs on yarn, jobs integrated with iceberg, hive, hbase, kafka, elasticsearch, etc.

Used components

  • Apache Hadoop 3.2
  • Kafka 2.0.0
  • Hbase 2.0.0
  • Hive 3.1.2
  • ELK 7.9.1
  • Iceberg 0.10.0
  • Flink 1.11.3

How to Run

  • Deploy a testing big data platform using docker-compose. Download here

  • Build each jobs and run it on yarn with yarn session or pre-job.

TODO

  1. testing ozone and alluxio.
  2. deep integration with iceberg and flink sql.
  3. running jobs on k8s.