Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Install openjdk in base ingestion image #6365

Merged

Conversation

frsann
Copy link
Contributor

@frsann frsann commented Nov 4, 2022

The acryldata/datahub-ingestion image is missing Java, meaning that the Kafka connect ingestion can't be run. This is fixed in the PR by installing openjdk 11 JRE in the base ingestion image. This fixes #6310 and possibly also #4741

The fix was tested like this:

docker build -t acryldata/datahub-ingestion-base  -f docker/datahub-ingestion/base.Dockerfile .
docker build -t acryldata/datahub-ingestion  -f docker/datahub-ingestion/Dockerfile .

docker run -it --entrypoint="/bin/bash" acryldata/datahub-ingestion 
$ java --version 
> openjdk 11.0.16 2022-07-19
> OpenJDK Runtime Environment (build 11.0.16+8-post-Debian-1deb11u1)
> OpenJDK 64-Bit Server VM (build 11.0.16+8-post-Debian-1deb11u1, mixed mode)

$ exit

docker run -it --user root -v /my/path/recipes:/temp acryldata/datahub-ingestion ingest run --dry-run -c /temp/kafka-connect-to-datahub-kafka.yml
> [2022-11-04 13:22:39,391] INFO     {datahub.cli.ingest_cli:167} - DataHub CLI version: 
> [2022-11-04 13:22:39,419] INFO     {datahub.ingestion.run.pipeline:174} - Sink configured successfully. 
> [2022-11-04 13:22:39,825] INFO     {datahub.ingestion.source.kafka_connect:877} - Connection to <URL> is ok
> [2022-11-04 13:22:39,990] INFO     {datahub.ingestion.run.pipeline:197} - Source configured successfully.
> [2022-11-04 13:22:39,991] INFO     {datahub.cli.ingest_cli:120} - Starting metadata ingestion
> ...
>  Pipeline finished successfully; produced X events in 9.25 seconds.

Checklist

  • The PR conforms to DataHub's Contributing Guideline (particularly Commit Message Format)
  • Links to related issues (if applicable)
  • Tests for the changes have been added/updated (if applicable)
  • Docs related to the changes have been added/updated (if applicable). If a new feature has been added a Usage Guide has been added for the same.
  • For any breaking change/potential downtime/deprecation/big changes an entry has been made in Updating DataHub

@github-actions github-actions bot added the devops PR or Issue related to DataHub backend & deployment label Nov 4, 2022
@github-actions
Copy link

github-actions bot commented Nov 4, 2022

Unit Test Results (build & test)

613 tests  ±0   609 ✔️ ±0   12m 0s ⏱️ +20s
151 suites ±0       4 💤 ±0 
151 files   ±0       0 ±0 

Results for commit af88300. ± Comparison against base commit 3080b9e.

Copy link
Collaborator

@anshbansal anshbansal left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@anshbansal anshbansal merged commit 9e86c6f into datahub-project:master Nov 4, 2022
anshbansal pushed a commit to acryldata/datahub that referenced this pull request Nov 4, 2022
anthonyhungnguyen pushed a commit to anthonyhungnguyen/zlp-datahub that referenced this pull request Nov 8, 2022
cccs-tom pushed a commit to CybercentreCanada/datahub that referenced this pull request Nov 18, 2022
cccs-tom pushed a commit to CybercentreCanada/datahub that referenced this pull request Nov 18, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
devops PR or Issue related to DataHub backend & deployment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Ingestion image is missing Java
2 participants