Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: enable multiquery #12

Merged
merged 11 commits into from
May 22, 2023
Merged

feat: enable multiquery #12

merged 11 commits into from
May 22, 2023

Conversation

ariadnafer
Copy link

@ariadnafer ariadnafer commented May 12, 2023

Description

Previous custom components was not allowing running multi-queries in the same SQL file. This PR merges all the queries into a single SQL script and replace the custom component with the BigqueryQueryJobOp google component

How has this been tested?

Please explain how you have tested the new changes.

Checklist

  • I have commented my code, particularly in hard-to-understand areas
  • I have successfully run the E2E tests, and have included the links to the pipeline runs below
  • I have added tests that prove my fix is effective or that my feature works
  • New and existing unit tests pass locally with my changes
  • I have updated any relevant documentation to reflect my changes
  • I have assigned a reviewer and messaged them

Pipeline run links:

https://console.cloud.google.com/vertex-ai/locations/europe-west2/pipelines/runs/xgboost-train-pipeline-20230516134111?project=dt-turbo-templates-dev
https://console.cloud.google.com/vertex-ai/locations/europe-west2/pipelines/runs/xgboost-prediction-pipeline-20230516135109?project=dt-turbo-templates-dev
https://console.cloud.google.com/vertex-ai/locations/europe-west2/pipelines/runs/tensorflow-train-pipeline-20230516135752?project=dt-turbo-templates-dev
https://console.cloud.google.com/vertex-ai/locations/europe-west2/pipelines/runs/tensorflow-prediction-pipeline-20230516142755?project=dt-turbo-templates-dev

@ariadnafer ariadnafer changed the title WIP: enable multiquery feat: enable multiquery May 16, 2023
@ariadnafer ariadnafer requested review from a user and felix-datatonic May 16, 2023 12:36
pipelines/src/pipelines/__init__.py Outdated Show resolved Hide resolved
@ghost ghost changed the base branch from main to develop May 17, 2023 13:11
@ghost ghost dismissed felix-datatonic’s stale review May 17, 2023 13:11

The base branch was changed.

@ghost ghost force-pushed the feature/enable-multiquery branch from 0b8b77f to 299f834 Compare May 17, 2023 13:11
pipelines/src/pipelines/__init__.py Outdated Show resolved Hide resolved
pipelines/src/pipelines/tensorflow/prediction/pipeline.py Outdated Show resolved Hide resolved
@ariadnafer ariadnafer force-pushed the feature/enable-multiquery branch from f53af72 to e8b492f Compare May 20, 2023 21:08
@teamdatatonic teamdatatonic deleted a comment May 22, 2023
@teamdatatonic teamdatatonic deleted a comment from ariadnafer May 22, 2023
@teamdatatonic teamdatatonic deleted a comment May 22, 2023
@teamdatatonic teamdatatonic deleted a comment from ariadnafer May 22, 2023
@teamdatatonic teamdatatonic deleted a comment from ariadnafer May 22, 2023
@teamdatatonic teamdatatonic deleted a comment from ariadnafer May 22, 2023
@teamdatatonic teamdatatonic deleted a comment May 22, 2023
@felix-datatonic
Copy link
Collaborator

/gcbrun

@felix-datatonic felix-datatonic requested review from felix-datatonic and a user May 22, 2023 07:34
Copy link

@ghost ghost left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants