Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

add TestLargeEphemeralCompilation #8376

Merged
merged 5 commits into from
Sep 18, 2023
Merged

add TestLargeEphemeralCompilation #8376

merged 5 commits into from
Sep 18, 2023

Conversation

MichelleArk
Copy link
Contributor

@MichelleArk MichelleArk commented Aug 12, 2023

resolves #8225
docs dbt-labs/docs.getdbt.com/#

Problem

We are missing a test to capture a failure case for ephemeral model cte injection, which was resolved in #8215

Solution

Convert the reproduction case created here https://github.com/jeremyyeo/ephemeral-invalid-sql-repro/tree/main/models, which was used to produce the failed run here: https://github.com/jeremyyeo/ephemeral-invalid-sql-repro/actions/runs/5667308405/job/15355782134

Checklist

  • I have read the contributing guide and understand what's expected of me
  • I have run this code in development and it appears to resolve the stated issue
  • This PR includes tests, or tests are not required/relevant for this PR
  • This PR has no interface changes (e.g. macros, cli, logs, json artifacts, config files, adapter interface, etc) or this PR has already received feedback and approval from Product or DX

@cla-bot cla-bot bot added the cla:yes label Aug 12, 2023
@github-actions
Copy link
Contributor

Thank you for your pull request! We could not find a changelog entry for this change. For details on how to document a change, see the contributing guide.

@codecov
Copy link

codecov bot commented Aug 12, 2023

Codecov Report

Patch coverage has no change and project coverage change: -0.05% ⚠️

Comparison is base (5182e3c) 86.61% compared to head (4d19f4c) 86.56%.

Additional details and impacted files
@@            Coverage Diff             @@
##             main    #8376      +/-   ##
==========================================
- Coverage   86.61%   86.56%   -0.05%     
==========================================
  Files         175      175              
  Lines       25595    25595              
==========================================
- Hits        22168    22156      -12     
- Misses       3427     3439      +12     
Flag Coverage Δ
integration 83.35% <ø> (-0.11%) ⬇️
unit 65.12% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

see 2 files with indirect coverage changes

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@MichelleArk
Copy link
Contributor Author

MichelleArk commented Aug 12, 2023

I checked out the commit (fecde23) prior to the fix: fdeccfa, but wasn't able to see this test fail.

I copied the setup from https://github.com/jeremyyeo/ephemeral-invalid-sql-repro/tree/main/models, which is what I believe was used to produce the failed run here: https://github.com/jeremyyeo/ephemeral-invalid-sql-repro/actions/runs/5667308405/job/15355782134

But to my understanding, this issue is adapter-agnostic and should be reproducible in our functional tests that use postgres.

@MichelleArk
Copy link
Contributor Author

Seeing the error in a failure when removing the fix introduced fdeccfa

from CI logs running on ubuntu: https://github.com/dbt-labs/dbt-core/actions/runs/6165034837/job/16732021940?pr=8376

--- RunResultError
{"data": {"msg": "Database Error in test passing (tests/passing.sql)\n  syntax error at or near \"with\"\n  LINE 14: ) with my_other_cool_cte as (\n             ^\n  compiled Code at target/run/singular_tests_ephemeral/tests/passing.sql"}, "info": {"category": "", "code": "Z024", "extra": {}, "invocation_id": "d02541ec-e8e1-4dae-a7c8-de0373a37874", "level": "error", "msg": "  Database Error in test passing (tests/passing.sql)\n  syntax error at or near \"with\"\n  LINE 14: ) with my_other_cool_cte as (\n             ^\n  compiled Code at target/run/singular_tests_ephemeral/tests/passing.sql", "name": "RunResultError", "pid": 4548, "thread": "MainThread", "ts": "2023-09-12T21:31:58.135065Z"}}

@MichelleArk MichelleArk marked this pull request as ready for review September 12, 2023 22:29
@MichelleArk MichelleArk requested a review from a team as a code owner September 12, 2023 22:29
@MichelleArk MichelleArk merged commit d597b80 into main Sep 18, 2023
50 checks passed
@MichelleArk MichelleArk deleted the 8225-ephemeral-test branch September 18, 2023 14:00
github-actions bot pushed a commit that referenced this pull request Sep 26, 2023
peterallenwebb added a commit that referenced this pull request Sep 29, 2023
* Add new get_catalog_relations macro, allowing dbt to specify which relations in a schema the adapter should return data about

* Implement postgres adapter support for relation filtering on catalog queries

* Code review changes adding feature flag for catalog-by-relation-list support

* Use profile specified in --profile with dbt init (#7450)

* Use profile specified in --profile with dbt init

* Update .changes/unreleased/Fixes-20230424-161642.yaml

Co-authored-by: Doug Beatty <[email protected]>

* Refactor run() method into functions, replace exit() calls with exceptions

* Update help text for profile option

---------

Co-authored-by: Doug Beatty <[email protected]>

* add TestLargeEphemeralCompilation (#8376)

* Fix a couple of issues in the postgres implementation of get_catalog_relations

* Add relation count limit at which to fall back to batch retrieval

* Better feature detection mechanism for adapters.

* Code review changes to get_catalog_relations and adapter feature checking

* Add changelog entry

---------

Co-authored-by: ezraerb <[email protected]>
Co-authored-by: Doug Beatty <[email protected]>
Co-authored-by: Michelle Ark <[email protected]>
QMalcolm pushed a commit that referenced this pull request Oct 9, 2023
QMalcolm pushed a commit that referenced this pull request Oct 9, 2023
* Add new get_catalog_relations macro, allowing dbt to specify which relations in a schema the adapter should return data about

* Implement postgres adapter support for relation filtering on catalog queries

* Code review changes adding feature flag for catalog-by-relation-list support

* Use profile specified in --profile with dbt init (#7450)

* Use profile specified in --profile with dbt init

* Update .changes/unreleased/Fixes-20230424-161642.yaml

Co-authored-by: Doug Beatty <[email protected]>

* Refactor run() method into functions, replace exit() calls with exceptions

* Update help text for profile option

---------

Co-authored-by: Doug Beatty <[email protected]>

* add TestLargeEphemeralCompilation (#8376)

* Fix a couple of issues in the postgres implementation of get_catalog_relations

* Add relation count limit at which to fall back to batch retrieval

* Better feature detection mechanism for adapters.

* Code review changes to get_catalog_relations and adapter feature checking

* Add changelog entry

---------

Co-authored-by: ezraerb <[email protected]>
Co-authored-by: Doug Beatty <[email protected]>
Co-authored-by: Michelle Ark <[email protected]>
@aranke aranke mentioned this pull request Jul 12, 2024
5 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[CT-2877] Convert test project for invalid ephemeral sql into dbt Core test
3 participants