Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Tests Being Put into Own "tests_schema" #168

Open
ernestoongaro opened this issue May 10, 2024 · 6 comments
Open

Tests Being Put into Own "tests_schema" #168

ernestoongaro opened this issue May 10, 2024 · 6 comments

Comments

@ernestoongaro
Copy link

ernestoongaro commented May 10, 2024

Hello! I have some standard tests in my project, starting with dbt-fabric 1.8 this is what's happening:

  1. Create a standard built in test
  2. Run that test
  3. Expect: test does not need to create a view or anything; it's transient
  4. Results, a view is attempted to be created. Seems to be related to this: https://github.com/microsoft/dbt-fabric/blame/adf9f8d6557e41eaae8ddacc14a08c057d67a687/dbt/include/fabric/macros/materializations/tests/helpers.sql#L5
EXEC('create view 
      dbt_eongaro_tests_schema.testview_4608
     as 
    
    

with all_values as (

    select
        region as value_field,
        count(*) as n_records

    from "sa_demo"."dbt_eongaro"."dim_customers"
    group by region

)

select *
from all_values
where value_field not in (
    ''AFRICA'',''MIDDLE EAST'',''ASIA'',''EUROPE'',''AMERICA''
)


;')
    select
      
      count(*) as failures,
      case when count(*) != 0
        then 'true' else 'false' end as should_warn,
      case when count(*) != 0
        then 'true' else 'false' end as should_error
    from (
      select * from 
      dbt_eongaro_tests_schema.testview_4608
    
    ) dbt_internal_test;

    EXEC('drop view 
      dbt_eongaro_tests_schema.testview_4608
    ;')

  
15:12:06 Opening a new connection, currently in state closed
15:12:06 fabric adapter: Using connection string: DRIVER={ODBC Driver 18 for SQL Server};SERVER=5xxoty5si6telax6vdkh6id534-fjebxqqv6kvexcr6vmlnwsjs2e.datawarehouse.pbidedicated.windows.net;Database=sa_demo;Authentication=ActiveDirectoryServicePrincipal;UID={ca9252f9-6361-4d5e-af31-8587eca1a429};PWD=***;encrypt=Yes;TrustServerCertificate=No;APP=dbt-fabric/1.8.2;ConnectRetryCount=1
15:12:08 fabric adapter: Connected to db: sa_demo
15:12:08 fabric adapter: Database error: ('42000', '[42000] [Microsoft][ODBC Driver 18 for SQL Server][SQL Server]The specified schema name "dbt_eongaro_tests_schema" either does not exist or you do not have permission to use it. (2760) (SQLExecDirectW)')
@ernestoongaro
Copy link
Author

It's not expected for tests to create views, and certainly not outside the specified schema. in this case my schema is dbt_eongaro and it's attempting to create them in dbt_eongaro_tests_schema

@ernestoongaro
Copy link
Author

ernestoongaro commented May 14, 2024

Hi @prdpsvs as requested via Slack here's a repro:

  1. Create a model my_model.sql:
with source_data as (

    select 'A' as something
    union all
    select 'B' as something

)

select *
from source_data
  1. create a yml file my_example.yml

version: 2

models:
    - name: my_model
      columns:
          - name: id
            tests:
                - not_null
                - accepted_values:
                    values:
                      - 'B'
                      - 'A'
  1. run dbt build --select my_model
  2. Observe that the test is attempted to be built in _tests_schema instead of and that actually it shouldn't try and execute EXEC('create view as tests are ephemral

error

('42000', '[42000] [Microsoft][ODBC Driver 18 for SQL Server][SQL Server]The specified schema name "dbt_eongaro_tests_schema" either does not exist or you do not have permission to use it. (2760) (SQLExecDirectW)')
  1. Note that the not_null test is perfectly fine as it does not try and build a view

@prdpsvs
Copy link
Collaborator

prdpsvs commented May 19, 2024

@ernestoongaro , I looked into these errors.

  1. Microsoft Fabric DW does not support nested CTE's. For that reason, the adapter cannot support ephermal tests.

  2. Adapter now will default these tests to dbt_test__audit (generated by dbt) schema to ensure consistency.

I will close this issue for now. Hope this helps.

prdpsvs added a commit that referenced this issue May 19, 2024
@prdpsvs prdpsvs mentioned this issue May 19, 2024
@prdpsvs prdpsvs closed this as completed May 20, 2024
@ThomsenS
Copy link

@prdpsvs I just updated to 1.8.4. The schema dbt_test__audit is not auto generated.

12:11:16 Database Error in test date_is_not_in_future_latest_inventory_movement_latest_inventory_movement (models\marts_marts_models.yml)
('42000', '[42000] [Microsoft][ODBC Driver 18 for SQL Server][SQL Server]The specified schema name "dbt_test__audit" either does not exist or you do not have permission to use it. (2760) (SQLExecDirectW)')
compiled Code at target\run\msbid\models\marts_marts_models.yml\date_is_not_in_future_latest_i_85e765eeda8d5db57964dc06f11dfe72.sql

@jtcohen6
Copy link
Contributor

jtcohen6 commented May 20, 2024

The schema dbt_test__audit is not auto generated.

By way of explanation: On every run, dbt-core will automatically check to see which schemas exist, and create those schemas for DAG nodes that are selected and "relational" (meaning, dbt is going to materialize this node as a database relation).

For some time, we've had conditional logic within dbt-core to generate schemas for tests if (and only if) "store failures" is enabled, via test config or the --store-failures flag.

I think there are two mitigation paths forward:

  1. dbt-fabric should use {{ target.schema }} instead of {{ node.schema }} as the location to create these views, under the assumption that the user is likely to be putting other models in their default configured schema, and dbt will create the schema accordingly.
  2. We investigate changing the logic in dbt-core so that test nodes' is_relational property is not conditional on should_store_failures (here). This is a change we'd want to make carefully, as it could have unrelated consequences.

We could change the default schema config of tests from dbt_test__audit to None. Given that it's been the former for many years (since v0.20 in 2021 IIRC), this would be a significant behaviour change and would require a migration plan.

Longer-term, it would be ideal for dbt-fabric use temp tables (scoped to the session) instead of persistent views (registered in a specific database/schema).

@prdpsvs I recommend reopening this issue in the meantime, while we sort out the appropriate mitigation.

@prdpsvs prdpsvs reopened this May 20, 2024
@prdpsvs
Copy link
Collaborator

prdpsvs commented May 31, 2024

@jtcohen6 , @ernestoongaro , I did not address this completely but dbt-fabric 1.8.6 creates target schema {{ target.schema }} if it does not exist.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

4 participants