[Bug]: dbt seed fails if Dremio space does not exist

[maxfirman]

Is there an existing issue for this?

• I have searched the existing issues

Current Behavior

Executing dbt seed fails if the Dremio space does not exist already. This is because it tries to create the seed view before creating the Dremio space.

Creating the Dremio space manually and re-running dbt seed succeeds.

Expected Behavior

Executing dbt seed should succeed even if the Dremio space does not already exist.

Steps To Reproduce

1. Create a new project with a seed targeting a Dremio space that does not already exist.
2. Execute dbt seed.

Environment

- OS: Ubuntu
- dbt-dremio: 1.5.0
- Dremio Software: 24.0.2 Enterprise

Relevant log output

No response

[fabrice-etanchaud]

It's related to #194 and to the twin_strategy :

by default twin_strategy = clone : for a materialization leading to a table Relation, a proxy view selecting from the table is (re)created in the corresponding custom folder based on the default and custom database+schema confs and the possibly overriden custom macros. But this is a 'light' view created by the adapter, not a regular dbt model. So dbt does not know at startup that it has to check if the view's path exists and create it overwise.

For time being, you can try to :

• set the twin_strategy to prevent or allow (not the default clone)
• and create a real model materialized as a view ontop if you really need it.

The only problem is for tables not in standard parquet/iceberg format. Indeed, the 'light' view reuse the write formating configuration of the underlying table to build the select with the good read formating options.

There may be a way to add a create_schema call in the apply_twin_strategy macro :
https://github.com/dremio/dbt-dremio/blob/8143c81a4e357ac90d1539192a8a6ee047d7e555/dbt/include/dremio/macros/materializations/twin_strategy.sql#L15C11-L15C30

[kenho811]

Is it possible that dbt-dremio automatically create the dremio space folder for us?

That would be quite useful for automation.