Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add more warehouses to database permissions #3943

Merged
merged 52 commits into from
Nov 1, 2023
Merged
Show file tree
Hide file tree
Changes from 49 commits
Commits
Show all changes
52 commits
Select commit Hold shift + click to select a range
32396db
Add more warehouses to database permissions
agnessnowplow Aug 17, 2023
081512e
Fix broken links
agnessnowplow Aug 17, 2023
a05aa94
Merge branch 'current' into database_permissions
mirnawong1 Aug 30, 2023
ea1fab2
Merge branch 'current' into database_permissions
mirnawong1 Aug 30, 2023
d0c8aaf
Update website/docs/reference/database-permissions/databricks-permiss…
mirnawong1 Sep 25, 2023
350c8c4
Merge branch 'current' into database_permissions
mirnawong1 Sep 25, 2023
2fd5835
Update website/docs/reference/database-permissions/redshift-permissio…
mirnawong1 Sep 25, 2023
92392ff
Update website/docs/reference/database-permissions/postgres-permissio…
mirnawong1 Sep 25, 2023
0ba6076
Update website/docs/reference/references-overview.md
mirnawong1 Sep 25, 2023
d5b0f09
Update website/docs/reference/references-overview.md
mirnawong1 Sep 25, 2023
327f3e9
Update website/docs/reference/references-overview.md
mirnawong1 Sep 25, 2023
a8718ee
Update website/docs/reference/references-overview.md
mirnawong1 Sep 25, 2023
c9f9b87
Merge branch 'current' into database_permissions
runleonarun Sep 28, 2023
ae794e0
Merge branch 'current' into database_permissions
mirnawong1 Oct 23, 2023
60c2d98
Merge branch 'current' into database_permissions
mirnawong1 Oct 24, 2023
ab020ef
Merge branch 'current' into database_permissions
mirnawong1 Oct 26, 2023
90a09bc
Merge branch 'current' into database_permissions
mirnawong1 Oct 31, 2023
b295d43
Merge branch 'current' into database_permissions
mirnawong1 Oct 31, 2023
864d215
Create about-database-permissions.md
mirnawong1 Oct 31, 2023
385bcdb
Update about-database-permissions.md
mirnawong1 Oct 31, 2023
14691d8
Update snowflake-permissions.md
mirnawong1 Oct 31, 2023
9ef7714
Update databricks-permissions.md
mirnawong1 Oct 31, 2023
6ff0cd3
Update databricks-permissions.md
mirnawong1 Oct 31, 2023
bcd6cc1
Update snowflake-permissions.md
mirnawong1 Oct 31, 2023
8c8140d
Update postgres-permissions.md
mirnawong1 Oct 31, 2023
e900d23
Update redshift-permissions.md
mirnawong1 Oct 31, 2023
425cd0f
Update redshift-permissions.md
mirnawong1 Oct 31, 2023
8441614
Update postgres-permissions.md
mirnawong1 Oct 31, 2023
6cd7589
Update databricks-permissions.md
mirnawong1 Oct 31, 2023
9142147
Merge branch 'current' into database_permissions
mirnawong1 Nov 1, 2023
7196b37
Update website/docs/reference/database-permissions/about-database-per…
mirnawong1 Nov 1, 2023
77beb97
Update website/docs/reference/database-permissions/about-database-per…
mirnawong1 Nov 1, 2023
5b75635
Update website/docs/reference/database-permissions/about-database-per…
mirnawong1 Nov 1, 2023
0bd0d32
Update sidebars.js
mirnawong1 Nov 1, 2023
3258937
Update about-database-permissions.md
mirnawong1 Nov 1, 2023
caa0c5d
Update sidebars.js
mirnawong1 Nov 1, 2023
e212d17
Update website/docs/reference/database-permissions/postgres-permissio…
mirnawong1 Nov 1, 2023
03ddf3e
Update snowflake-permissions.md
mirnawong1 Nov 1, 2023
dace850
Update sidebars.js
mirnawong1 Nov 1, 2023
b08a9af
Update website/docs/faqs/Warehouse/database-privileges.md
mirnawong1 Nov 1, 2023
b5b4795
Update connect-databricks.md
mirnawong1 Nov 1, 2023
1246101
Update connect-redshift-postgresql-alloydb.md
mirnawong1 Nov 1, 2023
3a20b9f
Merge branch 'current' into database_permissions
mirnawong1 Nov 1, 2023
4ec7ff6
Update sidebars.js
mirnawong1 Nov 1, 2023
28f376d
Update sidebars.js
mirnawong1 Nov 1, 2023
1a7ecd7
Merge branch 'current' into database_permissions
mirnawong1 Nov 1, 2023
d7fd7d4
Update website/docs/reference/database-permissions/snowflake-permissi…
mirnawong1 Nov 1, 2023
356c281
Update website/docs/reference/database-permissions/snowflake-permissi…
mirnawong1 Nov 1, 2023
a44a9c4
Update website/docs/reference/database-permissions/snowflake-permissi…
mirnawong1 Nov 1, 2023
f373cbe
Update website/docs/reference/database-permissions/about-database-per…
mirnawong1 Nov 1, 2023
6768a72
Update about-database-permissions.md
mirnawong1 Nov 1, 2023
14aa364
Update website/docs/reference/database-permissions/about-database-per…
mirnawong1 Nov 1, 2023
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
Expand Up @@ -26,6 +26,8 @@ Unity Catalog allows Databricks users to centrally manage all data assets, simpl

To learn how to optimize performance with data platform-specific configurations in dbt Cloud, refer to [Databricks-specific configuration](/reference/resource-configs/databricks-configs).

To grant users or roles database permissions (access rights and privileges), refer to the [example permissions](/reference/database-permissions/databricks-permissions) page.


To set up the Databricks connection, supply the following fields:

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -64,3 +64,5 @@ The Bastion server should now be ready for dbt Cloud to use as a tunnel into the
## Configuration

To learn how to optimize performance with data platform-specific configurations in dbt Cloud, refer to [Redshift-specific configuration](/reference/resource-configs/redshift-configs).

To grant users or roles database permissions (access rights and privileges), refer to the [Redshift permissions](/reference/database-permissions/redshift-permissions) page or [Postgres permissions](/reference/database-permissions/postgres-permissions) page.
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,7 @@ The following fields are required when creating a Snowflake connection
| Warehouse | The virtual warehouse to use for running queries. | `transforming` |


**Note:** A crucial part of working with dbt atop Snowflake is ensuring that users (in development environments) and/or service accounts (in deployment to production environments) have the correct permissions to take actions on Snowflake! Here is documentation of some [example permissions to configure Snowflake access](/reference/snowflake-permissions).
**Note:** A crucial part of working with dbt atop Snowflake is ensuring that users (in development environments) and/or service accounts (in deployment to production environments) have the correct permissions to take actions on Snowflake! Here is documentation of some [example permissions to configure Snowflake access](/reference/database-permissions/snowflake-permissions).

### Username / Password

Expand Down
4 changes: 2 additions & 2 deletions website/docs/faqs/Warehouse/database-privileges.md
Original file line number Diff line number Diff line change
Expand Up @@ -12,8 +12,8 @@ schema¹
* read system <Term id="view">views</Term> to generate documentation (i.e. views in
`information_schema`)

On Postgres, Redshift, and Snowflake, use a series of `grants` to ensure that
your user has the correct privileges.
On Postgres, Redshift, Databricks, and Snowflake, use a series of `grants` to ensure that
your user has the correct privileges. Check out [example permissions](/reference/database-permissions/about-database-permissions) for these warehouses.

On BigQuery, use the "BigQuery User" role to assign these privileges.

Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,35 @@
---
title: "Database permissions"
id: about-database-permissions
description: "Database permissions are access rights and privileges granted to users or roles within a database management system."
sidebar_label: "About database permissions"
pagination_next: "reference/database-permissions/databricks-permissions"
pagination_prev: null
---

Database permissions are access rights and privileges granted to users or roles within a database or data platform. They help you specify what actions users or roles can perform on various database objects, like tables, views, schemas, or even the entire database.


## Why are they useful
mirnawong1 marked this conversation as resolved.
Show resolved Hide resolved

- Database permissions are essential for security and data access control.
- They ensure that only authorized users can perform specific actions.
- They help maintain data integrity, prevent unauthorized changes, and limit exposure to sensitive data.
- Permissions also support compliance with data privacy regulations and auditing.

## How to use them
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is it worth adding here that permissions can be granted at various levels e.g. table, schema, warehouses, database etc. Also that each cloud vendor has a slightly different approach and naming to privelages so to check each vendors documentation for more specifics?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

great flag @rlh1994 , i've added some more context based on your sugg above!


- Users and administrators can grant and manage permissions using SQL statements or through the database system's interface.
mirnawong1 marked this conversation as resolved.
Show resolved Hide resolved
- Assign permissions to individual users or roles (groups of users) based on their responsibilities.
- Typical permissions include "SELECT" (read), "INSERT" (add data), "UPDATE" (modify data), "DELETE" (remove data), and administrative rights like "CREATE" and "DROP."
- Users should be assigned permissions that ensure they have the necessary access to perform their tasks without overextending privileges.

mirnawong1 marked this conversation as resolved.
Show resolved Hide resolved
Something to note is that each data platform provider might have different approaches and names for privileges. Refer to their documentation for more details.
## Examples

Refer to the following database permission pages for more info on examples and how to setup database permissions:

- [Databricks](/reference/database-permissions/databricks-permissions)
- [Postgres](/reference/database-permissions/postgres-permissions)
- [Redshift](/reference/database-permissions/redshift-permissions)
- [Snowflake](/reference/database-permissions/snowflake-permissions)
Original file line number Diff line number Diff line change
@@ -0,0 +1,20 @@
---
title: "Databricks permissions"
---

In Databricks, permissions are used to control who can perform certain actions on different database objects. Use SQL statements to manage permissions in a Databricks database.

## Example Databricks permissions

The following example provides you with the SQL statements you can use to manage permissions.

**Note** that you can grant permissions on `securable_objects` to `principals` (This can be user, service principal, or group). For example, `grant privilege_type` on `securable_object` to `principal`.

```

grant all privileges on schema schema_name to principal;
grant create table on schema schema_name to principal;
grant create view on schema schema_name to principal;
```

Check out the [official documentation](https://docs.databricks.com/en/data-governance/unity-catalog/manage-privileges/privileges.html#privilege-types-by-securable-object-in-unity-catalog) for more information.
Original file line number Diff line number Diff line change
@@ -0,0 +1,25 @@
---
title: "Postgres Permissions"
---


In Postgres, permissions are used to control who can perform certain actions on different database objects. Use SQL statements to manage permissions in a Postgres database.

## Example Postgres permissions

The following example provides you with the SQL statements you can use to manage permissions. These examples allow you to run dbt smoothly without encountering permission issues, such as creating schemas, reading existing data, and accessing the information schema.

**Note** that `database_name`, `database.schema_name`, and `user_name` are placeholders and you can replace them as needed for your organization's naming convention.

```
grant usage on database database_name to user_name;
grant create schema on database database_name to user_name;
grant usage on schema database.schema_name to user_name;
grant create table on schema database.schema_name to user_name;
grant create view on schema database.schema_name to user_name;
grant usage on all schemas in database database_name to user_name;
grant select on all tables in database database_name to user_name;
grant select on all views in database database_name to user_name;
```

Check out the [official documentation](https://www.postgresql.org/docs/current/sql-grant.html) for more information.
Original file line number Diff line number Diff line change
@@ -0,0 +1,25 @@
---
title: "Redshift permissions"
---

In Redshift, permissions are used to control who can perform certain actions on different database objects. Use SQL statements to manage permissions in a Redshift database.

## Example Redshift permissions

The following example provides you with the SQL statements you can use to manage permissions.

**Note** that `database_name`, `database.schema_name`, and `user_name` are placeholders and you can replace them as needed for your organization's naming convention.


```
grant usage on database database_name to user_name;
grant create schema on database database_name to user_name;
grant usage on schema database.schema_name to user_name;
grant create table on schema database.schema_name to user_name;
grant create view on schema database.schema_name to user_name;
grant usage on all schemas in database database_name to user_name;
grant select on all tables in database database_name to user_name;
grant select on all views in database database_name to user_name;
```

Check out the [official documentation](https://docs.aws.amazon.com/redshift/latest/dg/r_GRANT.html) for more information.
154 changes: 154 additions & 0 deletions website/docs/reference/database-permissions/snowflake-permissions.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,154 @@
---
title: "Snowflake permissions"
---

In Snowflake, permissions are used to control who can perform certain actions on different database objects. Use SQL statements to manage permissions in a Snowflake database.

## Set up Snowflake account

This section explains how to set up permissions and roles within Snowflake. In Snowflake, you would perform these actions using SQL commands and set up your data warehouse and access control within Snowflake's ecosystem.

1. Set up databases
```
use role sysadmin;
create database raw;
create database analytics;
```
2. Set up warehouses
```
create warehouse loading
warehouse_size = xsmall
auto_suspend = 3600
auto_resume = false
initially_suspended = true;

create warehouse transforming
warehouse_size = xsmall
auto_suspend = 60
auto_resume = true
initially_suspended = true;

create warehouse reporting
warehouse_size = xsmall
auto_suspend = 60
auto_resume = true
initially_suspended = true;
```

3. Set up roles and warehouse permissions
```
use role securityadmin;

create role loader;
grant all on warehouse loading to role loader;

create role transformer;
grant all on warehouse transforming to role transformer;

create role reporter;
grant all on warehouse reporting to role reporter;
```

4. Create users, assigning them to their roles

Every person and application gets a separate user and is assigned to the correct role.
mirnawong1 marked this conversation as resolved.
Show resolved Hide resolved

```
create user stitch_user -- or fivetran_user
password = '_generate_this_'
default_warehouse = loading
default_role = loader;

create user claire -- or amy, jeremy, etc.
password = '_generate_this_'
default_warehouse = transforming
default_role = transformer
must_change_password = true;

create user dbt_cloud_user
password = '_generate_this_'
default_warehouse = transforming
default_role = transformer;

create user looker_user -- or mode_user etc.
password = '_generate_this_'
default_warehouse = reporting
default_role = reporter;

-- then grant these roles to each user
grant role loader to user stitch_user; -- or fivetran_user
grant role transformer to user dbt_cloud_user;
grant role transformer to user claire; -- or amy, jeremy
grant role reporter to user looker_user; -- or mode_user, periscope_user
```

5. Let loader load data
Give the role unilateral permission to operate on the raw database
```
use role sysadmin;
grant all on database raw to role loader;
```

6. Let transformer transform data
The transformer role needs to be able to read raw data.

If you do this before you have any data loaded, you can run:
```
grant usage on database raw to role transformer;
grant usage on future schemas in database raw to role transformer;
grant select on future tables in database raw to role transformer;
grant select on future views in database raw to role transformer;
```
If you already have data loaded in the raw database, make sure also you run the following to update the permissions
```
grant usage on all schemas in database raw to role transformer;
grant select on all tables in database raw to role transformer;
grant select on all views in database raw to role transformer;
```
transformer also needs to be able to create in the analytics database:
```
grant all on database analytics to role transformer;
```
7. Let reporter read the transformed data
A previous version of this article recommended this be implemented through hooks in dbt, but this way lets you get away with a one-off statement.
```
grant usage on database analytics to role reporter;
grant usage on future schemas in database analytics to role reporter;
grant select on future tables in database analytics to role reporter;
grant select on future views in database analytics to role reporter;
```
Again, if you already have data in your analytics database, make sure you run:
```
grant usage on all schemas in database analytics to role reporter;
grant select on all tables in database analytics to role transformer;
grant select on all views in database analytics to role transformer;
```
8. Maintain
When new users are added, make sure you add them to the right role! Everything else should be inherited automatically thanks to those `future` grants.

For more discussion and legacy information, refer to [this Discourse article](https://discourse.getdbt.com/t/setting-up-snowflake-the-exact-grant-statements-we-run/439).

## Example Snowflake permissions

The following example provides you with the SQL statements you can use to manage permissions.

**Note** that `warehouse_name`, `database_name`, and `role_name` are placeholders and you can replace them as needed for your organization's naming convention.

```

grant all on warehouse warehouse_name to role role_name;
grant usage on database database_name to role role_name;
grant create schema on database database_name to role role_name;
grant usage on schema database.an_existing_schema to role role_name;
grant create table on schema database.an_existing_schema to role role_name;
grant create view on schema database.an_existing_schema to role role_name;
grant usage on future schemas in database database_name to role role_name;
grant monitor on future schemas in database database_name to role role_name;
grant select on future tables in database database_name to role role_name;
grant select on future views in database database_name to role role_name;
grant usage on all schemas in database database_name to role role_name;
grant monitor on all schemas in database database_name to role role_name;
grant select on all tables in database database_name to role role_name;
grant select on all views in database database_name to role role_name;
```

22 changes: 20 additions & 2 deletions website/docs/reference/references-overview.md
Original file line number Diff line number Diff line change
Expand Up @@ -51,9 +51,27 @@ Learn how to add more configurations to your dbt project or adapter, use propert
icon="computer"/>

<Card
title="Snowflake Permissions Artifacts"
title="Snowflake permissions artifacts"
body="Provides an example Snowflake database role permissions."
link="/reference/snowflake-permissions"
link="/reference/database-permissions/snowflake-permissions"
icon="computer"/>

<Card
title="Databricks permissions artifacts"
body="Provides an example Databricks database role permissions."
link="/reference/database-permissions/databricks-permissions"
icon="computer"/>

<Card
title="Redshift permissions artifacts"
body="Provides an example Redshift database role permissions."
link="/reference/database-permissions/redshift-permissions"
icon="computer"/>

<Card
title="Postgres permissions artifacts"
body="Provides an example Postgres database role permissions."
link="/reference/database-permissions/postgres-permissions"
icon="computer"/>

</div>
25 changes: 0 additions & 25 deletions website/docs/reference/snowflake-permissions.md

This file was deleted.

8 changes: 7 additions & 1 deletion website/sidebars.js
Original file line number Diff line number Diff line change
Expand Up @@ -936,7 +936,13 @@ const sidebarSettings = {
{
type: "category",
label: "Database Permissions",
items: ["reference/snowflake-permissions"],
items: [
"reference/database-permissions/about-database-permissions",
"reference/database-permissions/databricks-permissions",
mirnawong1 marked this conversation as resolved.
Show resolved Hide resolved
"reference/database-permissions/postgres-permissions",
"reference/database-permissions/redshift-permissions",
"reference/database-permissions/snowflake-permissions",
],
},
],
guides: [
Expand Down