Skip to content

Commit

Permalink
fix pathing to config
Browse files Browse the repository at this point in the history
  • Loading branch information
nicklausroach committed Apr 24, 2024
1 parent c733928 commit fc9d792
Show file tree
Hide file tree
Showing 4 changed files with 58 additions and 10 deletions.
59 changes: 53 additions & 6 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -94,17 +94,19 @@
<!-- USAGE EXAMPLES -->
## Usage

Currently, sprocketship expects a `.sprocketship.yml` file and a `procedures/` directory at the same level in a directory structure.
### Structure

Currently, sprocketship expects a `.sprocketship.yml` file in a `procedures/` directory.

```
├── dbt_models
│ ├── customers.sql
│ ├── products.sql
├── procedures
│ ├── admin
│ ├── useradmin
│ │ ├── create_database_writer_role.js
│ │ ├── create_database_reader_role.js
│ ├── development
│ ├── sysadmin
│ │ ├── create_temp_database.js
└── .sprocketship.yml
```
Expand All @@ -131,6 +133,52 @@ procedures:
...
```

### Directory-level Default Parameters

sprocketship allows providing default parameters at any given level of
your project. These defaults will be applied recursively to any procedures
defined in any of the subdirectories, unless overridden by a default in one
of the subdirectories.

```
procedures:
# for all procedures, default to the below database and schema
+database: !env_var SNOWFLAKE_DATABASE
+schema: !env_var SNOWFLAKE_SCHEMA
development:
# for all procedures in the development directory,
# default to using the sysadmin role
+use_role: sysadmin
create_temp_database:
args:
- name: Name of argument
type: Type of argument
default: (Optional) default value for the argument
returns: varchar
```

### File Frontmatter

Thanks to ABSQL, sprocketship also provides the ability to define parameters using file frontmatter. Suppose we have a file `create_database_writer_role.js`, we can define parameters for the stored procedure within the file using frontmatter:

```js
/*
database: my_database
schema: my_schema
language: javascript
execute_as: owner
use_role: sysadmin
*/
```

sprocketship will automatically parse and apply the parameters defined in the frontmatter to the stored procedure.

### Recommended Configuration

When setting up your sprocketship project, we recommend setting more general parameters (e.g., database, schema, language, etc.) in the `.sprocketship.yml` file, and anything that's specific to a given procedure should be defined in the file frontmatter of that procedure, such as the args or return type.

### Execution

From here, simply run

`$ sprocketship liftoff`
Expand All @@ -140,24 +188,23 @@ from the project directory (or provide the directory, e.g. `sprocketship liftoff
### Exhaustive Options for Stored Procedure Configuration

```
name: The name of the procedure
database: The name of the database where the procedure will be stored
schema: The name of the schema where the procedure will be stored
language: The language of the procedure definition
execute_as: caller or owner
use_role: The role you'd like to own the procedure
args:
- name: Name of argument
type: Type of argument
default: (Optional) default value for the argument
returns: The return type
returns: The return type, this can include the `NOT NULL` option
comment: Explanation of the procedure
```

## Support

sprocketship currently only supports Javascript-based stored procedures (Python support coming soon!). Additionally, there are a few options from the `CREATE STORED PROCEDURE` function that are not yet supported:

* `RETURNS <result-data-type> NOT NULL`
* `CALLED ON NULL INPUT | { RETURNS NULL ON NULL INPUT | STRICT }`
* `VOLATILE | IMMUTABLE` (deprecated)

Expand Down
2 changes: 1 addition & 1 deletion pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ build-backend = "setuptools.build_meta"

[project]
name = "sprocketship"
version = "1.0.0"
version = "1.0.1"
authors = [
{ name="Nicklaus Roach", email="[email protected]" },
]
Expand Down
4 changes: 2 additions & 2 deletions sprocketship/cli.py
Original file line number Diff line number Diff line change
Expand Up @@ -26,7 +26,7 @@ def main(ctx):
def liftoff(dir, show):
click.echo(click.style(f"🚀 Sprocketship lifting off!", fg="white", bold=True))
data = render_file(
os.path.join(dir, "procedures", ".sprocketship.yml"), return_dict=True
os.path.join(dir, ".sprocketship.yml"), return_dict=True
)
con = connector.connect(**data["snowflake"])
files = list(Path(dir).rglob("*.js"))
Expand Down Expand Up @@ -74,7 +74,7 @@ def build(dir, target):
Path(os.path.join(dir, target)).mkdir(parents=True, exist_ok=True)

data = render_file(
os.path.join(dir, "procedures", ".sprocketship.yml"), return_dict=True
os.path.join(dir, ".sprocketship.yml"), return_dict=True
)
files = list(Path(dir).rglob("*.js"))

Expand Down
3 changes: 2 additions & 1 deletion sprocketship/utils.py
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,8 @@ def extract_configs(data, path=""):

def get_file_config(path: Path, config: dict, dir: str):
filename = path.stem
keys = str(path.relative_to(dir)).split("/")[:-1] + [filename]
keys = ["procedures"] + str(path.relative_to(dir)).split("/")[:-1] + [filename]
print(keys)

file_config = {"path": str(path), "name": filename}
curr_config = config
Expand Down

0 comments on commit fc9d792

Please sign in to comment.