This project illustrates how to use the Soda framework to run data quality checks against Snowflake tables, entirely on your local machine.
The code is based on the Snowflake Quickstart Guide on Data Quality Testing with Soda.
localstack
CLI withLOCALSTACK_AUTH_TOKEN
environment variable setawslocal
CLI- LocalStack Snowflake emulator
snow
CLI withlocalstack
connection profile pointing to LocalStack Snowflake
To install the snow
CLI with a local connection profile, follow the instructions here.
It basically boils down to running this command:
snow connection add \
--connection-name localstack \
--user test \
--password test \
--account test \
--host snowflake.localhost.localstack.cloud
Run the following command to install dependencies for the project (into a local Python virtual environment):
make install
Start the LocalStack Snowflake emulator using the following command:
DOCKER_FLAGS='-e SF_LOG=trace' \
IMAGE_NAME=localstack/snowflake \
DEBUG=1 \
localstack start
The sample application provides Makefile targets to simplify the setup process.
Run the following command to initialize the environment and seed test data into local Snowflake:
make init
Once the test data has been set up, we can run the Soda data quality checks via this command:
make scan
The code in this project is licensed under the Apache 2.0 License.