Skip to content

Commit

Permalink
Merge branch 'master' of github.com:flyteorg/flytesnacks into ray
Browse files Browse the repository at this point in the history
  • Loading branch information
pingsutw committed Aug 15, 2022
2 parents 2989d6a + 2549a19 commit 983e404
Show file tree
Hide file tree
Showing 4 changed files with 12 additions and 8 deletions.
4 changes: 2 additions & 2 deletions cookbook/integrations/external_services/snowflake/README.rst
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
Snowflake
====
=========

Flyte backend can be connected with snowflake service. Once enabled it can allow you to query a snowflake service.
This section will provide how to use the Snowflake Query Plugin using flytekit python.
Expand All @@ -19,7 +19,7 @@ This plugin is purely a spec. Since SQL is completely portable there is no need


Configuring the backend to get snowflake working
---------------------------------------------
-------------------------------------------------
1. Make sure to add "snowflake" in ``tasks.task-plugins.enabled-plugin`` in `enabled_plugins.yaml <https://github.com/flyteorg/flyte/blob/master/deployment/sandbox/flyte_generated.yaml#L2296>`_

2. Add snowflake JWT token to Flytepropeller. `here <https://docs.snowflake.com/en/developer-guide/sql-api/guide.html#using-key-pair-authentication>`_ to see more detail to setup snowflake JWT token.
Expand Down
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
"""
Snowflake Query
############
###############
This example shows how to use a Flyte SnowflakeTask to execute a query.
"""
Expand Down
8 changes: 4 additions & 4 deletions cookbook/integrations/gcp/bigquery/README.rst
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
BigQuery
====
========

Flyte backend can be connected with BigQuery service. Once enabled it can allow you to query a BigQuery table.
Flyte backend can be connected with BigQuery service. Once enabled, it can allow you to query a BigQuery table.
This section will provide how to use the BigQuery Plugin using flytekit python.

Installation
Expand All @@ -15,9 +15,9 @@ To use the flytekit bigquery plugin simply run the following:

No Need of a dockerfile
------------------------
This plugin is purely a spec. Since SQL is completely portable there is no need to build a Docker container.
This plugin is purely a spec. Since SQL is completely portable, there is no need to build a Docker container.


Configuring the backend to get bigquery working
---------------------------------------------
------------------------------------------------
- BigQuery plugins are `enabled in flytepropeller's config <https://docs.flyte.org/en/latest/deployment/plugin_setup/gcp/bigquery.html#deployment-plugin-setup-gcp-bigquery>`_
6 changes: 5 additions & 1 deletion cookbook/integrations/kubernetes/k8s_spark/README.rst
Original file line number Diff line number Diff line change
Expand Up @@ -61,7 +61,11 @@ Spark needs a special service account (with associated role and role bindings) t
If you use IAM for Service accounts or GCP Workload identity, you need to update the service account to include this.

You can use ``Flyte cluster resource manager`` to manage creating the spark service account per namespace.
For this, you need to add the cluster resource templates as shown `here <https://github.com/flyteorg/flyte/tree/376f61acc37d885d17aa6b4d003db502c4cb6bcf/kustomize/overlays/eks/flyte/config/clusterresource-templates>`__ (refer to the ``*spark*.yaml`` files).
For this, you need to add the cluster resource templates as shown `here <https://github.com/flyteorg/flyte/tree/376f61acc37d885d17aa6b4d003db502c4cb6bcf/kustomize/overlays/eks/flyte/config/clusterresource-templates>`__ (refer to the ``spark.yaml`` files).

.. note::

Refer to `this <https://github.com/GoogleCloudPlatform/spark-on-k8s-operator/blob/master/docs/gcp.md>`__ guide to use GCP instead of AWS.

Step 2: Environment Setup
-------------------------
Expand Down

0 comments on commit 983e404

Please sign in to comment.