Skip to content

Commit

Permalink
update SQLAlchemy docs (#416)
Browse files Browse the repository at this point in the history
* update docs -- SQLAlchemy

Signed-off-by: Samhita Alla <[email protected]>

* resolve conflicts

Signed-off-by: Samhita Alla <[email protected]>
  • Loading branch information
samhita-alla authored Sep 20, 2021
1 parent feb61c8 commit a48c81b
Show file tree
Hide file tree
Showing 3 changed files with 12 additions and 61 deletions.
9 changes: 4 additions & 5 deletions cookbook/integrations/flytekit_plugins/sql/requirements.txt
Original file line number Diff line number Diff line change
Expand Up @@ -2,13 +2,13 @@
# This file is autogenerated by pip-compile with python 3.8
# To update, run:
#
# /Applications/Xcode.app/Contents/Developer/usr/bin/make requirements.txt
# /Library/Developer/CommandLineTools/usr/bin/make requirements.txt
#
attrs==21.2.0
# via scantree
certifi==2021.5.30
# via requests
charset-normalizer==2.0.5
charset-normalizer==2.0.6
# via requests
click==7.1.2
# via flytekit
Expand Down Expand Up @@ -36,7 +36,7 @@ flytekit==0.22.2
# via
# -r ../../../common/requirements-common.in
# flytekitplugins-sqlalchemy
flytekitplugins-sqlalchemy==0.22.2
flytekitplugins-sqlalchemy==0.22.3
# via -r requirements.in
greenlet==1.1.1
# via sqlalchemy
Expand Down Expand Up @@ -76,7 +76,7 @@ pathspec==0.9.0
# via scantree
pillow==8.3.2
# via matplotlib
protobuf==3.17.3
protobuf==3.18.0
# via
# flyteidl
# flytekit
Expand Down Expand Up @@ -119,7 +119,6 @@ six==1.16.0
# cycler
# flytekit
# grpcio
# protobuf
# python-dateutil
# responses
# scantree
Expand Down
45 changes: 0 additions & 45 deletions cookbook/integrations/flytekit_plugins/sql/sql-alchemy-remote.py

This file was deleted.

19 changes: 8 additions & 11 deletions cookbook/integrations/flytekit_plugins/sql/sql-alchemy.py
Original file line number Diff line number Diff line change
Expand Up @@ -4,17 +4,13 @@
SQLAlchemy is the Python SQL toolkit and Object Relational Mapper that gives application developers the full power and flexibility of SQL.
That being said, Flyte provides an easy-to-use interface to utilize SQLAlchemy to connect to various SQL Databases.
Flyte provides an easy-to-use interface to utilize SQLAlchemy to connect to various SQL Databases. In this example, we use a Postgres DB to understand how SQLAlchemy can be used within Flyte.
In this example, we'll use a Postgres DB to understand how you can use SQLAlchemy with Flyte.
This task will run with a pre-built container, and thus users needn't build one.
You can simply implement the task, then register and execute it immediately.
Install the following packages before running this example (works locally only):
* `Postgres <https://www.postgresql.org/download/>`__
* ``pip install flytekitplugins-sqlalchemy``
* ``pip install psycopg2``
.. note::
This example works locally only. For remote, you need to input a valid URI.
"""

# %%
Expand Down Expand Up @@ -52,7 +48,7 @@ def create_db(


# %%
# We define a ``run_query`` function to create a table and insert data entries into the table.
# We define a ``run_query`` function to create a table and insert data entries into it.
def run_query(
sql_query: str,
conn: psycopg2.extensions.connection,
Expand All @@ -73,7 +69,7 @@ def run_query(


# %%
# We define a helper function to call the required functions.
# We define a helper function.
def pg_server() -> str:
conn_info = {"database": "flights", "user": "postgres", "password": "postgres"}

Expand Down Expand Up @@ -131,7 +127,7 @@ def get_length(df: pandas.DataFrame) -> int:
and duration <= {{ .inputs.upper_duration_cap }}
""",
inputs=kwtypes(lower_duration_cap=int, upper_duration_cap=int),
task_config=SQLAlchemyConfig(uri=local_pg_server),
task_config=SQLAlchemyConfig(uri="postgresql://localhost/flights"),
)


Expand All @@ -147,4 +143,5 @@ def my_wf(lower_duration_cap: int, upper_duration_cap: int) -> int:
if __name__ == "__main__":
print(f"Starting pg server at {pg_server()}")
print(f"Running {__file__} main...")
print(my_wf(lower_duration_cap=600, upper_duration_cap=800))
print(f"Starting Postgres DB at {pg_server()}")
print(my_wf(lower_duration_cap=300, upper_duration_cap=800))

0 comments on commit a48c81b

Please sign in to comment.