-
Notifications
You must be signed in to change notification settings - Fork 2.5k
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
* DOC add pipeline examples * Add pipeline notebook to the example.rst file * retrigger checks
- Loading branch information
1 parent
bea7299
commit 23fd327
Showing
2 changed files
with
310 additions
and
1 deletion.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,308 @@ | ||
{ | ||
"cells": [ | ||
{ | ||
"cell_type": "markdown", | ||
"metadata": {}, | ||
"source": [ | ||
"# Pipeline examples\n", | ||
"\n", | ||
"This example show quickly how to use pipelines in `redis-py`." | ||
] | ||
}, | ||
{ | ||
"cell_type": "markdown", | ||
"metadata": {}, | ||
"source": [ | ||
"## Checking that Redis is running" | ||
] | ||
}, | ||
{ | ||
"cell_type": "code", | ||
"execution_count": 1, | ||
"metadata": {}, | ||
"outputs": [ | ||
{ | ||
"data": { | ||
"text/plain": [ | ||
"True" | ||
] | ||
}, | ||
"execution_count": 1, | ||
"metadata": {}, | ||
"output_type": "execute_result" | ||
} | ||
], | ||
"source": [ | ||
"import redis \n", | ||
"\n", | ||
"r = redis.Redis(decode_responses=True)\n", | ||
"r.ping()" | ||
] | ||
}, | ||
{ | ||
"cell_type": "markdown", | ||
"metadata": {}, | ||
"source": [ | ||
"## Simple example" | ||
] | ||
}, | ||
{ | ||
"cell_type": "markdown", | ||
"metadata": {}, | ||
"source": [ | ||
"### Creating a pipeline instance" | ||
] | ||
}, | ||
{ | ||
"cell_type": "code", | ||
"execution_count": 2, | ||
"metadata": {}, | ||
"outputs": [], | ||
"source": [ | ||
"pipe = r.pipeline()" | ||
] | ||
}, | ||
{ | ||
"cell_type": "markdown", | ||
"metadata": {}, | ||
"source": [ | ||
"### Adding commands to the pipeline" | ||
] | ||
}, | ||
{ | ||
"cell_type": "code", | ||
"execution_count": 3, | ||
"metadata": {}, | ||
"outputs": [ | ||
{ | ||
"data": { | ||
"text/plain": [ | ||
"Pipeline<ConnectionPool<Connection<host=localhost,port=6379,db=0>>>" | ||
] | ||
}, | ||
"execution_count": 3, | ||
"metadata": {}, | ||
"output_type": "execute_result" | ||
} | ||
], | ||
"source": [ | ||
"pipe.set(\"a\", \"a value\")\n", | ||
"pipe.set(\"b\", \"b value\")\n", | ||
"\n", | ||
"pipe.get(\"a\")" | ||
] | ||
}, | ||
{ | ||
"cell_type": "markdown", | ||
"metadata": {}, | ||
"source": [ | ||
"### Executing the pipeline" | ||
] | ||
}, | ||
{ | ||
"cell_type": "code", | ||
"execution_count": 4, | ||
"metadata": {}, | ||
"outputs": [ | ||
{ | ||
"data": { | ||
"text/plain": [ | ||
"[True, True, 'a value']" | ||
] | ||
}, | ||
"execution_count": 4, | ||
"metadata": {}, | ||
"output_type": "execute_result" | ||
} | ||
], | ||
"source": [ | ||
"pipe.execute()" | ||
] | ||
}, | ||
{ | ||
"cell_type": "markdown", | ||
"metadata": {}, | ||
"source": [ | ||
"The responses of the three commands are stored in a list. In the above example, the two first boolean indicates that the the `set` commands were successfull and the last element of the list is the result of the `get(\"a\")` comand." | ||
] | ||
}, | ||
{ | ||
"cell_type": "markdown", | ||
"metadata": {}, | ||
"source": [ | ||
"## Chained call\n", | ||
"\n", | ||
"The same result as above can be obtained in one line of code by chaining the opperations." | ||
] | ||
}, | ||
{ | ||
"cell_type": "code", | ||
"execution_count": 5, | ||
"metadata": {}, | ||
"outputs": [ | ||
{ | ||
"data": { | ||
"text/plain": [ | ||
"[True, True, 'a value']" | ||
] | ||
}, | ||
"execution_count": 5, | ||
"metadata": {}, | ||
"output_type": "execute_result" | ||
} | ||
], | ||
"source": [ | ||
"pipe = r.pipeline()\n", | ||
"pipe.set(\"a\", \"a value\").set(\"b\", \"b value\").get(\"a\").execute()" | ||
] | ||
}, | ||
{ | ||
"cell_type": "markdown", | ||
"metadata": {}, | ||
"source": [ | ||
"## Performance comparison\n", | ||
"\n", | ||
"Using pipelines can improve performance, for more informations, see [Redis documentation about pipelining](https://redis.io/docs/manual/pipelining/). Here is a simple comparison test of performance between basic and pipelined commands (we simply increment a value and measure the time taken by both method)." | ||
] | ||
}, | ||
{ | ||
"cell_type": "code", | ||
"execution_count": 6, | ||
"metadata": {}, | ||
"outputs": [], | ||
"source": [ | ||
"from datetime import datetime\n", | ||
"\n", | ||
"incr_value = 100000" | ||
] | ||
}, | ||
{ | ||
"cell_type": "markdown", | ||
"metadata": {}, | ||
"source": [ | ||
"### Without pipeline" | ||
] | ||
}, | ||
{ | ||
"cell_type": "code", | ||
"execution_count": 7, | ||
"metadata": {}, | ||
"outputs": [], | ||
"source": [ | ||
"r.set(\"incr_key\", \"0\")\n", | ||
"\n", | ||
"start = datetime.now()\n", | ||
"\n", | ||
"for _ in range(incr_value):\n", | ||
" r.incr(\"incr_key\")\n", | ||
"res_without_pipeline = r.get(\"incr_key\")\n", | ||
"\n", | ||
"time_without_pipeline = (datetime.now() - start).total_seconds()" | ||
] | ||
}, | ||
{ | ||
"cell_type": "code", | ||
"execution_count": 8, | ||
"metadata": {}, | ||
"outputs": [ | ||
{ | ||
"name": "stdout", | ||
"output_type": "stream", | ||
"text": [ | ||
"Without pipeline\n", | ||
"================\n", | ||
"Time taken: 21.759733\n", | ||
"Increment value: 100000\n" | ||
] | ||
} | ||
], | ||
"source": [ | ||
"print(\"Without pipeline\")\n", | ||
"print(\"================\")\n", | ||
"print(\"Time taken: \", time_without_pipeline)\n", | ||
"print(\"Increment value: \", res_without_pipeline)" | ||
] | ||
}, | ||
{ | ||
"cell_type": "markdown", | ||
"metadata": {}, | ||
"source": [ | ||
"### With pipeline" | ||
] | ||
}, | ||
{ | ||
"cell_type": "code", | ||
"execution_count": 9, | ||
"metadata": {}, | ||
"outputs": [], | ||
"source": [ | ||
"r.set(\"incr_key\", \"0\")\n", | ||
"\n", | ||
"start = datetime.now()\n", | ||
"\n", | ||
"pipe = r.pipeline()\n", | ||
"for _ in range(incr_value):\n", | ||
" pipe.incr(\"incr_key\")\n", | ||
"pipe.get(\"incr_key\")\n", | ||
"res_with_pipeline = pipe.execute()[-1]\n", | ||
"\n", | ||
"time_with_pipeline = (datetime.now() - start).total_seconds()" | ||
] | ||
}, | ||
{ | ||
"cell_type": "code", | ||
"execution_count": 10, | ||
"metadata": {}, | ||
"outputs": [ | ||
{ | ||
"name": "stdout", | ||
"output_type": "stream", | ||
"text": [ | ||
"With pipeline\n", | ||
"=============\n", | ||
"Time taken: 2.357863\n", | ||
"Increment value: 100000\n" | ||
] | ||
} | ||
], | ||
"source": [ | ||
"print(\"With pipeline\")\n", | ||
"print(\"=============\")\n", | ||
"print(\"Time taken: \", time_with_pipeline)\n", | ||
"print(\"Increment value: \", res_with_pipeline)" | ||
] | ||
}, | ||
{ | ||
"cell_type": "markdown", | ||
"metadata": {}, | ||
"source": [ | ||
"Using pipelines provides the same result in much less time." | ||
] | ||
} | ||
], | ||
"metadata": { | ||
"interpreter": { | ||
"hash": "84048e2f8e89effc8610b2fb270e4858ef00e9403d223856d62b05266db287ca" | ||
}, | ||
"kernelspec": { | ||
"display_name": "Python 3.9.2 ('.venv': venv)", | ||
"language": "python", | ||
"name": "python3" | ||
}, | ||
"language_info": { | ||
"codemirror_mode": { | ||
"name": "ipython", | ||
"version": 3 | ||
}, | ||
"file_extension": ".py", | ||
"mimetype": "text/x-python", | ||
"name": "python", | ||
"nbconvert_exporter": "python", | ||
"pygments_lexer": "ipython3", | ||
"version": "3.9.2" | ||
}, | ||
"orig_nbformat": 4 | ||
}, | ||
"nbformat": 4, | ||
"nbformat_minor": 2 | ||
} |