Skip to content

Commit

Permalink
DOC add pipeline examples (#2240)
Browse files Browse the repository at this point in the history
* DOC add pipeline examples

* Add pipeline notebook to the example.rst file

* retrigger checks
  • Loading branch information
Iglesys347 authored Jun 23, 2022
1 parent bea7299 commit 23fd327
Show file tree
Hide file tree
Showing 2 changed files with 310 additions and 1 deletion.
3 changes: 2 additions & 1 deletion docs/examples.rst
Original file line number Diff line number Diff line change
Expand Up @@ -10,4 +10,5 @@ Examples
examples/asyncio_examples
examples/search_json_examples
examples/set_and_get_examples
examples/search_vector_similarity_examples
examples/search_vector_similarity_examples
examples/pipeline_examples
308 changes: 308 additions & 0 deletions docs/examples/pipeline_examples.ipynb
Original file line number Diff line number Diff line change
@@ -0,0 +1,308 @@
{
"cells": [
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# Pipeline examples\n",
"\n",
"This example show quickly how to use pipelines in `redis-py`."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Checking that Redis is running"
]
},
{
"cell_type": "code",
"execution_count": 1,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"True"
]
},
"execution_count": 1,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"import redis \n",
"\n",
"r = redis.Redis(decode_responses=True)\n",
"r.ping()"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Simple example"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Creating a pipeline instance"
]
},
{
"cell_type": "code",
"execution_count": 2,
"metadata": {},
"outputs": [],
"source": [
"pipe = r.pipeline()"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Adding commands to the pipeline"
]
},
{
"cell_type": "code",
"execution_count": 3,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"Pipeline<ConnectionPool<Connection<host=localhost,port=6379,db=0>>>"
]
},
"execution_count": 3,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"pipe.set(\"a\", \"a value\")\n",
"pipe.set(\"b\", \"b value\")\n",
"\n",
"pipe.get(\"a\")"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Executing the pipeline"
]
},
{
"cell_type": "code",
"execution_count": 4,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"[True, True, 'a value']"
]
},
"execution_count": 4,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"pipe.execute()"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"The responses of the three commands are stored in a list. In the above example, the two first boolean indicates that the the `set` commands were successfull and the last element of the list is the result of the `get(\"a\")` comand."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Chained call\n",
"\n",
"The same result as above can be obtained in one line of code by chaining the opperations."
]
},
{
"cell_type": "code",
"execution_count": 5,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"[True, True, 'a value']"
]
},
"execution_count": 5,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"pipe = r.pipeline()\n",
"pipe.set(\"a\", \"a value\").set(\"b\", \"b value\").get(\"a\").execute()"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Performance comparison\n",
"\n",
"Using pipelines can improve performance, for more informations, see [Redis documentation about pipelining](https://redis.io/docs/manual/pipelining/). Here is a simple comparison test of performance between basic and pipelined commands (we simply increment a value and measure the time taken by both method)."
]
},
{
"cell_type": "code",
"execution_count": 6,
"metadata": {},
"outputs": [],
"source": [
"from datetime import datetime\n",
"\n",
"incr_value = 100000"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Without pipeline"
]
},
{
"cell_type": "code",
"execution_count": 7,
"metadata": {},
"outputs": [],
"source": [
"r.set(\"incr_key\", \"0\")\n",
"\n",
"start = datetime.now()\n",
"\n",
"for _ in range(incr_value):\n",
" r.incr(\"incr_key\")\n",
"res_without_pipeline = r.get(\"incr_key\")\n",
"\n",
"time_without_pipeline = (datetime.now() - start).total_seconds()"
]
},
{
"cell_type": "code",
"execution_count": 8,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Without pipeline\n",
"================\n",
"Time taken: 21.759733\n",
"Increment value: 100000\n"
]
}
],
"source": [
"print(\"Without pipeline\")\n",
"print(\"================\")\n",
"print(\"Time taken: \", time_without_pipeline)\n",
"print(\"Increment value: \", res_without_pipeline)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### With pipeline"
]
},
{
"cell_type": "code",
"execution_count": 9,
"metadata": {},
"outputs": [],
"source": [
"r.set(\"incr_key\", \"0\")\n",
"\n",
"start = datetime.now()\n",
"\n",
"pipe = r.pipeline()\n",
"for _ in range(incr_value):\n",
" pipe.incr(\"incr_key\")\n",
"pipe.get(\"incr_key\")\n",
"res_with_pipeline = pipe.execute()[-1]\n",
"\n",
"time_with_pipeline = (datetime.now() - start).total_seconds()"
]
},
{
"cell_type": "code",
"execution_count": 10,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"With pipeline\n",
"=============\n",
"Time taken: 2.357863\n",
"Increment value: 100000\n"
]
}
],
"source": [
"print(\"With pipeline\")\n",
"print(\"=============\")\n",
"print(\"Time taken: \", time_with_pipeline)\n",
"print(\"Increment value: \", res_with_pipeline)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Using pipelines provides the same result in much less time."
]
}
],
"metadata": {
"interpreter": {
"hash": "84048e2f8e89effc8610b2fb270e4858ef00e9403d223856d62b05266db287ca"
},
"kernelspec": {
"display_name": "Python 3.9.2 ('.venv': venv)",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.9.2"
},
"orig_nbformat": 4
},
"nbformat": 4,
"nbformat_minor": 2
}

0 comments on commit 23fd327

Please sign in to comment.