01 - Producer.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 🦀  Time to play with Apache Kafka®"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The first step is to install the required libraries, in our case `kafka-python` (and `pandas` to support visualization)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "%%bash\n",
    "pip install kafka-python\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "---\n",
    "\n",
    "## Kafka Producer\n",
    "\n",
    "Now we can create a Kafka Producer\n",
    "\n",
    "![Kafka Producer](images/producing.png)\n",
    "\n",
    "we need to have an Apache Kafka instance already available, if not you can easily create one on [Aiven.io](https://aiven.io/)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from kafka import KafkaProducer\n",
    "import json\n",
    "from config.kafka_config import *\n",
    "\n",
    "producer = KafkaProducer(\n",
    " bootstrap_servers=hostname+\":\"+str(port),\n",
    " security_protocol=\"SSL\",\n",
    " ssl_cafile=cert_folder+\"/ca.pem\",\n",
    " ssl_certfile=cert_folder+\"/service.cert\",\n",
    " ssl_keyfile=cert_folder+\"/service.key\",\n",
    " value_serializer=lambda v: json.dumps(v).encode('ascii'),\n",
    " key_serializer=lambda v: json.dumps(v).encode('ascii')\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Once the Producer is created, we can produce a couple events"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "producer.send(\n",
    " topic_name,\n",
    " key={\"id\":1},\n",
    " value={\"name\":\"👨 Francesco\", \"pizza\":\"Margherita 🍕\"}\n",
    ")\n",
    "\n",
    "producer.flush()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now let's produce one more event"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "producer.send(\n",
    " topic_name,\n",
    " key={\"id\":2},\n",
    " value={\"name\":\"👩 Adele\", \"pizza\":\"Hawaii 🍕+🍍+🥓\"}\n",
    ")\n",
    "\n",
    "producer.send(\n",
    " topic_name,\n",
    " key={\"id\":3},\n",
    " value={\"name\":\"👦 Mark\", \"pizza\":\"Choccolate 🍕+🍫\"}\n",
    ")\n",
    "\n",
    "producer.flush()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now let's produce another event <-- New Consumer Group"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "producer.send(\n",
    " topic_name,\n",
    " key={\"id\":4},\n",
    " value={\"name\":\"👨 Dan\", \"pizza\":\"Fries 🍕+🍟\"}\n",
    ")\n",
    "producer.flush()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.9.15"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}