added a new prom exporter cog

README.md

@@ -42,7 +42,9 @@ Cogs for the [RED](https://github.com/Cog-Creators/Red-DiscordBot/)-based [Homel
     - [Roleinfo](#roleinfo)
     - [Sentry](#sentry)
     - [Tags](#tags)
+    - [promExport](#promexport)
     - [Timeout](#timeout)
+    - [Topic](#topic)
     - [Verify](#verify)
     - [xkcd](#xkcd)
   - [License](#license)
@@ -77,6 +79,7 @@ A massive thank you to all who've helped out with this project ❤️
 ## Cog Summaries
 
 - **[AutoReact](#autoreact):** React to specific phrases with one or more emotes.
+- **[PromExport](#promExport):** Exposes a HTTP endpoint for exporting guild metrics in Prometheus format
 - **[AutoReply](#autoreply):** Automatically replies to messages that match a trigger phrase.
 - **[Convert](#convert):** Converts any unit to any another unit.
 - **[Custom-msg](#custom-msg):** Allows moderators to send/edit messages from the bot.
@@ -199,7 +202,7 @@ This cog converts a string of letters/numbers into large emote letters ("regiona
 
 This cog generates messages based on markov chains generated per-user.
 
-User messages will never be analysed unless the user explicitly opts in.  
+User messages will never be analysed unless the user explicitly opts in.
 It must also be enabled per-channel: `[p]markov channelenable`
 
 #### User commands
@@ -313,12 +316,12 @@ Configure Sentry DSN using `[p]set api sentry dsn,https://[email protected]/9`, then
 
 ### Tags
 
-Allow user-generated stored messages to be triggered upon configurable tags. Aliases can also be created to make 
-accessing information even quicker. Users can transfer tag ownership between themselves and even claim ownership of 
-abandoned tags. Ever use of a tag or alias is tracked. Same with ownership transfers of any kind. Statistics and 
-searching have not yet been implemented. 
+Allow user-generated stored messages to be triggered upon configurable tags. Aliases can also be created to make
+accessing information even quicker. Users can transfer tag ownership between themselves and even claim ownership of
+abandoned tags. Ever use of a tag or alias is tracked. Same with ownership transfers of any kind. Statistics and
+searching have not yet been implemented.
 
-- `[p]tag <tag>` - Triggers the specified tag. 
+- `[p]tag <tag>` - Triggers the specified tag.
 - `[p]tag search <query>` - Searches for a tag or alias (WIP).
 - `[p]tag create <tag> <content>` - Creates the specified tag which will reply with the provided content when triggered.
 - `[p]tag stats [user]` - Provides general stats about the tag system, or if a user is provided, about that user (WIP).
@@ -330,12 +333,24 @@ searching have not yet been implemented.
 - `[p]tag alias create <alias> <tag>` - Creates the specified alias to the specified tag.
 - `[p]tag alias delete <alias>` - Deletes the specified alias.
 
+### promExport
+
+This cog exposes a http server for serving metrics in prom format
+
+- `[p]prom_exporter set_port <port>` - Sets the port of the http server
+- `[p]prom_exporter set_address <address>` - Sets the address of the http server
+- `[p]prom_exporter set_poll_interval <interval>` - Sets the time between metrics updates
+- `[p]prom_exporter get_config ` - Shows the current running config
+
+
+
+
 ### Timeout
 
 Manage the timeout status of users.
 
-Run the command to add a user to timeout, run it again to remove them. Append a reason if you wish: `[p]timeout @someUser said a bad thing`  
-If the user is not in timeout, they are added. If they are in timeout, they are removed.  
+Run the command to add a user to timeout, run it again to remove them. Append a reason if you wish: `[p]timeout @someUser said a bad thing`
+If the user is not in timeout, they are added. If they are in timeout, they are removed.
 All of the member's roles will be stripped when they are added to timeout, and re-added when they are removed.
 
 - `[p]timeout <user> [reason]` - Add/remove a user from timeout, optionally specifying a reason.
@@ -397,3 +412,4 @@ If you use [VSCode](https://code.visualstudio.com/) you can use the tasks integr
 When suggesting changes, please [open an issue](https://github.com/rHomelab/LabBot-Cogs/issues/new/choose) so it can be reviewed by the team who can then suggest how and if the idea is to be implemented.
 
 When submitting changes, please [create a pull request](https://github.com/rHomelab/LabBot-Cogs/compare) targeting the main branch.
+

prom_export/__init__.py

@@ -0,0 +1,12 @@
+import time
+
+from redbot.core.bot import Red
+
+from .main import PromExporter
+
+
+async def setup(bot: Red):
+    prom = PromExporter(bot)
+    await prom.init()
+    await bot.add_cog(prom)
+

prom_export/info.json

@@ -0,0 +1,15 @@
+{
+    "author": [
+        "ripple (frzen chip lova)"
+    ],
+    "description": "Exposes a HTTP endpoint for exporting guild metrics in Prometheus format",
+    "short": "Exports guild metrics in Prometheus format",
+    "name": "Prom Exporter",
+
+    "tags": [
+        "metrics",
+        "prom"
+
+    ],
+    "min_bot_version": "3.5.1"
+}

prom_export/main.py

@@ -0,0 +1,120 @@
+import asyncio
+import logging
+import os
+import time
+
+import discord
+from redbot.core import Config, checks, commands
+from redbot.core.bot import Red
+
+from .prom_server import PrometheusMetricsServer, promServer
+from .stats import Poller, statApi
+
+logger = logging.getLogger("red.rhomelab.prom")
+logger.setLevel(logging.DEBUG)
+
+class PromExporter(commands.Cog):
+    """commands for managing the prom exporter"""
+
+    def __init__(self, bot: Red):
+        self.bot = bot
+        self.logger = logging.getLogger("red.rhomelab.prom")
+        self.logger.info("initalising")
+        self.address = "0.0.0.0"
+        self.port = 9000
+        self.poll_frequency = 1
+
+        self.config = Config.get_conf(self, identifier=19283750192891838)
+
+        default_global= {
+            "address": "0.0.0.0",
+            "port": 9900,
+            "poll_interval": 1
+        }
+        self.config.register_global(**default_global)
+
+
+    async def init(self):
+        self.address = await self.config.address()
+        self.port = await self.config.port()
+        self.poll_frequency = await self.config.poll_interval()
+        self.start()
+
+    @staticmethod
+    def create_server(address: str, port: int):
+        return promServer(address, port)
+
+    @staticmethod
+    def create_stat_api(prefix: str, poll_frequency: float, bot: Red, server: PrometheusMetricsServer) -> statApi:
+        return Poller(prefix, poll_frequency, bot, server)
+
+    @commands.group()
+    async def prom_exporter(self, ctx: commands.Context):
+        """New users must `enable` and say some words before using `generate`"""
+
+    @checks.is_owner()
+    @prom_exporter.command()
+    async def set_port(self, ctx: commands.Context, port: int):
+        """sets the port the prometheus exporter should listen on"""
+        self.logger.info(f"changing port to {port}")
+        self.port = port
+        await self.config.port.set(port)
+        self.reload()
+        await ctx.tick()
+
+
+    @checks.is_owner()
+    @prom_exporter.command()
+    async def set_address(self, ctx: commands.Context, address: str):
+        """sets the address the prometheus exporter should listen on"""
+
+        self.logger.info(f"changing address to {address}")
+
+        self.address = address
+        await self.config.address.set(address)
+        self.reload()
+        await ctx.tick()
+
+
+    @checks.is_owner()
+    @prom_exporter.command()
+    async def set_poll_interval(self, ctx: commands.Context, poll_interval: float):
+        """sets the poll interval to update the endpoint metrics"""
+
+        self.logger.info(f"changing poll interval to {poll_interval}")
+        self.poll_frequency = poll_interval
+        await self.config.poll_interval.set(poll_interval)
+        self.reload()
+        await ctx.tick()
+
+    @checks.is_owner()
+    @prom_exporter.command()
+    async def show_config(self, ctx: commands.Context):
+        """shows the current running config"""
+        addr = await self.config.address()
+        port = await self.config.port()
+        poll_interval = await self.config.poll_interval()
+
+        await ctx.send(f"{self.address}:{self.port}:{self.poll_frequency}")
+
+    def start(self):
+        self.prom_server = self.create_server(self.address, self.port)
+        self.stat_api = self.create_stat_api("discord_metrics", self.poll_frequency, self.bot, self.prom_server)
+
+        self.prom_server.serve()
+        self.stat_api.start()
+
+    def stop(self):
+        self.prom_server.stop()
+        self.stat_api.stop()
+        self.logger.info("stopped server process")
+
+    def reload(self):
+        self.logger.info("reloading")
+        self.stop()
+        self.start()
+        self.logger.info("reloading complete")
+
+    def cog_unload(self):
+        self.stop()
+        self.logger.info("cog unloading")

prom_export/prom_server.py

@@ -0,0 +1,73 @@
+import logging
+import socket
+import threading
+from functools import partial
+from socketserver import ThreadingMixIn
+from typing import Protocol
+from wsgiref.simple_server import WSGIRequestHandler, WSGIServer, make_server
+
+from prometheus_client import CollectorRegistry, make_wsgi_app
+
+from .stats import statApi
+
+
+class _SilentHandler(WSGIRequestHandler):
+    """WSGI handler that does not log requests."""
+
+    def log_message(self, format, *args):
+        """Log nothing."""
+
+class MetricsServer(Protocol):
+    def serve(self) -> None:
+        ...
+
+    def stop(self) -> None:
+        ...
+
+class PrometheusMetricsServer(MetricsServer, Protocol):
+    @property
+    def registry(self) -> CollectorRegistry:
+        ...
+
+class promServer(PrometheusMetricsServer):
+    def __init__(self, addr: str, port: int):
+        self.addr = addr
+        self.port = port
+
+        self.server_thread = None
+        self.server = None
+        self.logger = logging.getLogger("red.rhomelab.prom.server")
+        self._registry = self._create_registry()
+
+    def _create_registry(self) -> CollectorRegistry:
+        return CollectorRegistry()
+
+    def _get_best_family(self):
+        """Automatically select address family depending on address"""
+        infos = socket.getaddrinfo(self.addr, self.port)
+        family, _, _, _, sockaddr = next(iter(infos))
+        return family, sockaddr[0]
+
+    @property
+    def registry(self) -> CollectorRegistry:
+        return self._registry
+
+    def serve(self) -> None:
+        """Starts a WSGI server for prometheus metrics as a daemon thread."""
+
+        app = make_wsgi_app(self._registry)
+        self.logger.info(f"starting server on {self.addr}:{self.port}")
+        self.server = make_server(self.addr, self.port, app, handler_class=_SilentHandler)
+        self.server_thread = threading.Thread(target=partial(self.server.serve_forever, 0.5))
+        self.server_thread.daemon = True
+        self.server_thread.start()
+
+    def stop(self) -> None:
+        if not self.server_thread is None and not self.server is None:
+            self.logger.info("shutting down prom server")
+            self.server.shutdown()
+            self.server.server_close()
+            self.server_thread.join()
+
+        else:
+            self.logger.info("prom server not running, not stopping")