Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

mypy: Add type hints to pglookout [BF-1560] #106

Open
wants to merge 24 commits into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
24 commits
Select commit Hold shift + click to select a range
0d4ea2c
mypy: `pglookout/logutil.py` [BF-1560]
Mar 16, 2023
238eb76
mypy: `pglookout/pgutil.py` [BF-1560]
Mar 16, 2023
3db5684
mypy: `pglookout/current-master.py` [BF-1560]
Mar 17, 2023
2ce13ab
mypy: `pglookout/statsd.py` [BF-1560]
Mar 17, 2023
25617d1
mypy: `pglookout/common.py` [BF-1560]
Mar 16, 2023
016d14d
mypy: `pglookout/config.py` [BF-1560]
Mar 17, 2023
bf067b3
mypy: `pglookout/common_types.py` [BF-1560]
Mar 23, 2023
a462bf9
mypy: `pglookout/webserver.py` [BF-1560]
Mar 17, 2023
9b4e13f
mypy: `pglookout/cluster_monitor.py` [BF-1560]
Mar 17, 2023
45d413b
mypy: `pglookout/pglookout.py` [BF-1560]
Mar 23, 2023
6ce993e
mypy: Refactor `check_cluster_state` [BF-1560]
Mar 24, 2023
0d438e9
mypy: `test/test_pglookout.py` [BF-1560]
Mar 27, 2023
1f47155
mypy: New method to normalize config [BF-1560]
Mar 27, 2023
5d4ac6e
mypy: `version.py` [BF-1560]
Mar 27, 2023
51bb85e
mypy: `setup.py` [BF-1560]
Mar 27, 2023
e9834d4
mypy: Remove python2 dependency [BF-1560]
Mar 27, 2023
a3ad6f8
mypy: `conftest.py` [BF-1560]
Mar 27, 2023
7434873
Bump `Makefile` version to `2.1.0` [BF-1560]
Mar 27, 2023
30bcd0d
mypy: Compatible with Python 3.9 logging [BF-1560]
Mar 27, 2023
e4e771a
pylint: Enable more messages [BF-1560]
Mar 27, 2023
56c01a8
pylint: Fix broad exception raised [BF-1560]
Mar 28, 2023
09a15b1
pylint: Dev requirements are now pinned [BF-1560]
Mar 28, 2023
9a0abae
make: Dev reqs are now in isolation [BF-1560]
Apr 4, 2023
dac0aad
Merge remote-tracking branch 'origin/main' into sgiffard/BF-1560_add_…
Apr 5, 2023
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@ long_ver = $(shell git describe --long 2>/dev/null || echo $(short_ver)-0-unknow
generated = pglookout/version.py

PYTHON ?= python3
PYTHON_SOURCE_DIRS = pglookout/ test/
PYTHON_SOURCE_DIRS = pglookout/ test/ stubs/

all: $(generated)
: 'try "make rpm" or "make deb" or "make test"'
Expand All @@ -17,7 +17,7 @@ unittest: $(generated)
$(PYTHON) -m pytest

mypy: $(generated)
$(PYTHON) -m mypy
MYPYPATH=stubs $(PYTHON) -m mypy

flake8: $(generated)
$(PYTHON) -m flake8 $(PYTHON_SOURCE_DIRS)
Expand Down
33 changes: 22 additions & 11 deletions pglookout/logutil.py
Original file line number Diff line number Diff line change
Expand Up @@ -5,39 +5,50 @@
Copyright (c) 2015 Ohmu Ltd
See LICENSE for details
"""
from __future__ import annotations
alanfranz marked this conversation as resolved.
Show resolved Hide resolved

from logging.handlers import SysLogHandler
from typing import Final, TYPE_CHECKING

import logging
import logging.handlers
import os

with_systemd: bool = False
try:
from systemd import daemon # pylint: disable=no-name-in-module

with_systemd = True
except ImportError:
daemon = None
if not TYPE_CHECKING:
Mulugruntz marked this conversation as resolved.
Show resolved Hide resolved
daemon = None

LOG_FORMAT = "%(asctime)s\t%(name)s\t%(threadName)s\t%(levelname)s\t%(message)s"
LOG_FORMAT_SHORT = "%(levelname)s\t%(message)s"
LOG_FORMAT_SYSLOG = "%(name)s %(threadName)s %(levelname)s: %(message)s"
LOG_FORMAT: Final[str] = "%(asctime)s\t%(name)s\t%(threadName)s\t%(levelname)s\t%(message)s"
LOG_FORMAT_SHORT: Final[str] = "%(levelname)s\t%(message)s"
LOG_FORMAT_SYSLOG: Final[str] = "%(name)s %(threadName)s %(levelname)s: %(message)s"


def set_syslog_handler(address, facility, logger):
syslog_handler = logging.handlers.SysLogHandler(address=address, facility=facility)
def set_syslog_handler(address: str, facility: str | int, logger: logging.Logger) -> SysLogHandler:
if isinstance(facility, str):
facility_id: int = SysLogHandler.facility_names.get(facility, SysLogHandler.LOG_LOCAL2)
Mulugruntz marked this conversation as resolved.
Show resolved Hide resolved
else:
facility_id = facility
Comment on lines +31 to +34
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A change that is not really a change. Our README mentions:

``syslog_facility`` (default ``"local2"``)

Determines syslog log facility. (requires syslog to be true as well)

syslog_handler = SysLogHandler(address=address, facility=facility_id)
logger.addHandler(syslog_handler)
formatter = logging.Formatter(LOG_FORMAT_SYSLOG)
syslog_handler.setFormatter(formatter)
return syslog_handler


def configure_logging(level=logging.DEBUG, short_log=False):
def configure_logging(level: int = logging.DEBUG, short_log: bool = False) -> None:
# Are we running under systemd?
if os.getenv("NOTIFY_SOCKET"):
logging.basicConfig(level=level, format=LOG_FORMAT_SYSLOG)
if not daemon:
if not with_systemd:
print("WARNING: Running under systemd but python-systemd not available, systemd won't see our notifications")
else:
logging.basicConfig(level=level, format=LOG_FORMAT_SHORT if short_log else LOG_FORMAT)


def notify_systemd(status):
if daemon:
def notify_systemd(status: str) -> None:
if with_systemd:
daemon.notify(status)
1 change: 0 additions & 1 deletion pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -34,7 +34,6 @@ exclude = [
'pglookout/cluster_monitor.py',
'pglookout/common.py',
'pglookout/current_master.py',
'pglookout/logutil.py',
'pglookout/pglookout.py',
'pglookout/pgutil.py',
'pglookout/statsd.py',
Expand Down
1 change: 1 addition & 0 deletions stubs/__init__.pyi
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
# Copyright (c) 2023 Aiven, Helsinki, Finland. https://aiven.io/
1 change: 1 addition & 0 deletions stubs/systemd/__init__.pyi
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
# Copyright (c) 2023 Aiven, Helsinki, Finland. https://aiven.io/
19 changes: 19 additions & 0 deletions stubs/systemd/daemon.pyi
Original file line number Diff line number Diff line change
@@ -0,0 +1,19 @@
# Copyright (c) 2023 Aiven, Helsinki, Finland. https://aiven.io/

from __future__ import annotations

from typing import AnyStr, IO

def notify(
status: str,
unset_environment: bool = False,
pid: int = 0,
fds: IO[AnyStr] | None = None,
) -> bool:
"""
notify(status, unset_environment=False, pid=0, fds=None) -> bool

Send a message to the init system about a status change.
Wraps sd_notify(3).
"""
...
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Note this is a stubs. The docstring comes from the systemd library.