From 036eebb0ee0907629793af75d24b6282a8069435 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Alejandro=20Avil=C3=A9s?= Date: Fri, 10 Nov 2023 02:15:30 +0100 Subject: [PATCH] Add patching guide --- README.md | 4 +- doc/README.md | 89 ++++++++++++++++++++++++++++++++- doc/classes.md | 121 +++++++++++++++++++++++++++++++++++++++++++++ doc/enums.md | 57 ++++++++++++++++++++++ doc/forms.md | 120 +++++++++++++++++++++++++++++++++++++++++++++ doc/models.md | 130 +++++++++++++++++++++++++++++++++++++++++++++++++ 6 files changed, 517 insertions(+), 4 deletions(-) create mode 100644 doc/classes.md create mode 100644 doc/enums.md create mode 100644 doc/forms.md create mode 100644 doc/models.md diff --git a/README.md b/README.md index be3c3f3..07aa1f0 100644 --- a/README.md +++ b/README.md @@ -77,7 +77,7 @@ Runtime patching is a powerful and flexible strategy but it will lead to code th ## Development -In order to develop `indico-patcher`, you will need to install the project and its dependencies in a virtualenv. This guide assumes that you have the following tools installed and available in your path: +In order to develop `indico-patcher`, install the project and its dependencies in a virtualenv. This guide assumes that you have the following tools installed and available in your path: - [`git`](https://git-scm.com/) (available in most systems) - [`make`](https://www.gnu.org/software/make/) (available in most systems) @@ -91,7 +91,7 @@ git clone https://github.com/unconventionaldotdev/indico-patcher cd indico-patcher ``` -Before creating the virtualenv, you probably want to be using the same version of Python that the development of the project is targeting. This is the first version specified in the `.python-version` file and you can install it with `pyenv`: +Before creating the virtualenv, make sure to be using the same version of Python that the development of the project is targeting. This is the first version specified in the `.python-version` file and you can install it with `pyenv`: ```sh pyenv install diff --git a/doc/README.md b/doc/README.md index 63db714..16ad56f 100644 --- a/doc/README.md +++ b/doc/README.md @@ -1,3 +1,88 @@ -# How to use Indico Patcher +# Indico Patching Guide - +This guide describes how to use `indico-patcher` for multiple use cases as well as considerations to keep in mind when patching Indico. In this page, you can read general information applicable in all the use cases described in the rest of the guide. + +- [Terminology](#terminology) +- [Usage](#usage) +- [FAQs](#faqs) +- [Keep in mind](#keep-in-mind) + +The guide covers in detail the following use cases in dedicated pages: + +1. [Patching Indico classes](./classes.md) +2. [Patching SQLAlchemy models](./models.md) +3. [Patching WTForms forms](./forms.md) +4. [Patching Enums](./enums.md) + +## Terminology + +### Patch class + +A class that is decorated with the `@patch` decorator. It is uniquely used to patch an existing class in Indico and never to be imported and used directly. + +### Original class + +The Indico class that is being patched by a patch class, passed as argument to the `@patch()` decorator. + +### Patched class + +The class that results from the application of a patch class to an original class. + +### Patch + +A collection of attributes, methods and properties defined in a patch class that are injected into the original class. A patch is applied when the patch class is imported. + +## Usage + +Import the `patch` decorator from the `indico_patcher` module: + +```python +from indico_patcher import patch +``` + +Import also the original class to be patched: + +```python +from indico.modules.users.models.users import User +from indico.modules.users.models.user_titles import UserTitle +``` + +Define a patch class and decorate it with the `@patch()` decorator. The original class is passed as argument to the decorator. Since the patch class is not meant to be used directly, it is recommended to prefix its name with an underscore: + +```python +@patch(User) +class _User: + ... +``` + +This differs a bit in the case of enums, as the patch class must inherit from `Enum` or a subclass of it: + +```python +@patch(UserTitle) +class _UserTitle(RichIntEnum): + ... +``` + +Once the patch class is imported in the plugin, the original class will be patched. New members can be accessed as if they had been defined in the original class. All calls to existing members will be redirected to the ones defined in the patch class. For more specific usage details, please refer to the different pages of the guide. + +## FAQs + +### When are patches applied? + +The patches are applied when the patch classes are imported. This means that you need to import the patch classes in your plugin's `__init__.py` file or in any other file that is imported by it. In some cases, like having a `patches.py` file with only patch classes, your linter may complain that the import is unused. You can safely silence this warning. + +### Can the same class be patched multiple times? + +Yes, this is possible and it is useful or unavoidable in some cases. For instance, you may want to patch the same class in two different modules of your plugin. Or you may enable two different plugins that patch the same class. In both cases, the patches will be applied in the order in which the patch classes are imported. This means that if multiple patches are overriding the same class member, the last one will be applied. + +### What are some built-in tools to avoid patching Indico? + +Indico provides many signals that can be used to extend its functionality without patching it. You can find a list of all the available signals in [`indico/core/signals`](https://github.com/indico/indico/tree/v3.2.8/indico/core/signals). A particularly useful one is [`interceptable_function`](https://github.com/indico/indico/blob/v3.2.8/indico/core/signals/plugin.py#L121). You may also want to check [Flask signals](https://flask.palletsprojects.com/en/2.0.x/api/#signals) and [SQLAlchemy event hooks](https://docs.sqlalchemy.org/en/14/core/event.html). + +## Keep in mind + +> [!WARNING] +> Remember! With great power comes great responsibility. Patch Indico as little as possible and only when it is absolutely necessary. Always try to find a way to achieve what you want without patching Indico. + +> [!NOTE] +> The code examples and links to APIs in this guide make reference to Indico classes and dependencies as they are defined in `v3.2.8` of the codebase. Some class names, class locations and APIs may differ if you are using a different version of Indico. diff --git a/doc/classes.md b/doc/classes.md new file mode 100644 index 0000000..577799d --- /dev/null +++ b/doc/classes.md @@ -0,0 +1,121 @@ +# Patching Indico classes + +This page of the guide explains the general mechanism to add and override attributes, methods and properties in Indico classes. For more specific use cases, like patching [SQLAlchemy models](./models.md) or [WTForms forms](./forms.md), please refer to their respective pages of the guide. + +- [Add and override attributes](#add-and-override-attributes) +- [Add and override methods](#add-and-override-methods) +- [Add and override properties](#add-and-override-properties) + +## Add and override attributes + +```python +@patch(RHUserBlock) +class _RHUserBlock: + # Adds new attribute + error_message = L_('Action not allowed on this user.') +``` + +Add a new attribute to the original class by simply defining it in the patch class. In this example, an `error_message` attribute is added to `RHUserBlock`. + +```python +@patch(LocalRegistrationHandler) +class _LocalRegistrationHandler: + # Overrides existing attribute + form = CustomLocalRegistrationForm +``` + +Override an existing attribute in the original class by assigning a new value to it in the patch class. In this example, a plugin-defined form is assigned to be used by `LocalRegistrationHandler` instead of the original `LocalRegistrationForm`. + +## Add and override methods + +```python +@patch(RHUserBlock) +class _RHUserBlock: + # Adds new method + def _can_block_user(self): + return not self.user.merged_into_user +``` + +Add a new method to the original class by defining it in the patch class. For instance, a `_can_block_user` method is added to `RHUserBlock`. + +```python +@patch(RHUserBlock) +class _RHUserBlock: + # Overrides existing method + def _process_PUT(self): + if self._can_block_user(): + raise Forbidden(self.error_message) + return super()._process_PUT() + + # Overrides existing method + def _process_DELETE(self): + if self._can_block_user(): + raise Forbidden(self.error_message) + return super()._process_PUT() +``` + +Override an existing method in the original class by defining a method with the same name in the patch class. You can intercept calls to the original method and alter their result with `super()`. + +In this example, the PUT and DELETE process methods of the class `RHUserBlock` are intercepted to check if the user can be blocked with the newly added `_can_block_user()`. If not, an exception is raised with `error_message`. Otherwise, the original method is called. + +```python +@patch(User) +class _User: + # Adds new classmethod + @classmethod + def get_academic_users(cls): + return cls.query.filter(cls.title.in_({UserTitle.dr, UserTitle.prof})) + + # Overrides existing staticmethod + @staticmethod + def get_system_user(): + system_user = super().get_system_user() + logger.info('System user was retrieved.') + return system_user +``` + +Apply the same logic to add and override `@staticmethod`s and `@classmethod`s. + +> [!IMPORTANT] +> Overriding a method in the original class is fragile and can break in future versions of Indico if the original method is changed. A more reliable way to override a method in the original class is to intercept calls via the [`interceptable_function`](https://github.com/indico/indico/blob/v3.2.8/indico/core/signals/plugin.py#L121) signal. + +## Add and override properties + +```python +@patch(User) +class _User: + # Adds new property + @property + def is_academic(self): + return self.title in {UserTitle.dr, UserTitle.prof} +``` + +Add a new property to the original class by defining it in the patch class. Above, a new `is_academic` property is added to the `User` class. + +```python +@patch(Identity) +class _Identity: + # Overrides existing property descriptor method + @property + def data(self): + data = super().data + data.update({'tag': 'plugin'}) + return data + + # Overrides existing property descriptor method + @data.setter + def data(self, value): + raise RuntimeError('Cannot set data on identity') + + # Adds new property descriptor method + @data.deleter + def data(self): + raise RuntimeError('Cannot delete data on identity') +``` + +Override a property as well as its setter and deleter descriptor methods by defining them in the patch class. Like within methods, you can access the original property descriptor method from the patch class by using `super()`. + +In the example above, the `data` property of `Identity` is overridden to add a new key to the dictionary returned by the original property. The setter and deleter descriptors are also overridden. + +> [!NOTE] +> It is not currently possible to call `super()` on the `setter` and `deleter` descriptor methods of properties. diff --git a/doc/enums.md b/doc/enums.md new file mode 100644 index 0000000..5470005 --- /dev/null +++ b/doc/enums.md @@ -0,0 +1,57 @@ +# Patching Enums + +Enums in Indico are commonly used to define valid values in certain database columns and choices in form fields. This page of the guide explains how to patch Indico Enums to add members and carry over extra attributes. + +- [Add new members](#add-new-members) +- [Inject extra attributes](#inject-extra-attributes) + +## Add new members + +```python +@patch(SurveyState) +class _SurveyState(IndicoEnum): + # Adds a new member with value 101 + disabled = 101 +``` + +Add new members to the original Enum by defining them in the patch Enum. In this example, the `disabled` member is added to the `SurveyState` Enum with the value `101`. + +> [!IMPORTANT] +> Make sure to set a high-enough value to guarantee that values of patched members will not conflict with the values of present AND future members defined in the original Enum. This is especially important when patching Enums used in `pyIntEnum` columns. + +```python +@patch(SurveyState, padding=100) +class _SurveyState(IndicoEnum): + # Also adds a new member with value 101 + disabled = 1 +``` + +You can more easily guarantee that values of patched members will not conflict using the `padding` argument of the `@patch()` decorator. Specify this argument to pad the value of each member by that amount when patching them into the original Enum. + +```python +@patch(UserTitle, padding=100) +class _UserTitle(RichIntEnum): + # Adds new members and their associated human-readable titles + __titles__ = [None, 'Madam', 'Sir', 'Rev.'] + madam = 1 # value is 101 + sir = 2 # value is 102 + rev = 3 # value is 103 +``` + +You can also patch Indico-defined `RichEnum`s and their variants. In this example, new user titles are added. The `__titles__` attribute defines how they should be displayed in the user interface and will be carried over to the original Enum. + +## Inject extra attributes + +```python +# Overrides the __page_sizes__ of the original PageSize Enum +@patch(PageSize, padding=100, extra_args=('__page_sizes__',)) +class _PageSize: + __page_sizes__ = {**PageSize.__page_sizes__, **{ + 'A7': pagesizes.A7, + 'A8': pagesizes.A8, + }} + a7 = 1 + a8 = 2 +``` + +By default, only the attributes used by `RichEnums` properties are carried over to the original Enum (e.g. `__titles__`, `__css_classes__`). Declare any extra attribute that needs to be carried over in the `extra_args` argument of the `@patch` decorator. diff --git a/doc/forms.md b/doc/forms.md new file mode 100644 index 0000000..857015d --- /dev/null +++ b/doc/forms.md @@ -0,0 +1,120 @@ +# Patching WTForms + +Forms in Indico are defined using the [WTForms](https://wtforms.readthedocs.io/) library and most of them get rendered based on the fields defined in the form classes. Apply techniques covered in the [Patching Indico classes](./classes.md) page of this guide to add, modify, remove or reorder fields in forms, among other use cases. + +- [Add new fields](#add-new-fields) +- [Modify existing fields](#modify-existing-fields) +- [Remove existing fields](#remove-existing-fields) +- [Reorder fields](#reorder-fields) +- [Alter field validators](#alter-field-validators) +- [Alter field choices](#alter-field-choices) + +## Add new fields + +```python +@patch(EventDataForm) +class _EventDataForm: + # Adds new field (this usually requires adding a new column to the database) + is_sponsored = BooleanField(_('Sponsored'), [DataRequired()], widget=SwitchWidget()) +``` + +Define new fields in the patch class as you would in the original form class. You can add custom validation functions and `@genereated_data` methods like in non-patched form classes. + +> [!IMPORTANT] +> If the field is used to populate an SQLAlchemy model object, you will need to add the corresponding column in the database. For more information, please refer to the [Patching SQLAlchemy models](./models.md) page of the guide. + +> [!NOTE] +> New fields will be rendered by default at the end of the form. To customize the order of fields, please refer to the [reordering fields](#reorder-fields) section. + +## Modify existing fields + +```python +@patch(EventDataForm) +class _EventDataForm: + # Replaces the original field + description = StringField(_('Description'), EventDataForm.description.validators) +``` + +Redefine existing fields in the patch class, for instance, to change the type of a field. In this example, the original `TextAreaField` field is replaced by a `StringField` and the original validators are preserved. + +## Remove existing fields + +```python +@patch(EventDataForm) +class _EventDataForm: + # Removes the original field + del EventDataForm.url_shortcut +``` + +Remove fields by simply deleting them from the original form class. + +> [!IMPORTANT] +> If the field is used to populate an SQLAlchemy model object, you may need to remove the corresponding column in the database, to alter the column to be nullable, or to set a default value in the column definition. For more information, please refer to the [Patching SQLAlchemy models](./models.md) page of this guide. + +> [!NOTE] +> Patching for removing fields is not necessary, as you are deleting the attributes in the original form class directly. Deleting the attributes in the original form class from a patch class may still be a good idea to keep all modifications in the same code block. + +## Reorder fields + +```python +@patch(EventDataForm) +class _EventDataForm: + # Specifies the order of fields + def __iter__(self): + for field_name in self._fields: + if field_name in ('is_sponsored',): + continue + if field_name == 'title': + yield self._fields['is_sponsored'] + yield self._fields[field_name] +``` + +You have all the flexibility to reorder fields in a form with the `__iter__()` method. In this example, the newly added `is_sponsored` field, which would be displayed as the last field, is moved before the `title` field. + +## Alter field validators + +```python +@patch(EventDataForm) +class _EventDataForm: + def __init__(self, *args, **kwargs): + super().__init__(*args, **kwargs) + # Removes all validators from title field + self.title.validators = [] +``` + +Override the `__init__()` method of the original form class to alter the validators of existing fields. In this example, all validators are removed from the `title` field. + +```python +BANNED_SHORTCUTS = {'admin', 'indico', 'event', 'official'} + +@patch(EventDataForm) +class _EventDataForm: + # Overrides validator function + def validate_url_shortcut(self, field): + super().validate_url_shortcut(field) + if field.data in BANNED_SHORTCUTS: + raise ValidationError(_('This URL shortcut is not allowed.')) +``` + +Override custom validator functions to replace or extend validation conditions. In this example, the `validate_url_shortcut()` function is overridden to add an additional check for banned shortcuts. + +## Alter field choices + +```python +@patch(EventLanguagesForm) +class _EventLanguagesForm: + __disabled_locales__ = {'en_US', 'fr_FR'} + + # Overrides field choices + def __init__(self, *args, **kwargs): + super().__init__(*args, **kwargs) + self.default_locale.choices = [ + (id_, title) for id_, title in EventLanguagesForm.default_locale.choices + if id_ not in self.__disabled_locales__ + ] +``` + +Override the `__init__()` method of the form class to alter the choices of existing fields. In this example, some locales are removed from the list of choices in the `default_locale` field. + +> [!IMPORTANT] +> Choices are defined from Enums in `IndicoEnumSelectField`. You can more reliably alter available choices in those fields by patching the Enum class. For more information, please refer to the [Patching Enums](./enums.md) page of this guide. diff --git a/doc/models.md b/doc/models.md new file mode 100644 index 0000000..f4777e1 --- /dev/null +++ b/doc/models.md @@ -0,0 +1,130 @@ +# Patching SQLAlchemy models + +Indico defines its database schema using [SQLAlchemy](https://www.sqlalchemy.org/) model classes. This page of the guide explains how to patch the SQLAlchemy models defined in Indico. Apply techniques covered in the [Patching Indico classes](./classes.md) page of this guide to add new columns and relationships to existing models or constraints to existing tables. + +- [Add new columns and relationships](#add-new-columns-and-relationships) +- [Add and modify hybrid properties](#add-and-modify-hybrid-properties) +- [Add, remove and replace table constraints](#add-remove-and-replace-table-constraints) +- [Generate Alembic migration scripts for patched models](#generate-alembic-migration-scripts-for-patched-models) + +## Add new columns and relationships + +```python +@patch(User) +class _User: + # Adds new column and relationship + credit_card_id = db.Column(db.String, ForeignKey('credit_cards.id')) + credit_card = db.relationship('CreditCard', backref=backref('user')) +``` + +Define columns and relationships in the patch model class to have them added to the original model. In this example, a `credit_card_id` column and a `credit_card` relationship are added to the `User` model. + +```python +user = User.query.filter_by(id=1).one() +user.credit_card = CreditCard('XXXX-XXXX-XXXX-XXXX') +``` + +```python +users = User.query.filter(User.credit_card == None).all() +``` + +You can then use the new column and relationship to insert, update and query rows in the database as if they were defined in the original model class. + +> [!IMPORTANT] +> You will still need to apply the changes to the database schema via Alembic migration script. This is [done differently](#generate-alembic-migration-scripts-for-patched-models) for patched models than for Indico-defined and plugin-defined models. + +## Add and modify hybrid properties + +```python +@patch(Event) +class _Event: + # Adds a new hybrid property + @hybrid_property + def is_in_series(self): + return self.series_id is not None + + # Adds the expression for the new hybrid property + @is_in_series.expression + def is_in_series(self): + return ~self.series_id.is_(None) +``` + +Add new hybrid properties to the original model class by defining them in the patch class. In this example, a new `is_in_series` hybrid property is added to the `Event` model. + +```python +@patch(Event) +class _Event: + # Overrides an existing hybrid property + @hybrid_property + def event_message(self): + return '' + + # Overrides the setter for an existing hybrid property + @event_message.setter + def event_message(self, value): + pass + + # Overrides the expression for an existing hybrid property + @event_message.expression + def event_message(self): + return '' +``` + +You will override existing hybrid properties in the original model class by redefining them in the patch class. This also works for hybrid property setters, deleters and expressions. In this example, the `event_message` hybrid property is overridden to always return an empty string. + +## Add, remove and replace table constraints + +```python +@patch(RegistrationForm) +class _RegistrationForm: + RegistrationForm.__table__.append_constraint( + db.CheckConstraint(...) + ) +``` + +Add new constraints by calling the `append_constraint()` method of the `__table__` attribute in the original model class. + +```python +@patch(RegistrationForm) +class _RegistrationForm: + RegistrationForm.__table__.constraints -= { + c for c in RegistrationForm.__table__.constraints + if c.name == '' + } +``` + +You can also remove constraints by reassigning the `constraints` attribute of the `__table__` attribute in the original model class. In this example, the constraint with name `` is removed. The easiest way to find the name of the constraint to remove is to inspect the database schema directly (e.g. via `psql` command). + +```python +@patch(RegistrationForm) +class _RegistrationForm: + RegistrationForm.__table__.constraints -= { + c for c in RegistrationForm.__table__.constraints + if c.name == '' + } + RegistrationForm.__table__.append_constraint( + db.CheckConstraint(..., '') + ) +``` + +Apply both steps to replace a constraint. In this example, the constraint with name `` is first removed and a new one with the same name is then added. + +> [!IMPORTANT] +> You will still need to apply the changes to the database schema via Alembic migration script. This is [done differently](#generate-alembic-migration-scripts-for-patched-models) for patched models than for Indico-defined and plugin-defined models. + +> [!NOTE] +> Patching for modifying table constraints is not necessary, as you are calling `__table__` directly the original model class directly. Altering table constraints on the original form class from a patch class may still be a good idea to keep all modifications in the same code block. + +## Generate Alembic migration scripts for patched models + +Once new patches are defined in the plugin, generate the Alembic migration script that will record the database schema changes. Since the new columns and table constraints are defined in Indico core model classes, run this command: + +```sh +indico db migrate +``` + +The new Alembic migration script will be placed in the `indico/migrations/versions/` directory of the `indico` package. Now, move the new script to the migrations directory of the plugin, adjust its `down_revision` to point to the latest revision and run the migration script: + +```sh +indico db --plugin upgrade +```