Skip to content

Commit

Permalink
Improve documentation about parsing URLs in lxml_html_clean.
Browse files Browse the repository at this point in the history
  • Loading branch information
frenzymadness committed Oct 9, 2024
1 parent 0d1a6e1 commit 8ce436d
Show file tree
Hide file tree
Showing 3 changed files with 15 additions and 0 deletions.
4 changes: 4 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,10 @@

This project was initially a part of [lxml](https://github.com/lxml/lxml). Because HTML cleaner is designed as blocklist-based, many reports about possible security vulnerabilities were filed for lxml and that make the project problematic for security-sensitive environments. Therefore we decided to extract the problematic part to a separate project.

**Important**: the HTML Cleaner in ``lxml_html_clean`` is **not** considered appropriate **for security sensitive environments**. See e.g. [bleach](https://pypi.org/project/bleach/) for an alternative.

This project uses functions from Python's `urllib.parse` for URL parsing which **do not validate inputs**. For more information on potential security risks, refer to the [URL parsing security](https://docs.python.org/3/library/urllib.parse.html#url-parsing-security) documentation. A maliciously crafted URL could potentially bypass the allowed hosts check in `Cleaner`.

## Installation

You can install this project directly via `pip install lxml_html_clean` or as an extra of lxml
Expand Down
8 changes: 8 additions & 0 deletions docs/index.rst
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,14 @@ This project was initially a part of `lxml <https://github.com/lxml/lxml>`_. Bec
many reports about possible security vulnerabilities were filed for lxml and that make the project problematic for
security-sensitive environments. Therefore we decided to extract the problematic part to a separate project.

**Important**: the HTML Cleaner in ``lxml_html_clean`` is **not** considered appropriate **for security sensitive environments**.
See e.g. `bleach <https://pypi.org/project/bleach/>`_ for an alternative.

This project uses functions from Python's ``urllib.parse`` for URL parsing which **do not validate inputs**.
For more information on potential security risks, refer to the
`URL parsing security <https://docs.python.org/3/library/urllib.parse.html#url-parsing-security>`_ documentation.
A maliciously crafted URL could potentially bypass the allowed hosts check in ``Cleaner``.

Security
--------

Expand Down
3 changes: 3 additions & 0 deletions lxml_html_clean/clean.py
Original file line number Diff line number Diff line change
Expand Up @@ -185,6 +185,9 @@ class Cleaner:
Note that you may also need to set ``whitelist_tags``.
Note that URLs are parsed via functions from ``urllib.parse`` and
no input validation is performed.
``whitelist_tags``:
A set of tags that can be included with ``host_whitelist``.
The default is ``iframe`` and ``embed``; you may wish to
Expand Down

0 comments on commit 8ce436d

Please sign in to comment.