Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Setup Vale (prose linter) with a minimal unobtrusive configuration #281

Merged
Merged
Show file tree
Hide file tree
Changes from 12 commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 0 additions & 1 deletion .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -8,5 +8,4 @@ __pycache__
*notes-from-review.md
*.idea*
# Grammar / syntax checkers
.vale.ini
styles/
9 changes: 9 additions & 0 deletions .pre-commit-config.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -31,6 +31,15 @@ repos:
- id: codespell
additional_dependencies:
- tomli
exclude: >
(?x)^(
.*vale-styles.*
)$

- repo: https://github.com/errata-ai/vale
rev: v3.4.2
hooks:
- id: vale

ci:
autofix_prs: false
Expand Down
32 changes: 32 additions & 0 deletions .vale.ini
Original file line number Diff line number Diff line change
@@ -0,0 +1,32 @@
# Configuration file for using Vale in the python-package-guide repository
#
# To disable checks on parts of a MarkDown or HTML file, delimit the section
# using these HTML comments:
# to disabled Vale checks after this line: <!-- vale off -->
# to enable Vale checks after this line: <!-- vale on -->
#
# To disable checks based on MarkDown scope, see IgnoredScopes.
# To disable checks on certain HTML elements, see IgnoredClasses.
#
# More information about the configuration can be found here:
# https://vale.sh/docs/topics/config


# Path to the styles directory, where style rules are defined
StylesPath = vale-styles

# Path to a dictionary folders inside the StylesPath config subdirectory. This
# folder can contain two files, accept.txt and reject.txt, with one word per
# line. These words will be used to check for spelling mistakes in addition to
# the internal dictionary, if the 'Vale' ruleset is enabled (see below)
# See https://vale.sh/docs/topics/vocab/#folder-structure for more details
Vocab = sample


# Checks are defined in sections by file type, like the one below for
# MarkDown. In each section you can enable groups of style rules, defined in folders
# inside the StylesPath directory.
# Use 'Vale' to enable the internal style rules and checks.

[*.md]
BasedOnStyles = package-guide-test
1 change: 1 addition & 0 deletions conf.py
Original file line number Diff line number Diff line change
Expand Up @@ -135,6 +135,7 @@
"styles/write-good/README.md",
"styles/*",
".pytest_cache/README.md",
"vale-styles/*",
]

# For sitemap generation
Expand Down
2 changes: 1 addition & 1 deletion index.md
Original file line number Diff line number Diff line change
Expand Up @@ -71,7 +71,7 @@ The first round of our community-developed, how to create a Python package tutor

* [What is a Python package?](/tutorials/intro)
* [Make your code installable](/tutorials/installable-code)
* [Publish your package to (test) PyPi](/tutorials/publish-pypi)
* [Publish your package to (test) PyPI](/tutorials/publish-pypi)
* [Publish your package to conda-forge](/tutorials/publish-conda-forge)

:::
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -104,7 +104,7 @@ exist in the default Anaconda cloud channel.

:::{figure-md} pypi-conda-channels

<img src="../images/python-pypi-conda-channels.png" alt="Graphic with the title Python package repositories. Below it says Anything hosted on PyPI can be installed using pip install. Packaging hosted on a conda channel can be installed using conda install. Below that there are two rows. the top row says conda channels. next to it are three boxes one with conda-forge, community maintained; bioconda and then default - managed by the anaconda team. Below that there is a row that says PyPI servers. PyPI - anyone can publish to pypi. and test pypi. a testbed server for you to practice. " width="700px">
<img src="../images/python-pypi-conda-channels.png" alt="Graphic with the title Python package repositories. Below it says Anything hosted on PyPI can be installed using pip install. Packaging hosted on a conda channel can be installed using conda install. Below that there are two rows. the top row says conda channels. next to it are three boxes one with conda-forge, community maintained; bioconda and then default - managed by the anaconda team. Below that there is a row that says PyPI servers. PyPI - anyone can publish to PyPI. and test PyPI. a testbed server for you to practice. " width="700px">
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is so so great!! i'll try to review and get this merged tomorrow @flpm thank you again. we have so many open issues and pr's it's taking a bit longer than expected to merge things!

@kierisi i just wanted you to see this - notice that Filipe setup a rule for pypi to be PyPI and it catches that issue across our guidebook! we can now create a style guide that is checked in CI using this tool (and we can do the same on our website too)!!

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Because the current PyPI rule is set to warning and not an error, pre-commit will succeed in CI. But if the level: warning is replaced with level: error in PyPI.yml, pre-commit will fail .

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ahhhh ok perfect!


Conda channels represent various repositories that you can install packages from. Because conda-forge is community maintained, anyone can submit a recipe there. PyPI is also a community maintained repository. Anyone can submit a package to PyPI and test PyPI. Unlike conda-forge there are no manual checks of packages submitted to PyPI.
:::
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -91,14 +91,14 @@ represent on your PyPI landing page. These classifiers also allow users to sort
```

:::{figure-md} build-workflow
<img src="../images/python-package-development-process.png" alt="Graphic showing the high level packaging workflow. On the left you see a graphic with code, metadata and tests in it. those items all go into your package. Documentation and data are below that box because they aren't normally published in your packaging wheel distribution. an arrow to the right takes you to a build distribution files box. that box leads you to either publishing to testpypi or the real pypi. from pypi you can then connect to conda-forge for an automated build that sends distributions from pypi to conda-forge. " width="700px">
<img src="../images/python-package-development-process.png" alt="Graphic showing the high level packaging workflow. On the left you see a graphic with code, metadata and tests in it. those items all go into your package. Documentation and data are below that box because they aren't normally published in your packaging wheel distribution. an arrow to the right takes you to a build distribution files box. that box leads you to either publishing to TestPyPI or the real PyPO. from PyPI you can then connect to conda-forge for an automated build that sends distributions from PyPI to conda-forge. " width="700px">
lwasser marked this conversation as resolved.
Show resolved Hide resolved

You need to build your Python package in order to publish it to PyPI (or Conda). The build process organizes your code and metadata into a distribution format that can be uploaded to PyPI and subsequently downloaded and installed by users. NOTE: you need to publish a sdist to PyPI in order for conda-forge to properly build your package automatically.
:::

:::{figure-md}

<img src="../images/python-build-package/pypi-metadata-keywords-license.png" alt="This screenshot shows the metadata on pypi for the xclim package. on it you can see the name of the license, the author and maintainer names keywords associated with the package and the base python version it requires which is 3.8." width="400px">
<img src="../images/python-build-package/pypi-metadata-keywords-license.png" alt="This screenshot shows the metadata on PyPI for the xclim package. on it you can see the name of the license, the author and maintainer names keywords associated with the package and the base python version it requires which is 3.8." width="400px">

PyPI screenshot showing metadata for the xclim package.
:::
Expand Down
8 changes: 7 additions & 1 deletion pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -21,7 +21,13 @@ dependencies = [
"sphinxext-opengraph",
"sphinx-inline-tabs",
# for project cards
"matplotlib"
"matplotlib",
]

[project.optional-dependencies]
dev = [
# for checking style rules
"vale"
]

[tool.hatch.build.targets.wheel]
Expand Down
2 changes: 1 addition & 1 deletion tutorials/intro.md
Original file line number Diff line number Diff line change
Expand Up @@ -315,7 +315,7 @@ Then you can create a conda-forge recipe using the [Grayskull](https://github.co
[You will learn more about the conda-forge publication process here.](publish-conda-forge.md)

:::{figure-md} publish-package-pypi-conda-overview
<img src="../images/tutorials/publish-package-pypi-conda.png" alt="Graphic showing the high level packaging workflow. On the left you see a graphic with code, metadata and tests in it. Those items all go into your package. Documentation and data are below that box because they aren't normally published in your packaging wheel distribution. an arrow to the right takes you to a build distribution files box. that box leads you to either publishing to testPyPI or the real PyPI. From PyPI you can then connect to conda-forge for an automated build that sends distributions from PyPI to conda-forge." width="700px">
<img src="../images/tutorials/publish-package-pypi-conda.png" alt="Graphic showing the high level packaging workflow. On the left you see a graphic with code, metadata and tests in it. Those items all go into your package. Documentation and data are below that box because they aren't normally published in your packaging wheel distribution. an arrow to the right takes you to a build distribution files box. that box leads you to either publishing to TestPyPI or the real PyPI. From PyPI you can then connect to conda-forge for an automated build that sends distributions from PyPI to conda-forge." width="700px">

In the image above, you can see the steps associated with publishing
your package on PyPI and conda-forge. Note that the distribution files that PyPI requires are the [sdist](#python-source-distribution) and [wheel](#python-wheel) files. Once you are ready to make your code publicly installable, you can publish it on PyPI. Once your code is on PyPI it is straight forward to then publish to conda-forge. You create a recipe using the Grayskull package and then you open a pr in the conda-forge recipe repository. You will learn more about this process in the [conda-forge lesson](/tutorials/publish-conda-forge).
Expand Down
2 changes: 1 addition & 1 deletion tutorials/publish-pypi.md
Original file line number Diff line number Diff line change
Expand Up @@ -284,7 +284,7 @@ will remember them.
## Install your package from TestPyPI

Once your package upload is complete, you can install it from
TestPYPI. You can find the installation instructions on the TestPyPI
TestPyPI. You can find the installation instructions on the TestPyPI
landing page for your newly uploaded package.

:::{figure-md} testpypi-landing-page
Expand Down
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@flpm so is this file where we add words that might be unusual but we want to accept as a part of our vocabulary?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That is right, this is a way to customize the internal dictionary. But the dictionary checks are part of the internal Vale ruleset that is currently disabled in .vale.ini because it triggers lots of errors. A better way to approach this is to use individual rules like the PyPI one.

Empty file.
Empty file.
22 changes: 22 additions & 0 deletions vale-styles/package-guide-test/PyPI.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,22 @@
extends: substitution
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you also please help me understand how this file works? Is there a way we could just create a list of words that we'd always want to spell a certain way rather than having a yml file for each word (if that makes sense?). i may not understand.

so a list like this

pyopensci: pyOpenSci
PyOpenSci: pyOpenSci
pypi: PyPI
PYPI: PyPI
back end: back-end
backend: back-end

etc?
thank you again for this! i'm just trying to understand how this works!

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sure. There are multiple extension points that can be used to customize Vale.

This rule uses extends: substitution, which needs a section called swap that defines the mapping between incorrect strings and their correct versions.

The first part of the map can also be a regular expression. I had to use two regular expressions in the PyPI rule because otherwise the lowercase string pypi will match inside URLs https://pypi.org (false positive).

You are right about the list, to check for pyOpenSci case issues, you could either create a new rule like this one or even rename the file to something more generic like capitalizations.yml and extend the list here.

Likely you will run into similar false positives with the lowercase version of pyopensci matching URLs so you will need a regular expression to exclude them, but you could do something like I did for PyPI:

(?:\spyopenscy[\.,]?\s): pyOpenSci

(? ) tells Vale it is a regexp
\s matches a word boundary (space, tab)
[.,]? allow the existence of a period or comma

This regexp matches pyopensci and pyopensci. but not pyopensci.org

For other extension points the structure is slightly different, for example for extends: existence instead of swap you have tokens and that contains the list of words you want to warn users to avoid.

extends: existence
message: Consider removing '%s'
level: warning
ignorecase: true
tokens:
    - appears to be
    - arguably

Here are more details about the extension points https://vale.sh/docs/topics/styles/#extension-points

I put some example rules as examples (write-good ruleset) in ./vale-styles/write-good/ to illustrate what rules can do to and how the files are structured.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

thank you so much!! this makes sense. and i hadn't considered the word within a url so regex also makes a lot of sense to control for that!

message: Consider using '%s' instead of '%s'
level: warning
ignorecase: false
action:
name: replace
# swap maps tokens in form of bad: good
swap:
# lower case defined as regex to prevent false positives in URLs or other identifiers
- (?:\spypi[\.,]?\s): PyPI
- (?:\stestpypi[\.,;:]?\s): TestPyPI
- (?:\stest-pypi[\.,;:]?\s): TestPyPI
# other tests are defined with strings
- pyPi: PyPI
- pyPI: PyPI
- PYPI: PyPI
- PyPi: PyPI
- Pypi: PyPI
- testPyPI: TestPyPI
- testPYPI: TestPyPI
- TestPypi: TestPyPI
- TestPYPI: TestPyPI
Loading
Loading