Update dependencies in 3.0 to align with rasa-sdk

[carlad]

Proposed changes:
Partially closes #592

This brings dependencies in the pyproject.toml in line with those in rasa-sdk.
I used rasa as the primary dependency source, so most changes were made in the PR for rasa-sdk:
RasaHQ/rasa-sdk#648.

However, as there was a python 3.9 compatibility issue with black, a newer version is being used, hence the changes in this PR.

• add changelog.
• Updated formatting.

[ancalita]

To double-check, python 3.9 requires black higher than v19, so you went ahead and updated this to the version compatible with 3.9?
I would also double-check if there are any style changes that need to be updated in Engineering Guidelines in Notion.

[carlad]

@ancalita

To double-check, python 3.9 requires black higher than v19, so you went ahead and updated this to the version compatible with 3.9?

That's correct. When running black I got the following error:

poetry run black --check rasa_sdk tests
Usage: black [OPTIONS] [SRC]...
Try 'black -h' for help.

Error: Invalid value for '-t' / '--target-version': 'py39' is not one of 'py27', 'py33', 'py34', 'py35', 'py36', 'py37', 'py38'.

Some googling revealed it was a bug that was fixed in 21.12 21.x.x

[carlad]

@ancalita

I would also double-check if there are any style changes that need to be updated in Engineering Guidelines in Notion.

The only changes made by black were making sure lines didn't exceed 80 columns.

[usc-m]

LGTM

[ancalita]

The only changes made by black were making sure lines didn't exceed 80 columns.

@carlad Oh is there a way to override the 80 columns to the previous setting, I think it was 88 or 89? This way you won't have such a large diff?

[carlad]

@ancalita apologies but I was mistaken. I assumed it was line length, as I didn't realise we were working with an 88 char limit. Black and flake8 are already configured to do an 88 char line length,
see pyproject.toml and setup.cfg

I've had a look through the files again and it seems much of the reformatting has to do with listing arguments on multiple lines, rather than on one line, eg the diff for tests/test_server.py:

 monkeypatch.setattr(
-            sys.modules["rasa.model_training"], "train", mocked_training_function,
+            sys.modules["rasa.model_training"],
+            "train",
+            mocked_training_function,
         )

I looked through the black changelog but couldn't see anything related to this that was changed between v19 and v21.

Would bumping the version cause black to look at all files, rather than just files that had been changed?

[joejuzl]

Would bumping the version cause black to look at all files, rather than just files that had been changed?

I think black already looks at all the files.
Would be good to keep the same line length I think. We shouldn't have to reformat our whole repo every time we update black (it obscures the git diff a lot too).

[joejuzl]

The line length is defined under [tool.black] in pyproject.toml. Maybe it has to be configured differently now?

[carlad]

@joejuzl

Would be good to keep the same line length I think. We shouldn't have to reformat our whole repo every time we update black (it obscures the git diff a lot too).

It looks like line length wasn't the issue. Line length is still configured at 88 chars in the pyproject.toml. The reformatting split multiple arguments to multiple lines. Is this a convention?

[joejuzl]

It looks like line length wasn't the issue. Line length is still configured at 88 chars in the pyproject.toml. The reformatting split multiple arguments to multiple lines. Is this a convention?

@carlad seems like there's something funky going on here. black should try to keep things on 1 line if they fit - as per the documentation

[carlad]

there's something funky going on here.

@joejuzl indeed. Reading through the documentation I can't think of any reason black would be behaving this way except for a lower than default configured line length. Will probe further...

[carlad]

Update: This seems like a change in black 21.7b0.
When I use 19.10b0 there's no reformatting. All versions of 21.x apply the reformatting (and v20.x). We need to use v21 as previous versions have a bug when python3.9 is the target version (rasa-sdk has python 3.9 as a target version)
I couldn't find anything obvious in the changelog about this difference in behaviour. So should we just roll with it?

[carlad]

Update: Black Black now assumes that if there is a trailing comma for a list of items, all items should be on separate lines. And the new --skip-magic-trailing-comma flag only removes trailing commas. So we might indeed have to roll with it...

[carlad]

So in the end, on a new branch, I ran Black with the -skip-magic-trailing-comma flag.
This removed all trailing commas from single lines.
Then I ran black without the flag, and it left the single lines alone.
Then I merged the branch into this one.
Hoping this gives us what we want 🤞 🙏

[joejuzl]

Looks good 👍
Can we make sure the commits are squashed and the final message makes clear it's a black update and formatting?