Bump pyramid from 1.10.5 to 2.0 #237

dependabot · 2021-03-17T10:10:17Z

Bumps pyramid from 1.10.5 to 2.0.

Changelog

2.0 (2021-02-28)

No changes from 2.0b1.

2.0b1 (2021-02-20)

Break potential reference cycle between request and context. See Pylons/pyramid#3649

Remove update_wrapper from pyramid.decorator.reify. See Pylons/pyramid#3657

2.0b0 (2020-12-15)

Overhaul tutorials and update cookiecutter to de-emphasize request.user in favor of request.identity for common use cases. See Pylons/pyramid#3629

Improve documentation and patterns with builtin fixtures shipped in the cookiecutters. See Pylons/pyramid#3629

2.0a0 (2020-11-29)

Features

Add support for Python 3.9. See Pylons/pyramid#3622

The aslist method now handles non-string objects when flattening. See Pylons/pyramid#3594

It is now possible to pass multiple values to the header predicate for route and view configuration. See Pylons/pyramid#3576

Add support for Python 3.8. See Pylons/pyramid#3547

New security APIs have been added to support a massive overhaul of the authentication and authorization system. Read "Upgrading Authentication/Authorization" in the "What's New in Pyramid 2.0" chapter of the documentation for information about using this new system.

pyramid.config.Configurator.set_security_policy.

... (truncated)

Commits

7b18204 add pre-commented forward
9a64c50 mark 2.0-branch as no longer in progress
ea6ac9c Merge branch 'master' into 2.0-branch
a05fd83 prep 2.0
b2fd15e Merge branch 'master' into 2.0-branch
ea30508 Merge pull request #3657 from luhn/reify-sphinx
5b63fb8 prep 2.0b1
24ea786 Remove inspect.unwrap, not in Python 2.7 and it's a no-op.
f510281 Write tests.
7a055e7 Update changelog.
Additional commits viewable in compare view

Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.

Dependabot commands and options

You can trigger Dependabot actions by commenting on this PR:

@dependabot rebase will rebase this PR
@dependabot recreate will recreate this PR, overwriting any edits that have been made to it
@dependabot merge will merge this PR after your CI passes on it
@dependabot squash and merge will squash and merge this PR after your CI passes on it
@dependabot cancel merge will cancel a previously requested merge and block automerging
@dependabot reopen will reopen this PR if it is closed
@dependabot close will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
@dependabot ignore this major version will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
@dependabot ignore this minor version will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
@dependabot ignore this dependency will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)

What's changed

There are no such things as authentication policies and authorization policies anymore in Pyramid 2. Instead there are just security policies, which are like simplified authentication and authorization policies merged into one class. See Security and ISecurityPolicy.

checkmate.auth.AuthenticationPolicy is now checkmate.security.SecurityPolicy. AuthenticationPolicy was really just a subclass for configuring CascadingAuthenticationPolicy though, and the new SecurityPolicy is just the same thing. Before:
```
class AuthenticationPolicy(CascadingAuthenticationPolicy):
    def __init__(self):
        super().__init__(sub_policies=[APIHTTPAuth(), GoogleAuthenticationPolicy()])
```
After:
```
class SecurityPolicy(CascadingSecurityPolicy):
    def __init__(self):
        super().__init__(
            subpolicies=[HTTPBasicAuthSecurityPolicy(), GoogleSecurityPolicy()]
        )
```
checkmate.auth.CascadingAuthenticationPolicy is now checkmate.security.CascadingSecurityPolicy. But this is just the same thing: it takes a list of subpolicies and consults the authenticated_userid() method of each subpolicy taking the first policy to return something as the "effective" subpolicy. As before the remember() method takes an iface argument to select the subpolicy.

GoogleAuthenticationPolicy is now GoogleSecurityPolicy. The class also no longer inherits from Pyramid's SessionAuthenticationPolicy. Pyramid 1 came with a bunch of builtin authentication policies that you could either use directly or subclass to make your own. Pyramid 2 doesn't come with any builtin security policies: you have to implement your own security policy (implementing all of ISecurityPolicy's methods). Instead Pyramid 2 has a bunch of security policy helpers that correspond to the previously builtin authentication policies: SessionAuthenticationHelper, AuthTktCookieHelper, etc. Your custom security policy classes can use these helpers via composition instead of inheritance. So where GoogleAuthenticationPolicy subclassed the builtin SessionAuthenticationPolicy, GoogleSecurityPolicy now uses SessionAuthenticationHelper.

APIHTTPAuth is now HTTPBasicAuthSecurityPolicy and instead of subclassing BasicAuthAuthenticationPolicy it uses the extract_http_basic_credentials() helper function.
There are no such things as principals or access control lists anymore in Pyramid 2. Instead there are permissions (as before) and identities. checkmate.auth.AuthorizationPolicy (which made authorization decisions based on principals) is no more. Its functionality is now merged into the security policies. The way it works now is:
1. You put a @view_config(..., permission="foo") on a view
2. Pyramid calls your security policy's authenticated_userid() method to get the string value for request.authenticated_userid
3. Pyramid calls your security policy's identity() method to get the value for request.identity.
  
  An identity is supposed to be an object representing the authenticated user. Whereas authenticated_userid must be a string, identity can be any object (e.g. a User model object). It's a formalisation of a common informal pattern in Pyramid 1.x: apps would often set request.user to a User model object. identity doesn't have to be a model though, it can be a string, None, or whatever you want. AFAIK identity isn't used at all by Pyramid itself. Pyramid just gets request.identity from your security policy's identity() method and makes it available to the rest of your app's code as request.identity. In particular Pyramid passes request.identity to your security policy's permits() method to make authorization decisions (see below).
4. Pyramid calls your security policy's permits(request, context, permission) method which decides whether request has permission in context. SecurityPolicy.permits() is the equivalent of what AuthorizationPolicy.permits() used to be: it's what makes the actual authorization decision (whether to call the view or not). Your permits() method is just a normal Python method that does whatever it wants with request, context and permission to decide whether or not to allow the request. It doesn't have to have anything to do with ACLs. It can access request.authenticated_userid (which Pyramid set by calling your same security policy's authenticated_userid() method) and request.identity (which Pyramid set by calling your same security policy's identity() method).

seanh · 2021-04-06T13:23:42Z

checkmate/templates/admin/pages.html.jinja2

+<h2>Permissions</h2>
+<code>{{ request.identity.permissions }}</code>


There's no such thing as principals anymore in Pyramid 2, so changed this to display permissions instead

seanh · 2021-04-06T13:24:48Z

checkmate/views/ui/admin.py

@@ -13,15 +13,15 @@ def __init__(self, request):

    @view_config(
        renderer="checkmate:templates/admin/pages.html.jinja2",
-        effective_principals=[Principals.STAFF],
+        permission=Permissions.ADMIN,


There's no such thing as principals in Pyramid 2 anymore, so use a permission instead.

seanh · 2021-04-06T13:26:04Z

checkmate/views/ui/admin.py

    )
    def get(self):
        cookie = SimpleCookie()
        cookie.load(self.request.headers["Cookie"])

        return {"session": cookie["session"].value}

-    @view_config()
+    @forbidden_view_config()


Previously this was an additional view without the effective_principals=[Principals.STAFF] so it would be called when the STAFF principal was missing. Now, with permissions, you need a forbidden view instead

seanh · 2021-04-06T13:26:38Z

checkmate/views/ui/api/add_to_allow_list.py

@@ -41,7 +41,7 @@ class Meta:
    route_name="add_to_allow_list",
    request_method="POST",
    jsonapi={"schema": AllowRuleSchema()},
-    effective_principals=[Principals.STAFF],
+    permission=Permissions.ADD_TO_ALLOW_LIST,


There's no such thing as principals anymore in Pyramid 2 so use a permission instead. This also changes the 404 to a 403

seanh · 2021-04-06T14:03:11Z

checkmate/security.py

+class Permissions(Enum):
+    ADMIN = "admin"
+    CHECK_URL = "check_url"
+    ADD_TO_ALLOW_LIST = "add_to_allow_list"


Just one separate permission for each different security-protected view

seanh · 2021-04-06T14:03:38Z

checkmate/security.py

+class Identity(NamedTuple):
+    userid: str
+    permissions: List[str]


request.identity is a NamedTuple containing the authenticated user's ID and permissions

Does NamedTuple actually do anything with the type definitions here. I'm assuming it's just for documentation?

seanh · 2021-04-06T14:04:14Z

checkmate/security.py

+class CascadingSecurityPolicy:
+    def __init__(self, subpolicies):
+        self._subpolicies = subpolicies
+
+    def identity(self, request):
+        return self._effective_subpolicy(request).identity(request)
+
+    def authenticated_userid(self, request):
+        return self._effective_subpolicy(request).authenticated_userid(request)
+
+    def permits(self, request, context, permission):
+        return self._effective_subpolicy(request).permits(request, context, permission)
+
+    def forget(self, request, **kwargs):
+        return self._effective_subpolicy(request).forget(request, **kwargs)
+
+    def remember(self, request, userid, iface, **kwargs):
+        return self._get_specific_policy(iface).remember(request, userid, **kwargs)
+
+    def _effective_subpolicy(self, request):
+        for policy in self._subpolicies:
+            if policy.authenticated_userid(request):
+                return policy
+
+        return self._subpolicies[-1]
+
+    def _get_specific_policy(self, iface):
+        for policy in self._subpolicies:
+            if isinstance(policy, iface):
+                return policy
+
+        raise KeyError(
+            f"Could not find a policy matching the requested interface: {iface}"
+        )


This is just exactly the same thing as CascadingAuthenticationPolicy was before

seanh · 2021-04-06T14:04:37Z

checkmate/security.py

+class SecurityPolicy(CascadingSecurityPolicy):
+    def __init__(self):
+        super().__init__(
+            subpolicies=[HTTPBasicAuthSecurityPolicy(), GoogleSecurityPolicy()]
+        )


This is just exactly the same thing as AuthenticationPolicy was before

seanh · 2021-04-06T14:06:29Z

checkmate/security.py

+            return Identity(
+                userid, permissions=[Permissions.ADMIN, Permissions.ADD_TO_ALLOW_LIST]
+            )


So GoogleSecurityPolicy gives all Google-authenticated users (with a @hypothes.is email address) the ADMIN and ADD_TO_ALLOW_LIST permissions. Google-authenticated users can access the admin pages and the add-to-allow-list API

seanh · 2021-04-06T14:07:24Z

checkmate/security.py

+        if userid:
+            return Identity(userid, permissions=[Permissions.CHECK_URL])


HTTPBasicAuthSecurityPolicy gives basic auth-authenticated users the CHECK_URL permission so they can access the check API

seanh · 2021-04-06T14:17:41Z

tox.ini

-    ; Suppress warnings about an import of `imp` by Pyramid
-    ignore:the imp module is deprecated in favour of importlib


Pyramid 2 fixed this deprecation warning

seanh · 2021-04-06T14:24:55Z

tests/unit/checkmate/security_test.py

+        # The subpolicy whose remember() method we expect to get called,
+        # either subpolicy1 or subpolicy2.
+        subpolicy = locals()[subpolicy]


Some cleverness here. You can't use a fixture in a parametrize so instead I parametrized the test with the fixture's name and then retrieved the actual fixture by name from locals(). The alternative would be to split this into two separate tests with some duplication

I think this and some other things could be simplied by using an array as the fixture:

@pytest.fixture def subpolicies(self): return [ create_autospec(SubPolicy0, spec_set=True, instance=True), create_autospec(SubPolicy1, spec_set=True, instance=True) ]

Then this test could be:

@pytest.mark.parametrize( "iface,policy_index", [(SubPolicy0, 0), (SubPolicy1, 1)] ) def test_remember(self, iface, policy_index, policy, subpolicies): # The subpolicy whose remember() method we expect to get called, # either subpolicy0 or subpolicy1. subpolicy = locals()[policy_index]

You can achieve something similar with policy,_subpolicies instead

seanh · 2021-04-06T14:27:11Z

tests/unit/checkmate/security_test.py

+    @pytest.fixture(params=["subpolicy1", "subpolicy2", "both", None])
+    def effective_subpolicy(
+        self,
+        request,
+        subpolicy1,
+        subpolicy2,
+    ):
+        """Return the subpolicy that the test expects to be effective."""
+
+        # Any test that uses this parametrized `effective_subpolicy` fixture
+        # will get run four times...
+
+        if request.param == "subpolicy1":
+            # The first time the test is run subpolicy1.authenticated_userid()
+            # will return a userid but subpolicy2.authenticated_userid() won't.
+            subpolicy1.authenticated_userid.return_value = sentinel.policy1_userid
+            subpolicy2.authenticated_userid.return_value = None
+            # CascadingSecurityPolicy should choose subpolicy1 as the effective policy.
+            return subpolicy1
+
+        if request.param == "subpolicy2":
+            # The next time the test is run subpolicy1.authenticated_userid()
+            # *won't* return a userid but subpolicy2.authenticated_userid() will.
+            subpolicy1.authenticated_userid.return_value = None
+            subpolicy2.authenticated_userid.return_value = sentinel.policy2_userid
+            # CascadingSecurityPolicy should choose subpolicy2 as the effective policy.
+            return subpolicy2
+
+        if request.param == "both":
+            # The next time the test is run both subpolicies will return a userid.
+            subpolicy1.authenticated_userid.return_value = sentinel.policy1_userid
+            subpolicy2.authenticated_userid.return_value = sentinel.policy2_userid
+            # CascadingSecurityPolicy should choose subpolicy1 as the effective policy.
+            return subpolicy1
+
+        # The final time the test is run neither subpolicy will return a userid.
+        assert request.param is None
+        subpolicy1.authenticated_userid.return_value = None
+        subpolicy2.authenticated_userid.return_value = None
+        # CascadingSecurityPolicy should choose subpolicy2 as the effective policy.
+        return subpolicy2


This is a very straightforward usage of parametrized fixtures. It enables identity(), authenticated_userid(), permits() and forget() to all have just one test each, but each to be tested in the four different authentication situations

Bumps [pyramid](https://github.com/Pylons/pyramid) from 1.10.5 to 2.0. - [Release notes](https://github.com/Pylons/pyramid/releases) - [Changelog](https://github.com/Pylons/pyramid/blob/2.0/CHANGES.rst) - [Commits](Pylons/pyramid@1.10.5...2.0) Co-authored-by: dependabot[bot] <[email protected]> Signed-off-by: dependabot[bot] <[email protected]>

jon-betts · 2021-04-06T15:57:21Z

checkmate/security.py

+    if permission in policy.identity(request).permissions:
+        return Allowed("allowed")
+
+    return Denied("denied")


This could be a nice thing to add to the identity object so the caller can instead do:

request.identity.allows(permissions)

I'm assuming from your summary that this should be available already on the request object by the time this gets called.

Maybe. I'm not sure that's really the right thing to do though, as Pyramid already provides APIs for asking about permissions. There's the permission view config arg but also things like request.has_permission() and this goes through all the necessary Pyramid machinations to give the right answer. I think request.identity is just meant to represent the user, not provide an API for asking about permissions. request.identity is available to the whole app not just to the security policies

jon-betts

This looks good, and honestly easier to understand than before. The less Pyramid all up in your auth the better as far as I'm concerned.

I think the only part I'd suggest doing some more work on is the cascading policy tests. I think these are swimming against the tide of pytest a bit by insisting on using two separate fixtures for the sub-policies. I think this would be simplified by using an array of sub-policies, which could then be referenced by index. There are a few places this would likely help, or make it easier to script instead of repeating things.

This could either be explicit as a separate fixture as it is now, or it could be achieved by using policy._subpolicies.

There's also a suggestion to use the class itself as the template for sub-policies as it implements the interface.

It's nice because it cuts out some waffle
It's slightly dodgy because it means the tests are coupled to the interface defined in the code... not sure if that's actually a problem or not

jon-betts · 2021-04-07T18:08:07Z

checkmate/security.py

+
+        return Identity("", [])
+
+    def authenticated_userid(self, request):  # pylint:disable=no-self-use


Can you just declare this as a class method and ignore cls?

jon-betts · 2021-04-07T18:10:17Z

tests/unit/checkmate/security_test.py

+        pass
+
+    def forget(self, request, **kw):
+        pass


As CascadingSecurityPolicy implements the interface, you could create the sub-policies from children if it to avoid making this class. There's nothing saying you can put cascading security policies inside each other.

jon-betts · 2021-04-07T18:17:17Z

tests/unit/checkmate/security_test.py

+        # The subpolicy whose remember() method we expect to get called,
+        # either subpolicy1 or subpolicy2.
+        subpolicy = locals()[subpolicy]


I think this and some other things could be simplied by using an array as the fixture:

@pytest.fixture def subpolicies(self): return [ create_autospec(SubPolicy0, spec_set=True, instance=True), create_autospec(SubPolicy1, spec_set=True, instance=True) ]

Then this test could be:

@pytest.mark.parametrize( "iface,policy_index", [(SubPolicy0, 0), (SubPolicy1, 1)] ) def test_remember(self, iface, policy_index, policy, subpolicies): # The subpolicy whose remember() method we expect to get called, # either subpolicy0 or subpolicy1. subpolicy = locals()[policy_index]

You can achieve something similar with policy,_subpolicies instead

jon-betts · 2021-04-07T18:20:00Z

tests/unit/checkmate/security_test.py

+        """Return the CascadingSecurityPolicy instance to be tested."""
+        return CascadingSecurityPolicy([subpolicy1, subpolicy2])
+
+    @pytest.fixture(params=["subpolicy1", "subpolicy2", "both", None])


If we used an index based version as suggested above, this could be a list of active indices:

@pytest.fixture(params=[[0], [1], [0,1], []])

The body of this fixture could then probably be written as a single loop

dependabot bot added the dependencies Pull requests that update a dependency file label Mar 17, 2021

seanh self-requested a review March 17, 2021 17:23

dependabot bot force-pushed the dependabot/pip/pyramid-2.0 branch from f8be294 to eddcf74 Compare March 18, 2021 16:12

seanh force-pushed the dependabot/pip/pyramid-2.0 branch from eddcf74 to 74247b9 Compare March 19, 2021 14:38

seanh marked this pull request as draft March 19, 2021 14:38

seanh force-pushed the dependabot/pip/pyramid-2.0 branch 3 times, most recently from 058553c to dd6d7de Compare April 6, 2021 13:22

seanh reviewed Apr 6, 2021

View reviewed changes

seanh force-pushed the dependabot/pip/pyramid-2.0 branch from dd6d7de to 5e4d448 Compare April 6, 2021 13:33

seanh reviewed Apr 6, 2021

View reviewed changes

seanh force-pushed the dependabot/pip/pyramid-2.0 branch from 5e4d448 to bede20b Compare April 6, 2021 14:17

seanh marked this pull request as ready for review April 6, 2021 14:17

seanh reviewed Apr 6, 2021

View reviewed changes

seanh requested review from jon-betts and marcospri April 6, 2021 14:17

seanh force-pushed the dependabot/pip/pyramid-2.0 branch from bede20b to c054c7f Compare April 6, 2021 14:23

seanh reviewed Apr 6, 2021

View reviewed changes

seanh force-pushed the dependabot/pip/pyramid-2.0 branch from c054c7f to 8acd77f Compare April 6, 2021 14:37

jon-betts reviewed Apr 6, 2021

View reviewed changes

jon-betts approved these changes Apr 7, 2021

View reviewed changes

marcospri approved these changes Apr 9, 2021

View reviewed changes

seanh merged commit be14356 into main Apr 9, 2021

seanh deleted the dependabot/pip/pyramid-2.0 branch April 9, 2021 14:19

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Bump pyramid from 1.10.5 to 2.0 #237

Bump pyramid from 1.10.5 to 2.0 #237

dependabot bot commented on behalf of github Mar 17, 2021 •

edited by seanh

Loading

seanh Apr 6, 2021

seanh Apr 6, 2021 •

edited

Loading

seanh Apr 6, 2021

seanh Apr 6, 2021 •

edited

Loading

seanh Apr 6, 2021 •

edited

Loading

seanh Apr 6, 2021

jon-betts Apr 6, 2021

seanh Apr 6, 2021

seanh Apr 6, 2021

seanh Apr 6, 2021

seanh Apr 6, 2021

seanh Apr 6, 2021

seanh Apr 6, 2021 •

edited

Loading

jon-betts Apr 7, 2021

seanh Apr 6, 2021

jon-betts Apr 6, 2021 •

edited

Loading

seanh Apr 6, 2021 •

edited

Loading

jon-betts left a comment

jon-betts Apr 7, 2021

jon-betts Apr 7, 2021

jon-betts Apr 7, 2021

jon-betts Apr 7, 2021

		<h2>Permissions</h2>
		<code>{{ request.identity.permissions }}</code>

		if userid:
		return Identity(userid, permissions=[Permissions.CHECK_URL])

		; Suppress warnings about an import of `imp` by Pyramid
		ignore:the imp module is deprecated in favour of importlib


		return Identity("", [])

		def authenticated_userid(self, request): # pylint:disable=no-self-use

Bump pyramid from 1.10.5 to 2.0 #237

Bump pyramid from 1.10.5 to 2.0 #237

Conversation

dependabot bot commented on behalf of github Mar 17, 2021 • edited by seanh Loading

2.0 (2021-02-28)

2.0b1 (2021-02-20)

2.0b0 (2020-12-15)

2.0a0 (2020-11-29)

Features

What's changed

Choose a reason for hiding this comment

seanh Apr 6, 2021 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

seanh Apr 6, 2021 • edited Loading

Choose a reason for hiding this comment

seanh Apr 6, 2021 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

seanh Apr 6, 2021 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

jon-betts Apr 6, 2021 • edited Loading

Choose a reason for hiding this comment

seanh Apr 6, 2021 • edited Loading

Choose a reason for hiding this comment

jon-betts left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

dependabot bot commented on behalf of github Mar 17, 2021 •

edited by seanh

Loading

seanh Apr 6, 2021 •

edited

Loading

seanh Apr 6, 2021 •

edited

Loading

seanh Apr 6, 2021 •

edited

Loading

seanh Apr 6, 2021 •

edited

Loading

jon-betts Apr 6, 2021 •

edited

Loading

seanh Apr 6, 2021 •

edited

Loading