Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Can't follow to or from GoToSocial #2794

Closed
kvibber opened this issue Apr 7, 2023 · 5 comments · Fixed by #2812
Closed

Can't follow to or from GoToSocial #2794

kvibber opened this issue Apr 7, 2023 · 5 comments · Fixed by #2812
Labels
activitypub bug Something isn't working

Comments

@kvibber
Copy link
Contributor

kvibber commented Apr 7, 2023

Describe the bug
GoToSocial can retrieve individual posts from Bookwyrm, and can retrieve a Bookwyrm user's profile, but it cannot follow that user.

Bookwyrm cannot retrieve a GoToSocial user's profile at all and displays an error message when I search for one.

To Reproduce
On a GoToSocial instance, search for a Bookwyrm user. Then try to follow them.

On a Bookwyrm instance, search for a GoToSocial user. You can search for my test account at @[email protected]

Expected behavior
The GTS user should be able to follow the Bookwyrm user and vice versa

Instance
bookwyrm.social

Additional context
Add any other context about the problem here.

Using GoToSocial 0.8.0-rc1 and bookwyrm.social.

Logs show both of the following are throwing back 401 Unauthorized HTTP responses:
GTS' POST request to https://bookwyrm.social/inbox
Bookwyrm's GET request to https://gts.keysmash.xyz/users/kelsonv

This looks very similar to a bug I fixed in Takahe where it was rejecting signatures whose algorithm had been obfuscated by labeling it as "hs2019", which GoToSocial does according to https://datatracker.ietf.org/doc/html/draft-cavage-http-signatures-12
jointakahe/takahe#528

In the case of Takahe it was easy to just treat hs2019 as sha-256, since that's the only signature algorithm it accepts. I took a look at bookwyrm/signatures.py and I see Bookwyrm also supports incoming sha-512 signatures, so it'll be a bit more complicated than just adding or algorithm == 'hs2019' in verify_digest.

I can give this a try over the weekend and see if it works and fixes either (or both) issues.

@kvibber kvibber added the bug Something isn't working label Apr 7, 2023
@WesleyAC
Copy link
Member

WesleyAC commented Apr 7, 2023

Stack trace for searching for a GTS user from Bookwyrm:

web_1             | OK: /search
web_1             | Traceback (most recent call last):
web_1             |   File "/app/bookwyrm/views/helpers.py", line 78, in handle_remote_webfinger
web_1             |     user = models.User.objects.get(username__iexact=query)
web_1             |   File "/usr/local/lib/python3.9/site-packages/django/db/models/manager.py", line 85, in manager_method
web_1             |     return getattr(self.get_queryset(), name)(*args, **kwargs)
web_1             |   File "/usr/local/lib/python3.9/site-packages/django/db/models/query.py", line 435, in get
web_1             |     raise self.model.DoesNotExist(
web_1             | bookwyrm.models.user.User.DoesNotExist: User matching query does not exist.
web_1             | 
web_1             | During handling of the above exception, another exception occurred:
web_1             | 
web_1             | Traceback (most recent call last):
web_1             |   File "/app/bookwyrm/activitypub/base_activity.py", line 330, in resolve_remote_id
web_1             |     data = get_data(remote_id)
web_1             |   File "/app/bookwyrm/connectors/abstract_connector.py", line 250, in get_data
web_1             |     resp.raise_for_status()
web_1             |   File "/usr/local/lib/python3.9/site-packages/requests/models.py", line 1021, in raise_for_status
web_1             |     raise HTTPError(http_error_msg, response=self)
web_1             | requests.exceptions.HTTPError: 401 Client Error: Unauthorized for url: https://gts.keysmash.xyz/users/kelsonv
web_1             | 
web_1             | During handling of the above exception, another exception occurred:
web_1             | 
web_1             | Traceback (most recent call last):
web_1             |   File "/usr/local/lib/python3.9/site-packages/django/core/handlers/exception.py", line 47, in inner
web_1             |     response = get_response(request)
web_1             |   File "/usr/local/lib/python3.9/site-packages/django/core/handlers/base.py", line 181, in _get_response
web_1             |     response = wrapped_callback(request, *callback_args, **callback_kwargs)
web_1             |   File "/usr/local/lib/python3.9/site-packages/sentry_sdk/integrations/django/views.py", line 67, in sentry_wrapped_callback
web_1             |     return callback(request, *args, **kwargs)
web_1             |   File "/usr/local/lib/python3.9/site-packages/django/views/generic/base.py", line 70, in view
web_1             |     return self.dispatch(request, *args, **kwargs)
web_1             |   File "/usr/local/lib/python3.9/site-packages/django/views/generic/base.py", line 98, in dispatch
web_1             |     return handler(request, *args, **kwargs)
web_1             |   File "/usr/local/lib/python3.9/site-packages/csp/decorators.py", line 19, in _wrapped
web_1             |     r = f(*a, **kw)
web_1             |   File "/app/bookwyrm/views/search.py", line 48, in get
web_1             |     return endpoints[search_type](request)
web_1             |   File "/app/bookwyrm/views/search.py", line 106, in user_search
web_1             |     handle_remote_webfinger(query)
web_1             |   File "/app/bookwyrm/views/helpers.py", line 94, in handle_remote_webfinger
web_1             |     user = activitypub.resolve_remote_id(
web_1             |   File "/app/bookwyrm/activitypub/base_activity.py", line 337, in resolve_remote_id
web_1             |     data = get_activitypub_data(remote_id)
web_1             |   File "/app/bookwyrm/activitypub/base_activity.py", line 395, in get_activitypub_data
web_1             |     resp.raise_for_status()
web_1             |   File "/usr/local/lib/python3.9/site-packages/requests/models.py", line 1021, in raise_for_status
web_1             |     raise HTTPError(http_error_msg, response=self)
web_1             | requests.exceptions.HTTPError: 406 Client Error: Not Acceptable for url: https://gts.keysmash.xyz/users/kelsonv

@WesleyAC
Copy link
Member

WesleyAC commented Apr 7, 2023

I'm not super familiar with the code, but it looks as though Bookwyrm is seeing the 401, correctly falling back to a signed request, but then getting a 406 since the signed request isn't what GTS expects. I'm going to open a issue in the GTS repo for that so we can get some feedback from them.

WesleyAC added a commit to WesleyAC/bookwyrm that referenced this issue Apr 7, 2023
This is the header described in the ActivityPub spec, which should fix
some federation problems with GoToSocial and potentially other picky
services.

Related: bookwyrm-social#2794
WesleyAC added a commit to WesleyAC/bookwyrm that referenced this issue Apr 7, 2023
This is the header described in the ActivityPub spec, which should fix
some federation problems with GoToSocial and potentially other picky
services.

Related: bookwyrm-social#2794, superseriousbusiness/gotosocial#1676
WesleyAC added a commit to WesleyAC/bookwyrm that referenced this issue Apr 7, 2023
This is the header described in the ActivityPub spec, which should fix
some federation problems with GoToSocial and potentially other picky
services.

Related: bookwyrm-social#2794, superseriousbusiness/gotosocial#1676
@kvibber
Copy link
Contributor Author

kvibber commented Apr 9, 2023

I've put together a fix for just the hs2019 issue, where it will try to validate an hs2019 signature as sha256 and if that fails, try to validate it as sha512, and then throw the usual error. So Bookwyrm should be able to validate GoToSocial's signatures now.

It still doesn't solve the whole problem, but I think I've figured out why GoToSocial isn't validating Bookwyrm's signatures!

With that patch applied, I get the following on my test instance when trying to follow:

gotosocial | timestamp="09/04/2023 18:27:46.014" func=federation.(*federator).AuthenticateFederatedRequest level=DEBUG requestID=p0r4csw704000wywd66g msg="error parsing public key http://bw.builder.hyperborea.org/user/bookwyrm.instance.actor#main-key: cannot find publicKey with id: http://bw.builder.hyperborea.org/user/bookwyrm.instance.actor#main-key"

And when I hit that endpoint on the bookwyrm instance, I get the following JSON:

{"id": "http://bw.builder.hyperborea.org/user/bookwyrm.instance.actor", "type": "Person", "preferredUsername": "bookwyrm.instance.actor", "inbox": "http://bw.builder.hyperborea.org/user/bookwyrm.instance.actor/inbox", "publicKey": {"id": "http://bw.builder.hyperborea.org/user/bookwyrm.instance.actor/#main-key", "owner": "http://bw.builder.hyperborea.org/user/bookwyrm.instance.actor", "publicKeyPem": "-----BEGIN PUBLIC KEY-----\nMIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEA13GWgPpnbQUH+jT5dy3G\nQxUGuMgcL7GzkYZRtT6RIX++H/7fMhxRX2nxqqEWC/bc/tioZbrABtjRD4otVqLe\nghEkHCakwOVzYnyBs6Be7Nw7DYGsuBVmirgcVG8SK0bfoVowv069q+vei3VZlqeZ\nJobhFs+sLf0HAFWJbtLk1gvH3DMz2GJKDBSoFAK5hYsVLq4NpybynhvSi09ZlQEc\nRPCCe0YaDvecScQbRHIsOVseHVn5M3Zi9AviTwx2qgkEw7nWeDNMURFDd4QWquAZ\nI7nEOLO43fkxu8ohiY+3EddXW39VY61c7KQAx5ITRil8JMa+jLeE0IIGXSmN0kPj\nuQIDAQAB\n-----END PUBLIC KEY-----"}, "followers": "http://bw.builder.hyperborea.org/user/bookwyrm.instance.actor/followers", "following": "http://bw.builder.hyperborea.org/user/bookwyrm.instance.actor/following", "outbox": "http://bw.builder.hyperborea.org/user/bookwyrm.instance.actor/outbox", "endpoints": {"sharedInbox": "http://bw.builder.hyperborea.org/inbox"}, "icon": {}, "bookwyrmUser": true, "manuallyApprovesFollowers": false, "discoverable": false, "hideFollows": false, "@context": ["https://www.w3.org/ns/activitystreams", "https://w3id.org/security/v1", {"manuallyApprovesFollowers": "as:manuallyApprovesFollowers", "schema": "http://schema.org#", "PropertyValue": "schema:PropertyValue", "value": "schema:value"}]}

http://bw.builder.hyperborea.org/user/bookwyrm.instance.actor#main-key - What GTS is looking fow
http://bw.builder.hyperborea.org/user/bookwyrm.instance.actor/#main-key - What the JSON identifies the key as.

If GTS is looking for the exact ID in the JSON, that's probably where the communication is breaking down.

I'm not familiar enough with the spec to say which side should be normalizing this, though I'd tend to go with the robustness principle and say that both should.

@hughrun
Copy link
Contributor

hughrun commented Apr 10, 2023

@kvibber is right that we're using mismatched key ids and should fix it.
However, I believe the primary problem is caused by this issue: #2801

I've got an almost-working fix, but unfortunately whilst testing that, another problem with mentions has surfaced so I need to work out what's going on with that.

hughrun added a commit to hughrun/bookwyrm that referenced this issue Apr 10, 2023
Fixes bookwyrm-social#2801
Related to bookwyrm-social#2794

It is legitimate to use any url for the user's key id. We have been assuming this id is the user id plus a fragment (#key-id) but this is not always the case, notably in the case of GoToSocial it is at /key-id. This commit instead checks the remote user's information to see if the key id listed matches the key id of the message allegedly received from them.

Whilst troubleshooting this it also became apparent that there is a mismatch between Bookwyrm users' keyId and the KeyId we claim to be using in signed requests (there is a forward slash missing). Since everything after the slash is a fragment, this usually slips through but we should be consistent so I updated that.
@bahuma20
Copy link

Seeing the same issue when trying to POST a Follow activity to a users inbox from Mastodon 4.1.2 as well as my own server that uses activitypub-express 4.2.1.

Example POST https://books.theunseen.city/user/bahuma20/inbox:

{
  "@context": "https://www.w3.org/ns/activitystreams",
  "id": "https://mastodon.local/13218191-0689-4d45-8f3e-d9027c812959",
  "type": "Follow",
  "actor": "https://mastodon.local/users/bahuma20",
  "object": "https://books.theunseen.city/user/bahuma20"
}

Signature header:

keyId="https://mastodon.local/users/bahuma20#main-key",algorithm="rsa-sha256",headers="(request-target) host date digest content-type",signature="vjLkSuw9qYdz3DB2X/Am7uVPyFERQm4Qn+sJFPef0o0y5YmffSUUtQDCZxW1500JNqXxqNmU63QpXrFGIRM4pfASzWb5F9D9P18Xtm9Xt5uL0GVHH7y9u5w9EYBIdZUAOkYCehoiv4g1UBI06XCPXAqukd28Wk7DdJTL9Y7v5jBR8Vn4tYMPqH5LEU8W0XYX6ZozlKF+mT4UYSL36URyd+06HiJEAu3Ns+Xv4tao9SAWFpQgE6xHLcpaQp1MtQfibwWeC5TN8t1g6UwJf92ES3jmnPWY/kwcQSIPScZsuWgyKvvqX3Lt1r7pJ+T6PcHzwl9fMCJD3XDdPG6Snc9akw=="

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
activitypub bug Something isn't working
Projects
None yet
Development

Successfully merging a pull request may close this issue.

4 participants