Summary of Recent Changes (2024-02-22)

[bnewbold]

There have been a whole bunch of changes in the past week! This is a listicle post for devs in case they missed any of this.

Federation: we've started federating with small self-hosting PDS instances! See blog posts for details. Impact on devs:

• will be seeing accounts hosted on non-bsky PDS instances. horray!
• more likely that there will be PDS instances that can't be reached. hopefully a small number of non-active experimental/exploration accounts, not real accounts. but to date almost all active accounts have been on a relatively stable PDS
• possible shenanigans in repos and over firehose. the PDS reference implementation does a fair amount of validation of records and repository structure. with federation, folks have more direct control over their repository and can in theory blast out anything over the event stream websocket, or stuff junk in repo CAR files
• TIDs (in record keys) may be truely bogus

Lexicon flexibility and disabling validation: until this week, we have be conservative about what kinds of records we allow folks to write in to their atproto repositories on our PDS. this has been "off-specification" to reduce headaches, but with federation we can't prevent this any longer, and have thrown open the limits. Specifically, folks can disable validation of existing record schemas (app.bsky.*), and create records for new or invented Lexicons. Impact on devs:

• will see records on the firehose for known Lexicons, which are invalid in new and exciting ways. they might be invalid by Lexicon, or could also be invalid in Lexicon-compliant ways (like having bogus text facet rules, for bsky posts)
• will see records on the firehose for unknown or non-existant Lexicons

Account Migration: details here: https://github.com/bluesky-social/pds/blob/main/ACCOUNT_MIGRATION.md

Control over PLC: until now, the vast majority of Bluesky PDS accounts have not actually had direct control of their DID identity. As part of account migration, we now have an endpoint which lets users request signed PLC operations from their PDS instance. This allows them to add new rotation keys (under their own control), and from there can submit additional future operations to update their DID documents. Impact on devs:

• possible that there will be identity updates (like handle changes, migrations, signing key rotation) which don't go out over the relay firehose, because they were not instigated by the account's PDS. if you are validating signatures, be sure to actually resolve the DID document!
• possibility (danger!) of accounts getting permanently locked out of control over their identity

New #identity repo stream event type: we are working on some iterations to the repo event stream messages for identity and account state updates. in the near future we will probably end up with just three events: #commit (no change), #identity (all handle and DID updates), #account (PDS repo hosting lifecycle related, including deletions, PDS takedowns, etc). As a first step, we have a new basic #identity which is emitted any time the PDS knows that an account's DID document or handle has changed. This rolls the #handle and #migrate events. For now this event is basically empty, and downstream services should use it as a trigger to re-resolve both the handle and DID document. In the future we may include the handle, DID document, and identity tombstone status. There is no plan to sign these events; authority rests in the resolution process.

Preventing legacy blobs on record creation: in early 2023, we refactored the blob data type in atproto to include size in bytes, and be easier to identity in un-specified data. Until now, we preferred but did not require the new format. Now our PDS requires new records to use the new format, though there remain many old records in the network. Details at: https://atproto.com/specs/data-model#blob-type

Account creation on PDS/entryway website: to date, account creation has been driven by the createAccount endpoint, both from the mobile app and the web app. We are now starting to have account creation on the Bluesky servers happen on a regular web page, which is closer to how things will work with OAuth. This makes it possible to tweak how the flow and our anti-abuse measures work (like SMS phone validation, captchas, invite codes, etc) without an API iteration. The API endpoint will probably stick around in some form, but we may reduce or remove support. From a mobile app, it will be possible to bounce to webview, complete signup, and redirect back to the app.

Firehose cursor reset: discussed here: #2194

search.bsky.social shut down: previously announced, but finally implemented.

[mackuba]

will see records on the firehose for known Lexicons, which are invalid in new and exciting ways

Yup, already happened :p I saw a post with no text and one with reply being an array instead of an object ;)

[mackuba]

Is the identity event being sent now? I don't think I'm seeing it...

[bnewbold]

I believe it is! And should be getting passed through by Relay. They are not particularly frequent.

@dholms and/or @ericvolp12 can confirm

[mackuba]

Wasn't that supposed to be sent for each handle change though?

[rudyfraser]

Is there somewhere to read more about that account creation bit? Not sure I fully understand what that change is and how it will work when users are able to select any other server at sign up.

[bnewbold]

Unfortunately I don't have much details for you right now, in terms of specific URL signatures, parameters, etc. What we have now is a placeholder until OAuth, which is coming along and will hopefully land a bit after moderation services.

The short of it is that during "signup / login", users will provide either existing identity info (eg, a handle), or specify a PDS host. From there the app will bounce to a page on that host (in a webview on mobile) and go through some combination of signup, login, or "approve this app" if already signed in. This web page can use arbitrary auth; eg this is how 2FA or passkeys can be integrated, without being part of atproto itself. After completing that process, the web page (hosted on the PDS), will bounce back to the client app via a redirect URL (on mobile this bounces back to the app), with additional auth metadata, as is standard in OAuth.

From the standpoint of the app, the distinction between login and signup won't matter too much. Either way the user will bounce back to the app with auth metadata, and app use can continue.

This will require PDS instances to implement a basic web login flow (they have been basically API-only to date).

[K-Nksm]

First of all, I really appreciate your hard work to create a nice protocol!
I have a question about this.

Lexicon flexibility and disabling validation

Is there a guideline of bluesky PDS for third-party developers to follow?
Particularly, I'm curious about the limitations of the records (size, CRUD rate, etc).

I'm a hobby atproto developer making an AppView for blog service.
WhiteWind
This service creates its own collection (com.whtwnd.blog.entry) on PDS and provides a viewer for the records.
I'm worrying about breaking rules or harming your infrastructure without knowing.