What does a PDS implementation entail?

[bnewbold]

This is an informal overview of what a PDS does and what a "full" implementation covers.

A few folks are working on independent PDS implementations, either as earnest real-world infrastructure, or as learning projects. There are still a few roles and pieces of protocol that haven't stabilized, but those are increasingly "known unknowns" that can be pointed out.

Account and Identity

PDS instances host accounts for users, which require account management and lifecycle controls similar to any network server. While atproto identities (DIDs and handles) can in theory be entirely separate from the PDS, in practice the PDS is expected to help manage the user’s identity.

• account lifecycle: signup, deletion, migration
• account security: email verification, password reset flow, change email flow
• atproto identity resolution: DIDs, handles
• ability to takedown or suspend accounts
• secret key management: atproto signing key, and PLC rotation key
• PLC identity operations and endpoints: change handle; validate, sign, and submit PLC ops
• existing (2023 era) PDS client auth: login, sessions, app-passwords
• email delivery: for reset flows, as well as forwarding moderation mail
• output #identity and #account (coming soon) events on repo event stream
• usually the ability to manage a default handle namespace (base domain)
• upcoming OAuth client flows, including web sign-in pages, and optional MFA (more below)

In terms of specifications:

• complete identity resolution (DID and Handle)
• cryptographic key generation, signing, and validation (Cryptography)
• DID PLC operation signing (CBOR) (PLC)

Public Repositories

The defining feature of a PDS is hosting and managing public atproto repositories for accounts.

• manage MST tree and mutations
• generation of “diffs” between two repo versions, with only new records and MST nodes included
• repo event stream (”firehose”), including sequence numbers and backfill period
• import and export of repositories as CAR files
• Lexicon validation of created records (with option to override)

Optionally, PDS implementations can support “read-after-write” semantics. This requires application-specific awareness of Lexicons.

In terms of specifications:

• MST data structure implementation, including “diffs” (Repository)
• event stream (WebSocket) (Event Stream)
• Data Model: transforming between JSON and CBOR, limits other and schema-agnostic data validation (Data Model)
• Lexicon validation (Lexicon)
• TID generation (TID)

Blobs

Public media files (like images, audio, and video) are referenced from records, though not stored directly in the repository data structure. The PDS is responsible for managing blobs and serving up original files, though it is recommended for applications to serve up content to end users and clients via a CDN. Blobs are generally stored and managed per-account.

• storing and publicly serving blobs
• validate content type of blobs (mime-type)
• optionally detect sensitive metadata in blobs (eg, EXIF location data), and prevent upload unless client overrides
• tracking repository references: enforce restrictions on size and type; expire never-referenced blobs, and delete blobs after last referencing record is deleted
• blob upload and enumeration endpoints
• likely need to track and enforce limits and quotas
• ability to purge (takedown) individual blobs from an account

Client Authn/Authz and Service Proxy

In the current network architecture, almost all client requests go to the account’s PDS instance and are proxied on to other services as needed.
It is possible this will change at some point, with requests going directly from clients to other atproto services, but are not sure whether that would be an overall improvement, and have no specific plans to change that aspect of the architecture for now.

In the near future, the PDS will be full-fledged OAuth Authorization server, and will track any registered client software, will provide a web interface for individual accounts to approve (or reject) client or integration privileges, and will enforce scope limitations to API and repository access.

• generic service proxying (via atproto-proxy header)
• forwarding of appropriate HTTP headers
• web views for client/integration sign-in and approval flows
• per-account management for client/integration authorization and access scopes

In terms of specifications:

• service proxy header (Service Proxying)
• inter-service auth token generation (signing) and verification (Inter-Service Auth)
• OAuth for atproto (OAuth Proposal)

Preferences and Private State

PDS instances store arbitrary Lexicon-defined private preferences and configuration. This aids synchronization across multiple devices and clients, and ensures that this data can be exported and re-imported during PDS account migration.

• store preference data
• bulk import/export endpoints

Anti-Abuse

While we hope and expect labelers and product features to address most forms of content moderation and interpersonal disputes, there are some specific forms of harmful content and network abuse that need to be addressed at the PDS level.

• mechanisms to control and limit signups: invite codes, vouching, payment processing, CAPTCHAs, phone verification, or some other mechanism should be used to limit open signups by bots (eg, millions of fake accounts)
• per-account and global rate limits should be applied on almost all operations: HTTP requests, repository operations, identity operations, proxied requests, blob uploads, log-in attempts, etc
• either a built-in interface, or ability to “delegate” administration of accounts and blobs to an external moderation authority or service
• public contact mechanism for the administrative team for each instance (eg, ability to declare an email address)
• account security should be considered proactively

Future Work

• OAuth is coming (as discussed above)
• there is a good change that PDS instances will eventually be directly involved with private direct messaging (DMs), and this may involve PDS-to-PDS communication
• likewise, group-private content is likely to involve PDS instances and PDS-to-PDS communication
• live resolution of unknown Lexicons (eg, for validation at record creation time) is likely to be specified. Note that this means that relying entirely on code-generation from known repositories of Lexicon schemas is not likely to be sufficient for PDS instances (but should be sufficient for client apps and AppViews)
• generic email delivery is likely to be specified, with users controlling which services are allowed to send mail to them

[DavidBuchanan314]

Are there any clients (even if it's just a dev branch) that implement the client half of the OAuth proposal yet?

[bnewbold]

I think @matthieusieben has some early client stuff in his mega branch here: #1958

[ShreyanJain9]

PDS instances store arbitrary Lexicon-defined private preferences and configuration

What does this look like? Is there a spec for it anywhere?

(I assume this is how app.bsky.actor.getPreferences/putPreferences work, but curious to understand it better)

[bnewbold]

Good question! I thought we might but I don't think we do.

The getPreferences/putPreferences system has a bunch of sharp corners that have caused bugs/clobbering, and are difficult for clients to implement safely. They also push the preferences to the AppView, which in some ways isn't ideal.

We have been doing design work on a new more general system, which would probably be atproto-generic, be easier to work with, have clearer mapping to Lexicons for preferences, and potentially allow delegation of access to remote services. Don't have a final design or timeline for this work though.

[rudyfraser]

either a built-in interface, or ability to “delegate” administration of accounts and blobs to an external moderation authority or service

Could you elaborate on this? Is this like delegating another service to be able to takedown accounts on your PDS? Is that already included somewhere in the canonical TS impl?

[bnewbold]

The plan is to allow PDS instances to delegate moderation authority to one or more moderation services (basically labelers plus infrastructure moderation; Ozone is an example). The degree of delegation will probably be configurable as well. This would cover:

• account takedown and suspension
• blob takedowns (in context of an account)
• not takedowns of individual records

Depending on the degree of delegation, the PDS might just directly take action based on the mod service; allow retroactive overrides by local PDS operator team; forward on the action as a "recommendation" to PDS team, either via private email or as a "report" to an Ozone instance; just ignore the "recommendation". We're not sure we'll try to implement configs for all these options, they are just laying out the design space.

A related issue is the ability for labelers or full mod services to send "mod mail" to accounts notifying them of actions. Right now the Bluesky Ozone instance can do this directly for the Bluesky PDS instances, but we need to get that more federated, while not being a spam/harassment vector. One ideas would be to have mail from any delegated mod service be delivered, plus mail from any labelers the account directly subscribes to, and drop everything else.

One use-case we want to support is a PDS operator team using Ozone just to administer a single PDS, instead of messing around with CLI tools or API calls; this is basically what Bluesky does today. Another is to let small/medium PDS teams partially delegate mod power to a collective or external party for after-hours or certain high-harm categories of abuse.

There are some existing com.atproto.admin.* Lexicons that touch on these concept (like sendEmail and updateSubjectStatus) which we think we'll probably iterate on to get the scope and terminology correct.

[DavidBuchanan314]

Currently, when the client makes requests to, say, /xrpc/app.bsky.feed.getPostThread, there is no atproto-proxy header set. It seems like (for now) the PDS is expected to implicitly know that certain endpoints should be proxied to the bluesky appview. Is it supposed to always be like this, or will these requests be expected to use atproto-proxy too, in future?

[DavidBuchanan314]

Related: #2811 "Why does api.bsky.app have a fake DID?"

[yamarten]

I've heard that implicit proxy is a transient function.
#2568 (comment)