Why are there 3 different ways to embed a CID in a record?

[DavidBuchanan314]

Per the docs, there are 3 ways: lexicon type cid-link, string with format cid, and string with format uri and scheme ipld://.

Why though? Isn't the cid-link method ideal, since it gets stored as binary?

[devinivy]

In my view cids surface in two different ways.

The first is a qualifier to support an integrity check or strong reference. For example, in an app.bsky.feed.like record we include the uri of the liked record, but also qualify it by a cid so that if the record is modified, the liker has documented the specific record version they liked—useful if the record has changed in some considerable way. There's no intention to link to that content, and in fact the content isn't even expected to necessarily still exist. The string format cid supports this use case. Most of the time this is how app developers will interact with cids. In the JSON representaion there's no need to deal with the noise of { "$link": "bafycid" } or ask devs to think about ipld.

The second is as a stand-in for the content itself, where the cid is actually intended to be dereferenced in some way and content found on the other end. Right now this is predominantly relevant to sync. This appears in blobs in the context of a repo, where a blob is effectively part of the repo can be dereferenced using sync.getBlob?cid=&did=. It also appears in sync.subscribeRepos where e.g. a commit is expressed as a link and the content can be found inside the event's blocks field. These cid links that are part of sync also all happen to be expressed in dag-cbor where the link is more compact, as you noted.

As for the ipld:// representation, I can't think of any way atproto really intends to utilize that. But of course it's technically able to be expressed with a lexicon string format since it is a uri. There are no special semantics for that in the protocol, though, and I don't anticipate any will crop-up. Applications are welcome to use it if it's useful to them.

[bnewbold]

It's a good question! Agree with @devinivy's parallel response.

A couple related things to note, though they aren't really answers:

• some IPLD libraries/implementations/interfaces work by spidering CID Link refs (aka, tag 42 CBOR fields), and will consider a CAR file (or similar) incomplete if not all the referenced blocks can be fetched. if you use a string CID this doesn't happen.
• there is a bit of a big-picture vibe that references within a repo make more sense as a CID Link (binary), while references outside (to other repos) are string format. and considering blobs as "inside the repo". I could imagine things evolving at some point (maybe a later version of protocol or repo format?) to allow sub-records objects which are only referenced by a record, but should be considered part of the repo, retained and tracked by the PDS, included in CAR files (and commit events), etc. This would be a change from how things work right now (to be clear!), and maybe we wouldn't go that way, but is a door we could keep open for the future.
• when using the string CID as a version field, it is most commonly alongside an AT-URI (aka, the "strongref" object type). in theory (and maybe in the future?), we could have extended the AT-URI syntax to include the CID as part of the URI itself, as a "strong reference URI" (single string, not an object with two fields). This would be yet another type or variation on AT-URIs, and maybe would need a new lexicon string format type ("at-uri-versioned"?)
• reminder that CIDs encode metadata about whether they refer to an IPLD data object, or raw binary blob, or something like an IPFS file or directory of files (chunked). this is an extra layer of distinction on top of the link/string/URI stuff