Disappearing Posts in Bluesky

[TahomaSoft]

Hi,
I am using the API (using homebrewed python code) to post to my Bluesky-social account (blog.tahoma.com). They will briefly show up in my account (as seen in my web browser client for bluesky social), and then auto delete after about 60 seconds.

I have no clue as to why; any hints on where to start looking would be much appreciated.

Thanks,

Erik

P.S. I should note that I receive status code 200 from the API post actions.

[bnewbold]

What that sounds like to me is that you are hitting a bug in the read-after-write system:
https://docs.bsky.app/docs/advanced-guides/read-after-write

The PDS is maybe accepting your posts, but the appview isn't indexing them. So they display for a short period, but never get indexed so they disappear after that. We were worried about this situation when we designed that system, but I don't think we've seen any bugs or issues like this to date, so something new!

I downloaded your repo and don't see any problems with recent posts, which aren't showing up in the app. We can poke around more internally.

[TahomaSoft]

Thank you!Please let me know if I can assist your efforts.ErikSent from my iPhoneOn Apr 8, 2024, at 18:05, bnewbold ***@***.***> wrote: What that sounds like to me is that you are hitting a bug in the read-after-write system: https://docs.bsky.app/docs/advanced-guides/read-after-write The PDS is maybe accepting your posts, but the appview isn't indexing them. So they display for a short period, but never get indexed so they disappear after that. We were worried about this situation when we designed that system, but I don't think we've seen any bugs or issues like this to date, so something new! I downloaded your repo and don't see any problems with recent posts, which aren't showing up in the app. We can poke around more internally. —Reply to this email directly, view it on GitHub, or unsubscribe.You are receiving this because you authored the thread.Message ID: ***@***.***>

[bnewbold]

We manually "reindexed" your repository in the back end, and that seems to have fixed some of your earlier posts from a couple days ago.

Still not really sure what exactly went wrong, but might have been an issue with the non-TID record keys on some posts? We do want to iron out this sort of bug and ensure that developers can't "break" their repository in a way that can't be easily recovered from. If this happens again please do let us know; you can at-mention me directly here in github to ensure I notice the problem.

Hopefully implementing the "Enforce Record Key As Part Of Lexicon Validation" behavior (mentioned here: #2128) will help.

[TahomaSoft]

Thank you very much! I will try using a TID record key and some other ideas and will let you know how it works out.
And thanks for the record key validation golang code.

Regards,

Erik

[TahomaSoft]

Hello @bnewbold! I think I now have a clearer understanding of the problem and the solution (or at least a workaround on my end).

I wrote and implemented a record key conforming with the TID specs. While testing successive posts using the new key, I noticed I was having a similar problem to before, with a key observed difference this time.

Note that again, all transactions to atproto were receiving Status 200 messages.

My API-using program is written to queue up several entries to post internally, does some checks and adjustments to the post text as needed, and then sequentially posts them using '.../xrpc/com.atproto.repo.createRecord'.

While fiddling around trying to sort out the problem, I noticed that a succeeding post would essentially 'overwrite' or clobber the preceding one. By that, I mean the prior post would vanish from my view on bsky.app; only the new one would be visible. This would continue for n posts; the n^th post would cause post n-1 to vanish. This would continue until the end of the sequence, either through the normal progression of the queue, or if I killed the program while running. This is different than what I remember seeing before; then, all n seemed to vanish, or maybe a random one would remain.

I thought maybe it might be that I wasn't refreshing the token enough; tinkering with that didn't change the behavior. Then I noticed that the createdAt field was very close together; posts were being created within milliseconds to microseconds of each other with the corresponding time stamps. For example:

"createdAt": "2024-05-04T00:12:43.922101Z"
"createdAt": "2024-05-04T00:12:43.922170Z

a difference of 6.9 x 10 ^-5

However, if I put a time delay into my action loop of one or more seconds, it completely alleviated the problem. All posts 1...n show up and remain!

My conjecture is, going back to the Read-After-Write bug you mentioned, the App View and PDS don't interact well if the time difference between posts is only milliseconds to microseconds apart.

Quoting from the page you shared with me:

"The App View communicates the current state of its indices by setting the Atproto-Repo-Rev response header. This is set by the rev of the most recent commit that's been indexed from the requesting user's repository. If the PDS sees this header on a response, it will search for all records that it has locally that are more recent than the provided rev and determine if they affect the App View's response."

Maybe the temporal resolution of this doesn't work at a scale of time less than 10^-2 or so.

Regards.

Erik

P.S.

Looking at metadata for posts like the following is illustrative.
"indexedAt": "2024-05-05T21:32:16.332Z",