Post view: move cursor pagination to separate library, add backward pagination to PostQuery

[dullbananas]

The library uses tuple comparison.

The new page_back option causes the page to be before the cursor instead of after. It's possible to get the posts at the end by enabling it without providing the cursor (without this, going to the previous page would be impossible when viewing the empty page that appears when all posts have been viewed).

Includes the change that was in #4300

[phiresky]

Amazing! The changed code here looks clean. The code in the library seems complex. A few concerns. Could you maybe post some example queries this generates? The code in the library is a bit hard to follow. Especially I don't understand the ORDER BY row_number() OVER () DESC" and the need for a subquery.

[phiresky]

this should probably be renamed to something more descriptive? like diesel-tuple-pagination maybe. Also not sure what dessalines/nutomic think of depending on an external library like this, since it does seem fairly specific to lemmy.

[dullbananas]

I will not rename my library

[dessalines]

I just don't like the extra complication cursor pagination requires in general, with all of these pre-fetching of bounds, macros, etc. It makes post_view.rs so much harder to read and understand even to me.

Every example I can find online of cursor pagination is where you are sorting based off a single id or time column, rather than multiple filter and sort columns like we have. Its unfortunate that limit and offset are such a performance burden, since they require none of this complication, but I spose there's no way around it.

Whether that complicated cursor logic lives in lemmy, or elsewhere where it can be used by other projects, doesn't matter too much to me, but the latter seems better if it helps other diesel projects besides ours. We're dependent on a lot of diesel libraries that aren't made by its maintainer.

As far as the name, it doesn't bother me personally, but I could see other people not wanting to use it because of the name. For limited-scope libraries it makes sense to name the thing what it does, and it seems like a lot of other diesel projects are named things like diesel-.... or ...-diesel .

[phiresky]

For clarification, the prefetching is unrelated to the cursor pagination - I just added it in the same PR because it was a convenient related change. It's a performance boost for the post query in general with the "subscribed" filter, even for the first page where the pagination method is irrelevant. For the page-based pagination the same method could in fact be used for both sides of the page so the db doesn't have to join everything while trying to find the page start and end.

[phiresky]

since timestamptz is a 8-byte column but numeric is a variable-size column it might be good if this also returned of 8-byte size. for example bigint ((-extract(epoch from t) * 1000)::bigint)?

[dullbananas]

done

[dullbananas]

Amazing! The changed code here looks clean. The code in the library seems complex. A few concerns. Could you maybe post some example queries this generates? The code in the library is a bit hard to follow. Especially I don't understand the ORDER BY row_number() OVER () DESC" and the need for a subquery.

I just added better comments in the library, so it should be a little less confusing.

For normal forward pagination, it just sorts by the columns and adds a filter with tuple comparison. In this case, wrapping it in SELECT * FROM (...) does nothing and is only done because I only put the if statement around the part that adds the additional order-by clause.

The limit_and_offset_from_end option for backwards pagination is complicated because PostgreSQL has no built-in way to get the last rows instead of the first, so the library does this by reversing the inner query by replacing descending sorts with ascending sorts, then reversing the final result using SELECT * FROM (...) ORDER BY row_number() OVER () DESC. The 2nd reversal cancels out the 1st, but the limit is still backwards.

[dessalines]

Seems good to me, I'd want others to look at it tho too.

[dessalines]

@phiresky when you get a chance, look at this one too.