Support RecordsQuery pagination in Web5

[LiranCohen]

Add paginationMessageCid to the RecordsQueryResponse object, now when providing a limit, if there are additional results this cursor will be returned to be used in subsequent queries.

Example response with additional results:

{
  status: { code: 200, message: '...' },
  records: [...],
  paginationMessageCid: 'bayfreigu....'
}

Example request for additional results:

const results = await dwn.records.query({
  message: {
    filter: {
      schema: 'foo/bar'
    },
    pagination: { limit: 2, messageCid: 'bayfreigu....' }
  }
});

Added a pagination test, as well as a missing sorts test.

@frankhinek I'm not sure if this is the best place to test these particular paths, input would be appreciated.

[codesandbox]

Review or Edit in CodeSandbox

Open the branch in Web Editor • VS Code • Insiders

Open Preview

[github-actions]

TBDocs Report

✅ No errors or warnings

@web5/api

• Project entry file: packages/api/src/index.ts

---

Updated @ 2023-11-28T23:17:03.775Z - Commit: f51373f

[codecov]

Codecov Report

Merging #268 (5b81224) into main (34b915d) will increase coverage by 0.00%.
The diff coverage is 100.00%.

Additional details and impacted files

@@           Coverage Diff           @@
##             main     #268   +/-   ##
=======================================
  Coverage   91.77%   91.77%           
=======================================
  Files          73       73           
  Lines       15758    15764    +6     
  Branches     1448     1448           
=======================================
+ Hits        14462    14468    +6     
  Misses       1270     1270           
  Partials       26       26           

Components	Coverage Δ
api	96.69% <100.00%> (+<0.01%)	⬆️
common	95.00% <ø> (ø)
credentials	94.32% <ø> (ø)
crypto	100.00% <ø> (ø)
dids	88.75% <ø> (ø)
agent	88.08% <100.00%> (+<0.01%)	⬆️
identity-agent	56.81% <ø> (ø)
proxy-agent	58.43% <ø> (ø)
user-agent	55.22% <ø> (ø)

[frankhinek]

@LiranCohen Can you also update the README files as part of this PR once we settle on the web5.dwn.* API syntax?

• https://github.com/TBD54566975/web5-js/blob/main/README.md
• https://github.com/TBD54566975/web5-js/blob/main/packages/api/README.md

[frankhinek]

@LiranCohen In the hopes of providing the most intuitive developer experience, I was thinking that perhaps we name paginationMessageCid something that alludes to this value being the pagination cursor.

I was reading through the PR that originally introduced the pagination feature. I can understand the reasons for using the messageCid as the token but it seems like providing a more descriptive name to the value returned would have been more intuitive for developers using the feature. Particularly since so many other pagination implementations often return to the value as a token or cursor.

Thoughts?

[LiranCohen]

@frankhinek

I agree I think what we landed on wrt the cursor objects is less than optimal, especially looking at it again after some time.

I'm leaning towards a more generic naming, and we can define the explicitness of the values within docs/spec/comments.

Example:

// Query pagination object in descriptor
export type Pagination = {
  cursor?: string
  limit?: number
};

export type RecordsQueryResponse = {
  status: UnionMessageReply['status'];
  records?: Record[];
  cursor?: string;
};

If we really want to be explicit about the cursor being a messageCid, which could have some value, maybe something like cursorMessageCid?

[frankhinek]

Sounds good. How do we move forward? Is there an update to dwn-sdk-js and then web5-js? Or another approach you prefer?

[LiranCohen]

@frankhinek Originally was thinking of making the change here first to get it in, but then realized that it would be a hassle since the pagination object with messageCid is in the descriptor of the request.

I opened a PR in dwn-sdk-js to get this in and bubble it up to here.

[LiranCohen]

@frankhinek updated this PR and added the missing documentation. Although would love some suggestions on the wording, not my strongest suit.

Also, not sure that I saw a pattern anywhere to document enum values like the DateSort enum we expose for sorting. Any suggestions on where/how to call it out or is it ok to just call out the type name like I did?

[frankhinek]

LGTM 🚀