Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

INT-18028 Cursor-based pagination fetches first page after fetching last #506

Merged
merged 1 commit into from
Aug 30, 2023
Merged

INT-18028 Cursor-based pagination fetches first page after fetching last #506

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
54 changes: 16 additions & 38 deletions docs/_reference/cli-docs.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3441,62 +3441,40 @@ const perform = async (z, bundle) => {
If your API uses cursor-based paging instead of an offset, you can use `z.cursor.get` and `z.cursor.set`:

```js
// the perform method of our trigger
// ensure operation.canPaginate is true!

const performWithoutAsync = (z, bundle) => {
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Dropped the promise-based example as we don't do this elsewhere and it only makes it all seem more complex than it is.

return Promise.resolve()
.then(() => {
if (bundle.meta.page === 0) {
// first page, no need to fetch a cursor
return Promise.resolve();
} else {
return z.cursor.get(); // Promise<string | null>
}
})
.then((cursor) => {
return z.request(
'https://5ae7ad3547436a00143e104d.mockapi.io/api/recipes',
{
params: { cursor }, // if cursor is null, it's ignored here
}
);
})
.then((response) => {
// need to save the cursor and return a promise, but also need to pass the data along
return Promise.all([response.items, z.cursor.set(response.nextPage)]);
})
.then(([items /* null */]) => {
return items;
});
};
const perform = async (z, bundle) => {
let cursor;

// ---------------------------------------------------
// if fetching a page other than the first (first page is 0),
// get the cursor stored after fetching the previous page.
if (bundle.meta.page > 0) {
cursor = await z.cursor.get();

const performWithAsync = async (z, bundle) => {
let cursor;
if (bundle.meta.page) {
cursor = await z.cursor.get(); // string | null
// if the previous page was the last one and cursor is empty/null,
// return an empty array.
if (!cursor) {
return [];
}
Comment on lines +3452 to +3456
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The fix

}

const response = await z.request(
'https://5ae7ad3547436a00143e104d.mockapi.io/api/recipes',
{
// if cursor is null, it's sent as an empty query
// param and should be ignored by the server
// cursor typically is a param to pass along to the next request,
// or the full URL for the next page of items.
params: { cursor },
}
);

// we successfully got page 1, should store the cursor in case the user wants page 2
// after fetching a page, set the returned cursor for the next page,
// or empty/null if this was the last one.
await z.cursor.set(response.nextPage);

return response.items;
};

```

Cursors are stored per-zap and last about an hour. Per the above, make sure to only include the cursor if `bundle.meta.page !== 0`, so you don't accidentally reuse a cursor from a previous poll.
Cursors are stored per-zap and last about an hour. Per the above, make sure to only include the cursor if `bundle.meta.page > 0`, so you don't accidentally reuse a cursor from a previous poll.

Lastly, you need to set `canPaginate` to `true` in your polling definition (per the [schema](https://github.com/zapier/zapier-platform/blob/main/packages/schema/docs/build/schema.md#basicpollingoperationschema)) for the `z.cursor` methods to work as expected.

Expand Down