Pagination is not updating parameters

[Valdri]

Describe the bug

When running http with pagination, it does not update the parameters in the next run

my body parameters:

Pagination settings:

On the first request its sending parameters:
{ "date_from": 1, "get_unconfirmed_orders": true, "include_custom_extra_fields": true }
In the 2nd request, pagination should update the parameters to:
{ "date_from": 2, "get_unconfirmed_orders": true, "include_custom_extra_fields": true }

But it doesn't do that, it just sends a test without updating the parameters

1 request:

2 request:

Workflow file:

pagination_test.json

• n8n Version 1.25.1

[Joffcom]

Hey @Valdri,

Good spot, It looks like when making a post request the pagination is not functioning correctly. To be honest it isn't something I had thought about testing either as it is not often you see pagination being needed for post requests.

Do you have a link to the API documentation for what you are working with so we can test against that while fixing this?

[Valdri]

Hi @Joffcom,
I'm using getOrders, but you must have an account to use it.

If it helps, I can run a second instance of dev n8n in docker to test it.

[ihortom]

@Joffcom , it is not just POST request. The pagination got broken for me when upgraded from 1.21.1 to 1.24.1. It doesn't seem to work with GET as well and with different options - "Update parameter in Each Request" and "Response Contains Next URL". I have a few examples with different APIs and different pagination options if you need that.

[Joffcom]

@ihortom if you can share those it would be great we put a fix in 1.22.6 that fixed an issue with pagination that I have tested against 4 different APIs that are working.

I will run my test workflow this morning to make sure all is still good on the get requests

[Joffcom]

Have ran my test workflow on 1.25.0

✅ GitHub - Update Parameter in each request - Query > Page > {{ $pageCount +1}} | Complete when response is empty

✅ Mailchimp - Update a Parameter in Each Request - Query > Offset > {{ $response.body.lists.length * $pageCount }} | Complete when other > {{ $response.body.lists.length === 0 }}

✅ PokeAPI - Response contains next url > {{ $response.body.next }} | Complete when other > {{ $response.body.next === null}}

✅ Hubspot - Response Next URL > {{ $response.body.paging.next.link }} | Complete
when other > {{ !$response.body.paging }} | Interval between requests 1000

From my testing it looks good on recent versions when making GET requests. My test workflow can be found below, If you have information on an API that isn't working with the GET request @ihortom if you can provide the API docs as long as they offer trial accounts of some kind I can give them a test.

{
  "meta": {
    "templateCredsSetupCompleted": true,
    "instanceId": "8c8c5237b8e37b006a7adce87f4369350c58e41f3ca9de16196d3197f69eabcd"
  },
  "nodes": [
    {
      "parameters": {},
      "id": "36ee1cd7-a3c4-4c75-8ad1-5d17d63c2b7a",
      "name": "When clicking \"Test Workflow\"",
      "type": "n8n-nodes-base.manualTrigger",
      "typeVersion": 1,
      "position": [
        680,
        420
      ]
    },
    {
      "parameters": {
        "url": "https://pokeapi.co/api/v2/pokemon",
        "sendQuery": true,
        "queryParameters": {
          "parameters": [
            {
              "name": "limit",
              "value": "200"
            }
          ]
        },
        "options": {
          "pagination": {
            "pagination": {
              "paginationMode": "responseContainsNextURL",
              "nextURL": "={{ $response.body.next }}",
              "paginationCompleteWhen": "other",
              "completeExpression": "={{ $response.body.next === null}}",
              "limitPagesFetched": true,
              "maxRequests": 10
            }
          }
        }
      },
      "id": "ca4a60d0-7fe2-4f47-883a-60f4e950e8cf",
      "name": "Pagination Next URL",
      "type": "n8n-nodes-base.httpRequest",
      "typeVersion": 4.1,
      "position": [
        940,
        500
      ]
    },
    {
      "parameters": {
        "url": "  https://api.mailchimp.com/3.0/lists/",
        "authentication": "predefinedCredentialType",
        "nodeCredentialType": "mailchimpApi",
        "sendQuery": true,
        "queryParameters": {
          "parameters": [
            {
              "name": "count",
              "value": "1"
            }
          ]
        },
        "options": {
          "pagination": {
            "pagination": {
              "parameters": {
                "parameters": [
                  {
                    "name": "offset",
                    "value": "={{ $response.body.lists.length * $pageCount }}"
                  }
                ]
              },
              "paginationCompleteWhen": "other",
              "completeExpression": "={{ $response.body.lists.length === 0 }}"
            }
          }
        }
      },
      "id": "55bb2ba4-65d7-458d-9fef-7945bbc47d1f",
      "name": "MailChimp",
      "type": "n8n-nodes-base.httpRequest",
      "typeVersion": 4.1,
      "position": [
        940,
        340
      ],
      "credentials": {
        "githubApi": {
          "id": "5d6FIFd32OFSXlw3",
          "name": "GitHub account"
        },
        "mailchimpApi": {
          "id": "OcbI1jH1dbWYrUmv",
          "name": "Mailchimp account"
        }
      }
    },
    {
      "parameters": {
        "url": "https://api.github.com/repos/n8n-io/n8n/issues",
        "authentication": "predefinedCredentialType",
        "nodeCredentialType": "githubApi",
        "options": {
          "pagination": {
            "pagination": {
              "parameters": {
                "parameters": [
                  {
                    "name": "page",
                    "value": "={{ $pageCount +1 }}"
                  }
                ]
              }
            }
          }
        }
      },
      "id": "ac22abaa-c2bc-4e23-883a-8d5813ccfe89",
      "name": "GitHub",
      "type": "n8n-nodes-base.httpRequest",
      "typeVersion": 4.1,
      "position": [
        940,
        160
      ],
      "credentials": {
        "githubApi": {
          "id": "5d6FIFd32OFSXlw3",
          "name": "GitHub account"
        }
      }
    },
    {
      "parameters": {
        "url": "https://api.hubapi.com/crm/v3/objects/companies",
        "authentication": "predefinedCredentialType",
        "nodeCredentialType": "hubspotOAuth2Api",
        "sendQuery": true,
        "queryParameters": {
          "parameters": [
            {
              "name": "limit",
              "value": "2"
            }
          ]
        },
        "options": {
          "pagination": {
            "pagination": {
              "paginationMode": "responseContainsNextURL",
              "nextURL": "={{ $response.body.paging.next.link }}",
              "paginationCompleteWhen": "other",
              "completeExpression": "={{ !$response.body.paging }}",
              "requestInterval": 1000
            }
          }
        }
      },
      "id": "bc4137ba-4527-401b-a6d4-26744b5f6535",
      "name": "HubSpot",
      "type": "n8n-nodes-base.httpRequest",
      "typeVersion": 4.1,
      "position": [
        940,
        660
      ],
      "credentials": {
        "hubspotOAuth2Api": {
          "id": "NR2o8uiltVAGrIc9",
          "name": "HubSpot account"
        }
      }
    }
  ],
  "connections": {
    "When clicking \"Test Workflow\"": {
      "main": [
        [
          {
            "node": "GitHub",
            "type": "main",
            "index": 0
          },
          {
            "node": "MailChimp",
            "type": "main",
            "index": 0
          },
          {
            "node": "HubSpot",
            "type": "main",
            "index": 0
          },
          {
            "node": "Pagination Next URL",
            "type": "main",
            "index": 0
          }
        ]
      ]
    }
  },
  "pinData": {}
}

[Joffcom]

Internal ticket for the POST request issue is NODE-1079

[ihortom]

@Joffcom , I didn't realize that pagination for GET request was fixed in 1.25 as I did not notice any relevant issue addressed in that release. I tested the broken paginations in latest and can confirm they are working again indeed. Thanks!

[Joffcom]

That is good to hear, I think they were actually fixed before that but I would need to check notes to be sure. I have just happy it isn’t broken again 🙂

[malsatin]

Upgrading to 1.25.0 also fixed the bug for us. (Don't remember, which version we had before the update)

[janober]

Fix got released with [email protected]