Best approach for paginated API requests without page info in response body

iPaaS Core Incoming Configuration
1

I have an API which is returning paginated results by calling/products?page=1 for example. The response body however, is not returning any information about the pagination (if there is anything to paginate).

The response with results looks like:

{
    "Products": [
        {
            "Product": {
                "sku": "abc"
            }
        }
    ]
}

The result of a paginated page without results looks like:

{
    "Products": []
}

What would be considered to be the best approach for these kind of implementations where no pagination information is returned in the API response?

I was thinking of storing a pagination param in Alumio’s storage, and increasing it if the products count is >= 1, and resetting it when the count on the Products response array is 0.

There must be a smoother implementation or blueprint.

2

Hello @daniellefers ,

When creating an incoming one of approach used to dynamic pagination is to:
“Set Response decoder” to “JSON”, set up your your “Pattern to items”, in case for this example bellow we already get a direct array of data so thats why we are using the “@” accessor with “Follow pagination” set to “Increase query parameter”.
In your specific case the “Pattern to items” will be “Products”.
Alumio knows in this way to paginate till either you have no information left in the response or you hit the threshold for “Maximum number of pages to fetch”.
To note: using Response decoder also splits your incoming collection of data into individual tasks.
Here’s an export example:
test-pagination-dynamic_incoming.ndjson (642 Bytes)

Have fun :smiley:

1 Like