Celigo imports endpoint - Pagination Options for Large Datasets

Integrator.io
, ,
1

Hi,

I'm working with the integrator.io API, specifically the https://api.integrator.io/v1/imports endpoint, to fetch all imports for our client. However, I'm running into an issue when dealing with a large number of imports.

Current Situation:

  • I'm using Postman Flows to fetch and process the data from the /v1/imports endpoint.
  • When there are many imports, the response array is too large for Postman scripting to handle efficiently, resulting in out of memory errors.

My Question: Is there a way to retrieve this data in smaller chunks or use pagination with the /v1/imports endpoint? Specifically, I'm looking for options like:

  • Using query parameters such as 'limit' and 'skip' to paginate through the results. For example:
GET https://api.integrator.io/v1/imports?limit=100&skip=0
GET https://api.integrator.io/v1/imports?limit=100&skip=100

and so on. Or are there maybe any other sorts of pagination that I can use?

Thank you in advance.

2

I also found out that the

https://api.integrator.io/v1/imports

does not return all imports but only a maximum of 1000.

Is there a way to retrieve all imports via the API as well ?

3

@nuriensing for the question in your comment, you can retrieve more than 1000 by using the next page link in the response header. See article here: https://docs.celigo.com/hc/en-us/articles/19141347828763-Manage-pagination-in-a-REST-API-call.

I'm not sure on the first question, but am checking.

4

@nuriensing I confirmed this api currently doesn't support any type of limit parameter.

5

Thanks @tylerlamparter