Pagination issue with Amazon Financial Events Flow (bug?)

Integrator.io
1

Amazon Financial Events API Pagination Issue

Problem Description

I'm experiencing pagination issues with the Financial Events Amazon endpoint. When the flow runs, only a couple of pages are queried before the flow stops without any error messages.

Current Configuration

API Endpoint Details

  • Relative URI: /finances/v0/financialEvents?PostedAfter=2025-06-10T00:00:00.000Z&PostedBefore=2025-06-25T00:00:00.000Z
  • Paging Method: Next page token
  • Path to Next Page Token: payload.NextToken
  • URI for Subsequent Pages: /finances/v0/financialEvents?NextToken={{{encodeURI export.http.paging.token}}}&PostedAfter=2025-06-10T00:00:00.000Z&PostedBefore=2025-06-25T00:00:00.000Z
  • Path to Records: payload.FinancialEvents.RefundEventList

Comparison with Working Flow

I have another Amazon financial events flow using identical settings that works perfectly. The only difference is:

  • Working Flow: Path to Records points to payload.FinancialEvents.ShipmentEventList
  • Problematic Flow: Path to Records points to payload.FinancialEvents.RefundEventList

The working flow gathers all pages properly without issues.

API Response Structure

The Amazon endpoint doesn't allow specifying the financial event type as a query parameter - it returns all event types in a single response.

Simplified example of the response structure below:

{
  "payload": {
    "FinancialEvents": {
      "ShipmentEventList": [
        "// Shipment event data here"
      ],
      "RefundEventList": [
        "// Refund event data here"  
      ]
    },
    "NextToken": "string_or_null"
  }
}

Assumption About the Root Cause

Since ShipmentEventList events are more common than RefundEventList events, there may be pages where RefundEventList is empty or blank.

My hypothesis: The flow may be prematurely ending because it incorrectly assumes there are no more pages when RefundEventList is empty, even though a NextToken is still present in the response.

Can anyone confirm or deny this is the case? Is there a solution to this?

2

This does seem like a bug, and this is a great write-up. To confirm, can you enable debug logs on the export, run the flow, and then see if the last API call is the one where no results are found in the path?

3

Just tested it now. Yes, it seems that the last call is the one with the empty RefundEventList. Is there any type of workaround I can do in the meantime?

The image shows a debug log viewer for Amazon export with a list of HTTP requests, response codes, and a highlighted section in the detailed HTTP response body. (Captioned by AI)
The image shows a debug log viewer for Amazon export with a list of HTTP requests, response codes, and a highlighted section in the detailed HTTP response body. (Captioned by AI)900×418 57.2 KB

4

Yeah, don’t set a path to records and then use a preSavePage script on the export to essentially do the same thing.

function preSavePage (options) {
  
  let output = [];
  for (let d of options.data) {
    if (d.payload && d.payload.FinancialEvents && d.payload.FinancialEvents.RefundEventList && d.payload.FinancialEvents.RefundEventList.length > 0) {
      for (let r of d.payload.FinancialEvents.RefundEventList) {
        output.push(r);
      }
    }
  }
  
  return {
    data: output,
    errors: options.errors,
    abort: false,
    newErrorsAndRetryData: []
  }
}
1 Like
5

I also met with engineering, and we identified another workaround. If you update the export doc via API, you can add "continueOnEmptyResource": true under the paging object. This will let us know to keep going even if we get no result returned from the path. We will see about bringing this field to the UI.

"paging": {
  "method": "token",
  "path": "payload.NextToken",
  "pathLocation": "body",
  "continueOnEmptyResource": true
}
1 Like