Managing multiple records from Lookup step

Integrator.io
1

Currently I am working on a usecase where I am fetching data from small improvements API.

First, I'll be fetching data of each quarterly cycle which has it's own data. Using that cycle-id, I need to fetch all the users' data for that cycle in the lookup step. Currently the API returns all the users' data for that cycle in a single API call as an array of objects. The lookup step is automatically considering each user object as a separate record.

Sample cycle record

{
  id: "cycle-1",
  name: "cycle-name",
  created_at: 2025-01-01T00:00:000Z,
  ...
}

Sample Users data for each cycle

[
  {
     user_name: "user-1",
     user_details: {}
  },
  {
     user_name: "user-2",
     user_details: {}
  },
  ...
]

But when I run the flow, the lookup is only providing the first user object for each cycle-id due to which I'm only getting data of a single user per cycle in the entire flow. For above example, I'm only getting the user-1 in the imported data.

Couldn't figure out how to resolve this. Appreciate any help.

2

Hi @Manjunath_Kilari ,

Did you try mapping the data field to the users_data field in the Results Mapping section?
Could you please share the current Results Mapping configuration you’ve added? Also, can you confirm whether this issue is occurring in the Preview section or during Run Time?
Please note that the Preview section displays only one record in the UI by default.

3

That is exactly what I did. Here is the results mapping configuration

The image shows a user interface for editing results mapping, with 'data' mapped to 'user_data' under the 'Rules' section. (Captioned by AI)
The image shows a user interface for editing results mapping, with 'data' mapped to 'user_data' under the 'Rules' section. (Captioned by AI)1249×422 13.9 KB

It's occuring during runtime, even during Preview but as you said, the perview, by default will show one record.

Even during test runs, I'm only getting the first lookup record per cycle-id.

Also just to be clear, the Sample Users response is the actual HTTP Response from the API. In the lookup Preview, I'm getting each object as a separate record in a page.

4

@Manjunath_Kilari - Can you please share screenshot of the input and output of the results mapping? Please mask any sensitive data

5

Hey @sreevaniamara,

Here is the input and output data of the Results Mapping Preview after Lookup

Input

{
  "data": [
    {
      "id": "xxxxxxxx",
      "cycleId": "xxxxxx",
      "permissions": {
        "canNudgeManager": false,
        "canCreateObjective": false,
        "canNudgeUser": false
      },
      "user": {
        "id": "xxxxxxxxxx",
        "name": "xxxxxx",
      },
      "manager": null,
      "objectives": [
        {
          "id": "xxxxxxxxxx",
          "category": 1,
          "rating": -1,
          "completionPercentage": 0,
          "status": 0,
          "title": "xxxxxxxxxxxx",
          "visibility": "public",
          "weightingPercentage": 5
        },
        {
          "id": "xxxxxxxxxx",
          "category": 3,
          "rating": -1,
          "completionPercentage": 0,
          "icon": "o_target_alt",
          "status": 0,
          "title": "xxxxxxxxxx",
          "visibility": "public",
          "weightingPercentage": 5
        },
        {
          "id": "xxxxxxxxxxx",
          "category": 1,
          "rating": -1,
          "completionPercentage": 0,
          "icon": "o_cog",
          "status": 0,
          "title": "xxxxxxxx",
          "visibility": "public",
          "weightingPercentage": 5
        },
        {
          "id": "xxxxxxxxxxxxxxxxx",
          "category": 3,
          "rating": -1,
          "completionPercentage": 0,
          "icon": "o_dollar",
          "status": 0,
          "title": "xxxxxxxxxxxxx",
          "visibility": "public",
          "weightingPercentage": 5
        }
      ]
    }
  ],
  "errors": [
    {
      "code": "error_code",
      "message": "error message",
      "source": "application"
    }
  ],
  "ignored": false,
  "statusCode": 200,
  "dataURI": ""
}

Output

{
  "record": {
    "cycleId": "xxxxxxxxxxxxxxxxxxxxxxx",
    "cycleName": "2025 Q3 xxxxx",
    "statuses": [
      {
        "value": 0,
        "name": "Open",
        "color": "#eee"
      },
      {
        "value": 101,
        "name": "some_progress",
        "color": "#F62"
      },
      {
        "value": 102,
        "name": "half_done",
        "color": "#FFBB51"
      },
      {
        "value": 103,
        "name": "almost_done"
      },
      {
        "value": 104,
        "name": "completed"
      },
      {
        "value": 105,
        "name": "Greater than 100%"
      },
      {
        "value": 999,
        "name": "Private"
      }
    ],
    "ratings": [
      {
        "code": -1,
        "name": "Not rated"
      },
      {
        "code": 999,
        "name": "Private"
      }
    ],
    "startDate": "xxxxx",
    "endDate": "xxxx",
    "user_data": [
    {
      "id": "xxxxxxxx",
      "cycleId": "xxxxxx",
      "permissions": {
        "canNudgeManager": false,
        "canCreateObjective": false,
        "canNudgeUser": false
      },
      "user": {
        "id": "xxxxxxxxxx",
        "name": "xxxxxx",
      },
      "manager": null,
      "objectives": [
        {
          "id": "xxxxxxxxxx",
          "category": 1,
          "rating": -1,
          "completionPercentage": 0,
          "status": 0,
          "title": "xxxxxxxxxxxx",
          "visibility": "public",
          "weightingPercentage": 5
        },
        {
          "id": "xxxxxxxxxx",
          "category": 3,
          "rating": -1,
          "completionPercentage": 0,
          "icon": "o_target_alt",
          "status": 0,
          "title": "xxxxxxxxxx",
          "visibility": "public",
          "weightingPercentage": 5
        },
        {
          "id": "xxxxxxxxxxx",
          "category": 1,
          "rating": -1,
          "completionPercentage": 0,
          "icon": "o_cog",
          "status": 0,
          "title": "xxxxxxxx",
          "visibility": "public",
          "weightingPercentage": 5
        },
        {
          "id": "xxxxxxxxxxxxxxxxx",
          "category": 3,
          "rating": -1,
          "completionPercentage": 0,
          "icon": "o_dollar",
          "status": 0,
          "title": "xxxxxxxxxxxxx",
          "visibility": "public",
          "weightingPercentage": 5
        }
      ]
    }
  ]
  }
}
6

@Manjunath_Kilari - this seems to be sending the array downstream.
Is there a downstream response mapping hook where this data might be getting transformed?
Could you add a console.log(JSON.stringify(something)) to inspect the input passed to the response mapping hook at runtime?
(You may need to stringify the object if it's not already a string.)
Please check if the full data set is present in the input during runtime.

7

Hey @sreevaniamara, actually I think I figured out the issue. I was using a postResponseMap hook wherein I was only using user_data[0] for writing data to import step. I think that is where I went wrong. I'm currently trying to modify the import mapping and check if that works. I'll reply here if it works.

Thanks for your help, appreciate it!!

8

Glad to know you figured it out! Sounds like you're on the right track - let me know how it goes.

9

Hey @sreevaniamara, yep ran the flow after changing the post response map script and everything seems to be working as expected.

Thanks!!