Group Multiple Records

Integrator.io
, ,
1

I'm sure there is way to do this (I assume in transformation 2.0), but I cant seem to figure it out. I would like to group these two (or more) records together in one object array. The issue is when I bring it up in 2.0 it only gives me the first record to work with.

Can someone point me to where to go to group multiple records together that come out of SF like below?

{
  "page_of_records": [
    {
      "record": {
        "attributes": {
          "type": "IntPS__Eligibility__c",
          "url": "/services/data/v54.0/sobjects/IntPS__Eligibility__c/a0ATV000000Gcbz2AC"
        },
        "Id": "a0ATV000000Gcbz2AC",
        "Name": "1232641-01 ClearShare 2500 2024 (01/01/24-)",
        "IntPS__Begin_Date__c": "2024-01-01",
        "IntPS__Term_Date__c": null,
        "Health_Plan_Id__c": "a0FTV000001EmNZ",
        "IntPS__Plan_Type__c": "Healthshare"
      }
    },
    {
      "record": {
        "attributes": {
          "type": "IntPS__Eligibility__c",
          "url": "/services/data/v54.0/sobjects/IntPS__Eligibility__c/a0ATV000000Gdg22AC"
        },
        "Id": "a0ATV000000Gdg22AC",
        "Name": "1232641-01 eXp Advanced 2024 (01/01/24-)",
        "IntPS__Begin_Date__c": "2024-01-01",
        "IntPS__Term_Date__c": null,
        "Health_Plan_Id__c": "a0FTV000001EmaT",
        "IntPS__Plan_Type__c": "MEC"
      }
    }
  ]
}
2

@daveguderian you would need to use a preSavePage script because the Salesforce export does not have a group by feature just yet (like S3, FTP, HTTP connectors have). However, I'll caution it may not work 100% of the time and to avoid the chance of it not working sometimes, you will need to set a high page size on your export. The reason for this is that if you have 2 records that need to group together, but 1 record gets on page 1 and the other record gets on page 2, you won't be able to group them because the preSavePage script runs on a single page of data. As usual now, I use Celigo AI to build the script for me. Here is the output, script itself, and prompt used to create it.

5301043caa20740200000001-d816b3f503784f819a702f1dff5e6d4b.png
5301043caa20740200000001-d816b3f503784f819a702f1dff5e6d4b.png1652×1030 98.2 KB

Script:

function preSavePage(options) {
  // sample code that simply passes on what has been exported
  let groupedData = {};
  options.data.forEach(record => {
    const name = record.Name;
    if (!groupedData[name]) {
      groupedData[name] = {
        Name: name,
        plans: []
      };
    }
    groupedData[name].plans.push(record);
  });

  return {
    data: Object.values(groupedData),
    errors: options.errors,
    abort: false,
    newErrorsAndRetryData: []
  };
}

Prompt:

Can you group objects in the data array together that have the same name? The output per group should look like this: {Name: value, plans:[{},{},...]}

3

@daveguderian if you want a 100% fullproof way, then you can export the data to a file provider, such as S3, FTP, etc, then have another flow to run after the first flow that picks up that file, groups how you need, then does whatever you want. See S3 export example below:

5301043caa20740200000001-31af692801794f5399d3a12598352af0.png
5301043caa20740200000001-31af692801794f5399d3a12598352af0.png1624×1025 77.8 KB

4

Thank you Tyler. I am going to look at option #2 more this week! Thanks as always for the feedback!

5

Hey Tyler-

I wanted to circle back on this and see if there was anyway now to group the Salesforce records? I believe it was on the roadmap, but dont believe its been released yet.

Do you still recommend the above approach for grouping records out of SF? Below is a sample (note: all data is fake data via chatGPT) of the data I need to group together to be able to eventually import into S3 via one-to-many. This data below is originally returned in a lookup step.

{
  "page_of_records": [
    {
      "record": {
        "attributes": {
          "type": "Contact",
          "url": "/services/data/v54.0/sobjects/Contact/003Hp000030Fxi1IAC"
        },
        "FirstName": "Emma",
        "LastName": "Smith",
        "MiddleName": null,
        "Suffix": null,
        "Birthdate": "1986-02-04",
        "MailingStreet": "789 Maple Ave",
        "MailingCity": "Greenville",
        "MailingState": "Ohio",
        "MailingPostalCode": "44130",
        "Person_Code__c": "01"
      }
    },
    {
      "record": {
        "attributes": {
          "type": "Contact",
          "url": "/services/data/v54.0/sobjects/Contact/003Hp000030Fxi0IAC"
        },
        "FirstName": "Michael",
        "LastName": "Johnson",
        "MiddleName": null,
        "Suffix": null,
        "Birthdate": "2010-10-17",
        "MailingStreet": "789 Maple Ave",
        "MailingCity": "Greenville",
        "MailingState": "Ohio",
        "MailingPostalCode": "44130",
        "Person_Code__c": "03"
      }
    },
    {
      "record": {
        "attributes": {
          "type": "Contact",
          "url": "/services/data/v54.0/sobjects/Contact/003Hp000030FxhzIAC"
        },
        "FirstName": "Sophia",
        "LastName": "Johnson",
        "MiddleName": null,
        "Suffix": null,
        "Birthdate": "2012-04-11",
        "MailingStreet": "789 Maple Ave",
        "MailingCity": "Greenville",
        "MailingState": "Ohio",
        "MailingPostalCode": "44130",
        "Person_Code__c": "04"
      }
    },
    {
      "record": {
        "attributes": {
          "type": "Contact",
          "url": "/services/data/v54.0/sobjects/Contact/003Hp000030FxhyIAC"
        },
        "FirstName": "James",
        "LastName": "Johnson",
        "MiddleName": null,
        "Suffix": null,
        "Birthdate": "2016-12-12",
        "MailingStreet": "789 Maple Ave",
        "MailingCity": "Greenville",
        "MailingState": "Ohio",
        "MailingPostalCode": "44130",
        "Person_Code__c": "05"
      }
    },
    {
      "record": {
        "attributes": {
          "type": "Contact",
          "url": "/services/data/v54.0/sobjects/Contact/003Hp000030FxhxIAC"
        },
        "FirstName": "Liam",
        "LastName": "Johnson",
        "MiddleName": null,
        "Suffix": null,
        "Birthdate": "2021-12-12",
        "MailingStreet": "789 Maple Ave",
        "MailingCity": "Greenville",
        "MailingState": "Ohio",
        "MailingPostalCode": "44130",
        "Person_Code__c": "06"
      }
    }
  ]
}
6

@daveguderian correct the above is still the best option.

7

Hi Tyler, I'm facing the same issue—only one record is visible in the Function Input of hooks, even though there are two records under page_of_records in the lookup.