How to wait on a flow step until all records are processed

[tylerlamparter]

@Yonton_Mehler asked a great question:

“How can I have a flow that takes tens of thousands of records, imports them somewhere like a database or file, then waits on that step until everything is processed before moving on?”

We talked through a couple options, and I shared one approach — but I couldn’t resist actually testing it. Here’s what I came up with.

---

Why this matters

This is really useful if you need to insert a large batch of records and then run an action after all of those records are done.

The way I’ve normally handled this is by creating two flows:

1. One flow inserts records into a staging table.
2. That flow then triggers a second flow that merges staging into production.

That approach is straightforward and honestly easier to manage. But I wanted to see if it could be done inside a single flow. The trick is using the async helper.

---

Step 1: Get your records

Start with your source. In my test, I used a dummy source, but it could be Salesforce opportunities, NetSuite sales orders, or whatever you’re working with.

On this step, add a preSavePage script to capture pageIndex and recordIndex.

• Later, you can filter on pageIndex = 0 AND recordIndex = 0 to make a downstream step only run once (since only the very first record passes through).
• Heads up: if your flow branches, pages can split and arrive in a different order, so this trick may not work further downstream.

---

Step 2: Insert into your target system (the key step)

This is the step where you want the flow to wait before moving on.

Each flow step creates a child job. You need that child job ID so you can check its status later.

• Use a postResponseMap script on this step to grab the child job ID.
• Add that ID back into your data so the next step knows about it.

---

Step 3: Add a “waiter” step

Now it’s time to pause until everything finishes.

Use the async helper with the child job ID:

• It checks the job’s status.
• When it’s complete, the flow continues to the next step.

You can repeat this pattern multiple times if you want to add more “wait points” in your flow.

---

Script

function pageIndexAdder (options) {
  
  for (let [index,d] of options.data.entries()) {
    d.pageIndex = options.pageIndex;
    d.recordIndex = index;
  }
  
  return {
    data: options.data,
    errors: options.errors,
    abort: false,
    newErrorsAndRetryData: []
  }
}


function jobIdAdder (options) {
  
  for (let d of options.postResponseMapData) {
    d.jobIdToHold = options.job._id;
  }
  
  return options.postResponseMapData
}

---

Screenshots

{
  "status": "in_progress",
  "jobIdToHold": "{{batch_of_records.0.record.jobIdToHold}}",
  "batchSize": {{batch_of_records.length}},
  "results": [
    {{#each batch_of_records}}{"index": {{@index}} }{{^if @last}},{{/if}}{{/each}}
  ]
}

{
  "results": [
    {{#each data.results}}{"index": {{@index}} }{{^if @last}},{{/if}}{{/each}}
  ]
}

---

Flow ZIP

68b09bcce744dace43f3c532.zip (8.9 KB)