# How to export HTTP response as individual records **URL:** \ **Category:** Integrator.io\ **Tags:** export, lookup\ **Created:** [August 11, 2025, 5:22pm UTC](https://connective.celigo.com/t/how-to-export-http-response-as-individual-records/5246 "2025-08-11T17:22:07Z")\ **Posts on this page:** 8\ **Page:** 1
**Author:** ![Marco\_Esteban](https://avatars.discourse-cdn.com/v4/letter/m/bc8723/32.png) [@Marco\_Esteban](https://connective.celigo.com/u/Marco_Esteban)\ **Post date:** [August 11, 2025, 5:22pm UTC](https://connective.celigo.com/t/how-to-export-http-response-as-individual-records/5246/1 "2025-08-11T17:22:07Z")
Hi Everyone, I’m using a Lookup step and the HTTP response contains ~2,000+ data. In the run results it shows as a single successful record. **I set the path to records in HTTP response body** and when I preview they show up as individual records. ![image](https://us1.discourse-cdn.com/flex001/uploads/celigo/original/2X/1/16720b7d1b34efc98d8ba86b14cdf469abff9204.png) ![image](https://us1.discourse-cdn.com/flex001/uploads/celigo/original/2X/8/8761b666cdef79ae410a77fe7538cd52c93f1d7a.png) I might be missing a setting/config. But yeah, is there a way to have the lookup export the data as individual records? Thank you in advance! ---
**Author:** ![tylerlamparter](https://sea1.discourse-cdn.com/flex001/user_avatar/connective.celigo.com/tylerlamparter/32/9204_2.png) [@tylerlamparter](https://connective.celigo.com/u/tylerlamparter)\ **Post date:** [August 12, 2025, 3:18am UTC](https://connective.celigo.com/t/how-to-export-http-response-as-individual-records/5246/2 "2025-08-12T03:18:28Z")
The "1" you see there just means that the single source record successfully ran. It doesn't indicate how many results were returned from the lookup response (that's why you see "1" for export and "1" for lookup). In the next NetSuite step, you have 2405 successes, so I assume your response mapped `data` to something and then used a one-to-many setting on the NetSuite step. The one-to-many setting individually processes those records in the specified array. ---
**Author:** ![Marco\_Esteban](https://avatars.discourse-cdn.com/v4/letter/m/bc8723/32.png) [@Marco\_Esteban](https://connective.celigo.com/u/Marco_Esteban)\ **Post date:** [August 12, 2025, 3:39am UTC](https://connective.celigo.com/t/how-to-export-http-response-as-individual-records/5246/3 "2025-08-12T03:39:10Z")
Thanks for the explanation. Yes, I mapped the response data and used it in a one-to-many import. Just to confirm, would the lookup still be safe from the Celigo 5MB limit in this case? ---
**Author:** ![tylerlamparter](https://sea1.discourse-cdn.com/flex001/user_avatar/connective.celigo.com/tylerlamparter/32/9204_2.png) [@tylerlamparter](https://connective.celigo.com/u/tylerlamparter)\ **Post date:** [August 13, 2025, 2:23am UTC](https://connective.celigo.com/t/how-to-export-http-response-as-individual-records/5246/4 "2025-08-13T02:23:49Z")
It would not. The way this is set up could hit the 5 MB page size limit. Essentially, what you’re wanting is to change the context of your flow after the lookup to "re-recordize." By that, I mean you want the results of the lookup to now be the records going forward through the flow. We don’t have this functionality yet, but it’s one of the main 5 MB limitations we are looking to solve soon-ish. To avoid it for now: - Filter out lookup results that you don’t need. - Transform the data to remove fields you don’t need. - Make field names smaller so fewer bytes take up space on the page. If you still have the issue, you could do this: > [@Shopify Payout Transactions: response stream exceeded limit of 5242880 bytes](https://connective.celigo.com/t/shopify-payout-transactions-response-stream-exceeded-limit-of-5242880-bytes/109/2): > > @harishbakka I reviewed the ticket and the meeting recording and have a solution for you. This is actually the solution we use for our own integration apps. To summarize, you have an export that is getting Shopify payouts, then you have a lookup step that is getting the transactions for the exported payout id, and then lastly you are wanting to put each transaction into Google BigQuery. The issue you're running into is a 5 MB page size limit due to having so many transactions for a given payou… ---
**Author:** ![tylerlamparter](https://sea1.discourse-cdn.com/flex001/user_avatar/connective.celigo.com/tylerlamparter/32/9204_2.png) [@tylerlamparter](https://connective.celigo.com/u/tylerlamparter)\ **Post date:** [August 13, 2025, 2:24am UTC](https://connective.celigo.com/t/how-to-export-http-response-as-individual-records/5246/5 "2025-08-13T02:24:38Z")
Could you also share what Walmart apis and reports you are grabbing? I'm wondering if an async helper could be used here which would also work to avoid the 5 MB limit. ---
**Author:** ![Marco\_Esteban](https://avatars.discourse-cdn.com/v4/letter/m/bc8723/32.png) [@Marco\_Esteban](https://connective.celigo.com/u/Marco_Esteban)\ **Post date:** [September 4, 2025, 3:16pm UTC](https://connective.celigo.com/t/how-to-export-http-response-as-individual-records/5246/6 "2025-09-04T15:16:23Z")
Hi @tylerlamparter I think I’m all set with your suggestion to use a second flow with a listener. Would it be possible for the listener to batch the records into 100 before sending them to the next step? I’ve noticed that by default, each page only contains 20 records. I also tried increasing the export page size, but it seems the listener doesn’t follow that setting. Thank you! ---
**Author:** ![tylerlamparter](https://sea1.discourse-cdn.com/flex001/user_avatar/connective.celigo.com/tylerlamparter/32/9204_2.png) [@tylerlamparter](https://connective.celigo.com/u/tylerlamparter)\ **Post date:** [September 4, 2025, 10:23pm UTC](https://connective.celigo.com/t/how-to-export-http-response-as-individual-records/5246/7 "2025-09-04T22:23:59Z")
If you change the page size on the lookup (not export) to 100, then it should then send 100 per webhook call. ---
**Author:** ![Marco\_Esteban](https://avatars.discourse-cdn.com/v4/letter/m/bc8723/32.png) [@Marco\_Esteban](https://connective.celigo.com/u/Marco_Esteban)\ **Post date:** [September 10, 2025, 6:57am UTC](https://connective.celigo.com/t/how-to-export-http-response-as-individual-records/5246/8 "2025-09-10T06:57:45Z")
Cool. This works! Thank you @tylerlamparter