FAQ: How do I send a series of records in a large CSV file as multiple smaller files via FTP?

[tomsantiago]

In some cases, you may want to break a large file with many records into multiple smaller files during an FTP upload. You can do so by setting the page size to the number of records you'd like to have per file, and clicking Skip aggregation.

In this scenario, you’d like to send records in smaller groups. You have 500 lines from a NetSuite saved search in a CSV file and want to send those as 5 files on FTP with 100 lines each.

1. In your CSV export, set Page size to 100. Page size determines the number of records sent per page, (in this case 100 records per page). The Page size defaults to 20 records per page if left blank.
   

2. Check the Skip aggregation checkbox in your FTP import to create one file for each page.
   

3. To ensure that your files don’t get overwritten when sending via FTP, use a file name with a Unix timestamp of milliseconds. Using a timestamp of seconds is not always granular enough. You could also use SSS. Here’s more information on supported time and date format codes.

For more information, check out our article on using page size when processing large data sets.