We are trying to migrate a large amount (6 Million records) of data (Customers, Invoices, etc) from the source system to Netsuite using the Trial version of Celigo. We are unable to see the throughput that we would like. We have tried running in parallel threads and using concurrency. There is not much difference in performance whether we run in a single thread or multiple threads with concurrency. Is there a difference between the Trial version of the software and vs paid version? Does anyone have best practices document that describes how to configure Celigo and Netsuite for optimal data migration performance? Thanks in advance.
@chandrareddy Sorry for the delayed response! I've moved your post from under the University forum to the Platform (integrator.io) forum. When you are using the free trial plan, there's no limitation. For optimal data performance, you can follow guidelines in Fine-tune integrator․io for optimal performance and data throughput. See also, our Netsuite section for any details.
The bottleneck is most likely NetSuite. It only processes about one record per second per thread. More complex records can be slower. If you have lots of customizations, it'll be even slower. I've seen NetSuite required 10 to 20 seconds to save one record.
Each page of data is submitted to NetSuite as a unit. So, you need to be processing multiple pages to get real benefits from increased concurrency.
You should probably consider some CSV imports if you have SuiteCloud plus.