Getting duplicate item error for a non-existing item – Possible Race Condition?

Integrator.io
,
1

Hi,

I have a flow that processes orders received via a SAP Ariba listener. As part of the flow, new items are created in NetSuite if they do not exist yet.
The flow is an import step with a Path to Many on the items.

Now sometimes I get the following error:

Failed to save record because Uniqueness error - there is already an item with that name or name/parent combination.

This is what I observed:

  • The Celigo logs show that multiple items are being created at the same time (since an order contains multiple line items).
  • The NetSuite Lot Auto Numbering SuiteApp is enabled, which auto-generates item IDs.
  • In the Create NetSuite Item flowstep, I am not mapping the itemid/itemname, so these fields are automatically generated.
  • If I retry the error, the item is created successfully, which suggests a race condition issue.

I have not enabled auto-recover on rate limit errors, as this could slow down the flow. Would setting the batch size limit to 1 in the flowstep prevent this issue? And are there better alternatives to handle this concurrency issue?

3

I would try:

  1. Set your trace key on the export. This trace key is used to determine concurrency locking and auto resolving of errors.

    The image shows an advanced settings form with fields like "Page size," "Data URI template," and "Override trace key template," highlighted with a red arrow pointing to "{{{record._id}}}". (Captioned by AI)
    The image shows an advanced settings form with fields like "Page size," "Data URI template," and "Override trace key template," highlighted with a red arrow pointing to "{{{record._id}}}". (Captioned by AI)1148×465 25.3 KB

  2. Set override child record trace key template on import settings (if you do this you actually shouldn't need the concurrency id lock override).

    The image shows a NetSuite import editing interface with fields for operation type, existing record search, and advanced settings including Concurrency ID lock template, highlighted by red arrows. (Captioned by AI)
    The image shows a NetSuite import editing interface with fields for operation type, existing record search, and advanced settings including Concurrency ID lock template, highlighted by red arrows. (Captioned by AI)787×1096 58.9 KB

1 Like
4

Thanks, @tylerlamparter for your quick responses, you are a big MVP for Celigo :slight_smile:

To summarize and check if I understand it correctly:

In the export, I will set up a trace key to identify each record and prevent it from being processed twice. For this, I will use a Handlebars expression like {{{record._id}}} to uniquely identify the "order" record. This helps with concurrency locking.

The second step is using the override child record trace key template, which ensures that the child records (in this case, the items) are properly linked to the parent and handled uniquely.

However, the items in the order are already unique and are processed using a path-to-many setup. If I don’t use the override, does this then mean that Celigo send the objects as a whole under the hood, just to get my understanding correct.

5

Sorry, let me correct myself here. In your case, you would want to use the "Concurrency ID lock template" because you would want it to be set to "sku" or whatever unique field identifies an item. When you use this, we would then check for other records with the same concurrency ID and not send those records in the same request—essentially, we will process it serially. Now, this also only works across a page of data and does not work between pages of data. So if you have a high concurrency set on your NetSuite import and that import is receiving multiple pages of data at once and items are duplicated across pages, you could hit this issue. To solve for this, you probably also want to set the concurrency on your NetSuite import connection to 1.