Currently, I have a flow that exports sales order data from a NetSuite saved search and sends the output to an SFTP location in a CSV. I then have the flow update a flag on that sales order record in NS to indicate the record has been sent to the SFTP location (so I don’t transfer the same record twice).
My issue is that the entire CSV transfers before any of the flags are updated. If, the csv uploads successfully but the update flag step fails, I will unintentionally send those sales order records twice.
I’m trying to find a solution where the SFTP upload and flag update steps either both only succeed or only fail–not one failing and one succeeding.
Does anyone know if this is possible?
Okay, obvious first question. Why would the update flag fail?
Second Question: If the update flag is an issue, and the CSV flag is a guaranteed winner, then why not switch the steps, pull the records, mark them pulled, and THEN write the lines in the CSV file? You could even indicate the lines to write based on the response from the update.
Thanks for the response, Todd.
First question: I can’t say for sure because it hasnt happened in a little bit (thankfully) but I think what happened was the update flag step timed out.
Second question: Unfortunately, the CSV upload is not guaranteed to succeed.
Because neither is guaranteed, I was hoping I could run them like a database transaction and only commit either step based on the successful completion of both steps. And rollback either step if either fails. Any idea if something like that is possible?
If you put a step in-between the update of the record, using the output of the update record (or it's absence) to build the output that would end up in the CSV file, the CSV file would only contain the ones you KNOW you updated, then you could push the CSV file, regardless of time out errors, when it did finally go, you would only have the updated records.
It would work "sort of" like that. If the records fail to update, branch it to another flow, if succeeds, add it to the CSV list. On the alternate branch, provide yourself a report of all the ones that failed, or at least run them through a script that writes that information to the console.log so you can look it up later.
I would probably use the export type = once on the export, then upload the CSV file, then enable 'proceed regardless of error' for that step, then branch off where one branch is the happy path and the other is the error path. On the happy path, you do nothing since the record was already marked as exported from the beginning. On the unhappy path, I would unmark the record as exported so it gets picked up in the next run. Doing this would mean you only need to manage errors for when you couldn't uncheck the box back in NetSuite, but I'm not really sure how often that would realistically occur.
1 Like
Actually, I like this better. It’s done if it’s done, if there is an issue, then you address it.