Under Symptoms of an unresponsive application endpoint, it states:
Processing has halted – Although the flow’s status shows as In Progress, records are not being processed. (Success and Pages counts are not increasing when you refresh the Dashboard.)
This suggests that when a connection goes offline, the affected flow step can remain stuck in an In Progress state instead of failing.
My question is about handling this more gracefully. Suppose I have a flow with a webhook listener as the trigger. In a subsequent step, the flow uses a connection that is currently down. At that point, the step stays In Progress. However, what I would prefer is for that step to explicitly error out, so the flow can continue via branching logic—for example, routing the data to another step that uses a fallback connection.
Is there a way in Celigo to force a failure (or timeout) when a connection is unavailable, so that error handling or conditional branching can take over instead of the flow hanging indefinitely?
A possible workaround is to split the logic into two separate flows.
Flow 1 (ingestion flow)
This flow only receives the incoming data and persists it to a file storage (or similar staging location). Its sole responsibility is to capture and store the payload so the data is not lost, even if the downstream application or connection is unavailable.
Flow 2 (processing flow)
This flow runs on a schedule (for example, every 5 minutes) and reads the stored data from the file storage. It then processes and sends the data to the destination application. If the destination is temporarily unavailable, the flow can retry on the next scheduled run.
This approach effectively decouples ingestion from processing and allows the system to recover automatically from temporary connection or availability issues without losing data.
For now, flows won't error out because we want to be able to restart them when you bring the connection online again. I could see wanting the option to choose the behavior you want. Then it's up to you to retry the errors after bringing the connection online again, or we'd have to create a separate process to retry errors that were due to an offline connection after we later bring the connection back online.
