I had the question come up today in regards to disaster recovery and if Celigo has any built in mechanism's for exporting integrations. The two main use cases were:
- Is it possible to have an integration that runs that will download each integration (essentially like clicking the ... next to an integration and selecting Download integration) on a nightly basis and output to an AWS S3 bucket? This would ensure that we have an up-to-date copy for each integration.
- Is there a way (or an integration.... thinking marketplace) that could run and create snapshots of all integrations on a regular cadence (the end user could determine what cadence)? I think this would be incredibly valuable for many of Celigo's customers!
Thanks in advance!
I just published an integration to the Marketplace that automates both snapshot creation and external backup for disaster recovery.
What it does:
- One flow creates snapshots of all integrations on a defined schedule.
- A second flow downloads the corresponding integration ZIP files and sends them to an external storage system (e.g., S3, FTP, etc.).
A few things to note:
- Integrations with in-progress revisions will be skipped — those need to be completed or canceled before a snapshot can be created.
- The export flow does not include a final destination out of the box — you’ll need to add your own storage connection and ensure the file export step uses
zip.signedURL.blobKey as the blobKey.
You can find this under the name:
Automated integration snapshots & backup to external storage in the Marketplace.
Let me know if you want help getting it set up!
Here is an S3 import as an example:
1 Like
@tylerlamparter - This is absolutely wicked cool. Thank you so much for creating this! I am going to download it and try it out. I would HIGHLY recommend this for all Celigo clients (even if they don't use the export flow) I think having the automation to create a snapshot (instead of relying on the end user to remember) is absolutely critical. Thanks again!
No problem! We have been discussing having a native feature for scheduled snapshots, but this got me thinking, where would you want that to be set? As in, would you want to schedule it just at an account level or at an integration level? If it were me, I'd just have it at the account level.
I think what would make sense is to have the capability to schedule each integration individually, BUT also have the option to "Select All" and schedule them to run on the same cadence.
The reason I say this is because some integrations I am making changes to on a daily basis. These are very high touch integrations/flows and I would want to create snapshots of these on a daily or weekly basis. Other integrations I have had up and running for months and never touched, so snapshotting these on a monthly basis would be appropriate, so I think you want the ability to "group", but also the ability to carve off those that may need a higher frequency of snapshotting.
I did go out and look for the tonight and don't see it on my view. Not sure if thats just me, or if it perhaps needs to be re-published?
Disregard. I refreshed and see it now!
1 Like
