Amazon sqs / s3

Integrator.io
, ,
1

Hello , i am trying to post an xml doc via s3 (basic works).... but use a controlled filename and pass it to a next step (AWS-SQS) whicj in turn will publish a message with the file name.

Two questions:

1: has anyone been able to get aws-SQS to work via Celigo? if so how?

2: has anyone been able o pass the file name used by the S3 Step on to the next step?

2

Hi @anthonyrosenkrant627,
Thanks for letting us know about your integration scenario needs! We're looking into your first question and will get back to you shortly. For your second question, you should be able to do that. I'm checking with our PM team on this. If you could provide any more detail on this one, that would be very helpful!

3

@anthonyrosenkrant627-Anthony-Rosenkrantz">@anthonyrosenkrant627

When you create an S3 export step and choose "No" for Parse files being transferred, then in the Advanced section, you can enable the "File metadata only" setting. You'll get the file name and file last modified date.

Thanks

5301043caa20740200000001-94bfe2b50e4b4e4bac7c6264aa45cd2e.png
5301043caa20740200000001-94bfe2b50e4b4e4bac7c6264aa45cd2e.png398×506 17.5 KB

4

Hi @anthonyrosenkrant627, let us know if the solution Vili provided above doesn't work! Thanks and good luck!

5

I am trying todo to an S3 Import , and do not see those options

5301043caa20740200000001-688bad16dfd64032b2630498d3371671.jpg
5301043caa20740200000001-688bad16dfd64032b2630498d3371671.jpg2004×1253 359 KB

6

@anthonyrosenkrant627 Could you please elaborate about "passing the file name used in S3", is it about getting the file name after creating the file in S3?

Thanks

7

yes i would like to pass the "file-key{documentname.xml}" that is dynamically created on to the next step which needs to be an sqs step, as i am required to send an sqs message with the file name. (in our case document-name = item-name.xml).

core process flow

1: extract all items from netsuite (works)

2: create a new file for each item and post to s3. document-name = item-name.xml. (basics work) but i am unable to get the dynamically created file-key/file-name so i can pass it on to the next step

3: post a message to sqs for every file created within s3....

issues:

1:Ping works and post to SQS..... (good)

2: unable to access the dynamically created filename created/used in the previous step

3: unable to get restapi to talk to SQS various issues (weird as the ping works)

8

@anthonyrosenkrant627

The information you need is available inside a _json variable which is the complete response data from the import application (Check here for more info on responseData (see this article for more info). Here is an example of the responseData from my S3 test:

{
   "statusCode":200,
   "id":"5b2969b3c0b14e8581d04da63424b13b",
   "_json":{
        "name":"file-2020-08-11T20-14-06"
      },
   "errors":[
     ]
}

Do a response mapping in your S3 import step like so:

5301043caa20740200000001-32701eda001441e592e63c20457295f4.png
5301043caa20740200000001-32701eda001441e592e63c20457295f4.png868×134 9.06 KB

Then, in the next flow step field mapping, you can refer to that dynamically generated file name variable as "_json.name".

5301043caa20740200000001-165047a4a86e425fa6596a603edc836d.png
5301043caa20740200000001-165047a4a86e425fa6596a603edc836d.png669×137 9.42 KB

Thanks

9

Perfect thank you, Got it.

Now all that is left is trying to get AMAZON SQS to work. DO you have any information on SQS?

10

Hi @anthonyrosenkrant627,

Glad to hear that worked for you. We don't have any documentation in our Help Center on SQS, as it's not an application we explicitly support via an SQS application connector (adaptor). What information are you looking for on SQS?

11

i need to post a message to AWS-SQS .... and i am trying to do it via RESTAPI? what is interesting the connection-ping will connect and post a message but when i try via it restapi i get the error "The request signature we calculated does not match the signature you provided"

12

Hi @anthonyrosenkrant627,

  • Please ensure that your Connection base URI has a region name in it. (e.g: sqs.us-west-2.amazonaws.com). With a base URI like sqs..amazonaws.com integrator.io invokes special logic regarding the HMAC signature that this service needs.
  • integrator.io does generate a signature for every request, but it's done in the background. Unlike Postman, integrator.io doesn't take input from the user for signature calculation, but rather depends on the baseURI to identify the service you are connecting to (in this case, SQS) and calculates the signature accordingly.

What is the REST call (POST?) you are making? Can you share details (screenshots) of that and the Connection settings, please?

13

@anthonyrosenkrant627 -- I edited @courtneyjordan's earlier response, so just tagging you here explicitly to be sure you're notified.

Thanks,
Matt

14

Morning Jordan yes i am using Post, unfortunately i am still getting signature issues, is it recalculating the signature based on the relative url?

5301043caa20740200000001-b298c0d86d424569bd2ea5425673286c.png
5301043caa20740200000001-b298c0d86d424569bd2ea5425673286c.png2075×1253 354 KB

15

Hi @anthonyrosenkrant627,

Could you also go to your AWS-SQS connection and take a screenshot of that for us to see connection details? You can click into your integration tile to the Integration details page, then click Connections to see all connections used by that integration. In the Actions ... column, select Edit connection.

16

@anthonyrosenkrant627

Hi Anthony

I am facing the exact same issue and would be very helpful to know whether you managed to get this resolved?

Thanks

17

Hi Anthony,

Were you able to connect to SQS?

Thanks