Read PDFs & sending as Gmail attachments

[tharakaathukorala]

Hi there,

Is it possible to read PDF files from an SFTP and send them as Gmail attachments through Celigo?

If so can you kindly share a documentation with me?

Thanks

[kathyanaqueeman]

Hi @tharakaathukorala,

I think you asked a similar question a few years ago. The Google Mail connector doesn't support sending attachments, but you could do it through a universal HTTP connection. According to Google Mail's REST API, you can upload an attachment as multipart/form-data. Here are some steps you'll need to follow:

1. Create an HTTP connection
2. Google Mail upload attachments REST API instructions
3. Import files into an HTTP destination app
4. Upload a file or record as multipart/form-data

If you have any further questions, feel free to comment here or make a separate post in the community! Thanks!

[kathyanaqueeman]

A note: I've moved your post to the Troubleshoot custom flows section of the Community for more visibility.

[tharakaathukorala]

Thanks Kathyana.

[tylerlamparter]

@tharakaathukorala it's possible, but not easy. There is a template in the marketplace that I created for sending google mail emails with attachments, but it requires you to have the file represented as a base64 encoded string. So the main issue is converting the pdfs you get from your FTP server and converting it the base64 encoded string. We don't currently have a way to convert the blobKey you get into this. You would need to make an intermediate step to convert the blob to base64 encoded string like this: https://connective.celigo.com/t/faq-convert-a-blob-key-to-base64-string-for-import/358 Alternatively, if you have NetSuite, I could create a better method using NetSuite as an intermediary and post it here.

[tharakaathukorala]

Hi Tyler,

Appreciate your guidance.

We also already created an integration to send base64 encoded JSON data as attachments via a Gmail HTTP connector.

import sjcl from 'sjcl';
base64Encode: (str) => {
        return sjcl.codec.base64.fromBits(sjcl.codec.utf8String.toBits(str));
    }

Now we are stuck on sending PDFs through Gmail. According to my understanding, the Celigo does not support converting PDFs to base64 string.

I will further dig into your solution.

Thanks again.

[Sriram_Ramji]

Hi Tharaka, I noticed that you previously faced a similar issue when sending a PDF via email. I’m currently experiencing a related issue while sending an EDI file as an email attachment. Have you found any solution that might help?

[tylerlamparter]

What does your current flow look like? Are you parsing the edi file or just getting the file blob key for it?

[tharakaathukorala]

Hi experts,

The following steps were taken to overcome this issue.

Problem definition

Read the files and email them as attachments using Gmail.

Approach used to overcome the problem

• Read files from Celigo. (i.e SFTP)

• Do not parse the files. The files should simply be transferred raw as-is (i.e. PDFs, JPGs, etc.)

• Convert the binary file using AWS (Lambda function) into a base64 string. Write a simple lambda function to convert any binary file into a base64 string with your preferred language.

1. Create a connection for AWS within Celigo. And follow the configurations below.
   1. HTTP request => POST
   2. Sample URI => https://*******.lambda-url.eu-north-1.on.aws/api/v1/bin2base64
   3. Override request media type => Multipart / form-data
   4. HTTP request body => [
      {
      "name": "file",
      "filename": "{{record.fileMeta.fileName}}",
      "type": "attachment",
      "mime-headers": {},
      "value": "{{blob}}"
      }]
   5. Add the Blob key path => blobKey

• Get the base64 string for your binary file and merge it into the response mapping.

• Send the base64 string as attachments in Gmail.

1. Create a universal HTTP connection in Celigo for Gmail.
   1. Base URI => https://www.googleapis.com
   2. Auth type => Auth2.0
      1. Grant type => Authorization code
      2. Client ID => Add the client ID from the Google console.
         1. Ref - Learn about authentication and authorization  |  Google Workspace  |  Google for Developers , Gmail API Overview  |  Google for Developers , https://docs.celigo.com/hc/en-us/articles/21552253434523-Available-Google-Mail-APIs#UUID-129dc558-fd2d-1f17-7b44-d0defd1fee55
      3. Add the Client secret
      4. Send client credentials via => HTTP Body
      5. Authorization URL => https://accounts.google.com/o/oauth2/auth
      6. Access token URL => https://oauth2.googleapis.com/token
      7. Valid domain names => googleapis.com,accounts.google.com,oauth2.googleapis.com,google.com
      8. Send token => Authorization HTTP Bearer
   3. Scope => https://www.googleapis.com/auth/gmail.send
   4. Save & Authorize.
2. Use the HTTP Gmail connection to send the attachments.
   1. HTTP method => POST
   2. Request URL => https://www.googleapis.com/gmail/v1/users/me/messages/send
   3. HTTP headers => Content-Type: message/rfc822
   4. HTTP request body => {
      "raw": "{{data.[0].base64ConvertedEmailStr}}"
      }

Refer to the below sample block of code to get the base64ConvertedEmailStr.

import sjcl from 'sjcl';

functionName_PreMap: (options) => {
    let PDFData = options.data;
    const toEmails = options.settings.email_recipents;
    const fromEmail = 'from email address';
    const emailSubject = "Mail Subject";
    let emailTemplate = `<div style="background-color: #f9f9f9; padding: 20px 40px; margin: 0 100px">
<p> <span style="font-weight: 400;">Hello,</span> </p>
<p> <span style="font-weight: 400;">Your Message</span> </p>`;

    PDFData.forEach((file) => {
        let fileName = file.fileMeta.fileName;
    });

    emailTemplate += `<p> <span style="font-weight: 400;">This is an automated notification.</span> </p>
<p>Thank you.</p></div>`;

    let fileContentString = PDFData.map((content, index) => {
        return `Content-Type: application/pdf; charset="UTF-8"

Content-Transfer-Encoding: base64
MIME-Version: 1.0
Content-Disposition: attachment; filename="${content.fileMeta.fileName}"
Content-ID: <content_${index + 1}>

${content.data.base64}\n`;  // Assuming content is base64 encoded accordingly
}).join('--celigo_email_boundary\n');

    let emailStr = `Content-Type: multipart/mixed; boundary=celigo_email_boundary

MIME-Version: 1.0
to: ${toEmails}
from: ${fromEmail}
subject: ${emailSubject}

--celigo_email_boundary
Content-Type: text/html; charset="UTF-8"
MIME-Version: 1.0

${emailTemplate}

--celigo_email_boundary
${fileContentString}
--celigo_email_boundary--`;

    emailStr = Util.base64Encode(emailStr);
    if (emailStr.endsWith('=')) {
        emailStr = emailStr.replace(/=+$/, '');
    }

    options.data[0].base64ConvertedEmailStr = emailStr;

    return options.data.map((d) => {

        if (d.base64ConvertedEmailStr) {
            return {
                data: d
            }
        } else {
            return {
                data: null
            }
        }
    });
}

const Util = {
    base64Encode: (str) => {
        return sjcl.codec.base64.fromBits(sjcl.codec.utf8String.toBits(str));
    }
}

[tylerlamparter]

You no longer need a Lambda function and can use this:

FAQ: Convert a blob key to base64 string for import?

I built a different way to do this which is all in Celigo. If you go to the marketplace, you can find it under "Blobkey Conversion Helper". [The image shows a software interface with a list of integration apps and templates, featuring the 'Blobkey Conversion Helper' within the Celigo Marketplace. (Captioned by AI)] With this being said, you normally do not need to use this. Normally, if you need to send base64 to your endpoint, you'd do something like this if you already have a blobKey: If …

[tharakaathukorala]

Thank you very much, Tyler. That’s awesome.
We built the AWS solution a couple of years ago.