Error "footer not found in the template." when saving JSON file to FTP

Integrator.io
, , , ,
1

I've been working on a dead-simple JSON template for an FTP import and receiving error:

footer not found in the template.

Export has page size set to 1, import has batch size 1 and skip aggregation set so I know I'll only have a single record running through the template.
Here's the template:

{{#each batch_of_records}}
{{{jsonSerialize this.record}}}
{{/each}}

which in the editor preview (after deleting the 4 extra records IIO ads) yields valid JSON:

{"test":"SO-AU-6"}

But when the flow runs, the "footer not found in the template" error is thrown. I've searched for the error and can't find anything.

Anyone have any ideas?

2

@steveklett you're in luck because I've got the solution for you. Keep everything you have the same, but put this in the "JSON document" field.

{
  {{#each batch_of_records}}
    {{#each record}}{{#unless @first}},
    {{/unless}}"{{@key}}": {{{jsonSerialize this}}}{{/each}}{{/each}}
}

FYI, the "Batch size" field is to "limit the number of files processed in a single batch request" so you could unset what you have here.

3

@tylerlamparter thanks for the reply and working solution! I didn't realize #each applied to an object would enumerate the properties, good to know. With that said, I don't understand why my original template is invalid. As I understand it, I'm iterating over the batch_of_records array and serializing each of it's elements - that should work, no?

Can you elaborate a bit on why it's invalid?

4

@steveklett it looks like we're doing some validation to confirm the output is JSON, but that validation seems to partially occur prior to handlebar evaluation. When I use the below, it works fine and I've guessing it's because I've manually put in starting and ending brackets. I'll check internally for capturing this as a bug or enhancement.

{
  "recordOutput": {{#each batch_of_records}}{{{jsonSerialize this.record}}}{{/each}}
}
5

@tylerlamparter I appreciate the additional reply and information. This seems like a bug then. I thought maybe it was related to the use of jsonSerialize but it's not, even manually defined JSON will throw this runtime error. I have another flow with data that isn't structured the way the output JSON needs to be, so using jsonSerialize isn't an option. I'm now facing a situation where I can't build the JSON file I need to.

For example, this template will throw the misleading "footer not found in the template." error even though it generates completely valid JSON output.

{{#each batch_of_records}} {
  {{#with record}}
    "timestamp": "{{dateFormat}}",
    "poNumber": "{{{poNumber}}}"
  }{{/with}}
{{/each}}

That's a heavily trimmed version of what I really have. I'm wondering what's wrong with my approach that I'm struggling this much to produce a simple JSON file with IIO, I mean this is a dead-simple use case for an integration platform.

...

While we're on the topic, the unexpected requirement that a template must include {{#each batch_of_records}} should be removed, IMO. It probably was meant to be helpful, but it's not. {{#with batch_of_records.0.record}} should be allowed and is self documenting that the developer intends to work with a single record only, but this isn't possible; you MUST have {{#each batch_of_records}} in your template (as I'm sure you know). My $0.02...

6

Ran into this as well when trying to write json or xml from a variable as-is into a file upload. My trick to fool the validation is to put a non-existent handlebar behind the {{/each}}:

{{#each batch_of_records}}
{{{jsonSerialize this.record}}}
{{/each}}{{myfakefooter}}

7

Hi @steveklett

In general we need header and footer. In this case, you are having a single record. But in general, a page will have multiple records. So, while we iterate over each batch_of_records for each record, we need some header and footer outside of the {{each}} template to properly form the document. That is why we are having this validation.

Regarding second one, you can use the below JSON-
{ 
{{#each batch_of_records}} 
{{#with record}} 
"timestamp": "{{dateFormat}}", 
"poNumber": "{{{poNumber}}}" 
{{/with}} 
{{/each}} 
}
8

One more item I believe... you will need commas between the output records. I asked Celigo AI to help me with that one and you'll see that it added the "unless" helper function. It also made a couple of other changes that I'm not sure if you need in your context.

{
  "batch_of_records": [
    {{#each batch_of_records}}
    {
      {{#with record}}
      "timestamp": "{{dateFormat}}",
      "poNumber": "{{{poNumber}}}"
      {{/with}}
    }{{#unless @last}},{{/unless}}
    {{/each}}
  ]
9

@basvanditzhuijzen good one and saved the day for me, thank you!

10

@layakodam I appreciate the reply and explanation. I would agree that if there were > 1 records you would need this header/footer or outer container present. I think I've illustrated the point that this behind-the-scenes validation is problematic. If I create a template that generates valid JSON (or XML or ANYTHING) for my use case, that should be the end of it. Additional validation may have value in some cases, but as we can see it also can greatly get in the way.

Hopefully this thread will make it into a ticket or discussion on your side for possible feature improvement. Your solution of defining the object structure outside the #each statement and Bas's suggestion for a dummy footer tag get's me unblocked, so I (and my client) thank you both. :)

BTW, how did you do the inline code style in your post?

11

@tonycurcio Thanks for the contribution, but in my case that's not needed. Booo "AI", no more "AI"!!
(half kidding, but only half)

12

@steveklett, @layakodam shared her secret and it's writing your message in Slack and then copying it over to Zendesk. Something like this. I've been trying different options to write directly in Zendesk, but haven't figured it out. Behind the scenes, Zendesk is storing this as like.

13

Nice sleuthing. I also tried different markdown variants but failed. Shame, inline code styling can make things so much easier to read.