We have implemented a transformation step in our data load taking data and grouping it by the external ID. However, when it is getting to the transformation step. The step is reading the File but taking the Last External Id and assigning it to all the records and it is failing. When we did a smaller subset, it worked as expected however when there is more than one record with an external Id we see this strange process.
Grouping from Mock output
Transformation
In your transformation, your external ID field is referenced by *.[External Id]. You should hardcode the position you want returned like 0.[External Id] if you want the first one. Can you also add a sort by external id within the grouping section to ensure all rows are pre-sorted before going into the group by?
Tyler ,
Thank you for responding to the Issue. We had to delete the Grouping by External Id and then readd it and it is working as expected. THank you
Lance
Sounds good. I would make sure everything is sorted as well because the group by function alone assumes everything is pre-sorted based on the field you want to group on. If you have value abc, then xyz, then abc again, you would end up with two records for abc because your data isn't sorted.
