Hitachi Vantara Pentaho Community Forums
Results 1 to 4 of 4

Thread: Mismatched data getting loaded into table fields

  1. #1

    Default Mismatched data getting loaded into table fields

    Hello,

    I am relatively new to PDI but somehow struggling, I was able to compete one end to end transformation rather un-successfully.

    I am trying to load data from multiple text files to a table. So, I am getting file names and copying them using "Copy Rows to Result". Then I am picking each file one by one in the next transformation through "Get rows from result" and setting the properties to pick each file one by one by setting this transformation properties by checking the option "execute for every input row" in the Advanced tab.

    I am able to load each file in to the table, however, I noticed that the data is not going into the correct columns despite of mappings in the table output step look correct to me. I am pasting a snapshot of mappings from source and target here. Mappings are automatically picked in the table output step.

    Still when I see the data inserted in the able, I see that lot of values are showing under columns where they don't belong to. Am I missing something to check or making some mistake? Please let me know.

    Name:  ColMapps.png
Views: 39
Size:  14.7 KB

  2. #2
    Join Date
    Jun 2012
    Posts
    5,534

    Default

    You can name fields in a Text-File-Input step, but the fields are always accessed by position, even if your CSV file comes with field names in a header.
    So, if your CSV files don't have the same layout, you can't use the same mapping.
    So long, and thanks for all the fish.

  3. #3

    Default

    Quote Originally Posted by marabu View Post
    You can name fields in a Text-File-Input step, but the fields are always accessed by position, even if your CSV file comes with field names in a header.
    So, if your CSV files don't have the same layout, you can't use the same mapping.
    So, please let me know what to do/how can I achieve the goal?

    Regards,
    Sachin

  4. #4
    Join Date
    Jun 2012
    Posts
    5,534

    Default

    If it's just a small number of known layouts you can get away with preprocessing the files using Select-Values (rename, reorder) thus creating a canonical layout. Else you may be interested in Metadata-Injection. You can find demos of this technique in your samples folder and in this forum. And sometimes you're not alone with your problem.
    So long, and thanks for all the fish.

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •  
Privacy Policy | Legal Notices | Safe Harbor Privacy Policy

Copyright © 2005 - 2019 Hitachi Vantara Corporation. All Rights Reserved.