Hitachi Vantara Pentaho Community Forums
Results 1 to 4 of 4

Thread: Multiple table loads from multiple files

  1. #1
    Join Date
    Dec 2012
    Posts
    29

    Default Multiple table loads from multiple files

    Hi All


    I want to take many files and load data into the corresponding tables.The filename and table name are same .The text file and corresponding table will be the same and will have exact nof columns.


    The file is a text file seperated by " | ".
    I am using text file input to take files from the folder.These all will work.
    But the problem is i couldnt give the field name in the field tab because the text files contains different fields in each file.
    So when i am previewing, i am not able to see tha data or load into the table
    When i clik on show filename ,all filenames are displaying, But when i preview rows, i couldnt see the data.
    ktr file is attached.

    Error is Table output.0 - ORA-00928: missing SELECT keyword.

    But when i give a single file name and field tab is filled, data is loaded tio the table.
    Whats wrong with multiple files?

    Please help!!!
    Thank you.
    Attached Files Attached Files

  2. #2
    Join Date
    Apr 2016
    Posts
    4

    Default

    Hi Plearner,

    if you don't define any fields, I think it's clear that you can't see anything in the preview nor that you have any data, when the transformation is run. And if you process multiple files, they should be structural identical in order to be able to define fields, unless you define one string field that encompasses the whole row. But then you would need some sort of script step to treat the data etc. pp. In short, when you have files with different layouts, you would need specific transformations for each type.

    Kind regards
    Manfred Engelhart

  3. #3
    Join Date
    Aug 2011
    Posts
    360

    Default

    Quote Originally Posted by mengelhart View Post
    Hi Plearner,

    if you don't define any fields, I think it's clear that you can't see anything in the preview nor that you have any data, when the transformation is run. And if you process multiple files, they should be structural identical in order to be able to define fields, unless you define one string field that encompasses the whole row. But then you would need some sort of script step to treat the data etc. pp. In short, when you have files with different layouts, you would need specific transformations for each type.

    Kind regards
    Manfred Engelhart
    That's correct, but you will rather need 2 transfornations:
    -one that read file and load table
    - one that inject metadata for each filetypes with the metadata injection step

    (And surely one job to loop through your metadata config files)

  4. #4
    Join Date
    Dec 2012
    Posts
    29

    Default

    Thank you for the reply. I will go with metadata injection step.

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •  
Privacy Policy | Legal Notices | Safe Harbor Privacy Policy

Copyright © 2005 - 2019 Hitachi Vantara Corporation. All Rights Reserved.