Hitachi Vantara Pentaho Community Forums
Results 1 to 2 of 2

Thread: How to dynamically read the column name from Delimited File

  1. #1
    Join Date
    Jun 2014

    Default How to dynamically read the column name from Delimited File


    I have a requirement wherein we have to load the target table based on the input files (approx 10 files in a directory which are delimited file).
    NOTE: Target table has same no. of columns as there are in file.
    File 1 (delimited by "|") has columns (Column1, Column2, Column3) --> Target Table src_file1 (Column1,Column2,Column3)
    File 2 (delimited by "|") has columns (Column11, Column12, Column13,Column14, Column15) --> Target Table src_file2 (Column11,Column12,Column13,Column14,Column15)
    File 3 (delimited by "~") has columns (Column21, Column22, Column23,Column24) --> Target Table src_file2 (Column21,Column22,Column23,Column24)

    Task is : Read each file from directory --> Parse the file individually based on the delimiter --> Get their Column Name details --> Load the target table based on the Column Name

    My Work :
    Step 1. Through Get File Names , I am able to get all the files from directory
    Step 2. Through the temp table , I am able to get a mapping between filename and the delimiter used.
    So I have stored these details in parameter FILENAME, DELIMITER and Target table name in TARGET_TABLE_NAME parameter

    Step 3. Now, I am executing the transformation, which executes for every row and reads the parameter "FILENAME", "DELIMITER", and "TARGET_TABLE_NAME"

    Tansformation Details :
    Text File Input (which reads FILENAME and DELIMITER) parameters, however Fields Section is left blank.
    Table Output (which has TARGET_TABLE_NAME) parameter set, without specifying the database fields.

    However, While executing the transformation I am not able to retrieve the field_name dynamically from the file, and due to which I am not able to write anything to the target table.

    Could anyone please assist me in reading the fieldname dynamically.


  2. #2
    Join Date
    Apr 2008


    I suggest you look for posts or blogs about Metadata Injection step
    -- Mick --

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
Privacy Policy | Legal Notices | Safe Harbor Privacy Policy

Copyright © 2005 - 2019 Hitachi Vantara Corporation. All Rights Reserved.