Hitachi Vantara Pentaho Community Forums
Results 1 to 3 of 3

Thread: Read Variable length columns file

  1. #1

    Default Read Variable length columns file

    Hi All,
    I have a file with variable length columns separated by semicolon like this

    c1;c2;;c3;c4
    c1;c2;;c3;c4;c5;;c6

    I have tried to read it using csv file input and also with text input file. The problem is that these inputs use first row to get the number of fields and only get the data till that column for other rows also. Due to this, the complete row is not getting in which number of columns are greater than no of columns in the first row.

    So kindly let me know how to read complete rows in the above case.

    Thanks
    Shuja

  2. #2
    Join Date
    Apr 2008
    Posts
    1,771

    Default

    Hi Shuja.
    I would import the file as one 1 field only (just set the separator/delimiter to some odd character, like tab or ^), then use a split row step to create multiple fields using ; as separator.
    I would set the number of fields to create to a quite high number, then you could do an aggregate and after that delete empty fields.

    Hope this help.
    Mick

  3. #3
    Join Date
    Nov 2008
    Posts
    271

    Default

    Another possible approach, perhaps not very elegant, would be manually adding more fields in the grid. As you already noticed, when pressing the Get Field button, the step automatically detects the fields in the first row. But no one prevents you from adding other fields.

    HTH
    Andrea Torre
    twitter: @andtorg

    join the community on ##pentaho - a freenode irc channel

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •  
Privacy Policy | Legal Notices | Safe Harbor Privacy Policy

Copyright © 2005 - 2019 Hitachi Vantara Corporation. All Rights Reserved.