Hitachi Vantara Pentaho Community Forums
Results 1 to 10 of 10

Thread: Fixed length text file input with varying lengths per line

  1. #1
    Join Date
    Jul 2008
    Posts
    4

    Question Fixed length text file input with varying lengths per line

    Hi,
    I am new to this environment and am trying to figure out how to specify the field specs for a text file input where the field lengths vary for each line. For eg: the first line is a field header where the field sizes are different compared to the next line which has a batch header and subsequent data records.
    Any pointers towards appropriate documenattion / tutorials will be very helpful.
    Thanks in advance
    Arthi

  2. #2
    Join Date
    May 2006
    Posts
    4,882

    Default

    Fixed length text files with varying lengths ... it doesn't work like that

    Regards,
    Sven

  3. #3
    DEinspanjer Guest

    Default

    Search this forum, there are two threads with links to a similar solution.

  4. #4
    Join Date
    Sep 2007
    Posts
    834

  5. #5
    Join Date
    Jul 2008
    Posts
    4

    Default

    Thanks Maria, Will check that out.
    And Sven, had a feeling somone would comment on that ! :-)

  6. #6
    Join Date
    Jul 2008
    Posts
    4

    Default

    Been trying different things and realized that I can do this in a way without using any JavaScript.

    I realised that I can do this using a Switch case statement leading to multiple 'text file input' steps. In each of the text file inputs,
    I can filter the batches of records I do not need and specify the format for those specific set of lines that I need to process.

    This works fine, but I am curious why this approach hasn't been suggested .. Performance considerations etc? Any thoughts from anyone?

    Thanks for the help.

  7. #7
    Join Date
    Sep 2007
    Posts
    834

    Default

    The "Switch case" Step is new (3.1).
    I've only heard about it; never tried it.

  8. #8
    Join Date
    Nov 1999
    Posts
    9,729

    Default

    You could do it, read the whole line as a single field, then pick the code in a certain position.
    Switch/case on that code and then pick the fields apart in JavaScript with substrings.
    That would only read the data once (as opposed to multiple passes in the other filtering solutions) but it would still not be ideal.

    Matt

  9. #9
    Join Date
    Jul 2008
    Posts
    4

    Default

    So the primary negative is that the data would be read with each pass..?
    I was just wondering if there was any considerations that I was unaware of. Thanks for the response. This helps.

  10. #10
    DEinspanjer Guest

    Default

    Quote Originally Posted by arthinc View Post
    Been trying different things and realized that I can do this in a way without using any JavaScript.

    I realised that I can do this using a Switch case statement leading to multiple 'text file input' steps. In each of the text file inputs,
    I can filter the batches of records I do not need and specify the format for those specific set of lines that I need to process.

    This works fine, but I am curious why this approach hasn't been suggested .. Performance considerations etc? Any thoughts from anyone?

    Thanks for the help.

    Thats actually similar to what I suggested in this example: http://forums.pentaho.org/showpost.p...22&postcount=5
    In that one I used filters because the logic wasn't simple enough to be contained in a single switch step.

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •  
Privacy Policy | Legal Notices | Safe Harbor Privacy Policy

Copyright © 2005 - 2019 Hitachi Vantara Corporation. All Rights Reserved.