Hitachi Vantara Pentaho Community Forums
Results 1 to 14 of 14

Thread: Text File Parsing

  1. #1

    Default Text File Parsing

    I'm parsing a fixed text file based on a certain position in the file.
    Lets just say position 10-11 contain values 01 through 99.
    Is there an easy way parse the text file into 99 "Text file outputs" without having to create 99 "Filters" and "Select Values" ?

  2. #2
    Join Date
    Nov 1999
    Posts
    9,729

    Default

    Read the complete line and split it manually?

  3. #3

    Default

    Sounds like another way to say "NO !"

  4. #4
    Join Date
    Nov 1999
    Posts
    9,729

    Default

    That's exactly what I wanted to say initially.
    I have been working on a plugin (on and off) for advanced use-cases of your variable record format, but it's going to be an Enterprise Edition feature only.

  5. #5
    Join Date
    Apr 2008
    Posts
    4,696

    Default

    Depends on how ugly you want to let things get...

    For Filter / Select or for Switch/Case, you have to isolate the section of the line (Char 10-11) ... so... Add a constant to that, and use a single Text File Output with "Accept File Name from Field"

    I'm just not sure how clean that line processing to grab Char 10-11 would be.

  6. #6

    Default

    Finally somebody that knows what they're doing
    That actually worked !
    Instead of creating a {Filter/Select Values/Text file output} for each of the 99 possibilities I was able to create just one "Select values" and one "Text file output" by using the "Accept file name from field?" in the "Text file output"
    It created all of the text files based on the digit value and named the files ##.txt which is perfect.
    It was also at least 70% faster.
    Is there a way to control where the text files are created. Looks like it defaults to my Kettlle directory.
    I tried using the internal variables but that didn't work.

  7. #7
    Join Date
    Apr 2008
    Posts
    4,696

    Default

    I would expect that you can create a different field based on your File field, before the text file output.

    1) Add Constant (Path='/svr/pdi/output/')
    2) Calculator (Filename=Path+FileNum)

    Then you would have Filename =/svr/pdi/output/01 etc.

  8. #8

    Default

    Once the I select "Accept filename from field" the "Filename" box turns gray (not available) ?

  9. #9
    Join Date
    Nov 1999
    Posts
    9,729

    Default

    For obvious reasons one would think.
    If the filename is in an input field, where else would you want to define it?

  10. #10

    Default

    I'm looking to control the path/directory where the file is created.

  11. #11
    Join Date
    Apr 2008
    Posts
    4,696

    Default

    Then generate the FULL PATH to the file in one field, and use that field as the filename.

    The steps for this are in a previous post.

  12. #12

    Default

    For some reason it didn't click the first time I read it.
    You are correct !
    I added a constant value for my path and combined that with my 2 digit value and it worked perfectly.
    Thanks for all of your help.

  13. #13
    DEinspanjer Guest

    Default

    Have you checked the validity of the output files that were created using this method? What version of Kettle are you using?

    I was experiencing some problems with file truncation in Kettle 3.2 when using Text File Output with the option to read filenames from a field. I have a patch that I'm using locally, but I haven't checked it in yet because it had a bunch of other changes in it..

    If you don't have a problem, that is great, but if you do, then I'll get off my duff and get it checked in to the 3.2.1 stream first thing next week.

  14. #14

    Default

    I did check the output files and they were right on.
    I'm using version 3.1 build 826.

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •  
Privacy Policy | Legal Notices | Safe Harbor Privacy Policy

Copyright © 2005 - 2019 Hitachi Vantara Corporation. All Rights Reserved.