Hitachi Vantara Pentaho Community Forums
Results 1 to 6 of 6

Thread: Fixed Flat File with record split on 2 rows

  1. #1
    Join Date
    Jan 2014
    Posts
    4

    Question Fixed Flat File with record split on 2 rows

    Very new to Pentaho... I need to read a fixed flat file with the following format

    recA ----- fixed colum data here colums in format A -----
    recB ----- fixed colum data here colums in format B -----
    recA ----- fixed colum data here colums in format A -----
    recB ----- fixed colum data here colums in format B -----
    recA ----- fixed colum data here colums in format A -----
    recB ----- fixed colum data here colums in format B -----
    .
    .
    .

    I am trying to figure out a way to read the file and merge the recA/recB into one row and map the fields to that I can then select the fields I want to write to a database table

    so wind up with:
    recA ----- fixed colum data here colums in format A -----recB ----- fixed colum data here colums in format B -----
    recA ----- fixed colum data here colums in format A -----recB ----- fixed colum data here colums in format B -----
    recA ----- fixed colum data here colums in format A -----recB ----- fixed colum data here colums in format B -----

    Then I can write to a database table..

    Any suggestions?

  2. #2
    Join Date
    Apr 2008
    Posts
    1,771

    Default

    You can read the file with Fixed File Input step.
    http://wiki.pentaho.com/display/EAI/Fixed+File+Input

    Split the stream using the Filter Rows step, then do a Stream lookup to merge those 2 streams back together.
    -- Mick --

  3. #3
    Join Date
    Jan 2014
    Posts
    4

    Default

    Still struggling (I've been using Pentah for 2 days now). I have 2 streams, one with all recA rows and another with recB rows.. There is no unique key inormation in the data. I would need to merge row 1 from first stream with row 1 of second stream etc... I don't see how a Stream lookup would help.

  4. #4
    Join Date
    Apr 2008
    Posts
    4,696

    Default

    Number each row as it is read (you can do this within the Text File Input, or add an "Add Sequence" right after your input)
    Use a calculator step to calculate the record number [ (Add Sequence Value - 1) / 2 ]
    Filter Rows to split based on Record Type
    a) Process the RecType A record
    b) Process the RecType B record
    Join the rows using either Stream Lookup or Join Rows based on the record number calculated above
    **THIS IS A SIGNATURE - IT GETS POSTED ON (ALMOST) EVERY POST**
    I'm no expert.
    Take my comments at your own risk.

    PDI user since PDI 3.1
    PDI on Windows 7 & Linux

    Please keep in mind (and this may not apply to this thread):
    No forum member is going to do your work for you. We will help you sort out how to do a specific part of the work, as best we can, in the timelines that our work will allow us.
    Signature Updated: 2014-06-30

  5. #5
    Join Date
    Jun 2012
    Posts
    5,534

    Default

    347 days until Christmas, but what the heck
    Attached Files Attached Files
    So long, and thanks for all the fish.

  6. #6
    Join Date
    Jan 2014
    Posts
    4

    Default

    ENORMOUS HELP.. this is EXACTLY what I need... Thank you

Tags for this Thread

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •  
Privacy Policy | Legal Notices | Safe Harbor Privacy Policy

Copyright © 2005 - 2019 Hitachi Vantara Corporation. All Rights Reserved.