Hitachi Vantara Pentaho Community Forums
Results 1 to 6 of 6

Thread: Generating Multiple Rows of Data from a single row

  1. #1
    Join Date
    Jan 2009
    Posts
    3

    Default Generating Multiple Rows of Data from a single row

    I am having an issue with a scenario that I have not been able to find a solution without exiting to a java script.

    I have a job that retrieves a row of data from any type of data source. I then need to interrogate that row of data and then generate n number of rows from that initial row.

    Some of the data in the new rows will will be duplicates, some data will be new, and some data will be using the data that is in the row as seed data to perform calculations.

    From each of these rows, I may or may not need to generate another subset of rows. Again, I do not know the number in advance and I need to interrogate the data normally in the initial row.

    Can someone provide an example of how to handle the initial problem of reading a single row then generating multple rows from that row.

    If i can get that working, I could always do multiple passes to handle the second condition.

    Thank you in advance.

  2. #2
    DEinspanjer Guest

    Default

    Take a look in the samples/transformations directory of your Kettle installation for the "JavaScript - create new rows.ktr" example. It should get you started.

  3. #3
    Join Date
    Nov 1999
    Posts
    9,729

    Default

    Or use the "Clone Rows" step.
    Or join with a bunch of empty rows.

  4. #4
    Join Date
    Jul 2009
    Posts
    24

    Default Clone Rows Followed with Switch/Case

    We have been successful utilizing the Clone Rows as stated by Matt then using a Switch/Case Step to distibute the rows down paths that handle various scenarios that utilize the stream data similar to the situation you have described below. If a particular row does not need to generate output for a branch we simply filter them in to dummy steps so we can see a complete picture of the stream flow in our logs. Hope this helps.

  5. #5
    Join Date
    Jan 2009
    Posts
    3

    Default Solutions

    Thank you for the sugestions. I will examine the sample that was referenced.

    Just to make sure that I understand. the other suggestion

    If the max number of rows that I need to generate is 36, then generate that many and if after interrogation the row only needs 12 then simply filter down to those 12 ?

  6. #6
    Join Date
    Jul 2009
    Posts
    24

    Default Clarification

    That sounds like you have a good grasp of what we did in our flows. If you have 36 scenarios simply make 35 clones Original makes 36. Presumably each clone would follow a branch of logic that either results in a row or flows to no where. I am new with Pentaho, Matt may be able to verify this approach works. Our current transformation only creates 1 clone so I am not clear on how you can identify which clone it is to flow it down the correct path. We have a simple true/false switch/case for 2 streams. You could always nest the clones always flowing through a true false if the logic for the 36 rows can be eliminated after earlier steps in the flow.

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •  
Privacy Policy | Legal Notices | Safe Harbor Privacy Policy

Copyright © 2005 - 2019 Hitachi Vantara Corporation. All Rights Reserved.