Hitachi Vantara Pentaho Community Forums
Results 1 to 3 of 3

Thread: Strange bug using "Copy rows to result" in sub-transformation

  1. #1
    Join Date
    Aug 2016
    Posts
    239

    Default Strange bug using "Copy rows to result" in sub-transformation

    There is a very strange bug happening when trying to send information from a child transformation to a parent transformation.

    Background:

    I want to loop through a list of values. The parent transformation sends the loop to a child transformation (sub-transformation). For each iteration in the loop, the child transformation will do some manipulation and send back a count of all rows it processed. Finally, the parent transformation receives the counts from each iteration and calculates the sum total number of rows written (for all iterations in the loop).

    Problem:

    The order of the creation of hops makes a difference if the parent transformation crashes or not! The child transformation has 4 steps.

    Correct order when creating hops:

    1) Create hop between step 1 and step 2
    2) Create hop between step 2 and step 3
    3) Create hop between step 3 and step 4

    If you change this order, parent transformation crashes! For example:

    1) Create hop between step 3 and step 4
    2) Create hop between step 2 and step 3
    3) Create hop between step 1 and step 2

    Why does the order the hops were created cause the parent transformation to crash? When working in spoon, you can't even see what order the hops were created. You would have to deep dive into the xml to see it. Furthermore, it seems like data 'leaks' from intermediate steps in the child transformation to the parent transformation if the order of the hops are incorrect.
    Last edited by Sparkles; 11-14-2018 at 11:04 AM.

  2. #2
    Join Date
    Aug 2016
    Posts
    239

    Default

    The difference is found in the xml:

    Child v1 (correct order, parent transformation executes successfully)
    <order>
    <hop> <from>Dummy_Rows</from><to>Count_Rows</to><enabled>Y</enabled> </hop>
    <hop> <from>Count_Rows</from><to>Convert_To_Integer</to><enabled>Y</enabled> </hop>
    <hop> <from>Convert_To_Integer</from><to>Copy_Rows_To_Result</to><enabled>Y</enabled> </hop>
    </order>


    Child v2 (incorrect order, parent execution crashes with error "Aggregate subject field [COUNT] couldn't be found!")
    <order>
    <hop> <from>Convert_To_Integer</from><to>Copy_Rows_To_Result</to><enabled>Y</enabled> </hop>
    <hop> <from>Count_Rows</from><to>Convert_To_Integer</to><enabled>Y</enabled> </hop>
    <hop> <from>Dummy_Rows</from><to>Count_Rows</to><enabled>Y</enabled> </hop>
    </order>
    Last edited by Sparkles; 11-14-2018 at 11:05 AM.

  3. #3
    Join Date
    Aug 2016
    Posts
    239

    Default

    You can download the transformations here:

    child v1 (child_v1.ktr): https://drive.google.com/open?id=1Zn...iOhpGW0zHz08c8
    child v2 (child_v2.ktr): https://drive.google.com/open?id=1gh...Bw5iLKQ27etIhy
    parent (parent.ktr): https://drive.google.com/open?id=15P...7fxd3wwZcOvA9z

    everything packed together in zip (hop_order.zip): https://drive.google.com/open?id=1Hh...QZTe2e-52CN33x

    Child v1 has correct order (parent transformation executes successfully).
    Child v2 has incorrect order (parent transformation crashes).
    Last edited by Sparkles; 11-14-2018 at 11:05 AM.

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •  
Privacy Policy | Legal Notices | Safe Harbor Privacy Policy

Copyright © 2005 - 2017 Pentaho Corporation. All Rights Reserved.