Hitachi Vantara Pentaho Community Forums
Results 1 to 3 of 3

Thread: Is this a bug - Hanging transform

  1. #1
    Join Date
    Apr 2007

    Default Is this a bug - Hanging transform


    Say you have a table input step. you want to use it as a source for 2 other steps.

    The first - a data validator. or some other stream lookup step.
    The second a merge join.

    Say you have a small rowset size. ( 500 )

    So; you run it for say 50k records. It hangs; Because the data validator needs to read ALL the rows from the table input step, but the table input step is blocked because it cannot write any more than 500 to the merge join step.

    Errrm, maybe I need a pic Anyway, is it a bug? or is it user error (Caused by over zealous performance tuning) ?


  2. #2
    Join Date
    Dec 2009



    use the "Dummy" Step to combine two incoming streams... the merge join might cause a kind of deadlock situation

    ...hmm, after reading your question exactly I would try to use the Dummy step as first step after the "Table Input" and then broadcast your data to the two other steps from there....



  3. #3
    Join Date
    Apr 2007


    using a dummy step like that only gives you more rowset cache, and it could potentially hide the issue, but fundamentally it still has the same problem if you have enough records.
    However, Matt may well have fixed this now, in the sense that there is now an exception in this scenario, see this great description here:

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
Privacy Policy | Legal Notices | Safe Harbor Privacy Policy

Copyright © 2005 - 2019 Hitachi Vantara Corporation. All Rights Reserved.