Hitachi Vantara Pentaho Community Forums
Results 1 to 8 of 8

Thread: 3.0.0RC1: run a transformation three times, get different results each time

  1. #1

    Default 3.0.0RC1: run a transformation three times, get different results each time

    I ran a transformation in 3.0RC-1 three times, but I got different results each time as displayed in the attached pictures. I checked the problem, and found groupby had unknown problems. (I sorted before groupby).

    More, these results are all not right. the real results should be 280 after groupby,not 159,160,180.

    Has anyone meet the same problem?
    Attached Images Attached Images    

  2. #2
    Join Date
    May 2006
    Posts
    4,882

    Default

    Make a JIRA at http://jira.pentaho.org/browse/PDI attaching a simplified version of your transformation that still shows your problem (preferably using e.g. files as input)

    Regards,
    Sven

  3. #3

    Default

    I finally found the problem.

    It was not the problem of groupby, but the problem of merge join.

    If the stream was split (I needed exactly the same copies of the data, and I added a dummy step to make a copy of the stream), and then joined by the merge join step. Merger join ran smoothly in the whole process and reported the right number of rows (how many rows output by this step), however, 1/3 rows are repeated. Each time, number of repeated rows are not the same,and finally causes the groupby step reports different results as displayed in the beginning of the thread.

    put it simply, the merge join can deal with the split stream, but processed with errors.

    Both 3.00M2 and 2.5 should stop at around 1000 rows in this case.

    Hope it can be solved in the RC2.

  4. #4
    Join Date
    May 2006
    Posts
    4,882

    Default

    Raise a tracker to be sure

    The thing you see was already a known problem in 2.5.x

    Regards,
    Sven

  5. #5

    Default

    Quote Originally Posted by sboden View Post
    Raise a tracker to be sure

    The thing you see was already a known problem in 2.5.x

    Regards,
    Sven
    I wanna know whether it will be solved in RC2.

    I used the split stream very often, and now my solution is to save a serialized file.... very confused now.

  6. #6
    Join Date
    May 2006
    Posts
    4,882

    Default

    Raise a JIRA tracker ;-)

    In 2.5.x I belief it could not be fixed and there was the intention to popup a warning dialog when you did something like you do. In v3.0 I thought it was fixed, but as you can see I can be wrong.

    Raise tracker.

    Regards,
    Sven

  7. #7
    Join Date
    Nov 1999
    Posts
    9,729

    Default

    Indeed, please create JIRA cases folks. We can't work with fuzzy screen shots or general descriptions to fix bugs.
    At a minimum we should have a use-case if you want your problem to get fixed soon.

    Matt

  8. #8
    Join Date
    Nov 1999
    Posts
    9,729

    Default

    I don't know the exact use-case, but I bet it's related to this problem I fixed a few minutes ago : http://jira.pentaho.org/browse/PDI-287
    Yes, this gambling habit of mine is starting to become a problem. ;-)

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •  
Privacy Policy | Legal Notices | Safe Harbor Privacy Policy

Copyright © 2005 - 2019 Hitachi Vantara Corporation. All Rights Reserved.