Hitachi Vantara Pentaho Community Forums
Results 1 to 8 of 8

Thread: does GROUP BY work?

  1. #1

    Default does GROUP BY work?

    I just found out that my "group by" step does not work.
    More accurately, it works when number of rows coming into group by is small. In my case, it worked when there are 20 lines. But in my real life case, there are hundred thousand lines going into that step, and I can see quite clearly that the step starts processing before all rows come in. Now, how you group by anything until you have all the rows to begin with? And when I open up the generated table (which is generated by insert/update step) I see that the rows that are supposed to show aggregate values got overwritten over and over by update functionality.
    Did anybody spot this problem? I ran into it in 2.3, downloaded 3.1 (much cooler interface, btw) but the same problem persists.
    any help will be appreciated. I can work around it, but it will be an ugly solution.

  2. #2
    Join Date
    Nov 1999
    Posts
    9,729

    Default

    To the best of our knowledge, the "Group by" step works just fine if you make sure the data is sorted as indicated.

  3. #3

    Default GROUP BY requires sorting

    Thanks Matt, it works now. The crucial piece of information I was missing was that the data needs to be sorted in the exact order of GROUP BY in advance. While it might be obvious to many, it is not for me. I do not remember seeing the sorting requirement either in step description, or 3.0 User Guide. If I did not miss it by accident, you might consider updating the section in the next version of the documentation.
    Thanks for the prompt response again, it is very much appreciated.
    oytun

  4. #4
    Join Date
    Nov 1999
    Posts
    9,729

    Default

    Give me a break here. There is a big fat warning dialog that pops up telling you to make sure the data needs to be sorted.
    You need to click "I understand" too.

    Name:  i-understand.jpg
Views: 49
Size:  13.7 KB

    Matt

  5. #5

    Default

    I was expanding a pre-existing transformation which already included GROUP BY step, so nothing popped up. I was merely changing the configuration of GROUP BY step.

  6. #6
    Join Date
    Nov 1999
    Posts
    9,729

    Default

    Nice try. It gets shown every time you modify the step.

  7. #7

    Default

    No it doesn't. I just went in and modified it again, and showed nothing.
    Il upload my transformation in case you'd like to try for yourself.
    You do not have access to data, but nevermind.
    Just open group by statement, delete one of the criteria and close.
    Attached Files Attached Files

  8. #8
    Join Date
    Nov 1999
    Posts
    9,729

    Talking

    Obviously you can click the "[x] don't show this anymore" button, but that's your choice, not mine.

    By the way, feel free to file a bug report for those documentation updates.

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •  
Privacy Policy | Legal Notices | Safe Harbor Privacy Policy

Copyright © 2005 - 2019 Hitachi Vantara Corporation. All Rights Reserved.