Hitachi Vantara Pentaho Community Forums
Results 1 to 3 of 3

Thread: Behavior of Starting Multiple Copies of a Sort Step with Only Pass Unique Rows

  1. #1
    Join Date
    Mar 2006
    Posts
    170

    Cool Behavior of Starting Multiple Copies of a Sort Step with Only Pass Unique Rows

    Hi All,

    Just a question on the affects of starting multiple copies of a sort step with Only Pass Unique Rows checked.

    I do have the "merge join" step following the multiple copies started sort step but with the rows being distributed to the multiple sort steps and the unique being done in the sort step and NOT after the "merge join" step does it indeed return only one unique row as defined by the keys to sort on?

    I know it does when you only start 1 copy of the sort step with the unique flag set to true.

    Thoughts?

    PDI 3.2.3 GA on Linux

    Thanks

    Kent

  2. #2
    Join Date
    Nov 1999
    Posts
    9,729

    Default

    The uniqueness of the output will be for each step copy. So you need a "Sorted Merge" as well as a "Unique" step to complete the parallel exercise.

  3. #3
    Join Date
    Mar 2006
    Posts
    170

    Default

    Hey Matt,

    Thanks for the input.

    I figured that I would have to remove the sort's unique rows and move that operation to a "Unique Rows" step which follows the merge.

    Nonetheless glad I verified.

    Kent

Tags for this Thread

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •  
Privacy Policy | Legal Notices | Safe Harbor Privacy Policy

Copyright © 2005 - 2019 Hitachi Vantara Corporation. All Rights Reserved.