Hitachi Vantara Pentaho Community Forums
Results 1 to 3 of 3

Thread: Regarding Select Value step in kettle

  1. #1
    Join Date
    Apr 2016
    Posts
    1

    Exclamation Regarding Select Value step in kettle

    Hello,

    Is it good to use select value step many times in a dashboard kettle? Because if we are having 10 columns in the input stream and want to join through merge join step and after the merge join we need only 4 columns. So it is good to remove the remaining 6 columns?

    Regards,
    Har**** Saxena

  2. #2
    Join Date
    Aug 2016
    Posts
    279

    Default

    It could be potentially bad for performance, each step should be more "work". But removing fields could give great boost to some steps also. It can often help readability and easier debugging to remove junk fields no longer relevant. My answer would therefore be "it depends" :P In most cases though, I don't think it makes much difference in terms of performance.

  3. #3
    Join Date
    Apr 2008
    Posts
    4,671

    Default

    From what I recall, rebuilding the row is an expensive step.
    So your best bet would be to do all your joins, and then before sending the data to the dashboard, do one Select Values to rebuild the row the way you want it.

    As always, I suggest, build it both ways, and do a large dataset (Row Generator at 100,000 rows) comparison to see which one performs better.

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •  
Privacy Policy | Legal Notices | Safe Harbor Privacy Policy

Copyright © 2005 - 2017 Pentaho Corporation. All Rights Reserved.