Hitachi Vantara Pentaho Community Forums
Results 1 to 3 of 3

Thread: Pentaho sort rows default sort size

  1. #1
    Join Date
    Dec 2016
    Posts
    17

    Default Pentaho sort rows default sort size

    Hello,

    As pentaho sort row step has default size 1000000, does it mean that it reserve memory as per default size even if we have smaller records in input.
    Let say we have 10 records in input file and we put default size in sort row step, does it block memory for 1000000 records ?

  2. #2
    Join Date
    Dec 2016
    Posts
    17

    Default

    Hello,

    Could anyone provide their thoughts or help me on this?

  3. #3

    Default

    No, the Sort Rows step does not create a 1,000,000 row buffer at the start of the transformation, it uses only very little memory to initialize the step.

    Once the step has reached the maximum number of rows to keep in-memory, it will move those rows from a memory cache to a disk cache file.

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •  
Privacy Policy | Legal Notices | Safe Harbor Privacy Policy

Copyright © 2005 - 2019 Hitachi Vantara Corporation. All Rights Reserved.