Hitachi Vantara Pentaho Community Forums
Results 1 to 3 of 3

Thread: SortRows results in Out Of Memory Exception

Hybrid View

Previous Post Previous Post   Next Post Next Post
  1. #1

    Default SortRows results in Out Of Memory Exception

    Hi,

    I'm trying to apply sortrows with sort size (rows in memory) = 1000000 records on a table that has 8 million records. Transformation results in Java Heap out of memory exception. I tried with Free Threshold Memory(%) setting but still no luck.

    Please let me know how to resolve the issue.

    Rgds,
    Rajesh B

  2. #2
    Join Date
    May 2013
    Posts
    2

    Default

    Can anybody help on this??Even i am also getting the same issue in Kettle SortRows transformation but here my Input CSV file contains 6 Lacs records .

    Thank you...

    Regards
    Saran

  3. #3

    Default

    Hi!

    I see 2 ways to resolve this problem:

    1. Set sort size less than 1 000 000 rows: for example 50 000 rows or 10 000 rows
    2. Increase memory limit for JVM:
    - Open "Spoon.bat" in text editor
    - Try to change <if "%PENTAHO_DI_JAVA_OPTIONS%"=="" set PENTAHO_DI_JAVA_OPTIONS="-Xmx512m" "-XX:MaxPermSize=256m">
    (for example you can use <if "%PENTAHO_DI_JAVA_OPTIONS%"=="" set PENTAHO_DI_JAVA_OPTIONS="-Xmx1024m" "-XX:MaxPermSize=512m">)

    I hope it helps you.

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •  
Privacy Policy | Legal Notices | Safe Harbor Privacy Policy

Copyright © 2005 - 2019 Hitachi Vantara Corporation. All Rights Reserved.