Hitachi Vantara Pentaho Community Forums
Results 1 to 4 of 4

Thread: Pentaho with increases memory

  1. #1
    Join Date
    Oct 2014
    Posts
    18

    Default Pentaho with increases memory

    Hi
    I am working on some large input excel files, which on transformation showing "java heap out of memory exception".

    I changed the following things in spoon.bat


    if "%PENTAHO_DI_JAVA_OPTIONS%"=="" set PENTAHO_DI_JAVA_OPTIONS="-Xmx2g" -XX:MaxPermSize=256m"
    I changed "-Xmx256m" to "-Xmx2g".
    Problem : PDI is not able to start.

    When I changed it to "-Xmx1g" , PDI started but still throwing memory out of exception.
    My input excel file is of 30MB only.
    I have windows 8 and while running PDI with 1g, process stuck at 1.5GB memory consumption.

    Please suggest me the way to solve the problem

    Thanks in advance

    cheers,
    Yuvam,

  2. #2

    Default

    Hi Yuvam,


    When you are facing the problem in Transformation then you will change PENTAHO_DI_JAVA_OPTIONS in kitchen.bat file and if you are facing the problem in job then you will increase the cache size of pan.bat file
    First you can clear cache then you can change PENTAHO_DI_JAVA_OPTIONS=-Xmx256m change with -Xmx512m. this one

    It will be working.

  3. #3
    Join Date
    Feb 2014
    Posts
    1

    Default

    Hello. I am having almost the exact same problem. I can see in Task Manager that the jvm is using all of the 1Gb of space before giving out of memory error.
    Is there a way to have PDI process the rows without loading the whole spreadsheet first (if that is what is happening), and why would it take 1Gb of memory to load a 30 MB spreadsheet.
    Thanks!
    -Bruce

  4. #4
    Join Date
    Oct 2014
    Posts
    18

    Default

    Hi All,

    I make my PDI to run. I did following things.

    1- First I used 64 bit of java version.
    2- Changed PDI memory to run on 3 & 2 GB both are working fine.
    3- Now in Excel input I choose "Excel 2007 XLSX (Apache POI Streaming)"

    with this case, my PDI is working fine..
    Please try this condition and share your outcome.

    Thanks
    Yuvam

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •  
Privacy Policy | Legal Notices | Safe Harbor Privacy Policy

Copyright © 2005 - 2019 Hitachi Vantara Corporation. All Rights Reserved.