Hitachi Vantara Pentaho Community Forums
Results 1 to 7 of 7

Thread: Java out of memory error

  1. #1
    Join Date
    Oct 2016
    Posts
    24

    Default Java out of memory error

    Hi,
    i'm receiving this error when at end of my transformation (Insert step)

    my spoon.bat has:

    if "%PENTAHO_DI_JAVA_OPTIONS%"=="" set PENTAHO_DI_JAVA_OPTIONS="-Xms512m" "-Xmx1300m" "-XX:MaxPermSize=512m"

    There is a solution or pentaho is a ****?

  2. #2
    Join Date
    May 2014
    Posts
    358

    Default

    Most probably you are doing something in your transformation that loads all rows into memory. Without seeing the transformation I can't tell you where the problem is.

  3. #3
    Join Date
    Oct 2016
    Posts
    24

    Default

    this is my trasformation:

    trf_elab_t_dmt_all_ft_ra_charging_id.ktr

  4. #4
    Join Date
    Apr 2016
    Posts
    156

    Default

    Have you tried increasing the amount of memory used by Spoon in the Windows .bat above?

    Search this forum for many similar threads where people have already given exact details on how to change / what to change to.
    My runtime environment: MacOS, JDK 1.8u121, PDI 7.0

  5. #5
    Join Date
    Oct 2016
    Posts
    24

    Default

    you mean which value?
    "-Xms512m" or
    "-Xmx1300m" or
    "-XX:MaxPermSize=512m"


    but i have same error

  6. #6
    Join Date
    Apr 2008
    Posts
    4,696

    Default

    You have 30+ steps with fairly wide rows, each hop allowing 10000 rows to be kept in memory.
    Some of those steps are rebuilding the row, which keeps two copies of the row in memory.

    Instead of trying to do all the work in one transformation, try breaking the work down into more logical chunks.

    You could try increasing the -Xmx value, but that will only disguise the problem, not solve it -- this is an issue of how you designed your transformation.

  7. #7
    Join Date
    May 2014
    Posts
    358

    Default

    You have a lot of Select Values step which rebuild the row, as gutlez says. Maybe try to do more things in one step? Otherwise it doesn't look that bad. How big is the T_DMT_ALL_DM_TIME table?

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •  
Privacy Policy | Legal Notices | Safe Harbor Privacy Policy

Copyright © 2005 - 2019 Hitachi Vantara Corporation. All Rights Reserved.