Hitachi Vantara Pentaho Community Forums
Results 1 to 5 of 5

Thread: Error!!! Java Heap space and GC overhead limit exceed.

  1. #1

    Default Error!!! Java Heap space and GC overhead limit exceed.

    Can anyone help me to identify if these both the errors are same or different ?

    How to resolve these errors in Pentaho.

    Many thanks in advance.

    G_nish

    -------------------------------
    pdi-ce-6.0.1.0-386-stable
    java 1.8.0_101
    Window 7 (x86_64)
    timezone IST
    --------------------------

  2. #2
    Join Date
    Jan 2015
    Posts
    107

    Default

    Your job is running out of memory. The two errors are not the same obviously, but are both related to your Java process not having enough memory.

    You can check spoon.bat for the line that looks like
    if "%PENTAHO_DI_JAVA_OPTIONS%"=="" set PENTAHO_DI_JAVA_OPTIONS="-Xms1024m" "-Xmx2048m" "-XX:MaxPermSize=256m"
    Increase the Xmx value to allocate more memory. This *may* fix your problem, if the dataset is large and you need just a bit more memory to keep rows in. More likely, it will just delay the appearance of the error.

    If you are getting this error even with only a few rows, check for loops, large amounts of data being copied to results and plugins you might be using. Personally, I've had issues with too large resultsets being kept throughout a job and also Excel input step with large .xlsx files.

  3. #3

    Default

    Thanks Isha for the help !!!

  4. #4

    Default

    Hi,

    I have tried to re-execute the job but it is not resolved. I have set it to 3 gigabytes but the things not looks like resolve. Getting the same error " Java heap space ".
    Can you help more on this ?

    Any idea how to resolve GC Overhead limit error as well ???

    Thanks,

  5. #5
    Join Date
    Jan 2015
    Posts
    107

    Default

    You will need to post much more information for anyone here to be able to help you, as there can be many different causes for this error. If it is a simple job, you could upload it to the forum (remove any credentials from the job if present) for people to take a look.

    Otherwise, you'll have to narrow it down by yourself with a general troubleshooting procedure.

    • Have you tried with a small data input only?
    • Have you disabled/removed subjobs and transformations to see which one is the likely cause?
    • When you run the job and look at the metrics, what is happening at the point you get the error?
    • What does the log say?


    Regarding the GC error, it is caused by the same problem. It's just a matter of chance which one you run into first.
    GC = Garbage Collector, the process that tries to free up Java heap memory. If you're almost out of memory, it will run constantly, until Java triggers the overhead limit error, rather than hanging endlessly. If your job tries to allocate more memory than the GC has freed right after a run, you get the out of Java Heap space error instead.
    Last edited by Isha Lamboo; 02-16-2017 at 06:38 AM. Reason: added comment on GC overhead vs out of heap space

Tags for this Thread

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •  
Privacy Policy | Legal Notices | Safe Harbor Privacy Policy

Copyright © 2005 - 2019 Hitachi Vantara Corporation. All Rights Reserved.