Hitachi Vantara Pentaho Community Forums
Results 1 to 4 of 4

Thread: OutOfMemoryError

  1. #1

    Default OutOfMemoryError

    I have wrapped Kettle in a web application through which jobs are run. After having run various jobs, I notcced that I got an OutOfMemoryError.

    Exception in thread "Create CSCL Records.0 (Thread-2187)" java.lang.OutOfMemoryError: Java heap space
    at java.util.ArrayList.ensureCapacity(ArrayList.java:169)
    at java.util.ArrayList.add(ArrayList.java:351)
    at be.ibridge.kettle.core.database.Database.insertRow(Database.java:1808)
    at be.ibridge.kettle.trans.step.tableoutput.TableOutput.writeToTable(TableOutput.java:193)
    at be.ibridge.kettle.trans.step.tableoutput.TableOutput.processRow(TableOutput.java:73)
    at be.ibridge.kettle.trans.step.tableoutput.TableOutput.run(TableOutput.java:500)

    I noticed that Job, JobMeta, LocalVariables, kettleVariables, RepositoryDirectory, StepLoader, JobEntryLoader, RepositoryMeta, UserInfo, RepositoryDirectory did not have any attributes by which to release resources.

    • Are there some methods that I should be calling to release resources when a job as run?
    • Has Kettle been tested for memory leaks? I noticed that most of the Kettle apps are standalone in which a new JVM is allocated for each instance.
    Thanks

    BTW - I have subscribed to the kettle-developers but unable to post. Do I need a gmail account to use the forum?

  2. #2
    Join Date
    May 2006
    Posts
    4,882

    Default

    I have wrapped Kettle in a web application through which jobs are run. After having run various jobs, I notcced that I got an OutOfMemoryError.

    Exception in thread "Create CSCL Records.0 (Thread-2187)" java.lang.OutOfMemoryError: Java heap space
    at java.util.ArrayList.ensureCapacity(ArrayList.java:169)
    at java.util.ArrayList.add(ArrayList.java:351)
    at be.ibridge.kettle.core.database.Database.insertRow(Database.java:1808)
    at be.ibridge.kettle.trans.step.tableoutput.TableOutput.writeToTable(TableOutput.java:193)
    at be.ibridge.kettle.trans.step.tableoutput.TableOutput.processRow(TableOutput.java:73)
    at be.ibridge.kettle.trans.step.tableoutput.TableOutput.run(TableOutput.java:500)

    I noticed that Job, JobMeta, LocalVariables, kettleVariables, RepositoryDirectory, StepLoader, JobEntryLoader, RepositoryMeta, UserInfo, RepositoryDirectory did not have any attributes by which to release resources.


    Are there some methods that I should be calling to release resources when a job as run?
    No, not that I can think of except for job.endProcessing().

    [*]Has Kettle been tested for memory leaks? I noticed that most of the Kettle apps are standalone in which a new JVM is allocated for each instance.[/LIST]Thanks
    Not explicitly... and there are still 2 memory leaks in 2.5 for sure. 1 with variables (which will only be solved in 3.0) and another one is related to not calling job.endProcessing(). A bugtracker is open for the memory leaks.

    BTW - I have subscribed to the kettle-developers but unable to post. Do I need a gmail account to use the forum?
    It should work without gmail account.

    Regards,
    Sven
    Last edited by sboden; 09-03-2007 at 02:23 PM.

  3. #3

    Default

    Thanks - I am starting the job using job.start(), not job.execute(). I will change it to job.execute(), job.waitUntilFinished(), job.endProcessing()

    Please include reference to Kettle bugtracker so that I can access them.

  4. #4
    Join Date
    May 2006
    Posts
    4,882

    Default

    http://jira.pentaho.org/browse/PDI-18

    You can vote for it

    Regards,
    Sven

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •  
Privacy Policy | Legal Notices | Safe Harbor Privacy Policy

Copyright © 2005 - 2019 Hitachi Vantara Corporation. All Rights Reserved.