Hitachi Vantara Pentaho Community Forums
Results 1 to 2 of 2

Thread: Kitchen Out of Memory Problem

  1. #1
    Join Date
    Mar 2007
    Posts
    5

    Default Kitchen Out of Memory Problem

    Hi guys,

    I need help badly. I'm running Kitchen to load data from lots of texts files to the database. This runs smoothly on Spoon, but when I try to run it on Kitchen, a "java.lang.OutOfMemoryError: Java heap space" always occur.

    I tried to modify Kitchen.bat to increase the Java heap space:

    Modified on Kitchen.bat

    set OPT=-Xms512m -Xmx1024m -cp %CLASSPATH% -Djava.library.path=libswt\win32\ -DKETTLE_HOME="%KETTLE_HOME%" -DKETTLE_REPOSITORY="%KETTLE_REPOSITORY%" -DKETTLE_USER="%KETTLE_USER%" -DKETTLE_PASSWORD="%KETTLE_PASSWORD%"


    Still here is the error message:

    D:\Pentaho\Kettle-2.5.0>Kitchen.bat /rep:"kettle_sql2000" /job:"Job_GETDATA - 1
    Upload" /user:***** /pass:***** /level:Basic 1>>D:\store_data\log\run_datauploa
    d.log
    2007/04/03 19:45:16:823 GMT+08:00 [INFO] DefaultFileReplicator - Using "C:\DOCUM
    E~1\admin\LOCALS~1\Temp\vfs_cache" as temporary files store.
    Exception in thread "Table - CASH.0 (Thread-95)" java.lang.OutOfMemoryError:
    Java heap space
    at java.util.Arrays.copyOf(Unknown Source)
    at java.util.Arrays.copyOf(Unknown Source)
    at java.util.ArrayList.ensureCapacity(Unknown Source)
    at java.util.ArrayList.add(Unknown Source)
    at be.ibridge.kettle.core.database.Database.insertRow(Database.java:1807
    )
    at be.ibridge.kettle.trans.step.tableoutput.TableOutput.writeToTable(Tab
    leOutput.java:193)
    at be.ibridge.kettle.trans.step.tableoutput.TableOutput.processRow(Table
    Output.java:73)
    at be.ibridge.kettle.trans.step.tableoutput.TableOutput.run(TableOutput.
    java:500)


    When I look at the logs it generated, the Kitchen processing has finished.

    19:46:00,089 INFO [Kitchen] Kitchen - Finished!
    19:46:00,089 ERROR [Kitchen] Kitchen - Finished with errors
    19:46:00,089 INFO [Kitchen] Kitchen - Start=2007/04/03 19:45:15.480, Stop=2007/04/03 19:46:00.089
    19:46:00,089 INFO [Kitchen] Kitchen - Processing ended after 44 seconds.



    What does this mean? How do i prevent this? Hope somebody can help me. Thanks.

  2. #2
    Join Date
    Nov 1999
    Posts
    459

    Default

    Hi,

    it has something to do with the batch update and commit size in your table output step. Seems the JDBC driver can not hold the number of rows before the batch update.
    So limit the commit size or try to switch of batch updates.
    Is the number of rows different when you run as a single transformation or as a job?

    Good luck, Jens

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •  
Privacy Policy | Legal Notices | Safe Harbor Privacy Policy

Copyright © 2005 - 2019 Hitachi Vantara Corporation. All Rights Reserved.