PDA

View Full Version : OutOfMemoryError running a simple update



kettle_alinv
02-02-2006, 05:57 AM
Attachment: update.ktr (http://forums.pentaho.org/archived_att/kettle/update.ktr) Hi,


I'm trying to run a simple update process on ~200k records. After a while, the spoon process dies, without any explanation. Running it from a console, I managed to see that's about a "java.lang.OutOfMemoryError: Java heap space". I doubled the allocated jvm memory, but it only runs a little longer, then dies.



Is there anything I can do to pass this point?



Thanks,
Alin

MattCasters
02-02-2006, 06:14 AM
Hi Alin,

You can enable the console for Spoon too by uncommenting the java line in Spoon.bat and commenting out kettlew.
Out-of-memory errors are unfortunately not easy to handle.
The cause can be many things. For example if you use MySQL, a TableInput will load the complete table into memory. (this is not a Kettle issue but a driver problem)

So, perhaps you can tell us a bit about your configuration, so that we can figure out a way around this problem?

Thanks,

Matt

MattCasters
02-02-2006, 06:20 AM
>I attached the transformation I use.

I disagree ;-)

kettle_alinv
02-02-2006, 06:22 AM
Attachment: update.ktr (http://forums.pentaho.org/archived_att/kettle/update.ktr) Oops!

kettle_alinv
02-02-2006, 06:57 AM
I get the data from SQL Server and I put it into Oracle. Before this, I had a simple read/insert transformation for over 30 mil records and it didn't hit this problem.

I attached the transformation I use.

Thanks for your quick response,
Alin