Hitachi Vantara Pentaho Community Forums
Results 1 to 5 of 5

Thread: Several heap space errors causing frequent Spoon crashes

  1. #1

    Unhappy Several heap space errors causing frequent Spoon crashes

    Lately I have several heap space errors while working with Spoon that hung my machine until I killed javaw via task manager.

    The latest was the following: I have a transformation with one table input step set up with select * from my_large_table. Hit either of "Preview" or "Get fields". Machine is now hung.

    The previous situation: New transformation, insert two table input steps, one as select * from my_large_tableA, the other as select * from my_large_tableB, insert a merge diff step, drag hops from each table input to the merge diff. On the second hop the machine is now hung. I rebooted and did the same procedure. The crash was repeatable.

    The machine is Windows XP SP3, Java = 1.6.0_23 the database Mysql 4.0.18. <= Ancient db version is unavoidable until application upgrade.

    Any suggestions? I had no problems with Spoon until about 10 days ago when this started happening. It is also happening on two different machines with the above configuration.

    I am also getting lots of stack overflow errors trying to issue "Get fields" in several different transformations:
    Name:  Oops2011-01-26_14-43-59.jpg
Views: 45
Size:  56.5 KB
    Last edited by britebyte; 01-26-2011 at 03:48 PM. Reason: Added screenshot

  2. #2
    Join Date
    Mar 2008
    Posts
    140

    Default

    What version of PDI are you running?

  3. #3

    Default

    Sorry I forgot to put this in: 4.1.0GA. Thanks for taking a look.

  4. #4
    Join Date
    Mar 2008
    Posts
    140

    Default

    Can you enable the console on Spoon by editing the spoon.bat file and changing javaw to java. Then on the last line remove [start "Spoon" ] and add a pause to the end of the script. This will allow you to see what happens even after it crashes.

    How many columns are there in your large tables and do you ever get the StackOverflow issue with streams containing fewer columns?

  5. #5

    Default

    Thanks for your help. I should be more clear that I have at least two separate issues. I was hoping there was some folder of saved settings I could clear out so I mentioned them in one post.

    So to be clear: One transformation with two large table inputs freezes as soon as I try to connect them to a merge diff. Another one gets a stack overflow on checking it or getting fields.

    As per your post I enabled the console and this is the extra info I see (in both cases) but I don't think it helps very much:

    Code:
    INFO 31-01 14:10:25,218 - Spoon - Logging goes to file:///C:/DOCUME~1/Lola/LOCA LS~1/Temp/spoon_c21620e0-2d6d-11e0-af7c-1b4139bd1fd7.log WARN 31-01 14:10:30,578 - could not parse [vertical] as Align value WARN 31-01 14:10:31,609 - could not parse [vertical] as Align value WARN 31-01 14:10:31,796 - Cannot overlay element with id [trans] as it does not exist in the target document. WARN 31-01 14:10:31,796 - Cannot overlay element with id [job] as it does not e xist in the target document. Changed to: C:\Programs\Pentaho\data-integration\docs\English\welcome\kettle_doc ument_map.html INFO 31-01 14:10:34,531 - Version checker - OK
    After the stack overflow on the one transformation and the freeze on the other, nothing other than the above was printed.

    I have created a Jira case with one of the transformations attached, the one with the stack overflow on getting fields or checking the transformation: http://jira.pentaho.com/browse/PDI-5295

    As for the other transformation with the large tables - they only have about 15-20 columns each, just lots of rows - about 29,000,000 - and a large row size.

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •  
Privacy Policy | Legal Notices | Safe Harbor Privacy Policy

Copyright © 2005 - 2019 Hitachi Vantara Corporation. All Rights Reserved.