Hitachi Vantara Pentaho Community Forums
Results 1 to 7 of 7

Thread: Frequent crashes and memory leaks in Kettle 6.0.1 under Max OS X 10.10

  1. #1

    Default Frequent crashes and memory leaks in Kettle 6.0.1 under Max OS X 10.10

    Hello Pentaho Peeps!

    I'm evaluating Pentaho - Particularly as an ETL tool - Beginning with the free "community edition." (General Availability Release, v. 6.0.1.0-386). So far, it seems like it would be a powerful tool - If it was stable. Under Mac OS X (v. 10.10, AKA Yosemite) and JRE 1.8.0, and I'm finding Kettle almost unusable, it crashes so often. It appears to have a memory leak - It gives me the "beach ball of death," and when this happens, I've seen its memory consumption climb upwards of 6 GB - Mind you, this is not while running an ETL - It occurs while editing steps in the transformation.

    I've found no info on these forums describing the problems I'm having. But I cannot emphasize enough - I can't understand how anybody could use this tool with this level of instability. It's crashed on me about a dozen times today alone. It's just unproductive. I've used similar tools (like CloverETL) and I like Pentaho's features. But something seems really wrong here.

    Is anyone else experiencing this? Has anyone found a solution? Also, I'm wondering if this problem exists in the enterprise edition as well.
    Paul

  2. #2
    Join Date
    Mar 2015
    Posts
    190

    Default

    Quote Originally Posted by accretivepaul View Post
    Hello Pentaho Peeps!

    I'm evaluating Pentaho - Particularly as an ETL tool - Beginning with the free "community edition." (General Availability Release, v. 6.0.1.0-386). So far, it seems like it would be a powerful tool - If it was stable. Under Mac OS X (v. 10.10, AKA Yosemite) and JRE 1.8.0, and I'm finding Kettle almost unusable, it crashes so often. It appears to have a memory leak - It gives me the "beach ball of death," and when this happens, I've seen its memory consumption climb upwards of 6 GB - Mind you, this is not while running an ETL - It occurs while editing steps in the transformation.
    is it database repository or file repository? if database repository then is it in your local machine or in server? if file repository then you don't find these type of issues
    so we need more info regarding the same..

    Quote Originally Posted by accretivepaul View Post
    I've found no info on these forums describing the problems I'm having. But I cannot emphasize enough - I can't understand how anybody could use this tool with this level of instability. It's crashed on me about a dozen times today alone. It's just unproductive. I've used similar tools (like CloverETL) and I like Pentaho's features. But something seems really wrong here.
    why are you assuming wrong , seems to be you are the only one facing these type of problems. have you read carefully about pdi installation process/steps. you may be expert in CloverETL but not on PDI. so read carefully PDI installation.
    Last edited by ranala; 03-23-2016 at 02:16 AM.

  3. #3

    Default

    Quote Originally Posted by ranala View Post
    is it database repository or file repository? if database repository then is it in your local machine or in server? if file repository then you don't find these type of issues
    so we need more info regarding the same..
    To be specific, the problem seems more frequent when working from a repository on a server, although it still occurs when working in local .ktr files.
    Beyond that, not much rhyme or reason to it, other than that it occurs when I'm working in step dialogue boxes - For example, it's happened multiple times when selecting an operation in the Calculator step. When working locally, it mostly seemed to happen in steps that connect to a database, such as Database Lookup, Database Join, etc...

    Since you ask about database repositories, is this a known issue?

    Quote Originally Posted by ranala View Post
    why are you assuming wrong , seems to be you are the only one facing these type of problems. have you read carefully about pdi installation process/steps. you may be expert in CloverETL but not on PDI. so read carefully PDI installation.
    Frankly, I find it a little difficult to believe I'm the only one having this trouble, though I haven't found much info yet. But let's please stay on topic - The PDI installation instructions basically consist of "unzip the download and run," and knowing that this is clearly a memory leak issue, you'd have trouble persuading me that this was anything under my control unless you can name something very specific. I'm mostly trying to figure out if this issue is documented, if any workarounds exist, and perhaps most importantly, if the issue exists in the Enterprise edition.
    Last edited by accretivepaul; 03-23-2016 at 03:14 PM. Reason: fixing quotes

  4. #4
    Join Date
    Apr 2008
    Posts
    4,696

    Default

    Quote Originally Posted by accretivepaul View Post
    Frankly, I find it a little difficult to believe I'm the only one having this trouble, though I haven't found much info yet. But let's please stay on topic - The PDI installation instructions basically consist of "unzip the download and run," and knowing that this is clearly a memory leak issue, you'd have trouble persuading me that this was anything under my control unless you can name something very specific. I'm mostly trying to figure out if this issue is documented, if any workarounds exist, and perhaps most importantly, if the issue exists in the Enterprise edition.
    I've been using PDI for a number of years, and have never encountered what you are describing.
    I have not yet tried PDI 6.0.1.
    I have used Java 8.
    I have not used Mac, but have used Linux and Windows.

    Is it the software? Maybe. My first place to look for that would be http://jira.pentaho.com

    My understanding is that for the most part, EE and CE are the same software. EE includes a few extra plugins and supporting programs, but the core PDI files (jars, exes, launchers, etc) are the same. EE however has a support agreement, that you can call Pentaho and say "I think I found a memory leak, can you look into it and fix it?"

  5. #5

    Default

    Thanks. One thing I forgot to mention is that my colleagues using the tool in Windows have not experienced this issue. I tried it myself under a Windows 10 VM and didn't have this trouble, though I would prefer to work in OSX. I haven't found anything in JIRA yet but I'll keep looking, and if anyone has had similar issues, please reply.

  6. #6
    Join Date
    Apr 2008
    Posts
    4,696

    Default

    It's possible that it's an issue with the underlying SWT libraries then, which by necessity are different by platform (OSX SWT is a different library than Win64 SWT)
    I'd start a new bug report, and see where it goes.

  7. #7
    Join Date
    Jul 2009
    Posts
    476

    Default

    I use OSX, and I've had some problems in the past, but my current installation is working well. Several details will be different from yours, but maybe this will help:

    * PDI 5.4.0.1
    * Java 1.8
    * OSX 10.11 El Capitan
    * 8 GB memory
    * File-based repository
    * Deleted the plugins/pentaho-big-data-plugin folder (I don't use it, and it's big, too)

    I *think* upgrading to El Capitan helped, but that was several months ago, and my memory of that is a bit fuzzy.
    Last edited by robj; 03-24-2016 at 09:28 AM. Reason: Clarify that I don't use the big data plugin

Tags for this Thread

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •  
Privacy Policy | Legal Notices | Safe Harbor Privacy Policy

Copyright © 2005 - 2019 Hitachi Vantara Corporation. All Rights Reserved.