Hitachi Vantara Pentaho Community Forums
Results 1 to 32 of 32

Thread: [SOLVED] Kettle slow down on Ubuntu 11.04

  1. #1

    Thumbs up [SOLVED] Kettle slow down on Ubuntu 11.04

    Hello!

    I've been using Kettle on Ubuntu (versions 10 and 11). I've been using the beta version of 11.04. Since last Friday (one day after 11.04 official release), Kettle start to consuming a lot of I/O just after open a .ktr file, besides increase the amount of memory used. Even if a new file (a 'blank' file).

    Seemingly there are some type of lock in a thread:

    Name: org.apache.commons.vfs.cache.SoftRefFilesCache$SoftRefReleaseThread
    State: TIMED_WAITING on java.lang.ref.ReferenceQueue$Lock@4a0ece36
    Total blocked: 6 Total waited: 1,383

    Stack trace:
    java.lang.Object.wait(Native Method)
    java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:118)
    org.apache.commons.vfs.cache.SoftRefFilesCache$SoftRefReleaseThread.run(Unknown Source)

    (Or this is not a lock????)

    I've already tried a different user, reinstall Ubuntu (after format), but unsuccessfully.

    What I do?

    I'm using version 4.1.3 (compiled from sources) and Java version "1.6.0_24" (build 1.6.0_24-b07).

    Tks

  2. #2
    Join Date
    Nov 1999
    Posts
    9,729

    Default

    There's all sorts of trouble with 11.04. Canonical keep introducing more bugs with every release.
    I guess for them features are more important than stability.

    Then again, you didn't *have* to upgrade, did you ;-)

  3. #3
    Join Date
    Mar 2003
    Posts
    8,085

    Default

    Are you using the Sun-JDK or the OpenJDK? They are different codelines and personally for production systems I would only trust the Sun JDK.
    Get the latest news and tips and tricks for Pentaho Reporting at the Pentaho Reporting Blog.

  4. #4
    Join Date
    Nov 1999
    Posts
    9,729

    Default

    It's unrelated to that unfortunately. The performance problems comes from GTK+ sending thousands of refresh requests, even for an empty canvas.
    It might be related to the new X server they ship in 11.04. There are other issues with that, including severe mouse problems and instability.

    I'll look a bit further tomorrow though.

  5. #5
    Join Date
    Apr 2011
    Posts
    10

    Default

    I know it is redundant to post this but I too am having problems with Ubuntu 11.04 and Spoon. In addition to being unable to place steps, am also unable to save. When I click save the "operation in progress" dialog pops up and never goes away. I try to click cancel and close the dialog by clicking the x in the corner but neither work. I end up having to kill the process manually in the term.

    I tried adding the line
    unset UBUNTU_MENUPROXY

    in spoon.sh as per the recommendation in http://jira.pentaho.com/browse/PDI-6138 but that didn't work.
    I am downloading 4.2 M1 and am going to give that one a try.

    -Andy

  6. #6
    Join Date
    Apr 2011
    Posts
    10

    Default

    Just for fun I tried 4.2 M1 and installed both versions on Kubuntu and nothing works. I guess at this point my only option is to downgrade my os which probably means doing a fresh install. lame.

    -Andy

  7. #7
    Join Date
    Nov 1999
    Posts
    9,729

    Default

    Since I actually ran PDI 4.x on Kubuntu 11.04 in a VirtualBox VM without a problem (even ran Gnome & Unity to test) I'm fairly convinced it's related to the new X server that ships with Natty.
    There's some driver issue at work...
    Please let us know a bit more about your video card and configuration (card, driver version, X version, dual screen, compiz, Unity, Gnome, KDE, ...)
    Last edited by MattCasters; 05-03-2011 at 03:29 AM.

  8. #8
    Join Date
    Nov 1999
    Posts
    9,729

    Default

    I downgraded Xorg to version 1.9 on my test-box and certain things like double-click in the tree start to work again.
    The excessive refreshing and drag and drop are not fixed though.
    I found out that a bleeding edge NVidia driver was shipped with 11.04 so I'm going to try to downgrade that too.

    Again, the weird thing is that if I run the same PDI version in a VirtualBox VM under the same broken desktop, all is fine.

  9. #9
    Join Date
    Feb 2011
    Posts
    7

    Default

    same problem here (cannot drag steps onto canvas) Using "Ubuntu Classic", not unity. Dual monitors. using Natty from Maverick

  10. #10
    Join Date
    Nov 1999
    Posts
    9,729

    Default

    I have a dual monitor as well. I wonder if the problem is related.
    Well, if anything it further points to Xorg.

    I managed to slow down the refresh rate to a maximum of 20 frames/sec (coming down from +400 frames/sec) so that performance problem should be under control.
    As far as the Drag & Drop problem is concerned... I'm working on a work-around but it's tedious work :-(
    Last edited by MattCasters; 05-03-2011 at 03:27 PM.

  11. #11
    Join Date
    May 2011
    Posts
    3

    Default

    Hello!

    I'm experiencing same problem in Ubuntu 11.04, with Kettle 4.2.0-M1 (build date 2011-01-23 12.32.29), but I don't have multiple monitors. Java process starts eating CPU and system response becomes sluggish when a transformation is open.

  12. #12
    Join Date
    Nov 1999
    Posts
    9,729

    Default

    OK, I found a solution.

    The culprit seems to be what are called "Ayatana overlay scrollbars".
    Once I removed those all was well again.

    http://www.webupd8.org/2011/04/how-t...llbars-in.html

    or since they make Eclipse un-usable anyway, do as I did:

    Code:
    sudo apt-get remove overlay-scrollbar liboverlay-scrollbar-0.1-0
    Good riddance.

    Let me know ASAP if this works for you too.

    Matt

  13. #13
    Join Date
    Jun 2005
    Posts
    7

    Thumbs up

    I thank you for finding our fix in addition for getting rid of on the the many annoying "features" in 11.04.

  14. #14
    Join Date
    May 2011
    Posts
    3

    Default

    Quote Originally Posted by MattCasters View Post
    Let me know ASAP if this works for you too.
    It's working perfectly. Thank you very much.

  15. #15
    Join Date
    Feb 2009
    Posts
    10

    Default

    I was ready for a downgrade to 10.04 ... but the ayatana removal works also for me. Many thanks.

    For your info: monitoring the use of CPU and memory, i notice that when I launch Spoon the Xorg process remain under 3-4 % of CPU usage, but after opening a job, it goes over 95% ! Memory remain under 1.5%. (My laptop have a Core2 Duo CPU P8700, with 4Gb RAM and nVidia GeForce 9600M GS).

  16. #16
    Join Date
    Feb 2011
    Posts
    7

    Default

    Thanks Matt!!!! Good Form!

  17. #17
    Join Date
    Nov 1999
    Posts
    9,729

    Default

    The CPU spike is caused by Xorg asking constantly to refresh the graph.
    I have a patch ready for 4.2.0 to limit this to 20 frames/s under *all* conditions.

  18. #18
    Join Date
    Dec 2010
    Posts
    1

    Default Thank you!

    Thank god I found this thread, those scrollbars were bringing my i5 to it's knees.

    Thanks for the excellent detective work Matt!
    Last edited by djjuhasz; 05-06-2011 at 06:59 PM. Reason: fix spelling

  19. #19
    Join Date
    Jul 2010
    Posts
    1

    Default Works For Me

    Thanks for the hard work in figuring this one out Matt!

  20. #20
    Join Date
    Apr 2011
    Posts
    10

    Default

    Worked for me too. Thanks Matt!

  21. #21
    Join Date
    May 2011
    Posts
    3

    Default

    Quote Originally Posted by MattCasters View Post
    I have a patch ready for 4.2.0 to limit this to 20 frames/s under *all* conditions.
    Is that patch available?

    Thank you in advance.

  22. #22
    Join Date
    Nov 1999
    Posts
    9,729

    Default

    No, it isn't. I'm not even sure it's a good idea to release it since the problem clearly is with Ubuntu only.

  23. #23
    Join Date
    Aug 2008
    Posts
    563

    Default

    Thanks a lot for the fix!
    Best regards,
    Diethard
    ===============
    Visit my Pentaho blog which offers some tutorials mainly on Kettle, Report Designer and Mondrian
    ===============

  24. #24
    Join Date
    Feb 2011
    Posts
    3

    Default

    Thank's Matt, it works fine for me too.

  25. #25
    Join Date
    Jun 2011
    Posts
    1

    Default

    Quote Originally Posted by MattCasters View Post
    OK, I found a solution.

    The culprit seems to be what are called "Ayatana overlay scrollbars".
    Once I removed those all was well again.

    http://www.webupd8.org/2011/04/how-t...llbars-in.html

    or since they make Eclipse un-usable anyway, do as I did:

    Code:
    sudo apt-get remove overlay-scrollbar liboverlay-scrollbar-0.1-0
    Good riddance.

    Let me know ASAP if this works for you too.

    Matt
    One other way that works for me is to add the line
    export LIBOVERLAY_SCROLLBAR=0
    in spoon.sh script

  26. #26
    Join Date
    Nov 1999
    Posts
    9,729

    Default

    Thanks federico-, I'll add this line to spoon.sh for version 4.2.0

  27. #27
    Join Date
    Aug 2011
    Posts
    1

    Default

    Hi Matt,

    I am facing similar problem while using TightVNC viewer to connect to the server. When I try to drag steps to canvas, it just doesn't stay on the canvas.

    There is no overlay-scrollbar package installed on the server.

    sudo apt-get remove overlay-scrollbar liboverlay-scrollbar-0.1-0
    running this command as root displays error: Couldn't find package overlay-scrollbar.

    I tried adding export LIBOVERLAY_SCROLLBAR=0 in spoon.sh, but its still the same.

    Any idea what could be the problem other than overlay-scrollbar issue. Is there any specific setting that is required for VNC?

  28. #28
    Join Date
    Feb 2012
    Posts
    1

    Default

    Matt, Satya,

    I'm having the same issue as Satya. Running lucid, sun-jdk-6, don't have the overlay package installed. I'm connected using VNC if that makes any difference.

    I've tried all versions from 4.2.1 to 3.2, but all have the same issue. I can't drag any steps to the canvas.

    Any suggestions? Much appreciate the help.

    - Nirav

  29. #29
    Join Date
    Nov 1999
    Posts
    9,729

    Default

    Sorry guys, didn't see the same problem yet, not on VNC nor on RDC.
    The problem is likely unrelated to the Ubuntu display problem.

    File a JIRA bug report, make sure to pass in as many details about the environment as you can, perhaps it's possible to identify a cause and work it out.
    Saying "it doesn't work" isn't really helpful, you need to give details.

  30. #30
    Join Date
    Mar 2009
    Posts
    2

    Default

    I am also having the same problem when using kettle with VNC. If i understood correctly, it is a bug in Ubuntu...
    https://bugs.launchpad.net/ubuntu/+s....0/+bug/587856

    - Pasi
    Last edited by phelasuo; 04-19-2012 at 07:57 AM.

  31. #31
    Join Date
    Oct 2014
    Posts
    2

    Default

    Thank you very much for find the solution to this problem. I've looking for it for a long time and finally someone has solved it.

  32. #32
    Join Date
    Oct 2014
    Posts
    2

    Default

    Thank you very much for this implementation of the solution. I prefer deactivate the scrollbars inside PDI instead the complete operating system

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •  
Privacy Policy | Legal Notices | Safe Harbor Privacy Policy

Copyright © 2005 - 2019 Hitachi Vantara Corporation. All Rights Reserved.