Hitachi Vantara Pentaho Community Forums
Results 1 to 9 of 9

Thread: Custom Plugin - External JARs

  1. #1

    Exclamation Custom Plugin - External JARs

    Hi there,

    I'm currently developing an external step to work in transformations that pull data from a source system.

    The step in question needs to leverage some external libraries to communicate with that source system. I've been able to successfully load the plugin within PDI and can properly edit the dialog.

    However when I try to run the plugin I receive a NoClassDefFoundError exception as PDI tries to load the external JAR files it requires from my source.

    I've tried almost everything I can think of:
    1. Ensure to build the plugin JAR so that it includes the external JARs
    2. Physically add the external JARs to a /lib directory within the PDI plugin path and reference them with a modified plugin.xml


    Neither of these or their variations seem to work. I realize this is less of a PDI question and more of a Java development one, but just curious as to how one needs to go about this properly?

    Please let me know if you need any other information.

    Mike

  2. #2
    Join Date
    Nov 1999
    Posts
    9,729

    Default

    You can obviously throw the libraries in the libext/ folder somewhere.
    You can also list the libraries in your plugin.xml file.

    Code:
    <plugin ...>
      <libraries>
        <library name="lib/foo.jar" />
        <library name="lib/bar.jar" />
      </libraries>
    </plugin>
    In version 4 if you use annotations you can simply throw the class in a jar somewhere under the plugins folders. The external jars in the lib/ subfolder will then be automatically picked up.

  3. #3

    Default

    Quote Originally Posted by MattCasters View Post
    You can obviously throw the libraries in the libext/ folder somewhere.
    You can also list the libraries in your plugin.xml file.

    Code:
    <plugin ...>
      <libraries>
        <library name="lib/foo.jar" />
        <library name="lib/bar.jar" />
      </libraries>
    </plugin>
    In version 4 if you use annotations you can simply throw the class in a jar somewhere under the plugins folders. The external jars in the lib/ subfolder will then be automatically picked up.
    Hi Matt,

    Thanks a lot for getting back to me so soon - really appreciate it. I have opted to modify the plugin.xml file but am still getting the issue. I'm trying to connect to the Google Analytics web service.

    My plugin.xml file looks as follows:

    Code:
       <libraries>
        <library name="GoogleAnalyticsPlugin.jar"/>
        <library name="lib/gdata/*.jar" />
        <library name="lib/*.jar"/>
       </libraries>
    And I most definitely have the required jar files contained under /[Application Root]/plugins/steps/GoogleAnalyticsPlugin/lib. Can you think of anything else I'm doing wrong? Are wildcards not allowed for library names?

    Best,
    Mike

  4. #4
    Join Date
    Sep 2009
    Posts
    810

    Default

    Hi Mike,

    could you try to make sure that you do *not* include any jars in your plugin that Kettle already has in its libext folder? Since 4.0 I've seen the Kettle classloaders complaining about that. Happens easily if you use some commons library for example.

    The plugin libraries are going to be loaded on demand, your main jar is loaded (you see the dialog). So setting a breakpoint in your plugin might help debugging why the libs are not loaded.

    This post show how to breakpoint inside your plugin.

    http://type-exit.org/adventures-with...-in-voldemort/

    Cheers

    Slawo
    Last edited by slawomir.chodnicki; 07-02-2010 at 01:52 AM.

  5. #5

    Exclamation

    Quote Originally Posted by slawomir.chodnicki View Post
    This post show how to breakpoint inside your plugin.

    http://type-exit.org/adventures-with...-in-voldemort/
    Hi Slawo,

    First of all thanks very much for your posts they are honestly invaluable for helping to understand the 4 step classes and how they interact with PDI.

    Second, I have enabled breakpoints and know the exact library where it's failing which is not one contained within libext or liibswt.

    Any other thoughts or ideas? Happy to send you some of the source if you have the time or inclination...

    Thanks for the help

    Mike
    Last edited by mike.sukmanowsky; 07-02-2010 at 08:36 AM.

  6. #6
    Join Date
    Sep 2009
    Posts
    810

    Default

    Hi Mike,

    off the top of my head: If you start spoon on the command line and follow it's console output, do you see the plugin loader complaining? It usually logs errors that occur when loading plugin jars. If it does not, chances are that pdi is not aware of the jar it is supposed to load.

    If you know which jar is not loaded, would you try to explicitly list it in the libraries in the plugin.xml file? Maybe it has a funny name and the wildcard is not picking it up?

    One thing that got me once is that I edited the plugin.xml manually in Spoon's plugins folder and my build script kept overwriting it with a stale version from my eclipse project. Just be sure what plugin.xml file Kettle really sees :-)

    Cheers

    Slawo

  7. #7
    Join Date
    Sep 2009
    Posts
    810

    Default

    Oh, and if you'd like to share the source I'd sure be curious to have a look at it :-)

  8. #8

    Exclamation

    Quote Originally Posted by slawomir.chodnicki View Post
    off the top of my head: If you start spoon on the command line and follow it's console output, do you see the plugin loader complaining? It usually logs errors that occur when loading plugin jars. If it does not, chances are that pdi is not aware of the jar it is supposed to load.
    Hi Slawo - unfortunately it seems like with the release of PDI that I have you are unable to start from the console. After adding the debug parameters to spoon.sh and trying to run from the console I get the following message:

    Starting Data Integration using 'Spoon.sh' from OS X is not supported.
    Please start using 'Data Integration 32-bit' or
    'Data Integration 64-bit' as appropriate.
    Quote Originally Posted by slawomir.chodnicki View Post
    If you know which jar is not loaded, would you try to explicitly list it in the libraries in the plugin.xml file? Maybe it has a funny name and the wildcard is not picking it up?

    One thing that got me once is that I edited the plugin.xml manually in Spoon's plugins folder and my build script kept overwriting it with a stale version from my eclipse project. Just be sure what plugin.xml file Kettle really sees :-)

    I've also double-checked the plugin.xml file and tried wildcards as well as explicit naming of the JAR files and am still throwing the same exception. Frustrating!

    I've sent you a note regarding the source.

    Mike

  9. #9
    Join Date
    Sep 2009
    Posts
    810

    Default

    Hi Mike,

    I suppose you are using 4.0 :-)

    You can run spoon.sh on OSX if you care to edit it :-) For dev purposes I'd say that is OK to do :-)

    In short:

    1. Make sure the darwin section sets the libpath to libswt/osx64/swt.jar (now it displays the error message)
    2. Make sure to add -XstartOnFirstThread to the java options

    If that does not make sense, I'd be happy to attach a modified version of spoon.sh in a few hours (I don't have my usual machine around at the moment)

    *** Okies, here's my spoon.sh that starts 64bit spoon on osx... ***

    Cheers

    Slawo
    Attached Files Attached Files
    Last edited by slawomir.chodnicki; 07-02-2010 at 05:32 PM. Reason: Attached spoon.sh

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •  
Privacy Policy | Legal Notices | Safe Harbor Privacy Policy

Copyright © 2005 - 2017 Pentaho Corporation. All Rights Reserved.