Hitachi Vantara Pentaho Community Forums
Page 3 of 3 FirstFirst 123
Results 21 to 27 of 27

Thread: Plugins and a market place

  1. #21
    Nicholas Goodman Guest

    Default Re: Plugins and a market place

    On Nov 30, 2010, at 12:19 PM, codeoncoffee wrote:

    > I agree with you Matt. OSGI provides a wealth of options that aren't
    > all desirable at the same time. Sitting down and deciding how to


    With OSGi you get many things "for free" just by using it. The other thing one gets is lots of other smart people sorting out complicated issues like applications with depedencies across class loaders, etc.

    For instance, in a service engagement we're building a set of PDI Job plugins. The plugins all have some common functionality (base class) and then we extend them for the specific fcnality. Given the current plugin classloading mechanism, we can't actually install all this as a plugin.

    You can only have 1 plugin per plugin.xml, and each plugin has it's own classloader. Currently, you can't have a package (with a base class) either duplicated across all plugins (you'll get strange errors like Object X cannot be cast to Object X) or only in one (class not found from one plugin to another). The only way currently to solve this problem is to include the base classes, which are really part of the plugins, in the kettle libext and then have the individual plugins extended from that classloader.

    So... there's one piece of functionality that OSGi solves well. The ability for plugins to build "on top of each other" instead of just being able to build upon core Kettle. Enough to warrant a whole framework? Maybe not. All the same, use cases for the way OSGi handles these types of things are there.

    Nick

    --
    You received this message because you are subscribed to the Google Groups "kettle-developers" group.
    To post to this group, send email to kettle-developers (AT) googlegroups (DOT) com.
    To unsubscribe from this group, send email to kettle-developers+unsubscribe (AT) g...oups (DOT) com.
    For more options, visit this group at http://groups.google.com/group/kettle-developers?hl=en.

  2. #22
    Matt Casters Guest

    Default Re: Plugins and a market place

    Hi Nick,

    Your assertions are fortunately not true.
    Have a look at the KFF source code (http://is.gd/iNJ0b) for an example on
    how you can easily deploy a series of plugins.
    In that particular case we deploy 8 plugins (job entries and steps) from a
    single jar file.
    Shared dependency jar files can be placed in an adjacent lib/ folder.
    And best of all: no XML is involved ;-)

    Cheers,

    Matt

    2010/12/15 Nicholas Goodman <ngoodman (AT) bayontechnologies (DOT) com>

    > On Nov 30, 2010, at 12:19 PM, codeoncoffee wrote:
    >
    > > I agree with you Matt. OSGI provides a wealth of options that aren't
    > > all desirable at the same time. Sitting down and deciding how to

    >
    > With OSGi you get many things "for free" just by using it. The other thing
    > one gets is lots of other smart people sorting out complicated issues like
    > applications with depedencies across class loaders, etc.
    >
    > For instance, in a service engagement we're building a set of PDI Job
    > plugins. The plugins all have some common functionality (base class) and
    > then we extend them for the specific fcnality. Given the current plugin
    > classloading mechanism, we can't actually install all this as a plugin.
    >
    > You can only have 1 plugin per plugin.xml, and each plugin has it's own
    > classloader. Currently, you can't have a package (with a base class) either
    > duplicated across all plugins (you'll get strange errors like Object X
    > cannot be cast to Object X) or only in one (class not found from one plugin
    > to another). The only way currently to solve this problem is to include the
    > base classes, which are really part of the plugins, in the kettle libext and
    > then have the individual plugins extended from that classloader.
    >
    > So... there's one piece of functionality that OSGi solves well. The
    > ability for plugins to build "on top of each other" instead of just being
    > able to build upon core Kettle. Enough to warrant a whole framework?
    > Maybe not. All the same, use cases for the way OSGi handles these types of
    > things are there.
    >
    > Nick
    >
    > --
    > You received this message because you are subscribed to the Google Groups
    > "kettle-developers" group.
    > To post to this group, send email to kettle-developers (AT) googlegroups (DOT) com.
    > To unsubscribe from this group, send email to
    > kettle-developers+unsubscribe (AT) g...oups (DOT) com>
    > .
    > For more options, visit this group at
    > http://groups.google.com/group/kettle-developers?hl=en.
    >
    >



    --
    Matt Casters <mcasters (AT) pentaho (DOT) org>
    Chief Data Integration, Kettle founder, Author of Pentaho Kettle
    Solutions<http://www.amazon.com/Pentaho-Kettle-Solutions-Building-Integration/dp/0470635177>
    (Wiley <http://eu.wiley.com/WileyCDA/WileyTitle/productCd-0470635177.html>)
    Fonteinstraat 70, 9400 OKEGEM - Belgium - Cell : +32 486 97 29 37
    Pentaho : The Commercial Open Source Alternative for Business Intelligence

    --
    You received this message because you are subscribed to the Google Groups "kettle-developers" group.
    To post to this group, send email to kettle-developers (AT) googlegroups (DOT) com.
    To unsubscribe from this group, send email to kettle-developers+unsubscribe (AT) g...oups (DOT) com.
    For more options, visit this group at http://groups.google.com/group/kettle-developers?hl=en.

  3. #23
    Nicholas Goodman Guest

    Default Re: Plugins and a market place

    On Dec 15, 2010, at 11:05 AM, Matt Casters wrote:

    > Your assertions are fortunately not true.

    Love it when I'm wrong and it means a limitation isn't actually!!!

    > And best of all: no XML is involved ;-)

    Yeah... I saw some notes in the plugin loader that this was only for testing... Double checking looks like the general classpath search was the one that's marked as testing only. I got the sense that the annotations was the slightly off color method and that the plugin.xml was the primary way to build plugins. Are annotations now the preferred method?

    Does the annotation method bring in all annotated plugins into separate classloaders? ie, with XML you get one classloader per plugin. Now, it would seem you get either
    1) a single classloader for all annotated plugins
    -or-
    2) a single classloader per loaded .jar in plugins/ dir

    This is great news! Makes my day! Multiple plugins in a single jar!

    Nick

    --
    You received this message because you are subscribed to the Google Groups "kettle-developers" group.
    To post to this group, send email to kettle-developers (AT) googlegroups (DOT) com.
    To unsubscribe from this group, send email to kettle-developers+unsubscribe (AT) g...oups (DOT) com.
    For more options, visit this group at http://groups.google.com/group/kettle-developers?hl=en.

  4. #24
    Matt Casters Guest

    Default Re: Plugins and a market place

    Nick, there is no "preferred" method as we decided to support both methods
    of plugin development.
    However, since v4 it's obviously a lot more convenient to use annotations
    for its ease of use.
    There was a slight drawback to developing in Eclipse as documented over
    here: http://is.gd/iNMn1
    However, even that beats restarting Spoon a few hundred times to get a
    plugin developed ;-)
    The ease of use of just being able to unzip a file in plugins/ or drop a jar
    file in there was worth the effort we put into it I think.

    Plugins are added to the collective by whichever way works for you:
    plugin.xml, annotations, kettle-steps.xml, kettle-jobentries.xml and one day
    OSGi. From there on they play their role the same like all steps and job
    entries.
    And yes, they do indeed all use separate class loaders (Except for the SAP
    Input step I believe)

    Cheers,
    Matt

    2010/12/15 Nicholas Goodman <ngoodman (AT) bayontechnologies (DOT) com>

    >
    > On Dec 15, 2010, at 11:05 AM, Matt Casters wrote:
    >
    > > Your assertions are fortunately not true.

    > Love it when I'm wrong and it means a limitation isn't actually!!!
    >
    > > And best of all: no XML is involved ;-)

    > Yeah... I saw some notes in the plugin loader that this was only for
    > testing... Double checking looks like the general classpath search was the
    > one that's marked as testing only. I got the sense that the annotations was
    > the slightly off color method and that the plugin.xml was the primary way to
    > build plugins. Are annotations now the preferred method?
    >
    > Does the annotation method bring in all annotated plugins into separate
    > classloaders? ie, with XML you get one classloader per plugin. Now, it
    > would seem you get either
    > 1) a single classloader for all annotated plugins
    > -or-
    > 2) a single classloader per loaded .jar in plugins/ dir
    >
    > This is great news! Makes my day! Multiple plugins in a single jar!
    >
    > Nick
    >
    > --
    > You received this message because you are subscribed to the Google Groups
    > "kettle-developers" group.
    > To post to this group, send email to kettle-developers (AT) googlegroups (DOT) com.
    > To unsubscribe from this group, send email to
    > kettle-developers+unsubscribe (AT) g...oups (DOT) com>
    > .
    > For more options, visit this group at
    > http://groups.google.com/group/kettle-developers?hl=en.
    >
    >



    --
    Matt Casters <mcasters (AT) pentaho (DOT) org>
    Chief Data Integration, Kettle founder, Author of Pentaho Kettle
    Solutions<http://www.amazon.com/Pentaho-Kettle-Solutions-Building-Integration/dp/0470635177>
    (Wiley <http://eu.wiley.com/WileyCDA/WileyTitle/productCd-0470635177.html>)
    Fonteinstraat 70, 9400 OKEGEM - Belgium - Cell : +32 486 97 29 37
    Pentaho : The Commercial Open Source Alternative for Business Intelligence

    --
    You received this message because you are subscribed to the Google Groups "kettle-developers" group.
    To post to this group, send email to kettle-developers (AT) googlegroups (DOT) com.
    To unsubscribe from this group, send email to kettle-developers+unsubscribe (AT) g...oups (DOT) com.
    For more options, visit this group at http://groups.google.com/group/kettle-developers?hl=en.

  5. #25
    codeoncoffee Guest

    Default Re: Plugins and a market place

    Quick update. I've just committed a revised approach to getting OSGI
    in Kettle. This new implementation is completely hidden behind the
    current PluginRegistry api. With this commit, OSGI is just another
    avenue to getting your plugin registered with the current system.

    I will be adding a listener mechanism to the PluginRegistry so Kettle
    can become aware as plugins come and go and react appropriately. This
    provides the hot-swapping and loading.

    The rest of the functionality added by OSGI will be available to the
    plugin developers through traditional OSGI APIs. The MaxMind example
    step uses the Blueprint beans approach right now. So far I'm only
    registering Spoon as a service object but others will surely come.

    I'm reasonably satisfied with this current effort. I'd like to see
    more types proven-out other than steps, scheduler interface for
    instance would be a nice demo. And the current directory watching
    system needs to be replaced with Apache Karaf or just removed.

    -Nick B.

    On Dec 15, 2:28

  6. #26
    codeoncoffee Guest

    Default Re: Plugins and a market place

    The wiki has been updated with samples from the MaxMind Plugin:
    http://wiki.pentaho.com/display/EAI/OSGI-enabled+Kettle



    On Dec 21, 8:55

  7. #27
    codeoncoffee Guest

    Default Re: Plugins and a market place

    The wiki has been updated with samples from the MaxMind Plugin:
    http://wiki.pentaho.com/display/EAI/OSGI-enabled+Kettle

    On Dec 21, 8:55

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •  
Privacy Policy | Legal Notices | Safe Harbor Privacy Policy

Copyright © 2005 - 2019 Hitachi Vantara Corporation. All Rights Reserved.