Hitachi Vantara Pentaho Community Forums
Results 1 to 3 of 3

Thread: New plugin in svn, needs some TLC

  1. #1
    DEinspanjer Guest

    Default New plugin in svn, needs some TLC

    svn://source.pentaho.org/svnkettleroot/plugins/MaxMindGeoIPLookup/trunk/

    I checked in a plugin for looking up GeoIP information from a MaxMind
    database. I really only had time to get the plugin working for me,
    but it has the potential to be a very useful plugin once it gets
    cleaned up and generified a bit.

    There are TODOs in the source, but the major items that need work are:

    1. The fields list in the dialog is not currently hooked up to the
    Meta class. The user needs to be able to add fields to the list
    selecting exactly what kind of lookups to perform. It needs an enum
    of the available MaxMind fields in all the different databases. It
    could get very fancy with autogenerated lists and such, but just
    providing the user the ability to select the particular fields they
    want (e.g. country code, country name, region code, city name,
    timezone, DMA, etc.) would be good enough to be useful to people.

    2. It could use more validations to check things like whether the
    database they pointed at exists and is valid and such.

    3. The Class static synchronized reference to the memory cached
    database should be code reviewed. I've had a few cases where my very
    long running transformations have been deadlocking, and I haven't
    figured out whether that might be related to this plugin or something
    else. I tried to design it such that it would only keep one copy of
    the MaxMind DB in memory per JVM and would make that object available
    for garbage collection if there were no transformations actively using
    it, but I might have messed something up.

    MaxMind's Java API is OSS so I included it in the SVN, but some effort
    should be made to tag the source files with an appropriate licensing
    declaration (LGPL?). I will relinquish all copyright once a license
    notification is in place that is agreeable to MaxMind and Pentaho.
    --~--~---------~--~----~------------~-------~--~----~
    You received this message because you are subscribed to the Google Groups "kettle-developers" group.
    To post to this group, send email to kettle-developers (AT) googlegroups (DOT) com
    To unsubscribe from this group, send email to kettle-developers-unsubscribe (AT) g...oups (DOT) com
    For more options, visit this group at http://groups.google.com/group/kettle-developers?hl=en
    -~----------~----~----~----~------~----~------~--~---

  2. #2
    nicholas guzaldo Guest

    Default Re: New plugin in svn, needs some TLC

    The plugin sounds pretty groovy. So what are you guys not only great at
    making kettle AWESOME but now can read dreams?
    I'll be checking this guy out for sure down the road.

    Thanks,

    Nic

    On Thu, Apr 24, 2008 at 3:56 PM, DEinspanjer <deinspanjer (AT) gmail (DOT) com> wrote:

    >
    > svn://source.pentaho.org/svnkettleroot/plugins/MaxMindGeoIPLookup/trunk/
    >
    > I checked in a plugin for looking up GeoIP information from a MaxMind
    > database. I really only had time to get the plugin working for me,
    > but it has the potential to be a very useful plugin once it gets
    > cleaned up and generified a bit.
    >
    > There are TODOs in the source, but the major items that need work are:
    >
    > 1. The fields list in the dialog is not currently hooked up to the
    > Meta class. The user needs to be able to add fields to the list
    > selecting exactly what kind of lookups to perform. It needs an enum
    > of the available MaxMind fields in all the different databases. It
    > could get very fancy with autogenerated lists and such, but just
    > providing the user the ability to select the particular fields they
    > want (e.g. country code, country name, region code, city name,
    > timezone, DMA, etc.) would be good enough to be useful to people.
    >
    > 2. It could use more validations to check things like whether the
    > database they pointed at exists and is valid and such.
    >
    > 3. The Class static synchronized reference to the memory cached
    > database should be code reviewed. I've had a few cases where my very
    > long running transformations have been deadlocking, and I haven't
    > figured out whether that might be related to this plugin or something
    > else. I tried to design it such that it would only keep one copy of
    > the MaxMind DB in memory per JVM and would make that object available
    > for garbage collection if there were no transformations actively using
    > it, but I might have messed something up.
    >
    > MaxMind's Java API is OSS so I included it in the SVN, but some effort
    > should be made to tag the source files with an appropriate licensing
    > declaration (LGPL?). I will relinquish all copyright once a license
    > notification is in place that is agreeable to MaxMind and Pentaho.
    > >

    >


    --~--~---------~--~----~------------~-------~--~----~
    You received this message because you are subscribed to the Google Groups "kettle-developers" group.
    To post to this group, send email to kettle-developers (AT) googlegroups (DOT) com
    To unsubscribe from this group, send email to kettle-developers-unsubscribe (AT) g...oups (DOT) com
    For more options, visit this group at http://groups.google.com/group/kettle-developers?hl=en
    -~----------~----~----~----~------~----~------~--~---

  3. #3
    Matt Casters Guest

    Default Re: New plugin in svn, needs some TLC

    Hi Nic,

    We can't read minds yet, but we're already thinking about that specific
    functionality : http://jira.pentaho.org/browse/PDI-1028

    This work by Daniel is a follow up on an initial effort done by Pentaho
    partner BreadBoard BI (for use in JavaScript). I guess a lot of people
    contacted me in the past about MaxMind functionality and as such I'm happy
    that we'll have a plugin to play around with.

    All the best,

    Matt



    On Thursday 24 April 2008 22:59:13 nicholas guzaldo wrote:
    > The plugin sounds pretty groovy. So what are you guys not only great at
    > making kettle AWESOME but now can read dreams?
    > I'll be checking this guy out for sure down the road.
    >
    > Thanks,
    >
    > Nic
    >
    > On Thu, Apr 24, 2008 at 3:56 PM, DEinspanjer <deinspanjer (AT) gmail (DOT) com> wrote:
    > > svn://source.pentaho.org/svnkettleroot/plugins/MaxMindGeoIPLookup/trunk/
    > >
    > > I checked in a plugin for looking up GeoIP information from a MaxMind
    > > database. I really only had time to get the plugin working for me,
    > > but it has the potential to be a very useful plugin once it gets
    > > cleaned up and generified a bit.
    > >
    > > There are TODOs in the source, but the major items that need work are:
    > >
    > > 1. The fields list in the dialog is not currently hooked up to the
    > > Meta class. The user needs to be able to add fields to the list
    > > selecting exactly what kind of lookups to perform. It needs an enum
    > > of the available MaxMind fields in all the different databases. It
    > > could get very fancy with autogenerated lists and such, but just
    > > providing the user the ability to select the particular fields they
    > > want (e.g. country code, country name, region code, city name,
    > > timezone, DMA, etc.) would be good enough to be useful to people.
    > >
    > > 2. It could use more validations to check things like whether the
    > > database they pointed at exists and is valid and such.
    > >
    > > 3. The Class static synchronized reference to the memory cached
    > > database should be code reviewed. I've had a few cases where my very
    > > long running transformations have been deadlocking, and I haven't
    > > figured out whether that might be related to this plugin or something
    > > else. I tried to design it such that it would only keep one copy of
    > > the MaxMind DB in memory per JVM and would make that object available
    > > for garbage collection if there were no transformations actively using
    > > it, but I might have messed something up.
    > >
    > > MaxMind's Java API is OSS so I included it in the SVN, but some effort
    > > should be made to tag the source files with an appropriate licensing
    > > declaration (LGPL?). I will relinquish all copyright once a license
    > > notification is in place that is agreeable to MaxMind and Pentaho.

    >
    >



    --
    Matt
    ____________________________________________
    Matt Casters
    Chief Data Integration - Kettle founder
    Pentaho, Open Source Business Intelligence
    http://www.pentaho.org -- mcasters (AT) pentaho (DOT) org
    Tel. +32 (0) 486 97 29 37

    --~--~---------~--~----~------------~-------~--~----~
    You received this message because you are subscribed to the Google Groups "kettle-developers" group.
    To post to this group, send email to kettle-developers (AT) googlegroups (DOT) com
    To unsubscribe from this group, send email to kettle-developers-unsubscribe (AT) g...oups (DOT) com
    For more options, visit this group at http://groups.google.com/group/kettle-developers?hl=en
    -~----------~----~----~----~------~----~------~--~---

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •  
Privacy Policy | Legal Notices | Safe Harbor Privacy Policy

Copyright © 2005 - 2019 Hitachi Vantara Corporation. All Rights Reserved.