Hitachi Vantara Pentaho Community Forums
Results 1 to 7 of 7

Thread: HOW TO develop a new Job Entry

  1. #1
    Join Date
    Aug 2007
    Posts
    10

    Default HOW TO develop a new Job Entry

    Hi,

    I am trying to develop a new job entry: the aim of this job is to be able to integrate R statistical computing in PDI flow.
    For information about R, visit http://www.r-project.org

    I started developping two java classes, JobEntryRBox.java and JobEntryRBoxDialog.java, I created a subdirectory in plugins/jobentries called RBoxJobEntry and put plugin.xml, RBox.png and RBox.jar inside.
    My plugin.xml looks like this:

    <?xml version="1.0" encoding="UTF-8"?>
    <plugin
    id="RBoxPlugin"
    iconfile="RBox.png"
    description="R Plugin"
    tooltip="Integrate Statistical Plugin"
    category="JobEntry"
    classname="fr.agrostar.kettle.plugin.rbox.JobEntryRBox">
    <libraries>
    <library name="RBox.jar"/>
    </libraries>

    <localized_category>
    <category locale="en_US">Transform</category>
    <category locale="nl_NL">Transform</category>
    <category locale="fr_FR">Transformation</category>
    </localized_category>
    <localized_description>
    <description locale="en_US">R plugin</description>
    <description locale="nl_NL">Plugin R</description>
    <description locale="fr_FR">Plugin R</description>
    </localized_description>
    <localized_tooltip>
    <tooltip locale="en_US">This is a R plugin step allowing statistical computing</tooltip>
    <tooltip locale="fr_FR">Le plugin R permet d'integrer des calculs statistiques</tooltip>
    </localized_tooltip>
    </plugin>

    When I start my PDI 2.5.0 trying to create a new job, I have the following error

    java.lang.NullPointerException
    at be.ibridge.kettle.spoon.Spoon.addExpandBarItemLine(Spoon.java:2092)
    at be.ibridge.kettle.spoon.Spoon.refreshCoreObjects(Spoon.java:2045)
    at be.ibridge.kettle.spoon.Spoon.enableMenus(Spoon.java:5282)
    at be.ibridge.kettle.spoon.Spoon.addChefGraph(Spoon.java:8060)
    at be.ibridge.kettle.spoon.Spoon.newJobFile(Spoon.java:4074)
    at be.ibridge.kettle.spoon.Spoon$54.widgetSelected(Spoon.java:1529)
    at org.eclipse.swt.widgets.TypedListener.handleEvent(TypedListener.java:227)
    at org.eclipse.swt.widgets.EventTable.sendEvent(EventTable.java:66)
    at org.eclipse.swt.widgets.Widget.sendEvent(Widget.java:938)
    at org.eclipse.swt.widgets.Display.runDeferredEvents(Display.java:3673)
    at org.eclipse.swt.widgets.Display.readAndDispatch(Display.java:3284)
    at be.ibridge.kettle.spoon.Spoon.readAndDispatch(Spoon.java:964)
    at be.ibridge.kettle.spoon.Spoon.main(Spoon.java:7325)


    Can someone tell me what I am doing wrong ?

    Thanks for your help.

  2. #2
    Join Date
    May 2006
    Posts
    4,882

    Default

    I'm guessing it has something to do with your icon image RBox.png ... I would suggest to download the dummy job entry (which I don't find anymore for the moment) and check how it's done there.

    BTW are you sure you need a new job entry, and not a new step. Also note that you can't use graphical components from within ETL... it would still work in 2.5, but it won't work in v3.0 (currently).

    Regards,
    Sven
    Last edited by sboden; 08-06-2007 at 06:49 AM.

  3. #3
    Join Date
    Aug 2007
    Posts
    10

    Default

    I thought about developing either a job entry or a step and my conclusion was to develop a job entry because R processes a set of rows at once and I could not figure out I can match this with step philosophy that in my mind is more line based.
    But if you have advices on this, I would certainly change my mind.

    To start developing my plugin, I looked at how the DummyJobEntry was done: the directory contains DPL.png, dummyjob.jar and plugin.xml .
    I cannot see what I made differently in my RBox plugin.

    And if you can help me understand your remarks about v3.0 and graphical components, I would really appreciate, I did not get you at all.

  4. #4
    Join Date
    May 2006
    Posts
    4,882

    Default

    I thought about developing either a job entry or a step and my conclusion was to develop a job entry because R processes a set of rows at once and I could not figure out I can match this with step philosophy that in my mind is more line based.
    But if you have advices on this, I would certainly change my mind.
    Well... steps are row based, but job entries are actually about flow control, not really about processing data. Although the boundaries are fading a bit with the mysql bulk loader, ...

    To start developing my plugin, I looked at how the DummyJobEntry was done: the directory contains DPL.png, dummyjob.jar and plugin.xml .
    I cannot see what I made differently in my RBox plugin.
    Looking at the code it has to do with your icon file... but without anything to debug it's hard of course. I would suggest to put a break point on be.ibridge.kettle.spoon.Spoon.addExpandBarItemLine and start from there.

    And if you can help me understand your remarks about v3.0 and graphical components, I would really appreciate, I did not get you at all.
    Write a transformation using a javascript step using Alert (which show a dialog box) and run it in 2.5 and in 3.0... it will work in 2.5, for the moment not in 3.0... In general PDI is an ETL tool, it's not really to be used to display graphs, ...

    Regards.
    Sven

  5. #5
    Join Date
    Aug 2007
    Posts
    10

    Default

    Exactly what I thought when I see the mysql bulk loader, that is why ...

    On the last point, I completely understand the aim of an ETL and the plugin I would like to develop is only to make easier the integration of the statistical computing using R in batch mode not the graphical part of R.
    A example is typically extract a set of data, compute it to produce a forecast with this plugin and then store it in a database, making R part of the information production environment.

    Will give news soon !
    Thanks for your help.
    Benoit

  6. #6
    Join Date
    Sep 2006
    Posts
    10

    Default

    Hi,

    I have also been looking at trying to program a R-to-Kettle transformation step, I understand your reasoning about processing as a job, but there may be situations where you need to stats on groups of data, a bit like the group by transformation. Thats Why I was focussing on a transformation.

    My main problem has been getting my head around SWT ( since I am a server/ web interface based developer). I have tried several visual editor plugins for eclipse, but with little success. I was just wondering what do you use - or are you all hardcore write it by hand SWT coders?

    BTW a R Plugin would be really really useful for many users, as there are many R routines available for data mining etc.

    Regards

  7. #7
    Join Date
    May 2006
    Posts
    4,882

    Default

    We are all really "write it by hand SWT coders"... it's not that hard when you start from an existing job.

    The interfaces for job entries/steps have changed from 2.5 so you may need to do a small conversion. If you want to share the step/job entry you can do so for the moment at: http://wiki.pentaho.org/display/EAI/...ation+Plug-Ins

    Regards,
    Sven

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •  
Privacy Policy | Legal Notices | Safe Harbor Privacy Policy

Copyright © 2005 - 2019 Hitachi Vantara Corporation. All Rights Reserved.