Hitachi Vantara Pentaho Community Forums
Results 1 to 10 of 10

Thread: Integrating from a Java Object

  1. #1
    Join Date
    Aug 2006
    Posts
    14

    Default Integrating from a Java Object

    [I'm sure that this is a FAQ but darned if I can find anything on this subject. Apologies if I am being stupid...]

    I have a OLD legacy socket-based server written in C using an organization-specific request/reply protocol. This would feed data to a truly ghastly VB client on windows.

    As the first step in working out how to 'modernise' this system, I have written a small 'adaptor' Java object to drive this server through its request/reply cycle (in brief: opens socket; sends a request string/returns a result Java ArrayList; close socket and delete self).

    The question is: how to integrate this with Pentaho? I presume that I should use Kettle/spoon but I can't see how...

    I have this sketched out/prototyped with BIRT 2.1 reporting, but I'd love to compare with Pentaho (which looks much better, overall :-))

    Performance is an issue (application is actually a SCADA logging system with about 15 years worth of 1 second data stored away for each of roughly 200,000 sensors. One typically only reports on a SMALL selection of sensors from the last quarter or [at most] year but that is still potentially a lot of data to handle ;-))

    The performance issues argues against using Javascript and staying in Java as much as possible.

    OTOH at the moment I need to quickly get a feel for FUNCTIONALITY, [getting a quick demo to the "powers that be"] rather than concentrating of the performance issue (but this WILL become an issue later).

    Cheers,

    Alpheratz

  2. #2
    Join Date
    Nov 1999
    Posts
    459

    Default Re:Integrating from a Java Object

    Alpheratz,

    yes Kettle is the right place to start and build a plugin for your demand . An example Dummy plugin is available, see http://www.kettle.be/en/dload.htm (Source code, check it out and see how the Dummy plugin is built)

    Hope that helps,
    Jens

  3. #3
    Join Date
    Aug 2006
    Posts
    14

    Default Re:Integrating from a Java Object

    Thanks so much for the fast reply.

    I will check the source out...

    I haven't done so yet, but the other question I have concerns integration with Pentaho: is there a way to avoid the creation of a temporary file in the filesystem. My current understanding is that Pentaho can report against a file and that this file can be created as a result of a Kettle transformation (specified in the design studio).

    I'd just like to cut out the intermediate file.

    I have read that JFreeReport can report directly from a Swing TableModel, so could I create an instance of this as a result of the Kettle transformation?

    Thanks,

    Alpheratz

  4. #4
    Join Date
    Mar 2003
    Posts
    8,085

    Default Re:Integrating from a Java Object

    There are no temporary files needed, unless you tell the system to create some.

    Some Kettle-Operations may need temporary files (Sorting on large datasets, for instance), but the plain Kettle stuff in conjunction with the reporting components should keep everything in memory (and if it doesnt, I would count it as a bug). Only the resulting content might be written to the file system. (This can be avoided with the new reporting components of the current development version (not the release candidate).

    Regards,
    Thomas
    Get the latest news and tips and tricks for Pentaho Reporting at the Pentaho Reporting Blog.

  5. #5
    Join Date
    Aug 2006
    Posts
    14

    Default Re:Integrating from a Java Object

    Thanks for that, Thomas. Much appreciate your quick responses.

    I think I now understand that Kettle integrates with Pentaho through memory where feasible....

    I am still a little unclear about my situation, however...I need to interface to a Java POJO that gets its data across the network via a simple socket connection.

    Am I right in thinking that this has to be done with an Injector? Is there any example code that I can look at in the source or (better) examples?

    Cheers,

    Alpheratz

  6. #6
    Join Date
    Aug 2006
    Posts
    14

    Default Re:Integrating from a Java Object

    hi guys.

    No response to my last post?

    I can see how to use a POJO in BIRT and I think I understand BIRT's ODA infrastructure now...but I still haven't got my head around "the pentaho way"...and I would really like to!

    So: to reiterate...

    I have a OLD legacy socket-based server written in C using an organization-specific request/reply
    protocol. This feedsdata to a truly ghastly VB client on windows.

    As the first step in working out how to 'modernise' this system, I have written a small 'adaptor'
    Java object to drive this server through its request/reply cycle (in brief: opens socket; sends a
    request string/returns a result Java ArrayList; close socket and delete self).

    The question is: how to integrate this with Pentaho?

    Cheers,

    BOB

  7. #7
    Join Date
    Nov 1999
    Posts
    9,729

    Default Re:Integrating from a Java Object

    Since you post this here, I'll give you a solution in Pentaho Data Integration, but i guess you can write a Pentaho web Component that could do something similar.

    I guess the easiest to go about doing this is by writing your own plugin for PDI.
    This is probably easier than you would think at first. We have a Dummy Plugin available to help you get started: http://www.javaforge.com/proj/doc.do?doc_id=3701

    This will allow you to integrate your legacy server into the ETL framework.
    If you have input parameters to be set etc, you can use PDI to help you out there as well. Once it's done, you can re-use the new plugin object everywhere, effectively making the legacy server part of the normal data flow.

    HTH,

    Matt

    Post edited by: MattCasters, at: 08/21/2006 06:06

  8. #8
    Join Date
    Aug 2006
    Posts
    14

    Default Re:Integrating from a Java Object

    Thanks for the reply, Matt.

    I am posting here because that seemed to me to be the best place. Apologies if my ignorance offends :-)

    If 'here' is not the best place...I'd be grateful for a suggestion as to where would be more appropriate, I am still in the dark.

    Just did a search for "Web Component " across the Pentaho site using the search box at the top of the page and turned up nothing that seemed relevant...

    I'll have a look at the dummy plugin you suggested.

    many thanks.

  9. #9
    Join Date
    Nov 1999
    Posts
    9,729

    Default Re:Integrating from a Java Object

    Hey no worries,

    The problem really is that I'm a novice in the Pentaho realm myself. :-)
    I can only tell you how I would do it in the Data Integration way of things.

    However, it may very well be that implementing a new Pentaho component, extend class ComponentBase (pkg: org.pentaho.plugin) is more appropriate.
    (See source code of KettleComponent for example)

    I'm off very little help there, so posting to for example the BI Framework discussions http://www.pentaho.org/index.php?opt...owcat&catid=14 might be more efficient ;-)

    All the best,

    Matt

  10. #10
    Join Date
    Nov 1999
    Posts
    579

    Default Re:Integrating from a Java Object

    Pentaho is easily extended to use other sources of data. The simplest is to implement your own IPentahoResultSet and IPentahoconnection interfaces, and then a component (IComponent) that knows how give it parameters/settings.

    Simple examples abound in the Pentaho source code - but, for simplicity, get the Pentaho source code, and look at the following places:

    The best example of an implementation of IPentahoResultSet you could look at would be org.pentaho.core.connection.memory.MemoryResultSet.

    Then, you could have a quick look at the org.pentaho.plugin.core.HelloWorldComponent for a super-simple component.

    Depending on the circumstances, your easiest way may be to just write your own component (like hello world) that talks to your java class, and creates a MemoryResultSet. However, once it's in the platform as an IPentahoResultSet, you can create reports on it, play with it in a JavascriptRule, and many other things.

    I hope this helped,

    Marc

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •  
Privacy Policy | Legal Notices | Safe Harbor Privacy Policy

Copyright © 2005 - 2019 Hitachi Vantara Corporation. All Rights Reserved.