PDA

View Full Version : Integrating from a Java Object



alpheratz
08-10-2006, 03:10 AM
[I'm sure that this is a FAQ but darned if I can find anything on this subject. Apologies if I am being stupid...]

I have a OLD legacy socket-based server written in C using an organization-specific request/reply protocol. This would feed data to a truly ghastly VB client on windows.

As the first step in working out how to 'modernise' this system, I have written a small 'adaptor' Java object to drive this server through its request/reply cycle (in brief: opens socket; sends a request string/returns a result Java ArrayList; close socket and delete self).

The question is: how to integrate this with Pentaho? I presume that I should use Kettle/spoon but I can't see how...

I have this sketched out/prototyped with BIRT 2.1 reporting, but I'd love to compare with Pentaho (which looks much better, overall :-))

Performance is an issue (application is actually a SCADA logging system with about 15 years worth of 1 second data stored away for each of roughly 200,000 sensors. One typically only reports on a SMALL selection of sensors from the last quarter or [at most] year but that is still potentially a lot of data to handle ;-))

The performance issues argues against using Javascript and staying in Java as much as possible.

OTOH at the moment I need to quickly get a feel for FUNCTIONALITY, [getting a quick demo to the "powers that be"] rather than concentrating of the performance issue (but this WILL become an issue later).

Cheers,

Alpheratz

jbleuel
08-10-2006, 03:17 AM
Alpheratz,

yes Kettle is the right place to start and build a plugin for your demand . An example Dummy plugin is available, see http://www.kettle.be/en/dload.htm (Source code, check it out and see how the Dummy plugin is built)

Hope that helps,
Jens

alpheratz
08-10-2006, 02:43 PM
Thanks so much for the fast reply.

I will check the source out...

I haven't done so yet, but the other question I have concerns integration with Pentaho: is there a way to avoid the creation of a temporary file in the filesystem. My current understanding is that Pentaho can report against a file and that this file can be created as a result of a Kettle transformation (specified in the design studio).

I'd just like to cut out the intermediate file.

I have read that JFreeReport can report directly from a Swing TableModel, so could I create an instance of this as a result of the Kettle transformation?

Thanks,

Alpheratz

Taqua
08-10-2006, 05:03 PM
There are no temporary files needed, unless you tell the system to create some.

Some Kettle-Operations may need temporary files (Sorting on large datasets, for instance), but the plain Kettle stuff in conjunction with the reporting components should keep everything in memory (and if it doesnt, I would count it as a bug). Only the resulting content might be written to the file system. (This can be avoided with the new reporting components of the current development version (not the release candidate).

Regards,
Thomas

alpheratz
08-14-2006, 03:50 AM
Thanks for that, Thomas. Much appreciate your quick responses.

I think I now understand that Kettle integrates with Pentaho through memory where feasible....

I am still a little unclear about my situation, however...I need to interface to a Java POJO that gets its data across the network via a simple socket connection.

Am I right in thinking that this has to be done with an Injector? Is there any example code that I can look at in the source or (better) examples?

Cheers,

Alpheratz

alpheratz
08-21-2006, 01:50 AM
hi guys.

No response to my last post?

I can see how to use a POJO in BIRT and I think I understand BIRT's ODA infrastructure now...but I still haven't got my head around "the pentaho way"...and I would really like to!

So: to reiterate...

I have a OLD legacy socket-based server written in C using an organization-specific request/reply
protocol. This feedsdata to a truly ghastly VB client on windows.

As the first step in working out how to 'modernise' this system, I have written a small 'adaptor'
Java object to drive this server through its request/reply cycle (in brief: opens socket; sends a
request string/returns a result Java ArrayList; close socket and delete self).

The question is: how to integrate this with Pentaho?

Cheers,

BOB

MattCasters
08-21-2006, 01:58 AM
Since you post this here, I'll give you a solution in Pentaho Data Integration, but i guess you can write a Pentaho web Component that could do something similar.

I guess the easiest to go about doing this is by writing your own plugin for PDI.
This is probably easier than you would think at first. We have a Dummy Plugin available to help you get started: http://www.javaforge.com/proj/doc.do?doc_id=3701

This will allow you to integrate your legacy server into the ETL framework.
If you have input parameters to be set etc, you can use PDI to help you out there as well. Once it's done, you can re-use the new plugin object everywhere, effectively making the legacy server part of the normal data flow.

HTH,

Matt

Post edited by: MattCasters, at: 08/21/2006 06:06

alpheratz
08-21-2006, 03:39 AM
Thanks for the reply, Matt.

I am posting here because that seemed to me to be the best place. Apologies if my ignorance offends :-)

If 'here' is not the best place...I'd be grateful for a suggestion as to where would be more appropriate, I am still in the dark.

Just did a search for "Web Component " across the Pentaho site using the search box at the top of the page and turned up nothing that seemed relevant...

I'll have a look at the dummy plugin you suggested.

many thanks.

MattCasters
08-21-2006, 04:18 AM
Hey no worries,

The problem really is that I'm a novice in the Pentaho realm myself. :-)
I can only tell you how I would do it in the Data Integration way of things.

However, it may very well be that implementing a new Pentaho component, extend class ComponentBase (pkg: org.pentaho.plugin) is more appropriate.
(See source code of KettleComponent for example)

I'm off very little help there, so posting to for example the BI Framework discussions http://www.pentaho.org/index.php?option=com_simpleboard&Itemid=275&func=showcat&catid=14 might be more efficient ;-)

All the best,

Matt

mbatchelor
08-22-2006, 07:49 AM
Pentaho is easily extended to use other sources of data. The simplest is to implement your own IPentahoResultSet and IPentahoconnection interfaces, and then a component (IComponent) that knows how give it parameters/settings.

Simple examples abound in the Pentaho source code - but, for simplicity, get the Pentaho source code, and look at the following places:

The best example of an implementation of IPentahoResultSet you could look at would be org.pentaho.core.connection.memory.MemoryResultSet.

Then, you could have a quick look at the org.pentaho.plugin.core.HelloWorldComponent for a super-simple component.

Depending on the circumstances, your easiest way may be to just write your own component (like hello world) that talks to your java class, and creates a MemoryResultSet. However, once it's in the platform as an IPentahoResultSet, you can create reports on it, play with it in a JavascriptRule, and many other things.

I hope this helped,

Marc