PDA

View Full Version : Kettle for integration



merayo
04-19-2006, 11:09 AM
Matt,
I need an integration tool to syncronize data online. For this I need it to process rows for Insert, update and delete operations.
¿Have you done something like this before? ¿do you have any thoughts on supporting this type of input output native in Kettle?

MattCasters
04-19-2006, 02:48 PM
Please search the forum for similar answers to these kind of questions.

Thanks,
Matt

merayo
04-20-2006, 10:48 AM
I did. I'll get the delete step. I already made a program to create a automatically a journal for the tables I'm interested by triggering.
In Kettle I would have to use a field on each row to indicate wether its an Insert, Update or Delete and send some rows to the insert/update step and others to the Delete step.
This I would have to do for each table I want to replicate on-line, so instead of making this transformation for each table I imagine a step that did it alltoguether and decided to insert or delete depending on the value of one field of the Row, maybe run some JS.
¿Has anyone thought of a better solution?

MattCasters
04-20-2006, 11:02 PM
Hi Merayo,

If you already have the flag on the record from your trigger, then you can just put a filter on that and send the records to either a Delete or an Insert/Update step depending on the flag.
That is only 3 steps with the advantage that the filter condition can be just about anything.
IMHO it doesn't really warrant *another* new step that has to be maintained independently.

Do let me know if this works for you or if you run into trouble. There are many people facing this issue so I want to make sure we get this to work.

On a higher level (as a BI person) of-course I have to wonder if people are not using this to secretely create an evil ODS to report on. :-(

Thanks,
Matt