Transformation - Step by Step

[leandroconcon]

Hi, I'm needing to do a transformation where my STEP can read the informations from a text file, and inserts into database,with 4 steps to do the insertion and the transformation will execute one step per time, and not simultaneously

For example

FIXED FILE INPUT ----> TABLE OUTPUT 01
Ending all inserts, will go to the next step
----> TABLE OUTPUT 02
Ending all inserts, will go to the next step
----> TABLE OUTPUT 03
Ending all inserts, will go to the next step
----> TABLE OUTPUT 04
Ending all inserts, will go to the next step

Tks,

Concon

[sboden]

Is not really possible right now... unless you all of your steps in a seperate transformation in a job but that would take too much processing power and is very clumsy.

Regards,
Sven

[ngoodman]

I think you may be able to accomplish your needs with the "Blocking Step"

It waits for all rows before outputting rows. Place them between your steps and it might work for you.

This, in practice, only works with small datasets because it keeps it in memory or swaps.

[sboden]

Blocking step also only works once per execution, so you would still need to use "execute for every row" and execute the transformation per incoming row.

Regards,
Sven

[leandroconcon]

Blocking step also only works once per execution, so you would still need to use "execute for every row" and execute the transformation per incoming row.

Regards,
Sven

Is "execute for every row" a Step or a Step option?

[sboden]

an option in the transformation job entry.

Regards,
Sven

[ngoodman]

Sven may be correct that what you want to do is not possible. I did put together an example that looks like it. I'm not sure this is exactly what you want but it might work for your needs given some tweaking/etc.

http://www.nicholasgoodman.com/bt/bl...ows-in-kettle/

[a]http://www.nicholasgoodman.com/bt/blog/2008/06/25/ordered-rows-in-kettle[/a]

[philins]

Sven may be correct that what you want to do is not possible. I did put together an example that looks like it. I'm not sure this is exactly what you want but it might work for your needs given some tweaking/etc.

http://www.nicholasgoodman.com/bt/bl...ows-in-kettle/

[a]http://www.nicholasgoodman.com/bt/blog/2008/06/25/ordered-rows-in-kettle[/a]

I use ur example, It seems right.But I tried this method in my transformation, It seems not work.

[Maria Roldan]

Leandro
I understood that you wanted to insert in table 1, then in table 2, ...etc, but not in a row by row fashion. If I understood well, you could do that in four transformations, and call them one after the other from a main job.

[ngoodman]

I use ur example, It seems right.But I tried this method in my transformation, It seems not work.

Should the above solution not work for you, continue to look a the solution Sven suggested with "execute once for every row" This too is a perfectly valid approach to the splitting the job up into individual steps.