Hi All,

I have a clustered transformation that uses, 2 slaves and 1 master. I need to process 24,000,000 records. The transformation has a table input step, a javascript step and a file writing step. At the moment what I have done is I executed the table input step in master and passed the data to 2 Javascript slave copies and finally combined the data for wrting in the master again. So the table data is read in master and it has to be distributed to slaves. Is there a way I can run the table input step in the slave in such a way that the two table input slave copies reads different set of data using a partitioning logic? This could save the data distribution time over the netword and speed up the process.

I tried to create two copies of the table input step in 2 slaves with a partition logic in that step, but both the copies are reading the full data and we are getting the total rows doubled in the output.

Can any one share your thoughts regarding this point?

Regards,
Sunil