xml file Performance issue
Hi All,
i have to generate many xml files.
Infact, need to generate about a million xml files ,one for each member.
I am looping member ids in a job to generate xml and generating the files.
It caused heap size error "java.lang.OutOfMemoryError: Java heap space".
So i increased the setting in spoon.sh OPT =-Xmx1024m . But it didnt work out.
So i tried to run it in a job executer step.
Now the job is working fine.
But it is taking one hour to generate 1000 xml file.
Is there any way to improve the performance ?
Each file will take about 2-3 seconds to generate. And 1 million record means
a million selects to Db,i million file generation. My transformation have table input steps,add xml ,join xml, replace in string,text file out put steps.
i am not able to take the whole records like 'select * from members' because the xml file will not generate for one member.(Add xml file cannot do for one member data alone)
Any other logic to generate the xmls?
Any help would be great!!!
Thanks,
Vineetha P
