I'm using Spoon to read in a clients data and generate XML files that our product can read in to import but I'm getting java heap space errors. I tried setting the command line to get '-Xmx1024m' but that hasn't helped (running 32bit java so can't go above 1024). So I think I need to rewrite my logic.

Right now I read into about 3000 rows, each with about 100 fields, and then for each row I create the contents for about 30 different XML files using E4X into strings and then output the strings using text output step. I can only do about 500 rows before it crashes.

What's the best way to resolve this? I was thinking I'd create a job that calls my transformation a bunch of times and the actual transformation only processes 500 rows at a time. Any other suggestions?

Let me know. Thanks.