PDA

View Full Version : Saving Transformation in Spoon



kettle_anonymous
03-28-2006, 06:30 AM
Hi,


Kettle is really a nice project and exactly what I just need, but one thing makes question marks appearing on my face.



Why do you build the XML file on saving a tranformation by string concatination (getXML() in StepMeta)??



Why not using StringBuffer/StringBuilder? Or better, use a dom tree similar to loadXML()!



I've set up a very simple transformation. Just read data from one XML file (using my own plugin) and writing it into another XML file. No problem so far. But there are 648 values per row and thats a problem! When I save this transformation in Spoon it needs nearly 10 minutes. Loading the same transformation in Spoon is done in seconds.



The reason for this is the above mentioned string concatination. With each field information the loop in getXML() (XML Output) is getting slower and slower because the string is getting longer and longer. With each concat statement a string object must be created and the content be copied. And don't forget, the GC needs to clean up all this.



Is there a chance to change this behavior?



Kind regards,
Axel

MattCasters
03-28-2006, 09:46 AM
OK, I saw that one coming...
We'll change this ASAP.

I guess it's all Bruce Tate's fault, I listened and went for the simple approach.

I'll convert to StringBuffer. Hey, any help is welcome...

Anyone, anyone, .... Bueller?

;-)

Thanks again for the feedback.

Matt

MattCasters
03-28-2006, 12:33 PM
Just changed all the getXML() methods to StringBuffer.

A big thank you goes to regular expression magic and Eclipse.

I guess when the auto-build system comes back on-line, you'll be able to grab a new dev version.
Otherwise, do a SVN checkout for the time being and build yourself.

;-)

Matt

kettle_anonymous
03-29-2006, 01:57 AM
I got the change via subversion and built it on my system (Boah, a build script that runs just out of the box! :-o)

Now saving of my transformation as fast as I expected - just a second.

Many thanks!!
Axel

MattCasters
03-29-2006, 02:07 AM
>a build script that runs just out of the box

Hey, that was never the plan. I was doing myself a favor, not you guys!!

Just kidding, hope you find Kettle useful.

On a side-note, if you preview the 648 values you will witness a documented Windows bug... Scroll all the way to the right and you'll see the header of the grid disapear. Initially I filed this as a bug for the Eclipse SWT team, but they confirmed it was a Windows bug and SWT just used standard Windows components.

Cheers,

Matt