Hitachi Vantara Pentaho Community Forums
Results 1 to 5 of 5

Thread: Saving Transformation in Spoon

  1. #1
    Join Date
    Sep 2005
    Posts
    1,403

    Default Saving Transformation in Spoon

    Hi,


    Kettle is really a nice project and exactly what I just need, but one thing makes question marks appearing on my face.



    Why do you build the XML file on saving a tranformation by string concatination (getXML() in StepMeta)??



    Why not using StringBuffer/StringBuilder? Or better, use a dom tree similar to loadXML()!



    I've set up a very simple transformation. Just read data from one XML file (using my own plugin) and writing it into another XML file. No problem so far. But there are 648 values per row and thats a problem! When I save this transformation in Spoon it needs nearly 10 minutes. Loading the same transformation in Spoon is done in seconds.



    The reason for this is the above mentioned string concatination. With each field information the loop in getXML() (XML Output) is getting slower and slower because the string is getting longer and longer. With each concat statement a string object must be created and the content be copied. And don't forget, the GC needs to clean up all this.



    Is there a chance to change this behavior?



    Kind regards,
    Axel

  2. #2
    Join Date
    Nov 1999
    Posts
    9,729

    Default RE: Saving Transformation in Spoon

    OK, I saw that one coming...
    We'll change this ASAP.

    I guess it's all Bruce Tate's fault, I listened and went for the simple approach.

    I'll convert to StringBuffer. Hey, any help is welcome...

    Anyone, anyone, .... Bueller?

    ;-)

    Thanks again for the feedback.

    Matt

  3. #3
    Join Date
    Nov 1999
    Posts
    9,729

    Default RE: Saving Transformation in Spoon

    Just changed all the getXML() methods to StringBuffer.

    A big thank you goes to regular expression magic and Eclipse.

    I guess when the auto-build system comes back on-line, you'll be able to grab a new dev version.
    Otherwise, do a SVN checkout for the time being and build yourself.

    ;-)

    Matt

  4. #4
    Join Date
    Sep 2005
    Posts
    1,403

    Default RE: Saving Transformation in Spoon

    I got the change via subversion and built it on my system (Boah, a build script that runs just out of the box! :-o)

    Now saving of my transformation as fast as I expected - just a second.

    Many thanks!!
    Axel

  5. #5
    Join Date
    Nov 1999
    Posts
    9,729

    Default RE: Saving Transformation in Spoon

    >a build script that runs just out of the box

    Hey, that was never the plan. I was doing myself a favor, not you guys!!

    Just kidding, hope you find Kettle useful.

    On a side-note, if you preview the 648 values you will witness a documented Windows bug... Scroll all the way to the right and you'll see the header of the grid disapear. Initially I filed this as a bug for the Eclipse SWT team, but they confirmed it was a Windows bug and SWT just used standard Windows components.

    Cheers,

    Matt

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •  
Privacy Policy | Legal Notices | Safe Harbor Privacy Policy

Copyright © 2005 - 2019 Hitachi Vantara Corporation. All Rights Reserved.