Hitachi Vantara Pentaho Community Forums
Results 1 to 4 of 4

Thread: ETL performance deteriorates after upgrade to 4.xx

  1. #1
    Join Date
    Aug 2009
    Posts
    5

    Default ETL performance deteriorates after upgrade to 4.xx

    What are the top 3 reasons for a simple Text File Input transformation of 1M rows to fly in Spoon 3.2.0-GA with the average speed of 1500 r/s while crawling and eventually shutting down in 4.10 and 4.0.1? Both versions are installed in parallel directories on Win 03 SP2 with latest Java and both use identical databases on the same SQL Server 05 for repositories.

    I should add that this happens alongside with all kinds of java exceptions and pentaho errors like ‘A serious error occurred during job execution:
    Unexpected error occurred while launching entry [transformation_name.0]
    Unexpected error during transformation metadata load
    No repository defined!’

    Am I expected to define a repository for exported transformation now??

    Your help in getting me out of the woods with 4.xx is humbly appreciated!

  2. #2
    Join Date
    Nov 1999
    Posts
    9,729

    Default

    Performance should be the same or better in 4.0. That's all I can say without other details.

  3. #3
    Join Date
    Aug 2009
    Posts
    5

    Default

    Is there something - JDK, repository settings, memory allocation, etc. - that should be configured in a different way when moving from 3.x to 4.x? What other details should we look at to make this work?

  4. #4
    Join Date
    Aug 2009
    Posts
    5

    Default

    I got it.
    The "jre" folder has been hard-coded in Spoon.bat while with latest-greatest java it's called "jre6".
    Replacing
    pushd "%_PENTAHO_JAVA_HOME%\jre\bin"
    with
    pushd "%_PENTAHO_JAVA_HOME%\jre6\bin"
    has solved the problem.

    duh!

Tags for this Thread

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •  
Privacy Policy | Legal Notices | Safe Harbor Privacy Policy

Copyright © 2005 - 2019 Hitachi Vantara Corporation. All Rights Reserved.