Hitachi Vantara Pentaho Community Forums
Results 1 to 3 of 3

Thread: Performance of Java Expression Step vs. User Defined Java Class

  1. #1
    Join Date
    Jun 2007
    Posts
    233

    Question Performance of Java Expression Step vs. User Defined Java Class

    Does anyone have any info / hard data / examples (real world) that can show a difference between these two steps?

    Cheers

    The Frog
    Everything should be made as simple as possible, but not simpler - Albert Einstein

  2. #2
    Join Date
    Nov 2008
    Posts
    143

    Default

    Hey,

    Based on Roland's blog post, I added a UDJC test and got the following results:

    Code:
        Transformation   |  Rows per second  |  Improvement (%)
        -----------------+-------------------+-------------------
        Original         |   54558,3         |     --
        Cache getString()|   59980,8         |    9,94
        No Compatmode    |   79013,9         |   44,82
        no parseInt()    |   91558,3         |   67,82
        Optimize flow    |  100633,9         |   84,45
        Formula          |  143184,4         |  162,44
        Calculator       |  257003,3         |  371,06
        Java Expression  |  426803,2         |  682,29
        Java Class       |  470588,2         |  762,54
    I attached the transformations in case you choose to try 'em.

    BTW, I don't recommend running "10 - All in.ktr".

    Cheers,

    Renato
    Attached Files Attached Files
    Last edited by renatopb; 01-17-2011 at 01:53 PM. Reason: Added improvement percentage compared to the original transformation

  3. #3
    Join Date
    Jun 2007
    Posts
    233

    Default

    Renato, thankyou very very much for this. I really appreciate it. The UDJC step wasnt available the last time (a long time ago now) that I worked with PDI and was still in the development stage. I appreciate the sample trans as well. I am hoping to make a business case to my manager(s) that we need to move from an older legacy ETL tool and some older PDI scripts that both rely on JavaScript and migrate towards UDJC (and redesigning the ETL processes to be more robust at the same time). Thankyou, thankyou, thankyou.

    Cheers

    The Frog
    Everything should be made as simple as possible, but not simpler - Albert Einstein

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •  
Privacy Policy | Legal Notices | Safe Harbor Privacy Policy

Copyright © 2005 - 2019 Hitachi Vantara Corporation. All Rights Reserved.