Hitachi Vantara Pentaho Community Forums
Results 1 to 10 of 10

Thread: Reading Transformation from GIT repository takes time than usual

  1. #1
    Join Date
    Apr 2013
    Posts
    5

    Default Reading Transformation from GIT repository takes time than usual

    When i was trying to run a Job, the below KTR(KTR1) inside the job for example, is taking almost a minute, just to read and load the transformation.
    We are the triggering the job from AWS batch.
    This is the scenario for all the transformations inside the job. If one ktr is taking around 1 minute, if i have 10 transformations, it will be almost 10 minutes.
    Is this the expected behavior or something is fishy !?
    Pentaho 7.1 Community Edition

    2019/02/14 16:15:43 - KTR1 - Loading transformation from repository [KTR1] in directory [/public/Main_Folder/KTR_Folder]
    2019/02/14 16:16:39 - KTR1 - Using run configuration [Pentaho local]

    Thanks in advance
    Last edited by Narenso; 03-04-2019 at 05:24 PM.

  2. #2
    Join Date
    Aug 2016
    Posts
    279

    Default

    I don't understand exactly, are you executing code located on the same computer as kettle, or are you executing code externally from a git repository?

  3. #3
    Join Date
    Apr 2013
    Posts
    5

    Default

    Quote Originally Posted by Sparkles View Post
    I don't understand exactly, are you executing code located on the same computer as kettle, or are you executing code externally from a git repository?

    Code is available in local when the job is executed.

  4. #4
    Join Date
    Aug 2016
    Posts
    279

    Default

    So the problem is slow execution / startup regardless of what version control is used?

  5. #5
    Join Date
    Apr 2013
    Posts
    5

    Default

    Quote Originally Posted by Sparkles View Post
    So the problem is slow execution / startup regardless of what version control is used?
    Yes, You are correct.
    All the code is pulled from Git and placed in local directory (using Bamboo automated builds). meaning for kitchen the code is available local in the server.
    So we are doubting, there is some setting in PDI CE 7.1.0.0-12 that we are missing.

    BTW, Could this help - https://blog.twineworks.com/improvin...n-78d0803c559b

  6. #6
    Join Date
    Aug 2016
    Posts
    279

    Default

    Yes I had that article in mind, there's also been other discussions on the topic I think.

  7. #7
    Join Date
    Apr 2013
    Posts
    5

    Default

    Quote Originally Posted by Sparkles View Post
    Yes I had that article in mind, there's also been other discussions on the topic I think.
    Okay. Please share, if you get to know any update on this.
    And in the mean time, i would like to thank and appreciate you for spending your time on immediate replies.

  8. #8
    Join Date
    Aug 2016
    Posts
    279

    Default

    Sorry I don't have anything useful to come with I'll have a look around if I find that earlier discussion on this topic.

  9. #9
    Join Date
    Apr 2013
    Posts
    5

    Default

    Quote Originally Posted by Sparkles View Post
    Sorry I don't have anything useful to come with I'll have a look around if I find that earlier discussion on this topic.
    We ran the job in Debug mode and found that it is writing this for each transformation in the job before transformation starts.

    2019/03/08 14:55:17 - TR_TEST_TRANSFORMATION - Loading transformation from repository [TR_TEST_TRANSFORMATION] in directory [/directory]
    2019/03/08 14:55:18 - JB_TEST_JOB - Triggering heartbeat signal for JB_TEST_JOB at every 10 seconds
    2019/03/08 14:55:28 - JB_TEST_JOB - Triggering heartbeat signal for JB_TEST_JOB at every 10 seconds
    2019/03/08 14:55:38 - JB_TEST_JOB - Triggering heartbeat signal for JB_TEST_JOB at every 10 seconds
    2019/03/08 14:55:48 - JB_TEST_JOB - Triggering heartbeat signal for JB_TEST_JOB at every 10 seconds
    2019/03/08 14:55:58 - JB_TEST_JOB - Triggering heartbeat signal for JB_TEST_JOB at every 10 seconds
    2019/03/08 14:56:08 - JB_TEST_JOB - Triggering heartbeat signal for JB_TEST_JOB at every 10 seconds
    2019/03/08 14:56:18 - JB_TEST_JOB - Triggering heartbeat signal for JB_TEST_JOB at every 10 seconds
    2019/03/08 14:56:28 - JB_TEST_JOB - Triggering heartbeat signal for JB_TEST_JOB at every 10 seconds
    2019/03/08 14:56:38 - JB_TEST_JOB - Triggering heartbeat signal for JB_TEST_JOB at every 10 seconds
    2019/03/08 14:56:48 - JB_TEST_JOB - Triggering heartbeat signal for JB_TEST_JOB at every 10 seconds
    2019/03/08 14:56:58 - JB_TEST_JOB - Triggering heartbeat signal for JB_TEST_JOB at every 10 seconds
    2019/03/08 14:57:08 - JB_TEST_JOB - Triggering heartbeat signal for JB_TEST_JOB at every 10 seconds
    2019/03/08 14:57:18 - JB_TEST_JOB - Triggering heartbeat signal for JB_TEST_JOB at every 10 seconds
    2019/03/08 14:57:28 - JB_TEST_JOB - Triggering heartbeat signal for JB_TEST_JOB at every 10 seconds
    2019/03/08 14:57:38 - JB_TEST_JOB - Triggering heartbeat signal for JB_TEST_JOB at every 10 seconds
    2019/03/08 14:57:48 - JB_TEST_JOB - Triggering heartbeat signal for JB_TEST_JOB at every 10 seconds
    2019/03/08 14:57:58 - JB_TEST_JOB - Triggering heartbeat signal for JB_TEST_JOB at every 10 seconds
    2019/03/08 14:58:04 - TR_TEST_TRANSFORMATION - Starting transformation...(file=null, name=TR_TEST_TRANSFORMATION, repinfo=null)
    2019/03/08 14:58:04 - TR_TEST_TRANSFORMATION - Using run configuration [Pentaho local]
    2019/03/08 14:58:04 - TR_TEST_TRANSFORMATION - Using legacy execution engine
    2019/03/08 14:58:04 - TR_TEST_TRANSFORMATION - Transformation is pre-loaded.
    2019/03/08 14:58:04 - TR_TEST_TRANSFORMATION - nr of steps to run : 19 , nr of hops : 19
    2019/03/08 14:58:04 - TR_TEST_TRANSFORMATION - Dispatching started for transformation [TR_TEST_TRANSFORMATION]

  10. #10
    Join Date
    Aug 2016
    Posts
    279

    Default

    There have been other discussions pointing out some bugs in 7.1 which makes transformation start slowly:

    https://forums.pentaho.com/threads/2...time-to-start/

Tags for this Thread

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •  
Privacy Policy | Legal Notices | Safe Harbor Privacy Policy

Copyright © 2005 - 2017 Pentaho Corporation. All Rights Reserved.