Hitachi Vantara Pentaho Community Forums
Results 1 to 12 of 12

Thread: Flow of Job

  1. #1
    Join Date
    Apr 2014
    Posts
    6

    Default Flow of Job

    Hello,

    Job Image.doc

    Can anybody explain the flow of job, see the attached document.
    Is it parallel run/Sequential run..??

    If Transformation 1 is success then it will go success and Transformation 4 and 5 are running.
    After that Transformation 2 completed, where it will go ?? Again it will go for transformation 4 or 5 ??
    Kindly Explain the flow of Job step by step

    Best Regards,
    Satya.

  2. #2
    Join Date
    Dec 2010
    Posts
    193

    Default

    Hi Satya,

    By default , the job flow is sequential. By seeing the document, you have enabled 'Launch next entries in parallel'. So, you should be careful in your logic to execute the remaining steps based on your logic .
    Sathish
    Back to Pentaho


    'Be the best Pearl in the ocean of wisdom'

  3. #3
    Join Date
    Mar 2012
    Posts
    276

    Default

    job execution is sequential based on mapping hops , please refer to below link http://wiki.pentaho.com/display/EAI/Beginners+FAQ

  4. #4

    Default

    I think he's asking particularly, how the flow works once the three parallel first ones reach success step. Does it resume execution/flow once per each incoming flow? Does it queue them? Does it wait for the three to finish/arrive and simply execute once after?

  5. #5
    Join Date
    Sep 2013
    Posts
    235

    Default

    I would say try to create example and test it how it is works.
    Here is 2 jobs one sequential second is parallel.

    Note - job executor create a chain of job entries. It very similar to recursion tree traverse. Every branch is executed once. If success entry is meet - it is like a new tree - and a new travers.
    Imagine job is a tree grows from start entry.
    Image like you have for sequential tree traverse only one traverse client walking around every tree branch.
    As opposite to parallel - it is concurrently started traverse client for every branch from a tree root (job start).
    ex.kjb
    ex2.kjb

  6. #6

    Default

    Thanks!
    Sorry, english is not my primary language.
    Does that mean that job executer infers and queues an order independently of the order of arrival of the parallel entries at runtime?
    So everything after success step is going to be executed once for every paralel branch in a planned way. Therefore if third parallel branch is faster, it will not execute the whole new tree after parallel branch 1 and 2 finish and lead to a complete execution of the remaining part. Is it like that?

  7. #7
    Join Date
    Apr 2008
    Posts
    4,696

    Default

    To be absolutely clear:

    Start calls 1, 2, and 3.
    When 1 finishes, it calls 5 and 4.
    When 5 finishes, it calls 6 and 7.
    Then 4 finishes, and it calls 6 and 7.
    Meanwhile, 2 has finished, and it calls 5 and 4.
    5 finishes (the second time), it calls 6 and 7.
    4 might finish the second time before 6 finishes the first time...

    In short, 1 2 and 3 will be called one time each, and in parallel.
    4 and 5 will be called three times each, after 1 2 and 3 finish.
    6 and 7 will be called 6 times each (after each of the 4 and 5 runs), at times which will be next to impossible to determine.

  8. #8

    Default

    Quote Originally Posted by gutlez View Post
    To be absolutely clear:

    Start calls 1, 2, and 3.
    When 1 finishes, it calls 5 and 4.
    When 5 finishes, it calls 6 and 7.
    Then 4 finishes, and it calls 6 and 7.
    Meanwhile, 2 has finished, and it calls 5 and 4.
    5 finishes (the second time), it calls 6 and 7.
    4 might finish the second time before 6 finishes the first time...

    In short, 1 2 and 3 will be called one time each, and in parallel.
    4 and 5 will be called three times each, after 1 2 and 3 finish.
    6 and 7 will be called 6 times each (after each of the 4 and 5 runs), at times which will be next to impossible to determine.
    Sorry for making you being so explictly but now is perfectly illustrated. Simply first branch that finishes executes a new instance of the rest of the tree. When the second parallel initial branch finishes it does the same even if the former is already running as you answered in another post.

    I guess that in case 4 and 5 were not parallel executed, they could be in fact, given that success 1 is executed concurrently by two parallel branches

    Thanks!

  9. #9
    Join Date
    Apr 2008
    Posts
    4,696

    Default

    Quote Originally Posted by bizintreader View Post
    I guess that in case 4 and 5 were not parallel executed, they could be in fact, given that success 1 is executed concurrently by two parallel branches

    Thanks!
    I'm not quite sure I'm following you here, but yes, if 4 and 5 were not marked for parallel execution, they would still execute 3 times each, since 1, 2, and 3 are running in parallel.

  10. #10

    Default

    I meant that, assuming 1,2,3 took a similar amount of time to complete, they could make several instances of 4 (then 5) occur concurrently as 1,2,3 are parallel paths, isn't like that?

    Quote Originally Posted by gutlez View Post
    I'm not quite sure I'm following you here, but yes, if 4 and 5 were not marked for parallel execution, they would still execute 3 times each, since 1, 2, and 3 are running in parallel.
    I did not understand the 'since' part. Shouldn't be always executed three times each no matter parallel/sequential incoming. I guess you mean 3 times each at the same time, which answers the previous doubt

  11. #11
    Join Date
    Apr 2008
    Posts
    4,696

    Default

    Quote Originally Posted by bizintreader View Post
    I did not understand the 'since' part. Shouldn't be always executed three times each no matter parallel/sequential incoming. I guess you mean 3 times each at the same time, which answers the previous doubt
    No - if the Job is set up so that 1 -> 2 -> 3 Sequential, then 4 and 5 would be executed once each.

    Parallelism MATTERS at the job level. MY feeling with parallelism at the job level is: Don't do it unless you ABSOLUTELY need to, and then only do it with a nested job containing just those job steps that need to be parallel

  12. #12

    Default

    I mean 1,2,3 disposed in parallel but not enabling "parallel execution", so sequential after all. I think it's pretty safe to assume 4,5 would be executed 3 times each (I'd be shocked if that is not the expected behaviour).

    Parallel = Parallel disposed
    Concurrent = Parallel disposed and 'Run in parallel' enabled

    Summing up, given the document I understand:

    - If 1,2,3,4,5 are executed paralelly (as in the original picture) but not concurrently, 1 (or the first one by DOM parser) would be executed, then 4, then 5 (and so on), then 2>4,5 then 3>4,5. No isntances would be ever multiple instantiated.

    - If 1,2,3 are executed concurrently and they took similar amount of time to finish, 4 (and then 5) would be instantiated several times at once even if 4,5 are not set to run concurrently
    Last edited by bizintreader; 04-29-2014 at 05:19 PM.

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •  
Privacy Policy | Legal Notices | Safe Harbor Privacy Policy

Copyright © 2005 - 2019 Hitachi Vantara Corporation. All Rights Reserved.