Hitachi Vantara Pentaho Community Forums
Results 1 to 9 of 9

Thread: Job Entry Branching -- If you have two outbound hops, which hop is followed first?

  1. #1
    Join Date
    Sep 2014
    Posts
    175

    Default Job Entry Branching -- If you have two outbound hops, which hop is followed first?

    Hi there--

    Obviously the default (and probably preferred) behavior of job entries is that they happen sequentially.

    So when you have a path A->B and another path A->C ----- how does Pentaho determine which one happens first? Is it the hop you literally built first? Does an unconditional hop happen before a 'success' hop?


    I'm interested.

    This is particularly interesting when one of the paths is a loop (with a conditional box that will make it loop 5 times).



    Currently I have a job that goes A->B and
    A->C->(did C happen 5 times? not yet? then) -> A


    Essentially what happens currently is that A occurs, then B occurs, THEN C occurs, then A occurs again, then B, then C, then A ... and on.

    This is what I would like to happen, but I fear if I make some changes, something goofy might happen.

  2. #2
    Join Date
    Aug 2011
    Posts
    360

    Default

    Hi,

    Not sure but i think it occurs in order of the creation of the hops.
    Guess you should not rely on this to sequence your jobs!

    But in your case, why not do A-> B-> C-> (test for loop ) -> A ?
    Since you want always this order?

  3. #3
    Join Date
    Jun 2012
    Posts
    5,534

    Default

    Quote Originally Posted by Mathias.CH View Post
    Not sure but i think it occurs in order of the creation of the hops.
    Hops are added to a List when created, and that List is iterated to retrieve the successors of a job entry.
    Since a list is an ordered collection, the next job entries will be started in the same order the hops were created.

    I second your advice to not rely on this purely internal contract.
    Would be bad practice, not prefessional, a violation of the information hiding principle.
    So long, and thanks for all the fish.

  4. #4
    Join Date
    Sep 2014
    Posts
    175

    Default

    Quote Originally Posted by marabu View Post
    Hops are added to a List when created, and that List is iterated to retrieve the successors of a job entry.
    Since a list is an ordered collection, the next job entries will be started in the same order the hops were created.

    I second your advice to not rely on this purely internal contract.
    Would be bad practice, not prefessional, a violation of the information hiding principle.
    How would I achieve this instead then?

    As Mathias pointed out, why not do A->B->C->(test for loops)->A?

    The answer is simple. C is a logging step. It fires no matter what. Returning to A though should require B success.

    Therefore the hop to C is unconditional. But the hop from B should be success only. That why the single branch option doesn't exactly work.

  5. #5
    Join Date
    Jun 2012
    Posts
    5,534

    Default

    Can we have a bit of clarity on the duties of job entry C, first?
    You say C is for logging, but you also said C is for loop control.
    So what exactly does C?
    Maybe you have to switch from foot-controlled-loop (as Mathias suggested) to head-controlled (where C is running before A).
    So long, and thanks for all the fish.

  6. #6
    Join Date
    Sep 2014
    Posts
    175

    Default

    Quote Originally Posted by marabu View Post
    Can we have a bit of clarity on the duties of job entry C, first?
    You say C is for logging, but you also said C is for loop control.
    So what exactly does C?
    Maybe you have to switch from foot-controlled-loop (as Mathias suggested) to head-controlled (where C is running before A).

    My mistake; I mixed up B and C.

    B is for logging.

    A is the basic ETL process --- once that's complete I want B to fire no matter what immediately after --- it will log the success or failure of A, errors, inserts, log_text, etc.

    Then C checks if we need to run A again (depending on if we have another page of data).





    So A->B is unconditional, yet I don't want A->C to be unconditional (don't want to loop if there is some issue).


    I suppose I could go A->B->(check if Number of Errors variable from A > 0) -> C -> A. That would keep it linear.

  7. #7
    Join Date
    Aug 2011
    Posts
    360

    Default

    Quote Originally Posted by cool_runnings View Post
    My mistake; I mixed up B and C.

    B is for logging.

    A is the basic ETL process --- once that's complete I want B to fire no matter what immediately after --- it will log the success or failure of A, errors, inserts, log_text, etc.

    Then C checks if we need to run A again (depending on if we have another page of data).





    So A->B is unconditional, yet I don't want A->C to be unconditional (don't want to loop if there is some issue).


    I suppose I could go A->B->(check if Number of Errors variable from A > 0) -> C -> A. That would keep it linear.
    So you just need to set two path: one success path and one error path, depending on result of A:
    Sucess path: A --if success---> B--unconditional--> C---A.
    Error path: A (same entry) --- if error ---> B (another one) --> abort job.

  8. #8
    Join Date
    Jun 2012
    Posts
    5,534

    Default

    Here is an alternative design for loop control.
    Duplication of entries is avoided by saving the execution result for later.
    Attached Files Attached Files
    So long, and thanks for all the fish.

  9. #9
    Join Date
    Sep 2014
    Posts
    175

    Default

    Thank you both -- I think I prefer the non duplicate entries, but the idea that I shouldn't really hide step order info was a good one. I will reconfigure, thanks.

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •  
Privacy Policy | Legal Notices | Safe Harbor Privacy Policy

Copyright © 2005 - 2019 Hitachi Vantara Corporation. All Rights Reserved.