Hitachi Vantara Pentaho Community Forums
Results 1 to 3 of 3

Thread: What is the benefit of using 'Launch next entries in parallel' [4.4.0]

  1. #1
    Join Date
    May 2013
    Posts
    16

    Default What is the benefit of using 'Launch next entries in parallel' [4.4.0]

    I am confused about the difference between the following two styles. Can anybody help me? Thanks in advance!

    (1) Why the second style is faster? What is the mechanism? I feel the Job 1 to Job 3 also run in parallel in the style 1.

    (2) Is there anything I should consider when I use the second style? For example, can I still use customized variables like how I use them before?

    (3) In PDI 5.x or 6.x, is there still any difference between the two styles?





    Thank you!
    Last edited by AlbertWang; 03-17-2016 at 12:56 AM.

  2. #2
    Join Date
    Aug 2011
    Posts
    360

    Default

    Yes there is a difference.
    Style 1: jobs are launch in sequence, and the order is quite random (actually the order is the order the hops have been created)
    So for example job1 starts, when it is finished job3 starts, when its finished job2 starts.

    Style 2: entries are launched in parallel, so they all starts at the same time.

    So this is really different!
    Moreover, executing jobs in parallel is not always faster, it all depends on ressources (cpu, memory, db) that they use.

    If you have small jobs, not using much ressource, then it will be faster in parallel, since your cpu/db etc will be used at 100%.

    If your jobs already use much ressources (say 70% each), then it wont necessary be faster in parallel as they will need
    to share 100% of ressource between 3 jobs.

  3. #3
    Join Date
    May 2013
    Posts
    16

    Default

    Quote Originally Posted by Mathias.CH View Post
    Yes there is a difference.
    Style 1: jobs are launch in sequence, and the order is quite random (actually the order is the order the hops have been created)
    So for example job1 starts, when it is finished job3 starts, when its finished job2 starts.

    Style 2: entries are launched in parallel, so they all starts at the same time.

    So this is really different!
    Moreover, executing jobs in parallel is not always faster, it all depends on ressources (cpu, memory, db) that they use.

    If you have small jobs, not using much ressource, then it will be faster in parallel, since your cpu/db etc will be used at 100%.

    If your jobs already use much ressources (say 70% each), then it wont necessary be faster in parallel as they will need
    to share 100% of ressource between 3 jobs.
    Hi Mathias,

    Thank you very much for your reply. I have tested it and it is just like what you said :-)

    Thanks again and have a nice day!

    Regards,
    Albert

Tags for this Thread

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •  
Privacy Policy | Legal Notices | Safe Harbor Privacy Policy

Copyright © 2005 - 2019 Hitachi Vantara Corporation. All Rights Reserved.