Hitachi Vantara Pentaho Community Forums
Results 1 to 2 of 2

Thread: Parallel execution

  1. #1
    Join Date
    Jul 2007
    Posts
    247

    Default Parallel execution

    Hi everybody,

    I have some questions concerning the parallel execution of jobs and transformations.

    What I understood so far is that transformations are executed parallel by default. There is no such thing for jobs at the moment, if one wants to execute jobs in parallel one has to implement this by using an external scheduler.

    This that correct?

    Are there any plans to implement the parallel execution of jobs in PDI? Is there already an opened JIRA tracker for this?

    Thx in advance,
    Ben

  2. #2
    Join Date
    May 2006
    Posts
    4,882

    Default

    What I understood so far is that transformations are executed parallel by default. There is no such thing for jobs at the moment, if one wants to execute jobs in parallel one has to implement this by using an external scheduler.
    The steps in a transformation are executed in parallel. If you want to run transformations parallel you also have to use an external scheduler (because of the sequential processing of jobs ).

    Are there any plans to implement the parallel execution of jobs in PDI? Is there already an opened JIRA tracker for this?
    No, no plans yet I think... and it would probably not be implemented I think. In jobs you need "sequential" execution because you want to control flow... you can't control flow when you run in parallel. I can see that it could be handy to start up multiple independent transformations in a job concurrently but this would make job design pretty complex... when to run concurrently, when to run sequentially, ...

    Regards,
    Sven
    Last edited by sboden; 10-12-2007 at 04:39 AM.

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •  
Privacy Policy | Legal Notices | Safe Harbor Privacy Policy

Copyright © 2005 - 2019 Hitachi Vantara Corporation. All Rights Reserved.