Hitachi Vantara Pentaho Community Forums
Results 1 to 5 of 5

Thread: Loop over array of transformation names and execute them concurrently

  1. #1
    Join Date
    Jul 2014
    Posts
    10

    Angry Loop over array of transformation names and execute them concurrently

    I have a large number of transformations that are all names similarly that I current run concurrently. The problem is that it's becoming rather unwieldily:

    http://i1173.photobucket.com/albums/...ps312e5418.png

    My thought was to create an array of these transformation names and dynamically load the transformation and run it concurrently as before. Is this sort of thing possible in Data Integration? If so how would I go about doing this?

  2. #2
    njain111 Guest

    Default

    Would grouping related transformations into jobs is something would help you here?
    Like, Student, Department, Staff ... etc

  3. #3
    Join Date
    Apr 2008
    Posts
    4,696

    Default

    Also, have a look in the "Run All" in the samples directory.

  4. #4
    Join Date
    Jul 2014
    Posts
    10

    Default

    Quote Originally Posted by gutlez View Post
    Also, have a look in the "Run All" in the samples directory.
    I implemented this into my project but unfortunately it seems that the concurrency aspect is lost. They appear to run one at a time now. Any suggestions?

  5. #5
    njain111 Guest

    Default

    You can create a transformation in which either you can hardcode details about your transformation or get them dynamically as needed.
    Then pass this information to Job Executor, which invokes a generic job which in turn invokes generic transformation.
    To achieve concurrency you can execute multiple copies of 'Job Executor', copies==nr of transfromations you have.


    Quote Originally Posted by kyledecot View Post
    I implemented this into my project but unfortunately it seems that the concurrency aspect is lost. They appear to run one at a time now. Any suggestions?

Tags for this Thread

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •  
Privacy Policy | Legal Notices | Safe Harbor Privacy Policy

Copyright © 2005 - 2019 Hitachi Vantara Corporation. All Rights Reserved.