Hitachi Vantara Pentaho Community Forums
Results 1 to 5 of 5

Thread: Run 2 transformations at once

  1. #1
    Join Date
    Aug 2008
    Posts
    2

    Default Run 2 transformations at once

    Hi alll,

    I have scheduled a .bat file that executes a transformation.

    Code:
    rem GRP00.bat
    @echo off
    C:
    cd C:\Documents and Settings\A073619\Desktop\pdi-open-3.0.4-GA
    pan.bat /file:"C:\Documents and Settings\A073619\Desktop\pdi-open-3.0.4-GA\GRP00.ktr"

    this works fine but how can a run onother one after this has finished?

    I tried adding this after the code in the top.

    pan.bat /file:"C:\Documents and Settings\A073619\Desktop\pdi-open-3.0.4-GA\GRP00_Count.ktr"

    but it doesn't run anything, any ideas?

    Thanks for you're help in advance!!

  2. #2
    Join Date
    Jul 2007
    Posts
    1,013

    Default

    Just create a Job that runs your transformations and run it with kitchen.bat...

  3. #3

    Default Script to run in parallel -- (cygwin)

    This is a file I use to run multiple loaders in parallel on my production system.

    #!/bin/sh
    # Command : sh -x run_pan_loader_bat.sh REPORTING REPORTING kettle_rep "mydatabase"
    # Initialize Variables
    delay=5
    command_name="pan.bat /rep:admin /user:admin /pass:admin /logging:error /trans:"
    max_commands=7
    REPORTING_HOME="D:/REPORTING_HOME"
    loader_listing_dir="$REPORTING_HOME"

    db_user_id=$1
    db_user_password=$2
    use_db_name=$3
    db_server=$4

    # Exit shell with an error message (ala perl)
    die()
    {
    echo "Usage: $0 db_user_id db_user_password use_db_name db_server"
    echo "Example: $0 REPORTING REPORTING REPORTING \"mydatabase""
    echo $errorMsg
    exit -1
    }


    # go into a wait state if there $max_commands number of processes running
    wait_max_commands()
    {
    while [ `ps -elfW | grep "pan.bat" | wc -l` -ge $max_commands ]
    do
    echo "sleeezzzping"
    sleep $delay
    done
    }

    #Shell starts here

    if [ $# -lt 4 ]; then
    errorMsg="Three comand line parameters required"
    die
    fi


    #check for existence of the directory passed from the command line
    if [ ! -d "$loader_listing_dir" ]; then
    errorMsg="$loader_listing_dir is not a valid directory"
    die
    elif [ -f $loader_listing_dir/loader_listing.txt ]; then
    rm $loader_listing_dir/loader_listing.txt
    fi

    # builds the where clause for the loader list
    SQL_WHERE=`./sql_where_clause.sh`

    # if the SQL_WHERE is not empty then process the files
    if [ "$SQL_WHERE" != "" ]; then

    cd "$REPORTING_HOME/PENTAHO/PDI/252"

    sqlcmd.exe -U "$db_user_id" -P "$db_user_password" -d "$use_db_name" -S "$db_server" -h -1 -W<< @@@ > $loader_listing_dir/loader_listing.txt
    Set NoCount on
    select name
    from [kettle_rep].[dbo].[r_transformation]
    $SQL_WHERE
    order by 1
    go
    exit
    @@@

    echo "IMPLEMENT_DELETES" >> $loader_listing_dir/loader_listing.txt

    dos2unix $loader_listing_dir/loader_listing.txt
    files=`cat $loader_listing_dir/loader_listing.txt`


    #Loop through the files and spawn each with the pan command in background
    #The wait_max_commands will sleep when max_commands are running
    for file in $files
    do
    ./$command_name$file &
    wait_max_commands
    done

    #Do not proceed until the last pan is completed
    while [ `ps -elfW | grep "pan.bat" | wc -l` -ge 1 ]
    do
    echo "waiting for pan.bat to complete"
    sleep $delay
    done

    else

    #Return a failed status
    errorMsg="Three comand line parameters required"
    die
    fi

    #Return a success status
    exit 0

  4. #4
    Join Date
    Aug 2008
    Posts
    2

    Default

    Quote Originally Posted by tdidomenico View Post
    Just create a Job that runs your transformations and run it with kitchen.bat...
    Thanks, so I created a Job (attached screenshot), my questions is will both transformations run at the same time? Ideally that's what I would want.

    Thanks!
    Attached Images Attached Images  

  5. #5
    Join Date
    Jul 2007
    Posts
    1,013

    Default

    Not right away, but just right click on the "start" step and activate the "launch next entries in parallel" option. That should do it.

    Cheers!

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •  
Privacy Policy | Legal Notices | Safe Harbor Privacy Policy

Copyright © 2005 - 2019 Hitachi Vantara Corporation. All Rights Reserved.