US and Worldwide: +1 (866) 660-7555
Results 1 to 4 of 4

Thread: Execute the job file (kjb) using java API

  1. #1
    Join Date
    May 2012
    Posts
    3

    Default Execute the job file (kjb) using java API

    I have tried to modify the code which executes a transformation but got "StepLoader" class not found. Would someone be able to provide the exact code of how to execute the Job?


    public static void main(String args[])
    {
    try
    {
    LogWriter logWriter = null;
    try
    {
    logWriter = LogWriter.getInstance(LogWriter.LOG_LEVEL_BASIC); //$NON-NLS-1$
    }
    catch (Throwable t)
    {
    }

    KettleEnvironment.init();
    JobMeta jobMeta = new JobMeta(logWriter, args[0], null, null);
    Job job = new Job(logWriter, StepLoader.getInstance(), null, jobMeta);

    job.getJobMeta().setInternalKettleVariables(job);
    job.execute(args);
    job.waitUntilFinished();
    if (job.getErrors() > 0)
    {
    throw new RuntimeException( "There were errors during transformation execution." );
    }
    }
    catch (KettleException e)
    {
    System.out.println(e);
    }
    }

  2. #2
    Join Date
    Sep 2009
    Posts
    809

    Default

    Below method works for me, assuming the jars from kettle/lib and kettle/libext are in your classpath

    Code:
         /** 
         * This method executes a job defined in a kjb file
         * 
         * It demonstrates the following:
         * 
         * - Loading a job definition from a kjb file
         * - Setting named parameters for the job
         * - Setting the log level of the job
         * - Executing the job, waiting for it to finish
         * - Examining the result of the job
         * 
         * @param filename the file containing the job to execute (kjb file)
         * @return the job that was executed, or null if there was an error
         */
        public Job runJobFromFileSystem(String filename) {
            
            try {
                System.out.println("***************************************************************************************");
                System.out.println("Attempting to run job "+filename+" from file system");
                System.out.println("***************************************************************************************\n");
                // Loading the job file from file system into the JobMeta object.
                // The JobMeta object is the programmatic representation of a job definition.
                JobMeta jobMeta = new JobMeta(filename, null);
                
                // The next section reports on the declared parameters and sets them to arbitrary values
                // for demonstration purposes
                System.out.println("Attempting to read and set named parameters");
                String[] declaredParameters = jobMeta.listParameters();
                for (int i = 0; i < declaredParameters.length; i++) {
                    String parameterName = declaredParameters[i];
                    
                    // determine the parameter description and default values for display purposes
                    String description = jobMeta.getParameterDescription(parameterName);
                    String defaultValue = jobMeta.getParameterDefault(parameterName);
                    // set the parameter value to an arbitrary string
                    String parameterValue =  RandomStringUtils.randomAlphanumeric(10);
                    
                    String output = "Setting parameter "+parameterName+" to \""+parameterValue+"\" [description: \""+description+"\", default: \""+defaultValue+"\"]";
                    System.out.println(output);
                    
                    // assign the value to the parameter on the job
                    jobMeta.setParameterValue(parameterName, parameterValue);
                    
                }
                
                // Creating a Job object which is the programmatic representation of a job 
                // A Job object can be executed, report success, etc.
                Job job = new Job(null, jobMeta);
                
                // adjust the log level
                job.setLogLevel(LogLevel.MINIMAL);
    
    
                System.out.println("\nStarting job");
                
                // starting the job, which will execute asynchronously
                job.start();
                
                // waiting for the job to finish
                job.waitUntilFinished();
                
                // retrieve the result object, which captures the success of the job
                Result result = job.getResult();
                
                // report on the outcome of the job
                String outcome = "\nJob "+ filename+" executed with result: "+result.getResult()+" and " +result.getNrErrors()+" errors\n";
                System.out.println(outcome);
                
                
                return job;
    
    
            } catch (Exception e) {
                
                // something went wrong, just log and return 
                e.printStackTrace();
                return null;
            } 
            
    }
    Do you want to know more? Check out Adventures with Open Source BI

  3. #3

    Default

    Agreed, you likely need to specify the location of those library jars. You can do it using the java -cp flag

  4. #4
    Join Date
    May 2012
    Posts
    3

    Default

    Thank you for your help.

    I got it compiled and tried to execute it, I got the following. I am pretty much using your code. The kjb file works if the kitchen.sh script is used.


    The kjb file is using MYSQL but the error message below is referring to Oracle (database type with plugin id [Oracle] couldn't be found!). Some of the lines in tech1533_bsabpmtxl_support.kjb are as follows:

    <type>MYSQL</type>
    <attribute><code>EXTRA_OPTION_MYSQL.defaultFetchSize</code><attribute>500</attribute></attribute>
    <attribute><code>EXTRA_OPTION_MYSQL.useCursorFetch</code><attribute>true</attribute></attribute>
    <type>MYSQL</type>
    <attribute><code>EXTRA_OPTION_MYSQL.defaultFetchSize</code><attribute>500</attribute></attribute>
    <attribute><code>EXTRA_OPTION_MYSQL.useCursorFetch</code><attribute>true</attribute></attribute>
    <type>MYSQL</type>
    <attribute><code>EXTRA_OPTION_MYSQL.defaultFetchSize</code><attribute>500</attribute></attribute>
    <attribute><code>EXTRA_OPTION_MYSQL.useCursorFetch</code><attribute>true</attribute></attribute>
    ...
    <database>${DB_OLTP}</database>
    <database>pentaho_OLAP</database>
    <database>pentaho_oltp</database>


    Running "java Pentaho_Job_Run ${BASEDIR}/code/tech1533_bsabpmtxl_support.kjb /interfaces/etl/ext/recv/test.txt" gave me the following:

    ***************************************************************************************
    Attempting to run job /interfaces/etl/code/tech1533_bsabpmtxl_support.kjb from file system
    ***************************************************************************************
    May 4, 2012 3:27:18 PM org.apache.commons.vfs.VfsLog info
    INFO: Using "/tmp/vfs_cache" as temporary files store.
    org.pentaho.di.core.exception.KettleXMLException:
    Unable to load the job from XML file [/interfaces/etl/code/tech1533_bsabpmtxl_support.kjb]
    Unable to load job info from XML node
    Database type not found!

    at org.pentaho.di.job.JobMeta.<init>(JobMeta.java:704)
    at org.pentaho.di.job.JobMeta.<init>(JobMeta.java:664)
    at Pentaho_Job_Run.main(Pentaho_Job_Run.java:41)
    Caused by: org.pentaho.di.core.exception.KettleXMLException:
    Unable to load job info from XML node
    Database type not found!
    at org.pentaho.di.job.JobMeta.loadXML(JobMeta.java:997)
    at org.pentaho.di.job.JobMeta.loadXML(JobMeta.java:754)
    at org.pentaho.di.job.JobMeta.<init>(JobMeta.java:696)
    ... 2 more
    Caused by: java.lang.RuntimeException: Database type not found!
    at org.pentaho.di.core.database.DatabaseMeta.setValues(DatabaseMeta.java:544)
    at org.pentaho.di.core.database.DatabaseMeta.setDefault(DatabaseMeta.java:416)
    at org.pentaho.di.core.database.DatabaseMeta.<init>(DatabaseMeta.java:406)
    at org.pentaho.di.core.database.DatabaseMeta.<init>(DatabaseMeta.java:868)
    at org.pentaho.di.job.JobMeta.loadXML(JobMeta.java:853)
    ... 4 more
    Caused by: org.pentaho.di.core.exception.KettleDatabaseException:
    database type with plugin id [Oracle] couldn't be found!
    at org.pentaho.di.core.database.DatabaseMeta.findDatabaseInterface(DatabaseMeta.java:489)
    at org.pentaho.di.core.database.DatabaseMeta.getDatabaseInterface(DatabaseMeta.java:465)
    at org.pentaho.di.core.database.DatabaseMeta.setValues(DatabaseMeta.java:540)
    ... 8 more

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •