Hitachi Vantara Pentaho Community Forums
Results 1 to 3 of 3

Thread: Obtaining metric information of transformation in Java

  1. #1
    Join Date
    Mar 2014
    Posts
    2

    Default Obtaining metric information of transformation in Java

    Hi,

    I would like please some help obtaining metric information (number of lines read, written, etc.) of a finished transformation in my Java program. My program dynamically creates transformations then run them and finally show the detailed information about the finished transformations. I use the Kettle Java API version 4.4. This is my code simplified :


    KettleEnvironment.init(false);

    //this creates the TransMeta object and the XML file
    TransMeta transMeta = createTransformationPerQuery(t...);

    if (transMeta != null) {
    Trans transformation = new Trans(transMeta);

    // adjust the log level //
    transformation.setLogLevel(LogLevel.ERROR);

    // retrieve logging appender
    Log4jBufferAppender appender = CentralLogStore.getAppender();

    // retrieve logging lines for transformation
    String logText = appender.getBuffer(transformation.getLogChannelId(), false).toString();

    transformation.execute(new String[0]);

    // waiting for the transformation to finish
    transformation.waitUntilFinished();

    org.pentaho.di.core.Result result = transformation.getResult();

    // report on the outcome of the transformation
    logger.info(result.getNrErrors());
    logger.info(result.getNrLinesWritten()); // this is always 0
    logger.info(logText); //this is always empty


    This should work, right? What am I doing wrong?
    Thanks in advance!

  2. #2
    Join Date
    Feb 2014
    Posts
    6

    Default

    I've run into the same problem. transformation.getResult() returns a result object with no meaningful information except for getNrErrors() > 0 if there are errors. Everything else is 0 even if the individual steps read or write thousands of records.

    Is the Result object ONLY useful for determining Success/Failure?

  3. #3
    Join Date
    Mar 2014
    Posts
    2

    Default

    I've found a solution : You can obtain the metric information from the list of executed steps (StepMetaDataCombi object) of the transformation object. Hereafter a simplified example, hope it works for you !

    Code:
    Trans transformation = new Trans(transMeta);
    transformation.setLogLevel(LogLevel.ERROR);
    org.pentaho.di.core.Result result = transformation.getResult();
    
    //number of errors
    logger.info(result.getNrErrors() + " errors ");
                            
    List<StepMetaDataCombi> steps2 = transformation.getSteps();
    int k = 0;
    while (k < steps2.size()) {
        StepMetaDataCombi step = steps2.get(k);
        if (step.step.getStepname().startsWith("WRITESDB-")) {
                            //metric "written lines"
                logger.info( step.step.getLinesWritten() + " items transfered: ";
        }
        k++;
    }
    Now, I would like to obtain the detailed description of the errors (if there are errors) through my Java program and print them to the user. I didn't find a solution for that. Please share if you find something!

Tags for this Thread

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •  
Privacy Policy | Legal Notices | Safe Harbor Privacy Policy

Copyright © 2005 - 2019 Hitachi Vantara Corporation. All Rights Reserved.