Hitachi Vantara Pentaho Community Forums
Results 1 to 5 of 5

Thread: Tracker / Logger

  1. #1

    Default Tracker / Logger

    Hi, I have been going through the code to better understand how the tracker works so that I could provide my own implementation.

    It appears that ChefLog is the starting point for the Chef Tracker. It builds the tree that one sees in Spoon when a job executes.

    I also noticed KettleLogRecord, which appears to be the class that logs transformation outcome to the database. Is this class invoked whenever results are updated or on completion of the transformation? Is this also invoked for jobs? If not, could you please point me to the class that updates the job status to the log table.

    Kettle expects the log table to have the following structure

    CREATE TABLE KETTLE_LOG
    (
    ID_JOB INTEGER
    , JOBNAME VARCHAR2(50)
    , STATUS VARCHAR2(15)
    , LINES_READ INTEGER
    , LINES_WRITTEN INTEGER
    , LINES_UPDATED INTEGER
    , LINES_INPUT INTEGER
    , LINES_OUTPUT INTEGER
    , ERRORS INTEGER
    , STARTDATE DATE
    , ENDDATE DATE
    , LOGDATE DATE
    , DEPDATE DATE
    , REPLAYDATE DATE
    , LOG_FIELD CLOB
    )
    ;

    My thought is to add two new attributes. Node and ParentNode. Both would be of type integer.

    The JOB_ID would be set if there is NO parent JOB and would be passed to each sub-job / transformation. Can this be done ? I am assuming that it could. Not sure how this would work in a clustered environment.

    Node would simply be the number of the node in the job or transformation. Parent Node would be the number of the parent that invoked it.

    The implementation that logs the information would include node and parentnode in its output. This would allow one eto track progress and build the tree as in tracker.

    Given that monitoring / logging are aspects and implementations may wish to provide their own implementations (such as a JMX implementation) we would need a factory to get the implementation class.

    So - is the above in synch with 3.0 ? Is there some code that I may be able to leverage? If I provided the above implementation would this be something that you would consider incorporating into the mainline product?

    Is there any thought to using an IoC container such as Spring for the Kettle implementation?

  2. #2
    Join Date
    May 2006
    Posts
    4,882

    Default

    NO spring ... for the rest it may be better off to move it to the developer list.

    Regards,
    Sven

  3. #3

    Default

    Ok - how do I access the developer list ?

  4. #4
    Join Date
    May 2006
    Posts
    4,882

  5. #5

    Default

    I subscribed to the group however get an error when I try to post to it.

    You do not have permission to post to group kettle-developers. You may
    need to join the group before being allowed to post, or this group may not
    be open to posting.

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •  
Privacy Policy | Legal Notices | Safe Harbor Privacy Policy

Copyright © 2005 - 2019 Hitachi Vantara Corporation. All Rights Reserved.