Hitachi Vantara Pentaho Community Forums
Results 1 to 4 of 4

Thread: join logging tables

  1. #1

    Default join logging tables

    Hello all!

    I guess the best way of logging information in Kettle is to use the standard logging tables and -routines (so, the ones that can be found and turned on in the properties of both transformations and jobs). I now turned them on for jobs, job entries and transformations and get some nice logging in these 3 tables, so all works quite nicely.

    Now my question is: are there any "foreign keys" between these tables? Obviously no FK is created as a DB object but what I mean is more like: can I join these tables and if so, on which columns? E.g. if I want to see all transformations that were run during execution of a certain job. Strangely, I couldn't google this information. Someone must have had the same problem before?!?

    On another note, I would be interested in adding some information to these tables, e.g. some of the stuff that I can get through the "Get System Info"-step (e.g. command line arguments). Would you recommend that and how would you implement it?

    Thanks for your help!
    /D
    Last edited by d.hommrich; 03-20-2014 at 06:01 AM.

  2. #2
    Join Date
    Nov 1999
    Posts
    9,729

    Default

    The Log Channel Log table provides you with the exact execution lineage of transformations, jobs, steps, job entries, ... The log channel IDs tie the executions together.

    As for the second question, you could add extra columns to the log table which you populate in the transformation itself. You can use the batch id or the log channel id to update the column(s).

  3. #3

    Default

    Thanks for the answer, Matt! The log channel table and the channel_id's seem to be just what I was searching for.

    On the extra-columns/updating part of your answer: how can I get the current transformation's channel ID in the transformation? Seems like the "Get System Info"-step can't provide this. Will I need to write Java(Script) code to get it? Sounds like too common a task that it would be so well hidden?!?
    To be more concrete: In case of an error, I'd like to write some additional information (what caused the error?) into the transformation_step table. So both the error-string (possibly from "Get System Info" - "previous job entry log text") and the channel_id to tie the string to the correct line in the table would be necessary.
    Last edited by d.hommrich; 03-25-2014 at 06:04 AM.

  4. #4
    Join Date
    Aug 2012
    Posts
    2

    Default

    Hello!


    I have similar problem. I use petaho data integrator 5.0.1 CE. I tried to use the following log tables for:
    job, job entry, logging channel, tarnsformation, step


    When I run a job where logging is enabled then all those tables are populated with logging data. I can link job, logging channel, transformation and step information using CHANNEL_ID column. The problem is that job entry table does not match with other tabels. It also have CHANNEL_ID column but those values does not exists on logging channel table. Is this expected behaviour? If yes then what's the point of job entry logging table if it does not relate to other logging tables? If no can someone share his/her knowledge?

Tags for this Thread

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •  
Privacy Policy | Legal Notices | Safe Harbor Privacy Policy

Copyright © 2005 - 2019 Hitachi Vantara Corporation. All Rights Reserved.