Hitachi Vantara Pentaho Community Forums
Page 1 of 2 12 LastLast
Results 1 to 10 of 13

Thread: PDI logging via Log4J

  1. #1

    Default PDI logging via Log4J

    Hey guys,

    I'll quite happily admit to being pretty inept when it comes to logging. Does it already or is it possible to hook up the PDI logging to log4j so I can send the output to a syslog and so?

    IE: can I just put a log4j config file somewhere and have it pick it up?

    Ta

    Tom
    This is a signature.... everyone gets it.

    Join the Unofficial Pentaho IRC channel on freenode.
    Server: chat.freenode.net Channel: ##pentaho

    Please try and make an effort and search the wiki and forums before posting!
    Checkout the Saiku, the future of Open Source Interactive OLAP(http://analytical-labs.com)

    http://mattlittle.files.wordpress.co...-bananaman.jpg

  2. #2
    Join Date
    Nov 1999
    Posts
    9,729

    Default

    Hi Tom,

    PDI indeed uses Log4J as the back-end. The logger is named "org.pentaho.di" so you can pick that one up and do whatever you like with it.
    The default log4j.xml file is included in lib/kettle-engine.jar but you can provide your own copy.

    HTH,
    Matt

  3. #3

    Default

    Thanks Matt, moronic question number two, to override that file can I slap a replacement file somewhere else, or do i have to repackage the jar with my new log4j config in it?

    Cheers
    Tom
    This is a signature.... everyone gets it.

    Join the Unofficial Pentaho IRC channel on freenode.
    Server: chat.freenode.net Channel: ##pentaho

    Please try and make an effort and search the wiki and forums before posting!
    Checkout the Saiku, the future of Open Source Interactive OLAP(http://analytical-labs.com)

    http://mattlittle.files.wordpress.co...-bananaman.jpg

  4. #4
    Join Date
    Nov 1999
    Posts
    9,729

    Default

    OK, so you can actually add -Dlog4j.debug to your Java command options to figure out where your log4j.xml is taken from.
    Just for the fun of it I tried to put a log4j.xml file in the root of the PDI distribution and this is what I got back:

    Code:
    log4j: Trying to find [log4j.xml] using context classloader java.net.URLClassLoader@3ea981ca.
    log4j: Using URL [file:/home/matt/svn/kettle/trunk/distrib/log4j.xml] for automatic log4j configuration.
    If I remove the file this comes up:

    Code:
    log4j: Trying to find [log4j.xml] using context classloader java.net.URLClassLoader@3ea981ca.
    log4j: Using URL [jar:file:/home/matt/svn/kettle/trunk/distrib/launcher/../lib/kettle-engine.jar!/log4j.xml] for automatic log4j configuration.
    So I guess you simply put it next to spoon.sh/Spoon.bat somewhere.

    HTH,
    Matt

  5. #5

    Default

    Cool. Thanks.

    Final question, I also want to create a GELF plugin for extra logging stuff, where in the SVN does the syslog plugin live? or does it not?

    Ta

    Tom
    This is a signature.... everyone gets it.

    Join the Unofficial Pentaho IRC channel on freenode.
    Server: chat.freenode.net Channel: ##pentaho

    Please try and make an effort and search the wiki and forums before posting!
    Checkout the Saiku, the future of Open Source Interactive OLAP(http://analytical-labs.com)

    http://mattlittle.files.wordpress.co...-bananaman.jpg

  6. #6

    Default

    Turned out I was a fool, and looked in the folders labeled plugin, which as ITXpander showed me is a fallacy

    Quote Originally Posted by bugg_tb View Post
    Cool. Thanks.

    Final question, I also want to create a GELF plugin for extra logging stuff, where in the SVN does the syslog plugin live? or does it not?

    Ta

    Tom
    This is a signature.... everyone gets it.

    Join the Unofficial Pentaho IRC channel on freenode.
    Server: chat.freenode.net Channel: ##pentaho

    Please try and make an effort and search the wiki and forums before posting!
    Checkout the Saiku, the future of Open Source Interactive OLAP(http://analytical-labs.com)

    http://mattlittle.files.wordpress.co...-bananaman.jpg

  7. #7

    Default

    Maybe in the OP I should have been more specific, I would like to capture and farm off the output of the logs, like the stuff that would end up in the database or spat out to the terminal whilst transformations and jobs are running. After having talked with Slawo, he says this is in fact just piped out to Stdout and stderr, which is unfortunate to say the least

    Tom
    This is a signature.... everyone gets it.

    Join the Unofficial Pentaho IRC channel on freenode.
    Server: chat.freenode.net Channel: ##pentaho

    Please try and make an effort and search the wiki and forums before posting!
    Checkout the Saiku, the future of Open Source Interactive OLAP(http://analytical-labs.com)

    http://mattlittle.files.wordpress.co...-bananaman.jpg

  8. #8
    Join Date
    Nov 1999
    Posts
    9,729

    Default

    What kind of magic do you want to achieve with that Trout?
    Anyway, the PDI console appender is only added in case there is no other console appender present. It's called "ConsoleAppenderrg.pentaho.di" in case you were wondering.

    Please note that while PDI uses Log4J, Log4J itself is not fine-grained enough for PDI. The Logging registry (check the wiki) was created to offer more possibilities.
    Depending on what you exactly want to do there might be better options than using Log4J.

  9. #9

    Default

    Hey Matt,

    What I was trying to do, preferably within PDI was push the log outputs to GrayLog2(webbased/glorified syslog) so our support team can view the output easily and track errors. What I have prototyped is an alternative syslog step using the GELF(http://www.graylog2.org/about/gelf) protocol, that I shall commit back once I've tidied it up, this allows for extended syslog payloads with stackstraces etc. But the grail would be, to be able to point the PDI logging output to the Graylog server preferably without major/any upheaval. The server comes with a few java libs/log4j adapters so i figured if the output went via log4j I could capture it and post it off to the server, the theory works, cause PDI sends some stuff to it, just not what I need

    Ideas on a postcard,

    thanks

    Tom
    This is a signature.... everyone gets it.

    Join the Unofficial Pentaho IRC channel on freenode.
    Server: chat.freenode.net Channel: ##pentaho

    Please try and make an effort and search the wiki and forums before posting!
    Checkout the Saiku, the future of Open Source Interactive OLAP(http://analytical-labs.com)

    http://mattlittle.files.wordpress.co...-bananaman.jpg

  10. #10
    Join Date
    Nov 1999
    Posts
    9,729

    Default

    Tom,

    The logging text of a job or transformation executed in a job is available as part of the Result object.
    See also the Result.getLogText() method over here: http://wiki.pentaho.com/display/EAI/...ript+job+entry
    and my blog: http://www.ibridge.be/?p=181

    Getting the log text for the complete job so far isn't very hard to do either. You simply need to know the log channel ID of the root job (getParentJob() all the way to the top) and then ask the CentralLogStore: String logText = CentralLogStore.getAppender().getBuffer(logChannelId, false);

    Good luck,
    Matt

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •  
Privacy Policy | Legal Notices | Safe Harbor Privacy Policy

Copyright © 2005 - 2019 Hitachi Vantara Corporation. All Rights Reserved.