Hitachi Vantara Pentaho Community Forums
Results 1 to 10 of 10

Thread: How to force immediate write of log file

  1. #1
    Join Date
    Nov 2013
    Posts
    382

    Default How to force immediate write of log file

    We have a job running daily that hangs up from time to time. No error, no messages, nothing. Just hangs up waiting forever in a FUTEX WAIT status. Our system people has been trying to discover what happens for months without success.

    One of the problems is that we don't know where exactly hangs up because the log file is incomplete, we know for sure more data/steps have been processed/executed than those shown on the log. It's clear that the log file has not been written up to the last operation.

    We end up with a log file on the form

    2017/11/22 06:39:03 - Text file output.0 - Procesamiento finalizado (I=0, O=9, R=9, W=9, U=0, E=0)
    2017/11/22 06:39:03 - Blocking Step.0 -OpenJDK 64-Bit Server VM warning: ignoring option MaxPermSize=256m; support was removed in 8.0
    nov 22, 2017 7:06:04 AM org.apache.karaf.main.Main$KarafLockCallback lockAquired
    INFORMACIÓN: Lock acquired. Setting startlevel to 100

    In red the start of a new execution job just in the middle of a line written 20 min ago by the hang up job. Also, a log step informs us of what files have been written on the process and there are always more real files on the folders than shown on the corresponding logs, so it's obvious than the log file is not written entirely.

    Is there any way to force the log file to be written in a safe way?

  2. #2
    Join Date
    Jun 2012
    Posts
    5,534

    Default

    Practically, Kettle uses log4j 1.2.

    Theoretically, we should be able to configure the relevant log appender for immediate flushing.
    That's not a thing you want to do in production, though, but it might help your investigation of the mysterious case of hang ups.

    Tinkering with log4j.xml isn't something I do on a regular basis, so I can't point you to the exact spot right now.
    So long, and thanks for all the fish.

  3. #3
    Join Date
    Nov 2013
    Posts
    382

    Default

    Most probably adding a

    <param name="immediateFlush" value="true"/>

    to plugins/kettle5-log4j-plugin/log4j.xml

    Will have to test it

  4. #4
    Join Date
    Jun 2012
    Posts
    5,534

    Default

    Not sure if it's that easy, since unbuffered i/o should be default for log appenders.

    Good luck, though.

  5. #5
    Join Date
    Nov 2013
    Posts
    382

    Default

    Quote Originally Posted by marabu View Post
    Not sure if it's that easy, since unbuffered i/o should be default for log appenders.

    Good luck, though.
    Seems the default is set to false http://www.tutorialspoint.com/log4j/...ging_files.htm

  6. #6
    Join Date
    Nov 2013
    Posts
    382

    Default

    Ooooops!!! Nope, false to buffer, so true to inmediate!

    Reading too fast!

  7. #7
    Join Date
    Apr 2008
    Posts
    4,696

    Default

    Are you using PDI 6.0 or 6.1 by chance?

  8. #8
    Join Date
    Nov 2013
    Posts
    382

    Default

    No. We are using PDI 7.0

  9. #9
    Join Date
    Apr 2008
    Posts
    4,696

    Default

    So I re-read things again, and noticed that you appear to have two different logs writing on top of each other.
    Make sure you use a distinct log for each Job-Run.

    Blocking Step is trouble... It means to me that you likely have a design issue in your transformation, possibly a deadlock within the transformation.

    Here be dragons.

  10. #10
    Join Date
    Nov 2013
    Posts
    382

    Default

    Quote Originally Posted by gutlez View Post
    So I re-read things again, and noticed that you appear to have two different logs writing on top of each other.
    Make sure you use a distinct log for each Job-Run.
    It's the same job running every 15 minutes, so the log file is always the same. We created (for testing purposes) a diferent log file but the result was the same: the first (hanged up job) did not write the full operations done, there are missing lines.

    Blocking Step is trouble... It means to me that you likely have a design issue in your transformation, possibly a deadlock within the transformation.
    Yes, we know blocking step is extremely trouble. We use only when we need to write a header plus lines. Inn that particular case it was added after a Text File Output step just to be sure the file was already closed before proceding to the next step (a copy rows to result + a second transformation with a copy file step), in fact it is not there usually and I just picked up the log of a testing execution. No, there is no deadlock transformation problems, the problem is identified as an ftp related problem but cannot say exactly why neither how to solve it. At first we thought the file was transferred while another process (out of our control) tried to move it, but after adding as many controls as we could imagine we still have problems from time to time.

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •  
Privacy Policy | Legal Notices | Safe Harbor Privacy Policy

Copyright © 2005 - 2019 Hitachi Vantara Corporation. All Rights Reserved.