Hitachi Vantara Pentaho Community Forums
Results 1 to 10 of 10

Thread: errors

  1. #1
    Join Date
    Nov 2007
    Posts
    4

    Default errors

    Hi all,

    I use spoon 2.5.1 and I have this problem:

    I want to catch errors (code, description, job and step where error occured, time,...) that have occured during a job. something like error handling. but i need it in each step, not only in Java Script or table output. Idea is controling all error that occur and pass their description into a table or file.
    is there some way to do it?

    thanks for all ideas

  2. #2
    Join Date
    Nov 1999
    Posts
    9,729

    Default

    That's just crazy talk!

  3. #3

    Default

    Why is this crazy?

    I was about to post somewhat the same question, I didn't want to be able to trap every step, but to know what step in a job had failed if it doesn't have the "define error handling" config in its step properties, as this information will have to be passed to a third party as part of the interface, this third party will then mail the error to the appropriate user. (I realise kettle could do this but the customer wants control of all communication with users)

    My background is in programming, rather than dragging things around on a gui so I expect I'm looking at this the wrong way. I've trawled the forums, docs, wiki, tips, Mat's, Jens, Nick's blogs etc and I'm just not getting the error handling at all. Is there a widely used method of error handlign within Kettle, or a sample that I've overlooked?

    Also on one of the forum posts regarding error handling there is a post from yourself Mat pointing a forum user to kettle tip 9, but the url takes you to the community page now and I can't work out which tip I'm meant to be looking at, I was hoping thats the missing link!

  4. #4
    Join Date
    Nov 1999
    Posts
    9,729

    Default

    http://wiki.pentaho.org/display/COM/Technical+Tips

    Error handling obviously only makes sense at the step level, that's why it's crazy talk.

    http://www.ibridge.be/?p=32

    Matt

  5. #5

    Default

    Thankyou for the respose.

    OK, maybe I'm not explaining myself very well.

    Handling errors at the step level - I can shunt anything that failed on insert to a file for example, I understand that and it filts perfectly in most situations.

    However there are steps in a job that don't have error handing allowed on them but they can still fail - for example I have a sql script step in one interface that I am currently working on, its updating some keyed fields so can certainly fail. From what I can gather an error on this step will halt the transformation and control will be passed back out to the job that called the transform. What I can't work out is how I could get the error message that occured on that step into a file, is the error info passed back out to the job and how do I access it if it is?

    Cheers,

    stevens_ns

  6. #6
    Join Date
    Nov 1999
    Posts
    9,729

    Default

    Aha!

    At the job level, all job entries are doing "error handling" in the definition that you give it.
    You can draw 3 types of links (job hops) between job entries:

    1) unconditional (black color)
    2) follow when everything ran OK (green)
    3) follow when there was an error (red)

    Using that simple principle, you can design complete flows.

    HTH,

    Matt

  7. #7

    Default

    Hi Matt,

    Yes thanks that helps alot I had missed that in the docs, so I add another transform in the job which is accessed on false, however how can I get the error information to this transform so I can write it to a file? Are their some environment variables that can be read or something I need to setup?

  8. #8
    Join Date
    Nov 1999
    Posts
    9,729

    Default

    Logging to a file can be set in the job entries "Transformation" and "Job" (and a few others).
    You can even log to a database.
    You can even mail the logfiles as an attachment if you like. (Mail job entry)

    HTH,

    Matt

  9. #9

    Default

    right ok thanks, looks like I'll have to log to a database then read that back out to a file. The problem I have is that the error file must be in a specific format, which doesn't look possible from the transform job entry settings.

    Thanks again for your help

    Nicole

  10. #10

    Default

    Hello,

    Approaching a way to do this now, I was just wondering if the way I've chosen is the best way to do this.

    I've set error logging on in the transaction steps in the job, they output to a file.
    On error I want to read the lines of that error file in one transformation
    Then in another transform I read the row and append each formated error row to a env variable
    Then in the next transform I write this env variable as one error line to a file.

    Basically I want to loop through the error file, pick out a small portion of each error line and write each of these portions concatenated to one line to an error file.

    Cheers,

    stevens_ns

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •  
Privacy Policy | Legal Notices | Safe Harbor Privacy Policy

Copyright © 2005 - 2019 Hitachi Vantara Corporation. All Rights Reserved.