Hitachi Vantara Pentaho Community Forums
Results 1 to 13 of 13

Thread: Error hop executing always

  1. #1
    Join Date
    Nov 2013
    Posts
    382

    Default Error hop executing always

    Hi,
    I'm trying to define an error hop to deal with duplicate keys. It works because the process ends succesfully (without the error hop the process ends with an error). But the problem is the error step executes always, even when no duplicates (neither other errors) are present.

    According to Pentaho Wiki I should have an "Evaluation" option, but it is not on my Pentaho 4.4.0 ... all I have is "Edit hop", "Flip direction", "Disable hop", "Delete hop" and "Hop bulk change".

    Any idea what I'm doing wrong?

    Thks

  2. #2
    Join Date
    Jun 2012
    Posts
    5,534

    Default

    Hard to say what exactly you did wrong without seeing anything.
    So long, and thanks for all the fish.

  3. #3
    Join Date
    Nov 2013
    Posts
    382

    Default

    How can I copy the needed info? Would an Export to a .ktr file be ok? (it has al database connections/encripted pwd, etc so unsure ...)

    Thks

  4. #4
    Join Date
    Nov 2013
    Posts
    382

    Default

    Does this help in any way?
    Attached Images Attached Images

  5. #5
    Join Date
    Jun 2012
    Posts
    5,534

    Default

    Just tried myself, but no peculiarities over here.

    Name:  mem-db.jpg
Views: 210
Size:  31.6 KB
    So long, and thanks for all the fish.

  6. #6
    Join Date
    Nov 2013
    Posts
    382

    Default

    It seems that your "Error" step is also executed, it has a green tick. Try doing something there (writing a record somewhere) ...

    As I understand it the "Error" step shouldn't be executed if "Table output" step doesn't produce any error.

    If it executes always, how can I know there is "no error" ?

  7. #7
    Join Date
    Nov 2008
    Posts
    777

    Default

    Quote Originally Posted by DepButi View Post
    Does this help in any way?
    I think the problem is happening because it is a SQL Script step in the error branch. Notice that in your PDF screenshots, the metrics for that step say "Read 0" and "Written 1". Contrast that to marabu's screenshots where the Error branch (Dummy Step) says "Read 0" and "Written 0". Knowing that all steps (in transformations) start up at once and run concurrently, I believe the SQL Script will start up and fire immediately unless it is truly configured to run based on fields coming in from a row. How do you have your SQL Script step configured in this case? The answer to that question will most likely solve your problem.
    Last edited by darrell.nelson; 11-18-2013 at 03:18 PM. Reason: Clarity
    pdi-ce-4.4.0-stable
    Java 1.7 (64 bit)
    MySQL 5.6 (64 bit)
    Windows 7 (64 bit)

  8. #8
    Join Date
    Jun 2012
    Posts
    5,534

    Default

    Quote Originally Posted by darrell.nelson View Post
    the SQL Script will start up and fire immediately unless it is truly configured to run based on fields coming in from a row
    I second that.
    So long, and thanks for all the fish.

  9. #9
    Join Date
    Nov 2013
    Posts
    382

    Default

    Yes, my SQL Script writes a record on itself.

    I knew all steps start at once, but I expected -erroneously- error steps not starting at all if no errors were present. That was my mistake.

    So the question is now, how do I get the number of errors? Or how do I get the records in error? I need to configure an Error step able to tell me if there were errors or not.
    Which kind of step must I use? and how do I get the needed info?

    Thks

  10. #10
    Join Date
    Nov 2008
    Posts
    777

    Default

    Quote Originally Posted by DepButi View Post
    So the question is now, how do I get the number of errors? Or how do I get the records in error? I need to configure an Error step able to tell me if there were errors or not.
    Which kind of step must I use? and how do I get the needed info?
    There are lots of options. Here are a few:
    1. A Table Output step could write a record for each failing row; just be sure to include the record ID or some other key field so the table writes are traceable to the errant rows.
    2. An Excel Output step or a CSV/Text File Output step could do basically the same as 1. above except the output would go to a file.
    3. A Step Metrics step could tell you how many records were output by 1. or 2. above.
    Last edited by darrell.nelson; 11-18-2013 at 06:35 PM. Reason: Clarity
    pdi-ce-4.4.0-stable
    Java 1.7 (64 bit)
    MySQL 5.6 (64 bit)
    Windows 7 (64 bit)

  11. #11
    Join Date
    Nov 2013
    Posts
    382

    Default

    Thanks for the answers, it works as expected now, writing a log file with the erroneous records.

    The Metric Steps gives me the number of erroneous records when executing online.

    But I need also to check if there is any error at all to write a record on a table. Only ONE record saying 'Yes, there is at least one error.' or NO record at all if no errors.

    To clarify, I need:
    1. To write erroneous records. This is solved with Error step.
    2. Check if number of errors>0 and write ONE record on a table. No idea how to do that, tried several output options ... and I cannot figure out how to do so ...

    Edited to add: So the only problem is NOT to write if there are no errors. How to check the number and do nothing if 0 ?
    Last edited by DepButi; 11-19-2013 at 06:09 AM.

  12. #12
    Join Date
    Nov 2008
    Posts
    777

    Default

    Quote Originally Posted by DepButi View Post
    To clarify, I need:
    2. Check if number of errors>0 and write ONE record on a table. No idea how to do that, tried several output options ... and I cannot figure out how to do so ...

    Edited to add: So the only problem is NOT to write if there are no errors. How to check the number and do nothing if 0 ?
    Again there are options:
    1. Another Step Metrics step, this time on the "main" Table Output step, which will capture the Error Count. This could be followed by a Filter Rows step that accepts only a non-zero Error Count value.
    2. In your error branch, append an Add Sequence step, an "Identify last row in a stream" step, and then another Table Output step for the error summary record. If there were no errors, then that last Table Output step should not write a record.
    pdi-ce-4.4.0-stable
    Java 1.7 (64 bit)
    MySQL 5.6 (64 bit)
    Windows 7 (64 bit)

  13. #13
    Join Date
    Nov 2013
    Posts
    382

    Default

    Thanks for your answers and help.

    I had to add an "Add constant values" step to create my record and a "Formula" step to add a time stamp and everything is running smoothly now .

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •  
Privacy Policy | Legal Notices | Safe Harbor Privacy Policy

Copyright © 2005 - 2019 Hitachi Vantara Corporation. All Rights Reserved.