Hitachi Vantara Pentaho Community Forums
Results 1 to 10 of 10

Thread: Conditional Branching and mail sending

  1. #1
    Join Date
    Apr 2013
    Posts
    15

    Default Conditional Branching and mail sending

    Hi,
    i've got an question regarding the conditional branching in transformations. The conditional branch did not do what i'am expect. The mail is send each time.
    Name:  regex_branch.jpg
Views: 326
Size:  9.3 KB

    Here is an output of the execution:

    2013/11/19 13:25:25 - Spoon - Transformation opened.
    2013/11/19 13:25:25 - Spoon - Launching transformation [REGEX]...
    2013/11/19 13:25:25 - Spoon - Started the transformation execution.
    2013/11/19 13:25:26 - REGEX - Dispatching started for transformation [REGEX]
    2013/11/19 13:25:26 - Transformation metadata - Natural sort of steps executed in 0 ms (8 time previous steps calculated)
    2013/11/19 13:25:26 - Data Grid.0 - Finished processing (I=0, O=0, R=0, W=3, U=0, E=0)
    2013/11/19 13:25:26 - Write to log 2.0 -
    2013/11/19 13:25:26 - Write to log 2.0 - ------------> Linenr 1------------------------------
    2013/11/19 13:25:26 - Write to log 2.0 - FALSE
    2013/11/19 13:25:26 - Write to log 2.0 -
    2013/11/19 13:25:26 - Write to log 2.0 - datasegment = 111
    2013/11/19 13:25:26 - Write to log 2.0 - result = Y
    2013/11/19 13:25:26 - Write to log 2.0 -
    2013/11/19 13:25:26 - Write to log 2.0 - ====================
    2013/11/19 13:25:26 - Write to log 2.0 -
    2013/11/19 13:25:26 - Write to log 2.0 - ------------> Linenr 2------------------------------
    2013/11/19 13:25:26 - Write to log 2.0 - FALSE
    2013/11/19 13:25:26 - Write to log 2.0 -
    2013/11/19 13:25:26 - Write to log 2.0 - datasegment = 112
    2013/11/19 13:25:26 - Write to log 2.0 - result = Y
    2013/11/19 13:25:26 - Write to log 2.0 -
    2013/11/19 13:25:26 - Write to log 2.0 - ====================
    2013/11/19 13:25:26 - Write to log 2.0 -
    2013/11/19 13:25:26 - Write to log 2.0 - ------------> Linenr 3------------------------------
    2013/11/19 13:25:26 - Write to log 2.0 - FALSE
    2013/11/19 13:25:26 - Write to log 2.0 -
    2013/11/19 13:25:26 - Write to log 2.0 - datasegment = 113
    2013/11/19 13:25:26 - Write to log 2.0 - result = Y
    2013/11/19 13:25:26 - Write to log 2.0 -
    2013/11/19 13:25:26 - Write to log 2.0 - ====================
    2013/11/19 13:25:26 - Regex Evaluation.0 - Finished processing (I=0, O=0, R=3, W=3, U=0, E=0)
    2013/11/19 13:25:26 - Filter rows.0 - Finished processing (I=0, O=0, R=3, W=3, U=0, E=0)
    2013/11/19 13:25:26 - Write to log 2.0 - Finished processing (I=0, O=0, R=3, W=3, U=0, E=0)
    2013/11/19 13:25:26 - Mail Parameter.0 - Finished reading query, closing connection.
    2013/11/19 13:25:26 - Mail Parameter.0 - Finished processing (I=1, O=0, R=0, W=1, U=0, E=0)
    2013/11/19 13:25:26 - Mail.0 - Finished processing (I=0, O=0, R=1, W=1, U=0, E=0)
    2013/11/19 13:25:26 - Spoon - The transformation has finished!!

    Please can someone help me?
    I've attached the ktr file to this post.
    Any Ideas?
    Many thanks
    Ayhan
    Attached Files Attached Files

  2. #2
    Join Date
    Apr 2008
    Posts
    4,696

    Default

    First thing to learn in Transformation logic: Filter is not branching logic.

    The data flow in Transformations is like a stream - Filter is like a flow gate... Data will flow in the direction you tell it to, but the downstream steps still exist and will still run if they can...
    **THIS IS A SIGNATURE - IT GETS POSTED ON (ALMOST) EVERY POST**
    I'm no expert.
    Take my comments at your own risk.

    PDI user since PDI 3.1
    PDI on Windows 7 & Linux

    Please keep in mind (and this may not apply to this thread):
    No forum member is going to do your work for you. We will help you sort out how to do a specific part of the work, as best we can, in the timelines that our work will allow us.
    Signature Updated: 2014-06-30

  3. #3
    Join Date
    Apr 2013
    Posts
    15

    Default

    Thank you so much for your answer. How can i realise this conditional branching? Do you have any examples or suggestions?
    Many Thanks
    Ayhan

  4. #4
    Join Date
    Apr 2008
    Posts
    1,771

    Default

    I *think* that you need to use a job and split the "stream" to 2 different transformations.
    -- Mick --

  5. #5
    Join Date
    Apr 2013
    Posts
    15

    Default

    Quote Originally Posted by Mick_data View Post
    I *think* that you need to use a job and split the "stream" to 2 different transformations.
    Thank you very much Mick. How can you branch in a job? Do you have an example?
    Thank you.
    Ayhan

  6. #6
    Join Date
    Apr 2008
    Posts
    1,771

    Default

    Hi.
    Sorry but I don't have any example on hand.
    I did it once and used "Simple evaluation" step: http://wiki.pentaho.com/display/EAI/...28Job+Entry%29
    But I am not sure that it can be applied to your specific case.

    When you states that the mail is sent everytime.. what do you want the transformation to do?
    Send an email only if the whole transformation succeed without any errors?
    -- Mick --

  7. #7
    Join Date
    Apr 2013
    Posts
    15

    Default

    Quote Originally Posted by Mick_data View Post
    Hi.
    Sorry but I don't have any example on hand.
    I did it once and used "Simple evaluation" step: http://wiki.pentaho.com/display/EAI/...28Job+Entry%29
    But I am not sure that it can be applied to your specific case.

    When you states that the mail is sent everytime.. what do you want the transformation to do?
    Send an email only if the whole transformation succeed without any errors?
    Hi Mick,
    i want to collect all data (from the data grid), which doesnt match a regex, in an file and send this file in an email.
    It is also possible that all data are wellformed, so in this case there should not be send an e-mail. If there is one row, that doesnt match to the regex, this row should be stored in a file and send it to email. Data grid in my case is only for test purposes.
    I thought, that i can do this with filter row.
    How can i handle this?
    Many Thanks
    Ayhan

  8. #8
    Join Date
    Apr 2008
    Posts
    1,771

    Default

    If you want to Validate the data, then use Data Validator step: http://wiki.pentaho.com/display/EAI/Data+Validator

    You can send the *wrong* rows to a Text File.
    Then in a different transformation you can check number of rows in that file and decide if that file has to be attached to an email or not.
    -- Mick --

  9. #9
    Join Date
    Apr 2013
    Posts
    15

    Default

    Quote Originally Posted by Mick_data View Post
    If you want to Validate the data, then use Data Validator step: http://wiki.pentaho.com/display/EAI/Data+Validator

    You can send the *wrong* rows to a Text File.
    Then in a different transformation you can check number of rows in that file and decide if that file has to be attached to an email or not.
    Hi Mick,
    thank you so much. I'll try it. Although I do not know how to find out the number of rows, but I think that will be not so difficult.
    Thank you.
    Ayhan

  10. #10
    Join Date
    Apr 2008
    Posts
    4,696

    Default

    Some methods that I can think of:
    Group By (Count of cell values)
    Output steps metrics (How many rows were written by the Error step)
    Add Sequence (How many rows passed the Add Sequence Step - this can be placed inline before the write to file)

    The transformation Mail step is designed to send one email per row that it receives, so if you write all the errors to a file, and then group the rows to count them (getting a count of all errors), then add the information required for sending an email, then send the email, you should be good.

    In fact, you could even include the error file in the email.
    **THIS IS A SIGNATURE - IT GETS POSTED ON (ALMOST) EVERY POST**
    I'm no expert.
    Take my comments at your own risk.

    PDI user since PDI 3.1
    PDI on Windows 7 & Linux

    Please keep in mind (and this may not apply to this thread):
    No forum member is going to do your work for you. We will help you sort out how to do a specific part of the work, as best we can, in the timelines that our work will allow us.
    Signature Updated: 2014-06-30

Tags for this Thread

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •  
Privacy Policy | Legal Notices | Safe Harbor Privacy Policy

Copyright © 2005 - 2019 Hitachi Vantara Corporation. All Rights Reserved.