Hitachi Vantara Pentaho Community Forums
Results 1 to 3 of 3

Thread: Counting threads in Spoon

  1. #1
    Join Date
    Feb 2017
    Posts
    5

    Unhappy Counting threads in Spoon

    Hi,
    I'm writing a job with 4 transformation. One of them gets some variables, the other creates a session, the third one migrates some data from one system to another, and the fourth transformation will send an email message with "logs" of the data that was migrated.

    Name:  PIC1.jpg
Views: 35
Size:  6.3 KB

    On the third transformation I created two sequences: one counts records successfully transferred and the other tracks errors. I tried setting those values to a variable so my fourth transformation can read the variable and then include that info in the email message. This gave me an "unexpected error" on the set variable step at run time.

    Name:  PIC2.jpg
Views: 34
Size:  19.0 KBName:  PIC3.jpg
Views: 33
Size:  14.0 KB


    After being unsuccessful at getting the variables to be set, I tried using "select values" and "copy rows to result" in my third transformation, and then getting those results in the fourth transformation. The problem I am having with this is that the fourth transformation seems to be running for every thread in the third validation. (So, if I print out the value of "successCount" in the fourth transformation, then I get 1, and 2, and 3...

    Name:  PIC4.jpg
Views: 37
Size:  15.4 KB

    How can I get it to just run once, once the third transformation has finished running, and with the final value of the results?

    This should be pretty standard, but I can't figure it out.

    Thanks in advance for your help.

  2. #2
    Join Date
    Jun 2012
    Posts
    5,534

    Default

    Some thoughts in no particular order:

    • Large images should be zipped and attached as files to preserve the original resolution.
    • Get-Variables is one cool step - if no input rows are present it will generate a single row.
    • So many JavaScript steps in a single transformation make me feel sad.
    • Always make sure a Set-Variables step will read a single row.
    • Generally it's a good idea to state OS and Kettle version.
    • The Mail step in a transformation will send a mail for each input row - not so the Mail job entry.
    So long, and thanks for all the fish.

  3. #3
    Join Date
    Feb 2017
    Posts
    5

    Default

    Quote Originally Posted by marabu View Post
    Some thoughts in no particular order:

    • Large images should be zipped and attached as files to preserve the original resolution.
    • Get-Variables is one cool step - if no input rows are present it will generate a single row.
    • So many JavaScript steps in a single transformation make me feel sad.
    • Always make sure a Set-Variables step will read a single row.
    • Generally it's a good idea to state OS and Kettle version.
    • The Mail step in a transformation will send a mail for each input row - not so the Mail job entry.
    Thanks for your reply.. I really appreciate all the feedback. I'm learning pentaho as I go, so my "form" might not be optimal. All advice is well received. I will use the Mail Job entry.. I think (and hope) that will do exactly what I need to do.

    Thanks again.

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •  
Privacy Policy | Legal Notices | Safe Harbor Privacy Policy

Copyright © 2005 - 2019 Hitachi Vantara Corporation. All Rights Reserved.