Hitachi Vantara Pentaho Community Forums
Results 1 to 5 of 5

Thread: Rows Getting Duplicated Between Jobs

  1. #1
    Join Date
    Oct 2014
    Posts
    8

    Default Rows Getting Duplicated Between Jobs

    We have a Pentaho job that contains multiple job steps. We are having an issue were the results are getting duplicated. Keep in mind that I'm a Pentaho newbie and it may be operator error.

    Name:  mainjob.GIF
Views: 23
Size:  5.3 KB

    The transformation step populates the results with two rows using a copy rows to result steps
    Both Job and Job 2 call the same job which calls a transformations and dumps the results to a text file. Job writes to job1.txt and Job 2 writes to job2.txt

    Here are the results:

    job1.txt
    field1;field2
    row1_field1;row1_field2
    row2_field1;row2_field1

    job2.txt
    field1;field2
    row1_field1;row1_field2
    row2_field1;row2_field1
    row1_field1;row1_field2
    row2_field1;row2_field1
    Attached Files Attached Files

  2. #2
    njain111 Guest

    Default

    It seems something weird is going on 'Job' is copying your rows again to results, even though you have not mentioned it. It could be a feature or be a bug. Dont know.

    But as a work around connect your 'Job2' directly to 'Transformation' instead of connecting to 'Job' in you main job.
    This resolves the issue.

  3. #3
    Join Date
    Apr 2008
    Posts
    4,696

    Default

    Quote Originally Posted by njain111 View Post
    But as a work around connect your 'Job2' directly to 'Transformation' instead of connecting to 'Job' in you main job.
    This resolves the issue.
    Not if the actual use-case requires something to be completed in Job 1 before Job 2 happens.

    Usually, "Get Rows from Result" consumes those rows, so that the next job can't also "Get Rows from Result" ... So there may be something odd going on here.

    @sphlen - I think you may have found a bug. If you turn logging on on Job1 at Row Level, you'll see it is executing SubJob.ktr a second time, with the other parameter.
    Last edited by gutlez; 10-07-2014 at 02:51 PM.

  4. #4
    Join Date
    Oct 2014
    Posts
    8

    Default

    Quote Originally Posted by gutlez View Post
    @sphlen - I think you may have found a bug. If you turn logging on on Job1 at Row Level, you'll see it is executing SubJob.ktr a second time, with the other parameter.
    Thanks for the assistance everyone. What is the proper way to submit a bug?

  5. #5
    Join Date
    Jun 2012
    Posts
    5,534

    Default

    Here's the Kettle bug tracker, but be sure to attach a demo of the bug.
    Your attachment in this thread is working just fine over here - ubuntu 13.10, Windows 7.
    So long, and thanks for all the fish.

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •  
Privacy Policy | Legal Notices | Safe Harbor Privacy Policy

Copyright © 2005 - 2019 Hitachi Vantara Corporation. All Rights Reserved.