Hitachi Vantara Pentaho Community Forums
Results 1 to 2 of 2

Thread: Problem with Merge Rows (diff) step

  1. #1

    Default Problem with Merge Rows (diff) step

    I am having a problem comparing two fields using Merge Rows (diff) step. My "reference" row is only one field that contains a filename from a result set. The "compare" is a filename from a file directory. I just want to test if the filename from the "reference" row exists in the file directory. If I test this transformation on it's own substituting the result set with a "generate rows" step it works fine. When I run the job I get this error:

    2019/05/02 11:16:48 - cdr_file_from_directory.0 - TRIMMED FILE: cdr_StandAloneCluster-1_01_201905021745_3448712019/05/02 11:16:48 - Merge Rows (diff).0 - ERROR (version 7.1.0.0-12, build 1 from 2017-05-16 17.18.02 by buildguy) : Unexpected error
    2019/05/02 11:16:48 - Merge Rows (diff).0 - ERROR (version 7.1.0.0-12, build 1 from 2017-05-16 17.18.02 by buildguy) : org.pentaho.di.core.exception.KettleException:
    2019/05/02 11:16:48 - Merge Rows (diff).0 - Invalid layout detected in input streams, keys and values to merge have to be of identical structure and be in the same place in the rows
    2019/05/02 11:16:48 - Merge Rows (diff).0 -
    2019/05/02 11:16:48 - Merge Rows (diff).0 - We detected rows with varying number of fields, this is not allowed in a transformation. The first row contained 4 fields, another one contained 1 : [cdr_file String]
    2019/05/02 11:16:48 - Merge Rows (diff).0 -
    2019/05/02 11:16:48 - Merge Rows (diff).0 -
    2019/05/02 11:16:48 - Merge Rows (diff).0 - at org.pentaho.di.trans.steps.mergerows.MergeRows.processRow(MergeRows.java:99)
    2019/05/02 11:16:48 - Merge Rows (diff).0 - at org.pentaho.di.trans.step.RunThread.run(RunThread.java:62)
    2019/05/02 11:16:48 - Merge Rows (diff).0 - at java.lang.Thread.run(Thread.java:748)
    2019/05/02 11:16:48 - Merge Rows (diff).0 - Caused by: org.pentaho.di.core.exception.KettleRowException:
    2019/05/02 11:16:48 - Merge Rows (diff).0 - We detected rows with varying number of fields, this is not allowed in a transformation. The first row contained 4 fields, another one contained 1 : [cdr_file String]
    2019/05/02 11:16:48 - Merge Rows (diff).0 -
    2019/05/02 11:16:48 - Merge Rows (diff).0 - at org.pentaho.di.trans.step.BaseStep.safeModeChecking(BaseStep.java:2121)
    2019/05/02 11:16:48 - Merge Rows (diff).0 - at org.pentaho.di.trans.steps.mergerows.MergeRows.checkInputLayoutValid(MergeRows.java:262)
    2019/05/02 11:16:48 - Merge Rows (diff).0 - at org.pentaho.di.trans.steps.mergerows.MergeRows.processRow(MergeRows.java:97)
    2019/05/02 11:16:48 - Merge Rows (diff).0 - ... 2 more
    2019/05/02 11:16:48 - Merge Rows (diff).0 - Finished processing (I=0, O=0, R=2, W=0, U=0, E=1)
    2019/05/02 11:16:48 - see_if_file_received_from_sftp - Transformation detected one or more steps with errors.
    2019/05/02 11:16:48 - see_if_file_received_from_sftp - Transformation is killing the other steps!
    2019/05/02 11:16:48 - see_if_file_received_from_sftp - ERROR (version 7.1.0.0-12, build 1 from 2017-05-16 17.18.02 by buildguy) : Errors detected!
    2019/05/02 11:16:48 - select_only_one_field_from_file_directory.0 - Finished processing (I=0, O=0, R=1, W=1, U=0, E=0)
    2019/05/02 11:16:48 - cdr_file_from_directory.0 - Finished processing (I=0, O=0, R=1, W=1, U=0, E=0)
    2019/05/02 11:16:49 - see_if_file_received_from_sftp - ERROR (version 7.1.0.0-12, build 1 from 2017-05-16 17.18.02 by buildguy) : Errors detected!
    This makes no sense as you can see there is only one field coming in from either side. How could there be 4? Also I thought I could attach a transformation here....

    I have attached the transformation.

  2. #2
    Join Date
    Apr 2008
    Posts
    4,689

    Default

    The tools are telling you that one of the steps gives 4 columns.
    The other step only gives one column.

    This is not allowed.

    From the logs you've posted, we can see that each step is giving only one row.

    (You used to be able to post transforms, and on the new forum you can... just not here)
    Though... you can post the transform on the new forum, and post a link to it from here...

Tags for this Thread

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •  
Privacy Policy | Legal Notices | Safe Harbor Privacy Policy

Copyright © 2005 - 2019 Hitachi Vantara Corporation. All Rights Reserved.