Hitachi Vantara Pentaho Community Forums
Results 1 to 4 of 4

Thread: Problem with step Add a checksum

  1. #1
    Join Date
    Dec 2017
    Posts
    2

    Default Problem with step Add a checksum

    Hey,

    I am having a problem with the step "Add a checksum" in PDI version 8 community edition.

    I created a simple transformation that concatenate 2 fields with # delimiter and then create a md5 checksum on this new field.

    Examble:
    field1: a
    field2: b
    concat_fields: a#b

    The correct md5 Hex on the concat_fields value should be 6457c7988b74a5dd2057c7bf0905389e (based on PostgreSQL md5 function). However when selecting this field in add checksum step and running the transformation I am getting wrong value of 0cc175b9c0f1b6a831c399e269772661.

    I found out that although I chose the concat field in the checksum step, PDI is calculating the md5 only on the first field in the stream (field1) and gives the md5 of value "a" in this case.

    When I add another step of Select/Rename values and re-order the steam so the concat_fields is the first, then the md5 hash is correct.

    In previous versions this issue does not exist.

    Attached is a simple transformation.
    Attached Files Attached Files

  2. #2
    Join Date
    Jun 2012
    Posts
    5,534

    Default

    Thank you very much for presenting your findings.
    It's just as if this step never was tested.
    Actually, the unit tests seem to focus on the checksum algorithms, not on the step code.
    I prepared a demo and compared version 8 with my beloved version 4.4.

    Here's what I used as input:

    field1 field2 sep source
    a b # a#b


    And here's what I ended up with:

    Transformation 4.4 8.0
    Add-Checksum MD5(single field) ea9b... 0cc1...
    Add-Checksum MD5(multiple fields) 6457... 131f...
    Apache Commons DigestUtils md5Hex(single field) ea9b... 6457...
    Apache Commons DigestUtils md5Hex(multiple fields) 6457... 6457...

    Please, file a bug and provide the Jira link here.
    Or, if you don't want to go that way, say so and I'll do it.
    Attached Files Attached Files
    So long, and thanks for all the fish.

  3. #3
    Join Date
    Jun 2012
    Posts
    5,534

    Default

    Oops, my answer from a couple of hours ago isn't approved yet.

  4. #4
    Join Date
    Dec 2017
    Posts
    2

    Default

    Thanks a lot for your reply.

    I opened an bug on it and attached also your file:

    http://jira.pentaho.com/browse/PDI-16903

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •  
Privacy Policy | Legal Notices | Safe Harbor Privacy Policy

Copyright © 2005 - 2019 Hitachi Vantara Corporation. All Rights Reserved.