Hitachi Vantara Pentaho Community Forums
Results 1 to 9 of 9

Thread: Row Denormaliser Step not functioning correctly

  1. #1

    Default Row Denormaliser Step not functioning correctly

    I have some sorted data that I'm trying to denormalize but the step in Pentaho isn't working correctly.
    Here is a snapshot of the sorted data:



    And here is a snapshot of the Row Denormaliser Step as I've configured it:



    What I get is:


    There are no steps between the sorted data preview and the Row Denormaliser Step. I've also made sure that the field type of 'Number' is consistent with the field type of the output field of the previous step.
    What am I missing/getting wrong? Any ideas as to why it's not working?
    Last edited by dmukherji; 12-19-2014 at 12:18 PM.

  2. #2
    Join Date
    Jun 2012
    Posts
    5,534

    Default

    I suggest you isolate your problem in a transformation consisting of just two steps: "Data Grid" holding some test rows and "Row Denormaliser".
    If we can reproduce the unwanted result, we can explain what happened.
    You certainly don't want anybody to type in your test data from images, do you?
    So long, and thanks for all the fish.

  3. #3

    Default

    Thank you for the insight. I tried out what Maribu recommended - with 'test' data. I took a Data Grid step and input the data exactly the same as the output of the Table Input step - and it worked fine! But with the Table Input step, it breaks. Here are the screenshots:

    Transformation

    The Select Values step converts all field types to String.

    With Table Input Step:

    Table Input Step's Data:


    Final Output:


    With Data Grid Step:


    Data Grid Step's Data: (exact same as Table Input Step's Data)


    Final Output:



    I've hit a roadblock and don't understand how the table input step could be breaking the transformation. If anyone has any insight, please share!

  4. #4
    Join Date
    Jun 2012
    Posts
    5,534

    Default

    Make sure the Order-By clause in your Table Input step looks something like "ORDER BY jobid, phaseid ".
    So long, and thanks for all the fish.

  5. #5

    Default

    Just tried it - did not work. Although, I don't understand why I would need an 'Order By' clause in the SQL if I have a 'Sort Rows' step in the transformation.

  6. #6
    Join Date
    Jun 2012
    Posts
    5,534

    Default

    Sorry, didn't recall the Sort Rows step in your design.
    Often the database server packs more punch than your Kettle client.
    On the other side, you don't have to pay that much attention to collation order when using Sort Rows instead of an ORDER-BY clause, so suit yourself.
    Without sample data to reproduce the phenomenon on my machine there's not much I can do for you.
    Your configuration of the Denormalizer step is flawless, I should say.
    Good luck.
    So long, and thanks for all the fish.

  7. #7

    Default

    Further Testing:


    My database connection is that of an MS SQL Server 2008 R2 SP2 Express for the original issue. I have now tested the following:


    - Similar architecture for a PostgreSQL Server (2 groupings on the normaliser step): SUCCESS
    - Single grouping on the MS SQL Server with the original field types (without Select Values Step) as 'String': FAILURE


    It seems that this issue is localized to the use of a MS SQL Server connection. Creating a blocker JIRA ticket now on Pentaho - hopefully someone on the team will be able to reproduce the bug(?).

  8. #8
    Join Date
    Apr 2008
    Posts
    4,696

    Default

    Watch out for trims.
    If the data coming from MSSQL is 'EXP ' then Denormalizer will not match it with 'EXP' and finding no values that match, it will put the value to null.

    I've used MSSQL (2008) and MSSQL Express (2012) as a direct source to Denormalizer without any issue... So I doubt it's a bug.

  9. #9

    Default

    You sir, are a genius. Can't believe that thought escaped me. Stupid field type of char was padding blank spaces causing the issue - it is fixed now. Thanks so much! I've closed the JIRA ticket.

Tags for this Thread

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •  
Privacy Policy | Legal Notices | Safe Harbor Privacy Policy

Copyright © 2005 - 2019 Hitachi Vantara Corporation. All Rights Reserved.