Hitachi Vantara Pentaho Community Forums
Results 1 to 7 of 7

Thread: Can someone please look at PDI-137?

  1. #1

    Default Can someone please look at PDI-137?

    I't biting me over and over again.

    I think it was prematurely filed as 'Trivial' as well, as I now found out that the rows that are causing the preview to act weird sometimes cause the sort step to break the stream. So, this has now moved from "preview showing stuff wrongly" to "sort step does't work with certain input" (and besides, a preview not working correctly should be considered as serious, since it is the main tool developers work with).

    I was able to give an example for the first case (preview only), but not for the second case (preview and sort). I have attached it in the forum.

    I know it is difficult to find bugs without an example transformation, but maybe the first bug may lead to conclusions about the second.

    Thanks,
    Tobias

  2. #2
    Join Date
    Nov 1999
    Posts
    9,729

    Default

    This does not happen in 3.0.0-M2.
    That's as far as I read.

    Sven displayed great emotional strength and intelligence by asking:

    Try to add a transformation showing your problem. It helps a lot in debugging.
    If I would have to guess (and you leave us no choice), I would guess you are mixing rows with different data types.
    t_start probably contains a mixture of Integers and Numbers.

    This is not allowed in any version. Perhaps 3.0 behaves more consistent in the data conversion department, but I think that's beside the point here.

    HTH,

    Matt

  3. #3

    Default

    Quote Originally Posted by MattCasters View Post
    That's as far as I read.
    A strange thing to do, stopping to read in the middle of a bug report.

    Quote Originally Posted by MattCasters View Post
    If I would have to guess (and you leave us no choice), I would guess you are mixing rows with different data types.
    t_start probably contains a mixture of Integers and Numbers.
    I didn't send an example transformation because I wasn't able to build one that doesn't read data from my database. I admit however, that maybe even if you cannot connect to my database, an example might be helpful.

    I managed to strip my transformation down quite a bit, and to find a specific step that, when removed, does influence the result. If that specific step is the root of the problem, I don't know.

    I sent the transformation to Matts pentaho email address, since I don't want to have it show up here. Compare the two output files, the bad values show up at the bottom of after.txt.

    In any way, I fail to see where in my transformation I am mixing data types. Maybe you can spot it.

    Thanks,
    Tobias

  4. #4

    Default

    For reference:

    Matt found out that the problem was indeed with mixing types, but not of the obvious kind, because it was caused by a JS/Rhino problem:

    I had a JS script step like this

    if (somecondition)
    some_integer.setValue(another_integer)

    Notably, both Integers had the exact same type. However, setValue() silently converts the data to Number, leaving me with some rows that contained Integers (where the condition did evaluate to false), and some values as Numbers (where it evaluated to true).

    This issue is worked around in 3.0, but is not fixable in 2.5.x.
    The workaround for 2.5.x is not to use setValue() at all, or to add a redundant else block like so:

    if (somecondition)
    some_integer.setValue(another_integer)
    else
    some_integer.setValue(some_integer)

    Note: the original problem with the preview changing when one clocks on the column header to sort the column persists, but is now very minor. It is demonstrated by the attachment I gave in forum thread I referenced in the Jira task. It might not be worth fixing it as all, since it also does not appear in 3.0.0.

  5. #5

    Default

    Seeing how Kettle Ktr files contain no passwords, usernames, or other confidential information as far as I'm aware, I don't see why you couldn't have posted it here for us all to have a nosey at and see what point your were talking about in more detail. I'm sure it would help debug stuff quicker.
    Anyway, good stuff
    Tom

  6. #6

    Default

    It contains a lot of confidential information, most notably names, positions and salaries of employees from my workplace.

    But you're right, with a little bit of effort, I could have generated a transformation stripped of all confidential data. In fact, I was about to do it but Matt was quicker in finding the bug.

    I' always post detailed explanations of everything once the problem is solved. This has to be enough whenever I cannot disclose transformations or test data.

    Thanks,
    Tobias

  7. #7
    Join Date
    Nov 1999
    Posts
    9,729

    Default

    I for one was glad to see version 3.0 behave better here as well.

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •  
Privacy Policy | Legal Notices | Safe Harbor Privacy Policy

Copyright © 2005 - 2019 Hitachi Vantara Corporation. All Rights Reserved.