Hitachi Vantara Pentaho Community Forums
Results 1 to 11 of 11

Thread: replace whitespace bug query

  1. #1

    Unhappy replace whitespace bug query

    Please assist. I believe this to be a bug in PDI 3.1.0 -RC1.

    looking to replace varying whitespaces within product string field reading from CSV import with target MySQL 5.1.
    have used select values, sort and unique items prior to this javascripting before moving to staging mysql table.

    This is my current script used...
    product.replace(/\s+/g, " ");

    I have tried as alternative to test and it works:
    var string = "find and replace whitespaces"'
    string.replace(/\s+/g, " ");

    I have also tried and it works....
    product.replace("the", "testtext")

    as process of elimination theretore using a text string from CVv import and search to replace based on regular expression DOES NOT WORK.

    Can someone closer to the project or with experience please assist me in logging this as a bug item or alternatively point me to the source of this PDI object in pentaho.

    Thanks in advance.
    Tom

  2. #2
    Join Date
    May 2006
    Posts
    4,882

    Default

    Don't double post, it won't help you...

    With compatibility mode off it does work as I expect it to

    Code:
    var b = replace(a, "\\s+", " ");
    What you try to do with replace does NOT work, as that replace() method does not use regular expressions. It's not a bug

    Regards,
    Sven
    Last edited by sboden; 09-06-2008 at 03:13 PM.

  3. #3
    DEinspanjer Guest

    Default

    Okay. So first off, PLEASE don't open three or four threads for the same problem. If you need support that desperately, purchase a Pentaho subscription and you will get fantastic support from them.

    Now then, when I replied to your first message I had tested the method in non-compatible mode and it worked fine. Since you kept complaining about the problem I decided to see if there was any problems trying to do the replace in compatible mode.

    I did find an interesting issue with that. In compatible mode, product.replace() is not calling the JS replace method that you think it is, it is calling a Java method replace() on the Kettle JS step's compatibility Value object. That method doesn't take a regex, it just takes plain string.

    Attached is a test case that demonstrates two different ways of doing what you are trying, one with compatible mode and one without.
    Attached Files Attached Files

  4. #4

    Default

    awesome thanks. desperation stuff, apolagies for the double post, thanks a mil for your solution.

  5. #5

    Default

    Hey there, apolagies for the double post. thanks a million for the fix. the trick was in the method not taking the regexp. working perfectly now.
    Regards
    Tom

  6. #6
    Join Date
    Oct 2008
    Posts
    2

    Default Similar Problem But Example Does not Seem To Work

    Hey guys,

    Im running into a similar problem with whitespace removal. I used the well written example provided earlier but to no avail I cannot seem to get the space removed off the fields. I have gone into the file to validate that the space is actually a space character. Processing seems to run correctly but my output still shows the field has not been altered.

    I have also tried ltrim and rtrim commands to try and clean up the data but these did not work as well.

    I have enclosed a sample of the data and my transformation which will need to have the first step changed to point at mydata.txt as the input source. Note that I am using a fixed field width to start the transformation so this may be where my problem is occuring.

    Any help or ideas would be greatly appreciated!

    Thanks!

    Jeff Mayrand
    Attached Files Attached Files

  7. #7
    Join Date
    May 2006
    Posts
    4,882

    Default

    Read the complete thread ... don't use junk.replace() but just replace(). The replace you're using doesn't support regexes.

    Regards,
    Sven

  8. #8
    Join Date
    Oct 2008
    Posts
    2

    Default Worked perfect

    Thanks Sven that did it.

  9. #9
    Join Date
    Apr 2010
    Posts
    16

    Default

    When you give examples, this helps out SO much!!!! I am finding myself looking for examples to step through. I learn more by this then I have through my studies...

    With that being said, is there a repository of examples that I can download and walk through? So I can put them in my own example library?

  10. #10
    Join Date
    Apr 2008
    Posts
    4,696

  11. #11
    Join Date
    Apr 2010
    Posts
    16

    Default

    I have the book and I HIGHLY recommend it to those who are new to Kettle!

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •  
Privacy Policy | Legal Notices | Safe Harbor Privacy Policy

Copyright © 2005 - 2019 Hitachi Vantara Corporation. All Rights Reserved.