Hitachi Vantara Pentaho Community Forums
Results 1 to 6 of 6

Thread: Question Marks in Variable Values

  1. #1

    Default Question Marks in Variable Values

    Hello,

    I'm trying to process several rows of data from each of several url's, and am running into a problem where the variable chokes if it's value contains a question mark.

    I read a url from a database table where the ones I'm interested in are stored.
    I set a variable to the string that is the url.
    I use that variable in the next transformation in the Text File Input step.
    If the value of that variable does NOT contain a question mark, then it works fine.
    If the value of the variable DOES contain a question mark, then it chokes and uses the default value for the variable.
    Several of my ur's contain a question mark in the format: http://url?id1=01&id2=02
    I have tried setting the url in the database table to this: http://url$[3F]id1=01&id2=02
    in order to use the hex value of the question mark when setting the variable value, but that did not help.

    Any ideas how to use a question mark in the value of a variable?

    I am using pdi-ce-4.2.1-stable.

  2. #2

    Default

    I also tried splitting the url into a variable called part1, consisting of the string up to, but not including, the question mark,
    and a second variable called part2, consisting of the string after the question mark. And then in the file name cell of the Text File Input step having this:
    ${part1}$[3F]${part2}
    but that did not work either...

  3. #3
    Join Date
    Jun 2007
    Posts
    260

  4. #4

    Default

    Thanks for your response. It did not solve the problem, but by going down that road it allowed me to zero in on the problem. Here is what I did:

    Try 1 Used encodeURI() to encode the raw url to the encoded version, so it changes the / to %3A and the ? to %3F etc...
    As before, I put the url (now encoded) into a variable, and out that variable in the File/Directory cell of the Text File Input step
    Result: Now it DID change the variable for each interation of the Job that runs with a different url. That is a positive step forward, since it would not set the variable correctly with the ? in the url.
    HOWEVER, it DID NOT find the web page. It could not understand the encoding.
    So, I played with the encoding by removing the encoding of characters. The problem is the question mark.

    Try 2 I set the raw url to SUBSTITUTE([url_raw]; "?"; "%3F")
    Result: Now it DID change the variable correctly for each iteration of the Job
    and it DID find the base page (everything up to the question mark
    HOWEVER, it failed to find the page I need and instead gave me the default error page for that web site

    Try 3 I split my url into 3 parts. Part 1 is everything up to and not including the ?; Part 2 is everything after the ?; Part 3 is just the ?
    I set the File/Directory entry in the Text File Input step to be ${url_pt1}${qmark}${url_pt2}
    When I set the variable qmark to contain the string "?" it never sets the variable correctly and always tries to prcoess the default value (it chokes on it)
    When I set the variable qmark to contain the string "%3F" it does set the variable correctly, but it passes the literal %3F in the url so it does not find the page. Same if set to $[3f]

    So, I guess I have been able to try a few more things and I see the problem a little more clearly, but I am still have the same basic question:
    How can I pass a question mark as part of a url into the Text File Input step?

  5. #5
    Join Date
    Jun 2007
    Posts
    260

    Default

    I'm guessing here, but extend your splitting solution from this:
    ${part1}$[3F]${part2}
    to this
    ${part1}?{part2}

  6. #6

    Default

    I tried your suggestion but it made no difference. I have create a jira case for it:
    http://jira.pentaho.com/browse/PDI-7031

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •  
Privacy Policy | Legal Notices | Safe Harbor Privacy Policy

Copyright © 2005 - 2019 Hitachi Vantara Corporation. All Rights Reserved.