Hitachi Vantara Pentaho Community Forums
Results 1 to 11 of 11

Thread: CR and LF substituted in 5.3.0

  1. #1

    Default CR and LF substituted in 5.3.0

    Hi all,

    I created a simple transformation to get data from database and create a xml file based on those results with parameter Encoding=UTF-8. In previous version I used (Pentaho Data Integration 4.4.0) the output was charset=utf-8 but in new version I implemented (Pentaho Data Integration 5.3.0) the file is charset=us-ascii

    OS: Linux CentOS

    Thank you for any help.
    Magdalena

  2. #2

    Default

    Hi all,

    when I tried to change encoding, it works, however it doesn't work for UTF-8. The file contains header <?xml version="1.0" encoding="UTF-8"?> but the content itself is in us-ascii.

    In case, I am running the transformation in Spoon and using preview for XML output step, data are displayed correctly.

    Does anybody have an idea?

    Thank you
    Magdalena

  3. #3
    Join Date
    Apr 2008
    Posts
    4,696

    Default

    Post your transformation (ktr file, NOT screenshots), and maybe we can help sort it out.
    Otherwise, we'll just be shooting in the dark

  4. #4

    Default

    Unfortunately, I am not able to upload an attachment. There is some kind of php error
    "Unable to proceed with save while $errors array is not empty in class vB_DataManager_AttachmentFiledata in [path]/includes/class_dm.php on line 849"

    Thank you for any help.

    Magdalena
    Last edited by gutlez; 10-27-2015 at 11:01 AM. Reason: Removed link pointing to unverifiable host

  5. #5

    Default

    Is there a list of "verified hosts", so I can provide the link?

    Thanks a lot
    Magdalena

  6. #6
    Join Date
    Apr 2008
    Posts
    4,696

    Default

    Honestly, I've never had an issue with uploading here.
    Some people post links to dropbox, some post links to other sites.

    If the link looks spammy (eg. a short link similar to http//link.to ) the mods will remove it.

  7. #7

    Default

    I uploaded the file to Dropbox
    https://www.dropbox.com/s/7kmlvq1yhl...ation.ktr?dl=0

    I tried the original transformation in new PDI 6.0.0 and the result is the same as in PDI 5.3.0

    Could you please help me?

    Thank you
    Magdalena
    Last edited by minimalne4; 11-09-2015 at 05:28 AM.

  8. #8

    Default

    Could anybody please help me?

    Thank you
    Magdalena

  9. #9
    Join Date
    Jun 2012
    Posts
    5,534

    Default XML Output encoding

    Kettle honors your chosen encoding by emitting an XML declaration.
    AFAICS the XML document itself is written with the encoding of your JVM (file.encoding) which happens to be your system encoding by default.
    Try to start the JVM with -Dfile.encoding=UTF-8 for a change.
    So long, and thanks for all the fish.

  10. #10

    Default

    Dear marabu,

    unfortunately it didn't help. I added the parameted -Dfile.encoding=UTF-8 to spoon.sh file and it still generates xml file with encoding application/xml; charset=us-ascii

    Default system encoding is
    [root@my_server]# echo $LANG
    en_US.UTF-8

    I also added a "write to log" step into my transformation and logged ${file.encoding} variable. There is a value UTF-8 for this variable in the log file.

    Magdalena
    Last edited by minimalne4; 11-13-2015 at 09:29 AM.

  11. #11
    Join Date
    Jun 2012
    Posts
    5,534

    Default

    You know that variable file.encoding is set when the JVM is started.
    Did you terminate Spoon and start it anew?
    Cause I just tried my own medicine and it worked.
    So long, and thanks for all the fish.

Tags for this Thread

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •  
Privacy Policy | Legal Notices | Safe Harbor Privacy Policy

Copyright © 2005 - 2019 Hitachi Vantara Corporation. All Rights Reserved.