Hitachi Vantara Pentaho Community Forums
Results 1 to 5 of 5

Thread: Pentaho Spoon does not produce table output with 3-bytes utf8 characters correctly in

  1. #1
    Join Date
    Jul 2015
    Posts
    1

    Default Pentaho Spoon does not produce table output with 3-bytes utf8 characters correctly in

    I use one of tools in Pentaho Data Integration, called Spoon, to extract and load into new table on Amazon Redshift. The problem is Spoon cannot build the result table correctly with showing up data in database as question mark in each foreign language character, where it was inserted properly by another tool named SQL Workbench/J.


    I loaded same data with exactly same way to MySql, csv, excel output (which is one of features in Spoon to load data into many types of output data). It works well, showing up correct data in my local language.


    By googling about this question, I found that my language is using 3-bytes characters in UTF-8 which I'm not quite sure whether it's supported by Pentaho Spoon or not ? or anyone else also have this problem ?

    After testing by using sql command directly run in simple SQL editor, all letters and words has been added correctly. So it is a problem only when I do a transformation from one input, no matter what type of source, to table output.

    Name:  Untitled.jpg
Views: 658
Size:  43.3 KB

  2. #2

    Default

    I have the same issue Table Output step with Redshift db connection failing with error "String contains invalid or unsupported UTF8 codepoints. Bad UTF8 hex sequence a8".
    When I execute "insert into select from" command directly in Redshift - all working fine with no errors. So I assume Spoon is not processing rows correctly.

    Anybody have solution for that?

  3. #3
    Join Date
    Dec 2013
    Posts
    1

    Default

    Guys....within Table Output step...."Edit" your connection.....Go to "Options" on left panel, Add a parameter......."CHARSET" with value "UTF8".
    You'll be all set.

    You might run into "untranslatable characters" issue later on while loading data.......Let me know when you get there

  4. #4
    Join Date
    Jun 2016
    Posts
    181

    Default

    Quote Originally Posted by rohitv View Post
    Guys....within Table Output step...."Edit" your connection.....Go to "Options" on left panel, Add a parameter......."CHARSET" with value "UTF8".
    You'll be all set.

    You might run into "untranslatable characters" issue later on while loading data.......Let me know when you get there
    WRONG answer.
    The correct one is:


    • Make sure your MySQL table uses the UTF-8 encoding
    • Make sure that in the database connection details in Kettle following options are set:characterEncoding=utf8, characterSetResult=utf8,useUnicode=true.

  5. #5
    Join Date
    Sep 2016
    Posts
    8

    Default

    Quote Originally Posted by Gosforth View Post
    WRONG answer.
    The correct one is:


    • Make sure your MySQL table uses the UTF-8 encoding
    • Make sure that in the database connection details in Kettle following options are set:characterEncoding=utf8, characterSetResult=utf8,useUnicode=true.
    How do you do this? Where I can find this database connection details?

Tags for this Thread

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •  
Privacy Policy | Legal Notices | Safe Harbor Privacy Policy

Copyright © 2005 - 2019 Hitachi Vantara Corporation. All Rights Reserved.