Hitachi Vantara Pentaho Community Forums
Results 1 to 4 of 4

Thread: Rearranging the order of output fields truncates some fields

  1. #1
    Join Date
    Nov 2011

    Default Rearranging the order of output fields truncates some fields


    I'm using Kettle -Spoon Stable Release - 4.2.

    I have two steps in my transformation. One reads in an input file (CSV with , delimiters).
    The second merely outputs the same fields to a text file (CSV with , delimiter).

    This works fine if I don't change the order of the fields.

    When I change the order of the fields to output (in the fields tab of the "Text File Output" step, I move a field to the top using the Ctrl-Up sequence), that field that I moved gets truncated at the carriage returns. This does not happen when I output in the order it arrived.

    I also tried using the "Select & Alter" tab in a "Select Values" step to show all fields and change the order, but that also didn't work.

    The same behavior is also happening when I try to use the "Remove" tab in the "Select Values" step to not show some fields on output. The other fields it does show are truncated on CR.

    I must be doing a basic step incorrectly.

    -Thanks in Advance

  2. #2
    Join Date
    Sep 2009


    Hi Maggie,

    please add a sample transformation and small data file that shows the issue. You're much more likely to get responses then


  3. #3
    Join Date
    Nov 2011

    Default Posting my transformation file and my input and output files.

    Since I posted this the first time, I've learned that it isn't the rearranging that causes the truncation.
    I've reduced the problem to learn that if I do nothing, the fields are still getting truncated.
    My transformation just reads in the xml file and outputs it to a file.

    I've attached:

    the transformation file I'm using: maggie_transformation.ktr
    the xml input file: cq_out_style.xml
    the resulting output file: out_test.txt

    The "problem_summary" and "defect_journal" fields are getting truncated because they contain carriage returns
    (&#13) in the output file.

    Attached Files Attached Files
    Last edited by mstearns; 11-10-2011 at 03:30 PM.

  4. #4
    Join Date
    May 2011


    Hi maggie,

    I had the same problem. I did a workaround for the issue. Try this approach and see if that could help you. Try downloading the csvjdbc jar and add it to the JDBC filder of libext in the pentaho installation directory. What this API does is basically treat your csv file as a table. So then i used the table input step to select all fields from the transformation. Using a select values step you can select the fields that you require. You need to use the generic database as setting in the table input connection settings. I ve attached the screen shot of the connection settings tab. The last part of the connection URL is the location of your folder where the CSV files are present.

    Name:  connection_settings_for_csvJDBC.jpg
Views: 54
Size:  18.5 KB

    Last edited by sdas; 11-11-2011 at 07:30 PM.

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
Privacy Policy | Legal Notices | Safe Harbor Privacy Policy

Copyright © 2005 - 2019 Hitachi Vantara Corporation. All Rights Reserved.