Hitachi Vantara Pentaho Community Forums
Results 1 to 8 of 8

Thread: CSV Text File Input

  1. #1

    Default CSV Text File Input

    I'm having some difficulty with CSV Text File Input. It is keeping the surrounding quotes from the input file when it passes the row fields. And eventually these end up in the database. How can I get it to stop doing this? Right now I have the Delimited set to comma and the Enclosure set to double-quote mark which is what is present in the input file.

    Gerry

  2. #2
    Join Date
    Nov 1999
    Posts
    9,729

    Default

    You know the drill Gerry ;-)
    • Versions
    • Sample file
    • Sample transformation
    • bug report
    Thanks a lot for the ones you made so far!

    Matt

  3. #3

    Default

    Ok, here it is. I was hoping it was something simple, but no....

    And I've tried changing the Enclosure setting and it doesn't seem to make a difference.

    Gerry
    Attached Files Attached Files

  4. #4

    Default

    It looks like this bug got fixed in RC2. How can I get this fix for my RC1? Eg: patch? I really don't want to get into svn setup right now.

    Gerry

  5. #5
    Join Date
    Nov 1999
    Posts
    9,729

    Default

    Hi Gerry,

    download these 2 files:

    http://www.kettle.be/dloads/kettle-engine-3.0.jar
    http://www.kettle.be/dloads/kettle-ui-swt-3.0.jar

    Replace the files in the lib/ directory of your RC1/RC2 distribution.
    I'm updating these files once in a while, like right now.

    All the best,

    Matt

  6. #6

    Default

    Ok, it seems to be doing better with the Enclosure chars but now I see another problem.

    In one of the CSV files there are two values like this:
    "SOME TEXT (AN"," EXPLANATION)"

    The Preview gets an exception when it hits this line:
    Code:
    2007/10/18 18:02:17 - Trans - Transformation is pre-loaded.
    2007/10/18 18:02:17 - Trans - This is not a replay transformation
    2007/10/18 18:02:17 - Trans - I found 2 different steps to launch.
    2007/10/18 18:02:17 - Trans - Allocating rowsets...
    2007/10/18 18:02:17 - Trans -  Allocating rowsets for step 0 --> CSV infile input
    2007/10/18 18:02:17 - Trans -   prevcopies = 1, nextcopies=1
    2007/10/18 18:02:17 - Trans - Transformation allocated new rowset [CSV infile input.0 - dummy.0]
    2007/10/18 18:02:17 - Trans -  Allocated 1 rowsets for step 0 --> CSV infile input  
    2007/10/18 18:02:17 - Trans -  Allocating rowsets for step 1 --> dummy
    2007/10/18 18:02:17 - Trans -  Allocated 1 rowsets for step 1 --> dummy  
    2007/10/18 18:02:17 - Trans - Allocating Steps & StepData...
    2007/10/18 18:02:17 - Trans -  Transformation is about to allocate step [CSV infile input] of type [CsvInput]
    2007/10/18 18:02:17 - CSV infile input.0 - distribution activated
    2007/10/18 18:02:17 - CSV infile input.0 - Starting allocation of buffers & new threads...
    2007/10/18 18:02:17 - CSV infile input.0 - Step info: nrinput=0 nroutput=1
    2007/10/18 18:02:17 - CSV infile input.0 - output rel. is  1:1
    2007/10/18 18:02:17 - CSV infile input.0 - Found output rowset [CSV infile input.0 - dummy.0]
    2007/10/18 18:02:17 - CSV infile input.0 - Finished dispatching
    2007/10/18 18:02:17 - Trans -  Transformation has allocated a new step: [CSV infile input].0
    2007/10/18 18:02:17 - Trans -  Transformation is about to allocate step [dummy] of type [Dummy]
    2007/10/18 18:02:17 - dummy.0 - distribution activated
    2007/10/18 18:02:17 - dummy.0 - Starting allocation of buffers & new threads...
    2007/10/18 18:02:17 - dummy.0 - Step info: nrinput=1 nroutput=0
    2007/10/18 18:02:18 - dummy.0 - Got previous step from [dummy] #0 --> CSV infile input
    2007/10/18 18:02:18 - dummy.0 - input rel is 1:1
    2007/10/18 18:02:18 - dummy.0 - Found input rowset [CSV infile input.0 - dummy.0]
    2007/10/18 18:02:18 - dummy.0 - Finished dispatching
    2007/10/18 18:02:18 - Trans -  Transformation has allocated a new step: [dummy].0
    2007/10/18 18:02:18 - Trans - This transformation can be replayed with replay date: 2007/10/18 18:02:18
    2007/10/18 18:02:18 - Trans - Initialising 2 steps...
    2007/10/18 18:02:18 - dummy.0 - Running on slave server #0/1.
    2007/10/18 18:02:18 - CSV infile input.0 - Running on slave server #0/1.
    2007/10/18 18:02:18 - Trans - Step [CSV infile input.0] initialized flawlessly.
    2007/10/18 18:02:18 - Trans - Step [dummy.0] initialized flawlessly.
    2007/10/18 18:02:18 - CSV infile input.0 - Starting to run...
    2007/10/18 18:02:18 - Trans - Transformation has allocated 2 threads and 1 rowsets.
    2007/10/18 18:02:18 - dummy.0 - Starting to run...
    2007/10/18 18:02:18 - CSV infile input.0 - ERROR (version 3.0.0-RC2, build 299020 from 2007/10/18 09:15:15) : Unexpected error : 
    2007/10/18 18:02:18 - CSV infile input.0 - ERROR (version 3.0.0-RC2, build 299020 from 2007/10/18 09:15:15) : org.pentaho.di.core.exception.KettleFileException: 
    2007/10/18 18:02:18 - CSV infile input.0 - ERROR (version 3.0.0-RC2, build 299020 from 2007/10/18 09:15:15) : 
    2007/10/18 18:02:18 - CSV infile input.0 - ERROR (version 3.0.0-RC2, build 299020 from 2007/10/18 09:15:15) : Exception reading line using NIO: java.lang.NegativeArraySizeException
    2007/10/18 18:02:18 - CSV infile input.0 - ERROR (version 3.0.0-RC2, build 299020 from 2007/10/18 09:15:15) : 
    2007/10/18 18:02:18 - CSV infile input.0 - ERROR (version 3.0.0-RC2, build 299020 from 2007/10/18 09:15:15) : 
    2007/10/18 18:02:18 - CSV infile input.0 - ERROR (version 3.0.0-RC2, build 299020 from 2007/10/18 09:15:15) :     at org.pentaho.di.trans.steps.csvinput.CsvInput.readOneRow(CsvInput.java:282)
    2007/10/18 18:02:18 - CSV infile input.0 - ERROR (version 3.0.0-RC2, build 299020 from 2007/10/18 09:15:15) :     at org.pentaho.di.trans.steps.csvinput.CsvInput.processRow(CsvInput.java:83)
    2007/10/18 18:02:18 - CSV infile input.0 - ERROR (version 3.0.0-RC2, build 299020 from 2007/10/18 09:15:15) :     at org.pentaho.di.trans.steps.csvinput.CsvInput.run(CsvInput.java:349)
    2007/10/18 18:02:18 - CSV infile input.0 - Finished processing (I=30, O=0, R=0, W=29, U=0, E=1)
    2007/10/18 18:02:18 - dummy.0 - Finished processing (I=0, O=0, R=0, W=0, U=0, E=0)
    So does CSV File Input not ESCAPE special characters?

    Gerry

  7. #7

    Default

    Opened Jira tracker PDI-332 for this issue.

    Gerry

  8. #8
    Join Date
    Nov 1999
    Posts
    9,729

    Default

    So does CSV File Input not ESCAPE special characters?
    Nope, and I have no plans in that direction either.

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •  
Privacy Policy | Legal Notices | Safe Harbor Privacy Policy

Copyright © 2005 - 2019 Hitachi Vantara Corporation. All Rights Reserved.