Hitachi Vantara Pentaho Community Forums
Results 1 to 4 of 4

Thread: Text Files are Becoming Over Sized

  1. #1
    Join Date
    Mar 2008
    Posts
    8

    Default Text Files are Becoming Over Sized

    I export a table from Oracle. The table should be about 310 mb, and when I use Toad to export it is about that size. With Kettle it becomes 34 GB. The table has a lot of varchar2(4000) fields, although the actual data length is much smaller.

    I have set up trim in the query and in the text step. However it makes no difference.

    In an effort to isolate the issue, I created a query to output 9999 with varchar2(4000) datatype, but which were null. That query produced a 39 MB file, but the size should have been about 30-40 k.

    This is the query I used:
    select trim(cast(null as varchar2(4000) ) )as null_test from all_objects where rownum <10000

  2. #2
    DEinspanjer Guest

    Default

    So when you look at the text file, is it padded with spaces?
    Can you post the transformation?
    What version of Kettle are you using?

  3. #3
    Join Date
    Mar 2008
    Posts
    8

    Default

    3.0.4

    I found out that you need to click Minimal Width, to get rid of the space padding. It seems like not padding should be the default, but anyway it is working.

    Thanks.

  4. #4
    DEinspanjer Guest

    Default

    Alternatively, if you don't need any formatting at all, there is an option on the Content tab to use "fast output (no formatting)". I don't remember if this feature was in 3.0.4 though.

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •  
Privacy Policy | Legal Notices | Safe Harbor Privacy Policy

Copyright © 2005 - 2019 Hitachi Vantara Corporation. All Rights Reserved.