Hitachi Vantara Pentaho Community Forums
Results 1 to 4 of 4

Thread: kettle does not see one value in filter

  1. #1
    Join Date
    Nov 2015

    Default kettle does not see one value in filter

    hi All!

    that's my 1st post so, nice to be here

    I am currently giving a try to the tool,

    and facing a problem,

    when I load from CSV file one of Filters cannot see a value 'GLK3',

    other 4chars in other filter are treated fine,

    only this one is givin'g me pain in the ... neck,

    Also what is very strange is that counts don't fit,

    count from DB vs count in spoon giving different number (spoon is lower)

    like part of file is missing but I can still see some values in total commander CSV file preview,

    GLK3s are there!

    What the.... strange thing...

    Would be very glad to see someone replying


    PS. file weights ~3,1 GB
    Last edited by 38snebob; 11-18-2015 at 08:48 PM. Reason: forgot to mention

  2. #2
    Join Date
    Apr 2008


    Difficult to know without looking at the data itself.
    Which filter are you talking about? Is Filter Rows step?

    How do you count your rows in the db?
    Do you export that data into a csv file and then use PDI to read the file?
    You can always use a Table input to read file from your db.
    -- Mick --

  3. #3
    Join Date
    Nov 2015


    Hi Mick,

    I think I know what might be going on,

    the system is 32 bit,

    so files should not be bigger than 2GB ?

    I had a quick test, and split files deeper into 'Year' also,

    and counts fit!

    The source system for me is old SAP,

    i hit count there directly on DB tab.

    then I copy files over net to local PC,

    and run transformations locally.

    And the filter is simple 'Filter rows' from 'Flow' submenu.

    Last edited by 38snebob; 11-21-2015 at 12:56 PM.

  4. #4
    Join Date
    Apr 2008


    Watch out with filtering on text.

    "GLK3" is not the same as " GLK3" which is different from "GLK3 "
    I would recommend turning trim on to setting "Both" in your Text File Input. Also make sure that you have Lazy Conversion off.

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
Privacy Policy | Legal Notices | Safe Harbor Privacy Policy

Copyright © 2005 - 2019 Hitachi Vantara Corporation. All Rights Reserved.