Hitachi Vantara Pentaho Community Forums
Results 1 to 5 of 5

Thread: new text input steps - question

  1. #1

    Default new text input steps - question

    Hi, I am enjoying the speed increases from lazy conversion in the new text input steps, but miss the functionality included in the original text input step. For example, reading in compressed files, error handling, etc...

    Are there plans to merge the 'lazy conversion' features to the original text input step, or add the other functionality into the new steps?

    Thanks,
    Drew
    -Drew

  2. #2
    Join Date
    May 2006
    Posts
    4,882

    Default

    I don't know all but I doubt there will be a "full merge" of them... The newer steps were created to be more speedy and less feature-full/cluttered.

    Lazy conversion maybe.

    Regards,
    Sven
    Last edited by sboden; 12-11-2007 at 06:17 PM.

  3. #3
    Join Date
    Nov 1999
    Posts
    9,729

    Default

    Hi Drew, we will add a number of features, but only the things that are strictly needed and are not getting in the way of performance.
    For example, reading compressed files is something we won't do since it is not possible to use Non-blocking IO (NIO) with this. VFS is out too.
    Only local files can be read with this step.

    In short: since we have a step that works (Text File Input) for the obscure cases, I'm not sure it's a good idea to port each and every feature.
    The algorithm is nice and clean and fast now, so I would hesitate to mess too much with it.

    All the best,

    Matt

  4. #4

    Default

    Thanks Sven & Matt. Appreciate your responses and that makes sense.

    Also, if I got to vote, I'd request the following (minor) features be added to the new steps for convenience sake (with the assmuption that it wouldn't impact performance):

    - limit rows
    - regex/wildcard
    - show file content
    - insert tab character

    Thanks again.

    -Drew
    -Drew

  5. #5
    Join Date
    May 2006
    Posts
    4,882

    Default

    Make JIRA's for them... some do get implemented

    Regards,
    Sven

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •  
Privacy Policy | Legal Notices | Safe Harbor Privacy Policy

Copyright © 2005 - 2019 Hitachi Vantara Corporation. All Rights Reserved.