Hitachi Vantara Pentaho Community Forums
Results 1 to 4 of 4

Thread: Reading only one row from multiple text files

  1. #1

    Question Reading only one row from multiple text files

    I'm reading a lot of text files in a folder with the text file input.

    I'm using the wildcard ".*txt" to read all of them.

    I want to know how to read only one row of each file (for example: row 3 of each one).

    How do I do that?

  2. #2
    Join Date
    Jun 2012
    Posts
    5,534

    Default

    Make your transformation read only a single file and provide the filename as a parameter.
    In a job, use a transformation to detect files using Get-File-Names and pipe them out with Copy-Rows-To-Result.
    Then, add your transformation, adjust settings using options "Copy previous results to parameters" and "Execute for every input row".
    Enjoy.
    So long, and thanks for all the fish.

  3. #3
    Join Date
    Aug 2016
    Posts
    290

    Default

    You could do a sample rows step, specifying row nr 3. But that wouldn't close the stream so you'd still have to read the entire file. The only way I've found to close the stream (to finish faster) is to use User Defined Java Class where you can close a stream any way you want. You'd still have to read the lines up to nr 3 though.

  4. #4
    Join Date
    Jun 2012
    Posts
    5,534

    Default

    Reading a certain row from a text file is best done by specifying the total number of rows to read and the number of rows to skip in Text-File-Input.
    A lot can be done with scripting, besides spoiling a true dataflow design by unnecessarily introducing procedural code.

Tags for this Thread

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •  
Privacy Policy | Legal Notices | Safe Harbor Privacy Policy

Copyright © 2005 - 2019 Hitachi Vantara Corporation. All Rights Reserved.