Hitachi Vantara Pentaho Community Forums
Results 1 to 8 of 8

Thread: Beginner - could use some guidance

  1. #1
    Join Date
    May 2007
    Posts
    128

    Default Beginner - could use some guidance

    Ok - so through reading here and the manual (by far pentaho's best documentation to date ) I was able to preform a simple query run against database1 and insert the results into database2 on different servers.

    BIG step for us here!

    Anyway, my first real task I could use some guidance on how to go about it - what to make a transformation vs job, etc.

    Here is the task (this is on a windows box - CSV files are loaded into a folder by another process):

    1. Transform files so that only one column of data remains (either A or B - will be transformation specific)
    2. Populate second column (B) with a string of text for each row (same string for every row)
    3. Save processed file in a different folder and delete original file
    4. Before or After step 3, insert rows into a database
    That is all! I know step 4 from the other transformation I did, but am not 100% sure on what steps to use for the rest of the process and would appreciate anyone's help in this learning process.

    Thanks!

  2. #2
    Join Date
    Jan 2008
    Posts
    22

    Default One method

    A Transform/Calculator or Transform/Select step could probably be used to choose between A and B in the source, depending on your criteria...

    An Input/Generate Rows could generate your new 'B' column...

  3. #3
    Join Date
    May 2007
    Posts
    128

    Default

    On the CSV input - can I have it work on any file in the folder or does it have to have a specific name? The files will have a different name each day (date is in the name).


    Also, on the Input/Generate rows, I also need to insert the current timestamp in mysql format (yyyy-mm-dd hh:mm:ss) - any input on how to accomplish this?

  4. #4
    Join Date
    May 2007
    Posts
    128

    Default

    Actually it appears as though generate rows won't work - because it [obviously] generates rows, not columns.

    Further help is much appreciated.


    edit: AH! Add Constants is what I needed - now how do I add a date field w/ the current datestamp within the add constants?
    Last edited by elgabito; 04-10-2008 at 01:30 PM.

  5. #5
    Join Date
    May 2006
    Posts
    4,882

    Default

    Use "Get System Info" step.

    Regards,
    Sven

  6. #6
    Join Date
    May 2007
    Posts
    128

    Default

    Quote Originally Posted by sboden View Post
    Use "Get System Info" step.

    Regards,
    Sven
    nvm - got it
    Last edited by elgabito; 04-10-2008 at 04:33 PM.

  7. #7
    Join Date
    May 2007
    Posts
    128

    Default

    Quote Originally Posted by sboden View Post
    Use "Get System Info" step.

    Regards,
    Sven

    Thanks! It wasn't working earlier - but now it is working perfectly!

  8. #8
    Join Date
    May 2007
    Posts
    128

    Default

    Ok - almost got it. Last question (hopefully).

    When I get to the delete files step - it gives me the following errors:

    Code:
    2008/04/10 16:19:03 - Delete original files - Processing argument [C:\Documents and Settings\k.b\My Documents\Connection Device Disconnects\Alltel Unprocessed].. wildcard [.*.csv] ?
    2008/04/10 16:19:03 - Delete original files - Processing folder [C:\Documents and Settings\k.b\My Documents\Connection Device Disconnects\Alltel Unprocessed]
    2008/04/10 16:19:03 - org.pentaho.di.job.entries.deletefiles.JobEntryDeleteFiles$TextFileSelector@ae2d66 - Deleting file [file:///C:/Documents and Settings/k.b/My Documents/Connection Device Disconnects/Alltel Unprocessed/Alltel_032508.csv] ...
    2008/04/10 16:19:03 - org.pentaho.di.job.entries.deletefiles.JobEntryDeleteFiles$TextFileSelector@ae2d66 - Deleting file [file:///C:/Documents and Settings/k.b/My Documents/Connection Device Disconnects/Alltel Unprocessed/alltel_040108.csv] ...
    2008/04/10 16:19:03 - Delete original files - ERROR (version 3.0.2, build 538 from 2008/02/06 13:13:19) : Could not process [C:\Documents and Settings\k.b\My Documents\Connection Device Disconnects\Alltel Unprocessed], exception: Could not delete "file:///C:/Documents and Settings/k.b/My Documents/Connection Device Disconnects/Alltel Unprocessed/Alltel_032508.csv".
    I am running Windows XP Pro - I wouldn't think it would be a permissions issue...?


    edit: Looks very similar to this:

    http://forums.pentaho.org/showthread.php?t=53862

    I can't tell if it was ever resolved.
    Last edited by elgabito; 04-10-2008 at 04:41 PM.

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •  
Privacy Policy | Legal Notices | Safe Harbor Privacy Policy

Copyright © 2005 - 2019 Hitachi Vantara Corporation. All Rights Reserved.