Hitachi Vantara Pentaho Community Forums
Results 1 to 2 of 2

Thread: Trying to delete source file from a job - failing miserably!

  1. #1
    Join Date
    Nov 2017

    Default Trying to delete source file from a job - failing miserably!

    Hi folks,

    Hoping someone can help, im trying to run a transformation, which pulls in a file from a particular folder. What im trying to achieve is that once the transformation has completed, the original file from folder is deleted, so when next file gets dropped in there it wont process that, and likewise wont keep processing the same file over and over.

    I have tried a few steps and did the token google but havent had much joy...

    Ive tried including the block at this step until complete step in the transformation and i've tried using a job to pump the info into another transformation so the other transformation can handle the deletion of the original, but all i can seem to do much. As i'm going to be taking potentially multiple files through to be processed i only want those which have been procesed to be deleted so i think i'm making it more complex than it needs to be.

    I can get the file to be deleted in the transformation step but i think when it deletes the source file it breaks the transformation, i get constant errors coming up of
    2017/11/07 20:54:44 - Process files.0 - ERROR (version, build 1 from 2017-05-16 17.18.02 by buildguy) : We can not find source file [C:\PDI\Data\Input\input.xml]
    2017/11/07 20:54:44 - Process files.0 - ERROR (version, build 1 from 2017-05-16 17.18.02 by buildguy) : Because of an error, this step can't continue:
    2017/11/07 20:54:44 - Process files.0 - We can not find source file [C:\PDI\Data\Input\input.xml]

    but it has successfully deleted.

    I assume because its running everything in one go its deleted it before its completed using it and keeps trying to delete it after its been deleted as the jobs running.......

    Many thanks in advance for whats probably just my semi understanding of what its doing, but complete failure to come up with a way to make it work Name:  Workflow.jpg
Views: 145
Size:  12.7 KB Name:  job.jpg
Views: 148
Size:  9.2 KB

  2. #2
    Join Date
    Jun 2012


    Even though a screenshot technically is an image, it's not a good idea to insert it as an image here.
    The forum software expects an image to be still informative after lossy compression, which holds for a meme but not for a screen.
    Either you keep a screenshot to roughly VGA dimensions or you attach it as a zipped file using advanced editor mode.

    As to what's going wrong, I guess you just mix control flow with data flow - a bad thing to do.
    Text-File-Output doesn't swallow the rows coming in, it just sends them downstream for further processing.
    If your next step is Process-Files and there are multiple rows with the same filename to be deleted, the first row will have the file deleted and the step will fail on every row thereafter.
    So long, and thanks for all the fish.

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
Privacy Policy | Legal Notices | Safe Harbor Privacy Policy

Copyright © 2005 - 2019 Hitachi Vantara Corporation. All Rights Reserved.