Hitachi Vantara Pentaho Community Forums
Results 1 to 3 of 3

Thread: Create unique file name

  1. #1
    Join Date
    May 2007
    Posts
    3

    Default Create unique file name

    I have a job with two transformations and the last transformation create a output file and is marked as "Execute for every input row" in the job.

    The problem is that the output file is owerwritten every time the last transformation execute.
    How can I create an unique name for each output file?

    I have tried to pass a different value to an env-variable each time the transformation runs but I
    don't know if that is possible?

    Regards
    Magnus

  2. #2
    DEinspanjer Guest

    Default

    For my system I do what you mention with variables.

    job_master invokes:
    a trans to get the list of input files to operate on and passes that to result set
    a job named job_one_file for every row in result set

    job_one_file invokes:
    a trans that runs a bit of JS to create a variable to be used for the output file based on part of the name of the input file
    a trans that opens the input file passed in to it and writes the data to the output file named with the variable

    If you don't want to get that elaborate, you could also use the run date/time (both checkboxes in the Text File Output step).

    I've wondered in the past why Kettle didn't have a built in variable for batch_id or run_id or something but I never needed it enough to file a request for it. If you think that would be the best solution I encourage you to file a Jira.

  3. #3
    Join Date
    Apr 2008
    Posts
    3

    Default

    I agree with the timestamps solution... that is what we have done here in the office and it is quick solution although not particularily clean if you launch multiple jobs in the same second of time.

    Best use is when you run the same process daily and dont want to overwrite the files just add a 20080422_nameoffile.ext prefix

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •  
Privacy Policy | Legal Notices | Safe Harbor Privacy Policy

Copyright © 2005 - 2019 Hitachi Vantara Corporation. All Rights Reserved.