Hitachi Vantara Pentaho Community Forums
Results 1 to 5 of 5

Thread: Using XML Input with newest file in directory

  1. #1
    Join Date
    Sep 2008
    Posts
    3

    Question Using XML Input with newest file in directory

    Hi, I've started using spoon/kettle 3.0.4 two days ago. I've used "XML Input" with "InsertUpdateRows" successfuly but now I want to import the newest xml file of a directory only. How can I do that?

    My idea was to use "Get File names", sort the list of names with "Sort Rows" and limit (how?) to one row -- and give "XML Input" that name. But it looks like only "Text input" can handle a filename coming from input stream... What can I do?

    Ans how could I use the newest file only if its name contains the current date (there's something like 20080826 in the filename)?

    Please give me some hints - spoon is a great tool and I believe there is a way to do what I want... :-)

    thanks,
    F.

  2. #2

    Default

    Hi F.,

    How you start your ETL process? Can you execute a shell script in front which can provide parameters to Spoon/Kitchen/Pan? Or do you use it within J2EE?

    On a professional operation system I would do it with
    export MY_FILENAME=`ls -t | head -1`
    and give that into Kettle...

    Christoph
    21 is only half the truth

  3. #3
    Join Date
    Sep 2008
    Posts
    3

    Default

    Hello,

    The plan is to use kitchen.sh (in unix) to start the process. How can I access the ENV variable in spoon when defining the job/transaction? Is it "Get variable"?

    I think... there will be a way to use the variables content in "XML Input" by pressing Ctrl-Space, right?

    Can I simulate the name in spoon by defining a"Set variable" to be able to test without having to restart spoon over and over?

    Thanks for your help!

  4. #4
    Join Date
    May 2006
    Posts
    4,882

    Default

    variables in PDI are not "environment variables" if you were in that assumption

    For the moment you can use positional arguments to kitchen/pan and you can retrieve them with "get system info" step. There are alternatives like loading property files and setting its value with "set variable" step.

    Regards,
    Sven

  5. #5
    Join Date
    Sep 2008
    Posts
    3

    Default thanks, I will try that

    thanks a lot! (thanks was too short )

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •  
Privacy Policy | Legal Notices | Safe Harbor Privacy Policy

Copyright © 2005 - 2019 Hitachi Vantara Corporation. All Rights Reserved.