PDA

View Full Version : HOW: multiple text files output to multiple text files?



kettle_anonymous
02-04-2006, 05:29 PM
I've got the "Transformation" to work.

But now I want to apply it to multiple text files and have each loaded text file end up generating another text file with a different extension.

EXAMPLE:

xxx_s1.txt will load
run through a JavaScript
then try and output
xxx_s1.mel

But the output is going to one "mel" file instead of many. I tried to get "Chef" working, but I couldn't figure out how to increment a variable in a file name.

HOW:

MattCasters
02-04-2006, 11:59 PM
Actually, I'm not sure if this is possible at the moment.
I have been thinking about a "Loop or Foreach" job entry in Chef, but it needs a bit more thought.

However, it shouldn't be difficult to script using environment variables.

Something like :

for filename in $(ls *_s1.txt)
do
pan.sh -file=processit.ktr
done

Make sure to set the environment variable filename in pan.sh like this:

-DFILENAME=${filename}

Hope this helps.

Matt

dangerahead
02-06-2006, 11:02 AM
I'm running this in CYGWIN. It loops through the files, but gives me a "bad command" error. I have the ktr job in the same folder as the to-be-operated-on text files.

SCRIPT I'M RUNNING:

for filename in $(ls sd456789_s*.xyz)
do
pan.sh -file=kettle03.ktr
done


Is this right? how is the value of "filename" being passed to the spoon job?

kettle_anonymous
02-06-2006, 01:18 PM
UPDATE: I couldn't get pan.sh to work on CYGWIN on my 64Bit machine. Dunno why. So I went to DOS.

I got ONE KTR to work under the DOS command Pan.bat!

How do I modify the KTR with the command line? say changing the file wildcard?

kettle_anonymous
02-06-2006, 03:39 PM
The job just keeps writing to the same file over and over. So everything is getting written over a thousand times.

Is there a setting to prevent that?

MattCasters
02-07-2006, 11:30 AM
Some quick pointers:
- use this format under dos (cygwin too): /file:kettle03.ktr
- add the $filename variable to the java virtual machine so that kettle can read it:
add option -DFILENAME=$filename
- specify input file name like %%FILENAME%%
- specify output file name like %%FILENAME%%.out

Hope this helps,

Matt