PDA

View Full Version : force enclosure on text output



kettle_anonymous
02-20-2006, 11:58 PM
Hi,
First of all: great peace of work!


I want to create a CSV file using the text output step with all fields enclosed with double quotes. However any string I put in the 'enclosure' field will not return in the output file. How can I force double qoutes as enclosures?



Cheers,
Patrick

MattCasters
02-21-2006, 12:33 AM
At this moment you can't.
We're collecting a good number of change requests so this one can be added to the bunch. In a couple of weeks we should be well under way to implment this one too ;-)

--> Tracker: Change Request - [# 1585] Force enclosure on text output

Thanks you for the kind remarks and your patience in this case.

Matt

kettle_pstnotpd
02-21-2006, 12:46 AM
Thank you for the quick reply.
It will be worth the wait. For now Spoon solves a lot of my problems already :-)

Cheers,
Patrick

MattCasters
02-21-2006, 02:42 AM
No problem Patrick.
I was just trying to be efficient by not putting enclosures around Strings when it is not needed. In retrospect, I guess it isn't that hard to add a checkbox to put it there in all cases ;-)

In case you or anyone else feels the need to implement this, the package is called

be.ibridge.kettle.trans.step.textfileoutput

Kind regards,
Matt

kettle_pstnotpd
02-21-2006, 11:55 PM
Hi Matt,
Had a look at the source code. It looks like it's basically in there, but now related to the availability of a "seperator". Line 284 in TextFileOutput.java. As far as I can tell it returns the string if no seperators exists, or puts the enclosure around it if they do....

It's kind of hell to get this to build (NB5) otherwise I would have tested it myself.

Cheers,
Patrick

MattCasters
02-22-2006, 12:09 AM
I thought it would be easy in Netbeans because the build.xml file exists.
Never tried it myself.

What we want to do in the source-code is:
1) Add a checkbox in the Dialog class saying something like : Force encolure around strings [x]
2) Add a boolean to the Meta class, make sure it gets loaded and saved from XML and Repository. Also make sure the default is set to false for backward compatibility.
3) around line 284 in TextFileOutput simply write:
if (!meta.isEnclosureForced() && seppos<0)
{
retval=v.toString();
}
else
{
retval=meta.getEnclosure()+v.toString()+meta.getEnclosure();
}

Better yet, throw the meta.isEnclosureForced() check up for performance reasons.

Good luck.

Matt

kettle_pstnotpd
02-22-2006, 01:15 AM
I&#39;ll give it a try as soon as I get NB up and running ;-)
Keep you posted.

Cheers,
Patrick

kettle_anonymous
02-22-2006, 03:07 AM
Hi Matt,
I got NB working and changed the code to check for meta.getEnclosure()!=null. It now seems to work for stringtypes which is good enough for me at the moment.

Cheers,
Patrick

MattCasters
02-22-2006, 03:51 AM
Good news,

Would it be possible to post something on the forum (in a new topic) to infom others of how you imported Kettle into NetBeans 5.0?
Also, using enclosures only seem to make sense for Strings, so I guess this is as should be.
If you feel up to it, commit the chances to trunk, OK?

Thanks a bunch!

Matt

kettle_pstnotpd
02-22-2006, 10:21 AM
NB was actually quite easy, I started a new "Java Project with Existing Ant Script" and added the source directory. Only minor annoyance now is that &#39;Run Main Project" doesn&#39;t know which one to start, so I copied the "bin" entries to the root and ran from there.

About the "fix" do you think the &#39;seperator&#39; code should stay in there in this case? I.e. should a field with seperators always have forced enclosures? As far as I&#39;m aware import utilities consider string enclosures mandatory for all strings (including those with seperators)

And, ehm... how do I commit to the trunk? (sorry, new at javaforge)

Cheers,
Patrick

kettle_anonymous
02-23-2006, 04:06 AM
Patrick,

If you run the ant target "distrib" you get the full distribution package under the distrb/ directory. That should solve the "run" problems.

Please note that this is not a fix, but rather an enhancement. Full backward compatibility has to be ensured. Therefor, again, I strongly suggest adding a flag to allow the user to force enclosures around strings.

But don&#39;t worry about it. In a couple of weeks most the requests should be done.

To be able to commit to the subversion repository, you need an account and be member or the Kettle project. You also need a subversion client like TortoiseSVN and a plugin for Netbeans. (subclipse for the Eclipse users)
Then you can checkout the project and commit patches.

All the best,
Matt

kettle_pstnotpd
02-23-2006, 04:54 AM
Thanks, I&#39;ll give it a go.
Distrib indeed does work better....

Cheers,
Patrick