View Full Version : How to implement IS_NUMERIC, IS_ALPHANUMERIC in filter step?
04-20-2006, 11:18 PM
I have this text file with columns. I have no problem retrieving them. however, some columns are weird in such a way that they have non-numeric chars in columns that should have numeric characters. I am performing a calculation so when these rows hit the calculator, there is an error.
"Division can only be done with numeric data!"
I havent seen the actual rows that causes the error but the error message clearly says that it is fed by non numeric data.
How can I implement the IS_NUMERIC filter? i cant see something like this in the filter step. Are there any way?
- Cholo -
04-21-2006, 12:35 AM
I never came across anything *this* bad ;-)
You mean you don't know whether the characters in a string are numeric in nature or not?
Loop over the characters in the String and see if they are all 0-9, minus, decimal, etc.
Then just add a flag IS_NUMERIC Y or N, do a filter on that.
04-23-2006, 04:57 PM
Well yeah! the data is really bad! It has about one hundred rows that are erroneous. I think it was an error during a system migration years back. The system cannot be corrected because it can cause the company additional money to change something. So what we are doing is downloading the data from the erroneous server and we transform it using kettle. the transformed data is then used in our data warehousing project.
04-24-2006, 12:28 AM
I thought about this, maybe we can add a couple of methods to the Value class that do this for you:
isStringNumeric(), isStringAlphanumeric(), ...
If you like this, add a feature request for it in the trackers.
It would make things easier for the people that come after you and have to solve the same puzzle.
In any case thanks for the feedback!