I am reading 100 million records from a file using kettle. Now i have to filter out some rows. As the number of checks are variable, i can not use javascript step to filter out. Moreover, i can not use comma separated list in a variable because in that case i have to break that list for every row.

For e.g. i have to filter out all the request having gif, jpg etc. but the number is not constant and i do not want to hard code it. After that i want to insert into the database only those rows which do not have image types.

Can anybody help me?