PDA

View Full Version : Market Basket in Weka



pro_naveen
04-05-2009, 10:01 PM
Hey Pals

I have been working in WEKA for the past 1 month but i,am not able to implement "MARKET BASKET" in this particular tool.

Can any one please help me how to go with it?

Thank you.

Mark
04-06-2009, 05:22 AM
Hi,

Weka has a number of association rule learning algorithms. You can access them from the "Associate" panel in the Explorer. For market basket analysis you should try the Apriori algorithm. You will need to code your data in a specific way - each attribute (representing an item in the basket) needs to be declared as nominal with a single value (e.g. "true" or "present" or something similar). Absence of an item in the basket is indicated by specifying a missing value (i.e. ? in the ARFF file).

Hope this helps.

Cheers,
Mark.

pro_naveen
04-13-2009, 10:54 PM
Thank you mark for your previous advise. I,am trying to implement market basket using association rule only. i just wanted to ask about the Predictive modelling in weka. How can be a predictive model generated.

Mark
04-15-2009, 06:05 PM
Hi pro_naveen,

Weka has many predictive learning algorithms. They can be classified as either supervised (an explicit target column is used to direct the construction of the model) or unsupervised (no explicit target exists). The former further includes methods for classification (discrete target values) and regression (numeric target). The latter covers clustering algorithms. What sort of predictive task do you have?

Cheers,
Mark.

scottmac99
04-26-2009, 09:38 PM
Hi,

You will need to code your data in a specific way - each attribute (representing an item in the basket) needs to be declared as nominal with a single value (e.g. "true" or "present" or something similar). Absence of an item in the basket is indicated by specifying a missing value (i.e. ? in the ARFF file).

Hope this helps.

Cheers,
Mark.

That answers my question, thank you !! :) My data are coded as 0 = not chosen and 1 = chosen. I tried formatting the fields as text in the .csv file but Weka still insisted on regarding them as numeric. Then I replaced 0 with No and 1 with Yes, and Weka insisted on finding rules to predict No, which is the opposite of what I want. I'll try recoding the data as 0=blank and 1=yes and see what happens. Scott

pro_naveen
05-13-2009, 09:52 AM
Respected Sir,

i was said to generate a Predicitve model using weka. I just wanted the help-about how to generate a predicitve model using this tool?
Please help me out Mr. Mark.

Thank you.

Mark
05-13-2009, 09:53 PM
Hi,

Weka has many algorithms for learning predictive models (both for classification and regression). Take a look at the documentation that comes with Weka and the Wiki (both Pentaho's and on Sourceforge).

Cheers,
Mark.

pro_naveen
05-16-2009, 09:58 AM
Hi Mark,
I have been trying to generate a predictive model and also i have partially understood it. If suppose we have a data set in "ARFF", we preprocess it and then we shift to the 'classify' and here we apply the classifier algorithm. On the right click in the name of applied algorithm we get a pop down menu which has a option save model. Is this model can be considered as the predictive model or we have to further perform some processing again on the data set to get the final predictive model.

Please help mark if possible please explain with an example.

Thanking you
Naveen.

pro_naveen
05-16-2009, 10:00 AM
Hey pals,

I just want to know how to proceed with the knowledge flow. How to start with it and what is its purpose?

Thanks
Naveen;)

Mark
05-17-2009, 02:17 AM
Yes, saving out the model from the Explorer gives you a predictive model that can be loaded and used in the future. If you have a lot of pre-processing that you do to your data then the best approach is to use a weka.classifiers.meta.FilteredClassifier (instead of using the pre-process panel of the Explorer for pre-processing). The FilteredClassifier allows you to specify a filter and a classifier to use. The filter gets applied to the data before it is passed to the classifier. Both (trained) filter and classifier get encapsulated and saved when you save the model. So, in future applications of the model you get the filtering process automatically. It is possible to apply an arbitrary number of filters by specifying that the base classifier for a FilteredClassifier is itself another FilteredClassifier (and so on).

Hope this helps.

Cheers,
Mark.

pro_naveen
05-18-2009, 01:28 PM
Hi Mark,
I wanted to know more about Predictive modelling?can you just brief me with the steps involved in generating the model from initial stage using WEKA. Lets consider the classifier :- "Bayes". can you please site an example of (say) health insurance or fraud detection or something else.
I would be helpful if you explain me out with the example.

I also wanted to know how will you use the trained dataset to check the test data using "WEKA". I normally use the CROSS VALIDATION. i use 10 folds only.

So i kindly request you to please explain me these two problems which i,am facing.

thank you
sir,

sincerely
Naveen

helene0618
06-11-2009, 05:54 AM
hi,

so if we want to analysis a database, the first we must to regroupe these attributes ,which i want to analysis, in a table and give a right type columnes?

Mark
06-11-2009, 05:30 PM
Hi Helene,

Weka can connect to relational databases. It has an SQL Viewer application (accessible from the Explorer) that can be used to run SQL queries against a database. The result of a query is converted to Weka's internal Instances format and loaded into the Explorer ready for use.

Cheers,
Mark.

helene0618
06-12-2009, 03:28 AM
thanks, mark .yestoday i connecte weka with sql .

i have another question: in the PDI we can add wekascoring, it's not obligtoire use it? for exemple in weka we can get the data in use query sql, so we can directly analysis with data and get information .

and what's differents between explorer and experimenter? all of them we can use scoring to analysis the data.

my problème is i don't know what's data i analysis, how to analysis, and how i can get the information important for my database. i know these question too general

helene

Mark
06-12-2009, 07:06 AM
Hi Helene,

The WekaScoring plugin for PDI is designed primarily as a tool for deploying a data mining model. I.e. after you have completed analysis and selected a final model that performs best on your data, you can export the model from Weka as a binary serialized object and then load that into the WekaScoring plugin in PDI. This model can then be used to append predicted scores to data rows before they are loaded into a data warehouse for example.

The Weka Explorer is designed primarily (as the name suggests) for exploring your data and quickly trying various learning algorithms. The Experimenter, on the other hand, is a tool for performing principled, large-scale experimental comparisons between multiple learning algorithms (with perhaps various parameter settings) on one or more data sets. The Experimenter supports repeated runs of cross-validation, collation of evaluation metrics and the presentation of results in tabular format complete with paired t-tests to detect statistically significant differences in performance between algorithms.

Cheers,
Mark.

helene0618
06-12-2009, 09:20 AM
ok, thanks again mark.

i must use preprocess to open datasource. many documentation talk about filter, but i always don't claire with it. can you explique more details in it?
if i have not apply the filter, when i clic in associate-->start, there are an message : cannot handle numeric attributes. so every time i choose the filter --> discretize and apply it. this is work with these file example in weka, but if i use my database, there are always an error. because in my table there are type date, char etc , aussi there are value null in my table.

so my question is each time we juste analysis one type value?

ps: i hope you can understand what i say^^

helene

Mark
06-13-2009, 04:19 AM
Hi Helene,

Yes, the association rule learning algorithms in Weka can only operate on nominal attributes.

As for database compatibility, the DatabaseUtils.props file for your particular database has entries that map particular db types to Weka's attribute types. If there are specific field types in your tables that are not represented then you can just add further mappings into the file.

Cheers,
Mark.

helene0618
06-15-2009, 05:14 AM
ok, thanks again mark

helene

helene0618
06-15-2009, 05:31 AM
how to add datatime in the "databaseutils.props". there data type time and date, but havn't the datetime same as in sql.

helene

Mark
06-15-2009, 05:13 PM
Try adding a line:

datetime=8

You might have to have it in all capital letters if that doesn't seem to work:

DATETIME=8

Cheers,
Mark.

helene0618
06-16-2009, 04:35 AM
ok, thanks.

helene