PDA

View Full Version : SVM output classification as probability:



neub
01-17-2008, 07:12 AM
Hello,

I've trained a SVM classifier with two classes that works correctly but the problem is that my output is binary, and not between [0-1].

This is okay if it was for a classification but i want the ouput of the SVM to be the input of probabilistic classification schema (BayesNetwork), therefore it would be great to have value between [0-1] depending on the distance from the support vectors.

I've read in Witten2005 (Chap.10 - p.410):


Logistic regression models can be fitted to the support
vector machine output to obtain probability estimates.
But i don't really know how to do this?

PS: a simple feature to include in Weka should be to have confusion matrix in percent for a better visualisation.

Mark
01-17-2008, 04:24 PM
Hi,

In the Explorer's classify panel: select SMO, click on the configuration summary to bring up the GenericObjectEditor, and set "buildLogisticModels" to "true".

At the command line: pass the "-M" option to SMO.

HTH.

Cheers,
Mark.

neub
01-17-2008, 07:06 PM
Thank you very much, It's exactly this

it was in front of my eyes but I never thought about using this option :)

simko
05-03-2010, 03:03 PM
Hi, all

I need a posterior probability output for SMO. I am pretty new with SVM in general and as I understand SVM outputs class label. If we have two classes: -1 and 1, the SMO classifier will output -1 or 1 for every train and test instance.

f(x) = -1 or 1

But I would need output as:
f(x) = -1 with 60% probability.

I think I can do that with logistic model in weka's SMO implementation. Is this true?

I did an experiment where I turned buildLogisticModels on true but I dont understand the Explorer's output. Is this explained anywhere?

Many thanks!
Simon

Mark
05-03-2010, 05:23 PM
Hi Simon,

Yes, turning on the option to fit logistic models to the output of SMO will give you posterior probability estimates for the classes.

The evaluation metrics that are produced in Weka's Explorer are explained in detail in the book that was written to accompany Weka - Ian H. Witten and Eibe Frank. Data Mining: Practical Machine Learning Tools and Techniques. 2005. Morgan Kaufmann. Amazon has it for a reasonable price.

Cheers,
Mark.