Hello fellow researchers,

recently, I classified my dataset through the Auto-Weka classifier and it gave me the Random Forest with these arguments as best result: [-I, 10, -K, 0, -depth, 0].
The accuracy was of 97.6879% and the AUC was 0.999. You can see these results on the following code:

Code:
=== Run information ===


Scheme:       weka.classifiers.meta.AutoWEKAClassifier -seed 123 -timeLimit 1 -memLimit 1024 -nBestConfigs 1 -metric errorRate -parallelRuns 1
Relation:     NewOriginais
Instances:    173
Attributes:   16
              feat1
              feat2
              feat3
              feat4
              feat5
              feat6
              feat7
              feat8
              feat9
              feat10
              feat11
              feat12
              feat13
              feat14
              feat15
              class
Test mode:    evaluate on training data


=== Classifier model (full training set) ===


best classifier: weka.classifiers.trees.RandomForest
arguments: [-I, 10, -K, 0, -depth, 0]
attribute search: null
attribute search arguments: []
attribute evaluation: null
attribute evaluation arguments: []
metric: errorRate
estimated errorRate: 0.017341040462427744
training time on evaluation dataset: 0.002 seconds


You can use the chosen classifier in your own code as follows:


Classifier classifier = AbstractClassifier.forName("weka.classifiers.trees.RandomForest", new String[]{"-I", "10", "-K", "0", "-depth", "0"});
classifier.buildClassifier(instances);




Correctly Classified Instances         169               97.6879 %
Incorrectly Classified Instances         4                2.3121 %
Kappa statistic                          0.9568
Mean absolute error                      0.0686
Root mean squared error                  0.1417
Relative absolute error                 19.0028 %
Root relative squared error             33.4188 %
Total Number of Instances              173     


=== Confusion Matrix ===


  a  b  c   <-- classified as
 10  1  1 |  a = LLC
  0 99  0 |  b = LCM
  1  1 60 |  c = LF


=== Detailed Accuracy By Class ===


                 TP Rate  FP Rate  Precision  Recall   F-Measure  MCC      ROC Area  PRC Area  Class
                 0.833    0.006    0.909      0.833    0.870      0.861    0.996     0.928     LLC
                 1.000    0.027    0.980      1.000    0.990      0.977    0.999     0.999     LCM
                 0.968    0.009    0.984      0.968    0.976      0.962    0.999     0.998     LF
Weighted Avg.    0.977    0.019    0.976      0.977    0.976      0.963    0.999     0.994     
Temporary run directories:
C:\Users\GUILH_~1\AppData\Local\Temp\autoweka3435236811599207287\




For better performance, try giving Auto-WEKA more time.
Tried 39 configurations; to get good results reliably you may need to allow for trying thousands of configurations.




Time taken to build model: 58.81 seconds


=== Evaluation on training set ===


Time taken to test model on training data: 0 seconds


=== Summary ===


Correctly Classified Instances         169               97.6879 %
Incorrectly Classified Instances         4                2.3121 %
Kappa statistic                          0.9568
Mean absolute error                      0.0686
Root mean squared error                  0.1417
Relative absolute error                 19.0028 %
Root relative squared error             33.4188 %
Total Number of Instances              173     


=== Detailed Accuracy By Class ===


                 TP Rate  FP Rate  Precision  Recall   F-Measure  MCC      ROC Area  PRC Area  Class
                 0.833    0.006    0.909      0.833    0.870      0.861    0.996     0.928     LLC
                 1.000    0.027    0.980      1.000    0.990      0.977    0.999     0.999     LCM
                 0.968    0.009    0.984      0.968    0.976      0.962    0.999     0.998     LF
Weighted Avg.    0.977    0.019    0.976      0.977    0.976      0.963    0.999     0.994     


=== Confusion Matrix ===


  a  b  c   <-- classified as
 10  1  1 |  a = LLC
  0 99  0 |  b = LCM
  1  1 60 |  c = LF
Then, I tried to reproduce this output on the Command Line Interface through the input:

Code:
java weka.classifiers.trees.RandomForest -I 10 -K 0 -depth 0 -s 123 -t "C:\Users\guilh_000\Dropbox\Mestrado\Resultados\Originais (novo)\NewOriginais.arff"
However, the results were a little different:

Code:
Options: -I 10 -K 0 -depth 0 


RandomForest


Bagging with 10 iterations and base learner


weka.classifiers.trees.RandomTree -K 0 -M 1.0 -V 0.001 -S 1 -do-not-check-capabilities


Time taken to build model: 0 seconds
Time taken to test model on training data: 0 seconds


=== Error on training data ===


Correctly Classified Instances         170               98.2659 %
Incorrectly Classified Instances         3                1.7341 %
Kappa statistic                          0.9677
Mean absolute error                      0.0651
Root mean squared error                  0.1318
Relative absolute error                 18.042  %
Root relative squared error             31.0928 %
Total Number of Instances              173     




=== Detailed Accuracy By Class ===


                 TP Rate  FP Rate  Precision  Recall   F-Measure  MCC      ROC Area  PRC Area  Class
                 0.833    0.006    0.909      0.833    0.870      0.861    0.998     0.974     LLC
                 1.000    0.014    0.990      1.000    0.995      0.988    1.000     1.000     LCM
                 0.984    0.009    0.984      0.984    0.984      0.975    1.000     0.999     LF
Weighted Avg.    0.983    0.011    0.982      0.983    0.982      0.975    1.000     0.998     




=== Confusion Matrix ===


  a  b  c   <-- classified as
 10  1  1 |  a = LLC
  0 99  0 |  b = LCM
  1  0 61 |  c = LF






=== Stratified cross-validation ===


Correctly Classified Instances         142               82.0809 %
Incorrectly Classified Instances        31               17.9191 %
Kappa statistic                          0.6539
Mean absolute error                      0.1726
Root mean squared error                  0.2948
Relative absolute error                 47.7941 %
Root relative squared error             69.502  %
Total Number of Instances              173     




=== Detailed Accuracy By Class ===


                 TP Rate  FP Rate  Precision  Recall   F-Measure  MCC      ROC Area  PRC Area  Class
                 0.083    0.025    0.200      0.083    0.118      0.089    0.780     0.176     LLC
                 0.929    0.203    0.860      0.929    0.893      0.740    0.935     0.943     LCM
                 0.790    0.108    0.803      0.790    0.797      0.685    0.937     0.888     LF
Weighted Avg.    0.821    0.156    0.794      0.821    0.805      0.675    0.925     0.870     




=== Confusion Matrix ===


  a  b  c   <-- classified as
  1  4  7 |  a = LLC
  2 92  5 |  b = LCM
  2 11 49 |  c = LF
Am I missing something here, maybe a wrong input argument?

Thanks in advance!

Cheers,

Guilherme Freire