Hitachi Vantara Pentaho Community Forums
Results 1 to 2 of 2

Thread: Difference between "Percent_correct" and "Weighted_avg_true_positive_rate" metrics

  1. #1
    Join Date
    Jun 2016
    Posts
    1

    Default Difference between "Percent_correct" and "Weighted_avg_true_positive_rate" metrics

    I'm using the Weka Experimenter to evaluate the performance of a number of learning algorithms on a single dataset using 10 times 10-fold-cross-validation. When extracting the results from the experiment I get identical results for both the "Percent_correct" (i.e. accuracy) and the "Weighted_avg_true_positive_rate" (i.e. recall or sensitivity) metric. Even the standard deviation is identical (see attached table).

    Is this really correct? Accuracy and recall are calculated totally different so I have a hard time understanding how they can be identical for all 16 algorithms that I evaluate. I have verified these results on different versions of Weka (3.6.14 and 3.7.7) and also directly using my own Java program relying on the Weka API.

    The data in the dataset consists of 4 ordinal class variables (see confusion matrix figure for more info).

    I hope someone can shed some light on this. Thanks in advance.

    /M

    Name:  Skärmavbild 2016-06-07 kl. 22.12.18.jpg
Views: 65
Size:  20.1 KBName:  Skärmavbild 2016-06-07 kl. 22.11.58.jpg
Views: 66
Size:  50.6 KB

  2. #2
    Join Date
    Aug 2006
    Posts
    1,741

    Default

    Yes, weighted average tp rate, weighted average recall and accuracy are the same thing. In the multiclass case, the information retrieval metrics are computed by treating each class in turn as the positive class and all others combined as the negative class. The weighted average tp rate is computed by taking each tp rate and multiplying it by the size of the class (i.e. cancel out the denominator) and then dividing by the total class count sum.

    Cheers,
    Mark.

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •  
Privacy Policy | Legal Notices | Safe Harbor Privacy Policy

Copyright © 2005 - 2019 Hitachi Vantara Corporation. All Rights Reserved.