F-beta

score is the weighted harmonic mean of precision and recall. It’s a more generalized version of F1 score

is a factor that determines how many times more important Recall is than precision in the score.

• If is close to 0, it skews towards precision
• If , it is F1 score
• If (a large number), it skews towards sensitivity

Precision-Recall Tradeoff

precision = the proportion of true positives out of all positive predictions = TP / (TP + FP) Recall = the proportion of true positives out of all actual positives = TP / (TP + FN)

• If False Negative is worse than False Positive (minimize Type II error), the model requires a high Recall, so that it could catch as many positive cases as possible. We should tailor our F-beta to incline towards Recall ()
  • Diagnosing sick patients
  • Detecting malfunctioning parts in a spaceship
• If False Positive is worse than False Negative (minimize Type I error), the model requires a high Recall. We should tailor our F-beta to incline towards precision ()
  • Sending promotional material in the mail to potential clients (we don’t want to send to many people that won’t be interested)