From d5d85c08c745caeff36b83adb775724f63dfbbcf Mon Sep 17 00:00:00 2001 From: ben-jaynes <1btjaynes@gmail.com> Date: Fri, 22 May 2026 12:57:45 -0700 Subject: [PATCH] vault backup: 2026-05-22 12:57:45 --- .../Model Validation and Evaluation.md | 13 ++++++++++++- 1 file changed, 12 insertions(+), 1 deletion(-) diff --git a/Wiki/Machine Learning/Model Validation and Evaluation.md b/Wiki/Machine Learning/Model Validation and Evaluation.md index 79e6342..23c81ae 100644 --- a/Wiki/Machine Learning/Model Validation and Evaluation.md +++ b/Wiki/Machine Learning/Model Validation and Evaluation.md @@ -32,4 +32,15 @@ Cross entropy loss is used for multi-class problems and due to the log that is u ### Accuracy Accuracy is defined as the percentage of predictions that the model gets correct. $$\frac{\text{num correct predictions}}{\text{num incorrect predictions}}$$ -This metric is simple but can often hide information about how a model performs on a dataset. One particular limitation is when the dataset is imbalanced since a model can have a high accuracy while predicting the minority class incorrectly most of the time. This is common in datasets relating to disease, especially when outputs such as whether someone has cancer \ No newline at end of file +This metric is simple but can often hide information about how a model performs on a dataset. One particular limitation is when the dataset is imbalanced since a model can have a high accuracy while predicting the minority class incorrectly most of the time. This is common in datasets relating to disease, especially when outputs such as whether someone has cancer need to be accurately predicted. If a model is biased towards the majority class there will likely be many false negatives that accuracy is not able to detect. +### Precision +$$\frac{TP}{TP + FP}$$ +### Recall + +### F-Measure +The F-measure is used to balance both the precision and recall metrics to better evaluate how a model performs. This helps to get a better metric on imbalanced datasets since it can help a model train for precision and recall. +$$F_{\beta} = (1 + \beta^2)\frac{{\text{precision} * \text{recall}}}{\beta^2 * \text{precision} + \text{recall}}$$ +$\beta$ is used as a way to tune whether precision or recall is emphasized in the F-measure. When $\beta > 1$ recall is emphasized and when $\beta < 1$ precision is emphasized. + +When $\beta = 1$ the F-measure is also called the F1-measure which is the most commonly used variant. This is where precision and recall are both weighted equally in the metric. +### Kappa