05_Machine_Learning

Tags: #◇CSINTSY #◇Computational Thinking #◇Computer Science #◇Artificial Intelligence #◇Machine Learning #◇Intelligent Systems #◇Probability #◇Data Analysis

---

◇Overview

Machine Learning (ML) is a field of AI where computers learn from data without being explicitly programmed for specific tasks.

◇Details / Theory

5. Machine Learning Reviewer

◇Overview

Machine Learning (ML) is a field of AI where computers learn from data without being explicitly programmed for specific tasks.

◇Types of Machine Learning

1. Supervised Learning

Data: Labeled dataset. Input ($X$) and correct Output ($Y$) are provided. Goal: Learn a mapping function $Y = f(X)$ to predict outputs for new inputs. Examples:
Classification: Predicting a category (e.g., Spam vs. Not Spam, Dog vs. Cat).
Regression: Predicting a continuous value (e.g., House Price, Temperature). Algorithms: Decision Trees, Naive Bayes, Linear Regression, Support Vector Machines.

2. Unsupervised Learning

Data: Unlabeled dataset. Only Input ($X$) is provided. Goal: Find hidden structures or patterns in the data. Examples:
Clustering: Grouping similar data points (e.g., Customer Segmentation).
Dimensionality Reduction: Simplifying data while keeping important info. Algorithms: K-Means Clustering, PCA.

3. Reinforcement Learning

Data: Interaction with an environment (States, Actions, Rewards). Goal: Learn a policy to maximize cumulative reward. Examples: Game playing (Chess, Mario), Robot control. Algorithms: Q-Learning.

◇Evaluation Metrics (for Classification)

Confusion Matrix

A table comparing predicted vs. actual labels.

Formulas

1. Accuracy: Overall correctness.
   $\text{Accuracy} = \frac{TP + TN}{TP + TN + FP + FN}$
   Warning: Misleading for imbalanced datasets (e.g., 99% benign, 1% cancer).

2. Precision: Quality of positive predictions. "Of all the ones I labeled positive, how many were actually positive?"
   $\text{Precision} = \frac{TP}{TP + FP}$

3. Recall (Sensitivity): Coverage of actual positives. "Of all the actual positives, how many did I find?"
   $\text{Recall} = \frac{TP}{TP + FN}$

4. F1 Score: Harmonic mean of Precision and Recall. Balances both.
   $F1 = 2 \times \frac{\text{Precision} \times \text{Recall}}{\text{Precision} + \text{Recall}}$

◇Feature Engineering

The process of using domain knowledge to extract features from raw data.
Feature Selection: Choosing the most relevant features (e.g., removing irrelevant columns). Feature Extraction: Creating new features (e.g., calculating BMI from Height and Weight). One-Hot Encoding: Converting categorical data (Red, Blue) into binary vectors (Red=[1,0], Blue=[0,1]). TF-IDF: A statistical measure used in text mining to evaluate how important a word is to a document in a collection.

◇Solved Example: Calculating Metrics

Scenario: A model predicts whether an email is Spam (Positive) or Not Spam (Negative).
Confusion Matrix:
TP (Spam correctly identified): 80 TN (Not Spam correctly identified): 900 FP (Not Spam labeled as Spam): 20 FN (Spam labeled as Not Spam): 10

Calculations:

1. Accuracy:
   $\frac{TP + TN}{Total} = \frac{80 + 900}{80 + 900 + 20 + 10} = \frac{980}{1010} \approx 0.97 \text{ or } 97\%$

2. Precision (for Spam):
   $\frac{TP}{TP + FP} = \frac{80}{80 + 20} = \frac{80}{100} = 0.80 \text{ or } 80\%$
   Interpretation: When the model says "Spam", it is correct 80% of the time.

3. Recall (for Spam):
   $\frac{TP}{TP + FN} = \frac{80}{80 + 10} = \frac{80}{90} \approx 0.89 \text{ or } 89\%$
   Interpretation: The model catches 89% of all actual spam emails.

4. F1 Score:
   $2 \times \frac{0.80 \times 0.89}{0.80 + 0.89} = 2 \times \frac{0.712}{1.69} \approx 2 \times 0.421 = 0.842$

---

---

◇Common Pitfalls

Confusing accuracy (overall correct) with precision (correct among predicted positive) and recall (correct among actual positive). Using accuracy alone on imbalanced datasets — F1 score gives a better picture. Mixing up supervised (labeled data), unsupervised (no labels), and reinforcement (reward signals) learning paradigms.

◇Summary

TL;DR — ML splits into supervised (classification/regression), unsupervised (clustering), and reinforcement learning. Evaluation uses confusion matrices to compute accuracy, precision, recall, and F1 score. Feature engineering and proper train/test splits are essential.

◇Links

Related: Source / Reference:

---

Made by 2ru17

Please contact me through discord or my the university email for any problems regarding the reviewer. Thanks!

◇References & Navigation

• Previous Topic: ◇Adversarial Games
• Next Topic: ◇Decision Tree
• Related Notes: ◇Machine Learning Models