ABW505 Python与预测分析期末终极题库：代码填空 + 算法手算 (含真题解析)

Analysis:

This is a classic Python interview question. The default parameter list=[] is created once at function definition time, not every time the function is called.

• First call extend_list(10) → Default parameter initialized as [], then append 10 → [10]
• Second call extend_list(123, []) → Passes a new list [], independent of default → [123]
• Third call extend_list('a') → Reuses the already-modified default parameter [10] → [10, 'a']

Key Concept: Python default parameters are evaluated at function definition time. Mutable objects retain their state across function calls.

Execution Trace:

Result: x = 4

Analysis:

This uses Python 3's Extended Unpacking syntax:

• a takes the first element → 10
• c takes the last element → 50
• *b collects all remaining middle elements and packs them as a list → [20, 30, 40]

Note: *b collects into a list, not a tuple.

Analysis:

(i) Create 4x4 Matrix:

• np.arange(1, 17) creates array from 1 to 16 (excluding 17)
• .reshape(4, 4) reshapes it into 4x4 matrix

(ii) Null Vector & Indexing:

• np.zeros(10) creates zero vector
• Key Point: Python uses 0-based indexing, so 7th value is at index 6
• null_vector[6] = 10 updates the 7th position

(iii) Checkerboard Pattern:

• Initialize all-zero matrix
• checkerboard[1::2, ::2] = 1 → Odd rows, even columns set to 1
• checkerboard[::2, 1::2] = 1 → Even rows, odd columns set to 1
• Slicing syntax: [start:stop:step]

Key Concepts:

• Numpy array creation and reshaping
• Indexing rules (0-based vs 1-based)
• Slicing with step parameter

Analysis:

Method 1: Slicing Reversal (Recommended):

• [::-1] is Python's slice reversal syntax
• start:stop:step, step of -1 means traverse backwards

Method 2: np.flip() function:

vector_reversed = np.flip(vector)

Method 3: np.flipud() function:

vector_reversed = np.flipud(vector)

Key Concepts:

• Python slice reversal syntax [::-1]
• Numpy array manipulation functions

Analysis:

*args (Variable Positional Arguments):

• Collects all positional arguments into a tuple
• Here: args = (1, 2, 3)
• Can be used like a normal tuple: iterate, index, sum(), etc.

**kwargs (Variable Keyword Arguments):

• Collects all keyword arguments into a dictionary
• Here: kwargs = {'multiplier': 10, 'label': 'test'}
• Use .get() method for safe access, avoiding KeyError

Execution Flow:

1. sum(args) → sum((1, 2, 3)) → 6
2. kwargs.get('multiplier', 1) → Returns 10 (default 1 if not present)
3. 6 * 10 = 60

Key Concepts:

1. While Loop Termination Condition:

while target_num != guess_num and guess_counter < max_attempts:

• Both conditions must be true to continue looping
• Exits when guessed correctly OR out of attempts

2. break Statement:

• Terminates loop early without waiting for condition check
• Used when guess is correct

3. Variable Increment:

guess_counter += 1

• Equivalent to guess_counter = guess_counter + 1

4. Boundary Condition Check:

• guess_counter < max_attempts ensures no overflow
• Don't show "Try again" on last attempt

Common Mistakes:

• Forgetting to increment guess_counter, causing infinite loop
• Wrong termination condition (using or instead of and)
• Not checking if guessed correctly outside loop

Code Analysis:

1. Data Preparation:

• Use lists to store city names and happiness indices
• Order must match (cities and values aligned by position)

2. Create Figure:

• plt.figure(figsize=(10, 5)) sets canvas size (10 inches wide, 5 inches tall)

3. Draw Horizontal Bar Chart:

• plt.barh() is for horizontal bars (h = horizontal)
• plt.bar() is for vertical bars
• color='skyblue' sets fill color
• edgecolor='navy' sets border color

4. Add Labels:

• xlabel() - X-axis label
• ylabel() - Y-axis label
• title() - Chart title
• fontsize - Font size
• fontweight='bold' - Bold text

5. Grid Lines:

• plt.grid(axis='x') shows only X-axis grid
• linestyle='--' dashed line
• alpha=0.7 70% transparency

6. Display Figure:

• plt.tight_layout() auto-adjusts layout to prevent label overlap
• plt.show() displays the figure

Common Exam Points:

• Distinguish between bar() and barh()
• Spelling of label parameters (xlabel, ylabel, title)
• Color parameter names (color, edgecolor)

Task (i): Propose Manual Expert Rules

Observation from first three rows:

• GPA 3.85 (High) → Prediction = Yes
• GPA 2.85 (Low) → Prediction = No
• GPA 3.50 (High) → Prediction = Yes

Pattern Identified:

• High GPA (≥ 3.50) → Yes
• Low GPA (< 3.50) → No

Proposed Expert Rule (Example):

Rule 1 (Based on GPA threshold):
IF GPA >= 3.50 THEN Prediction = Yes
ELSE Prediction = No

Or rule based on Grade:

Rule 2 (Based on Grade):
IF Grade is 'A' or 'A-' THEN Prediction = Yes
ELSE Prediction = No

Recommend Rule 1, because GPA is a numerical feature, more objective and stable.

Task (ii): Identify Predictions

Based on Rule 1: IF GPA >= 3.50 THEN Yes, ELSE No

Final Answers:

1. Student 710: No
2. Student 595: No (Grade A but GPA 3.30 < 3.50)
3. Student 540: Yes ✅
4. Student 630: No

If using Rule 2 (Based on Grade):

IF Grade is 'A' or 'A-' THEN Yes
ELSE No

Final Answers (Rule 2):

1. Student 710: No
2. Student 595: Yes ✅
3. Student 540: No
4. Student 630: No

Key Concepts Summary:

1. Rule Induction: Observe patterns from known samples, propose hypotheses
2. Rule Validation: Rules should correctly classify training data (first 3 rows)
3. Rule Application: Apply rules to new samples one by one, output predictions
4. Rule Conflicts: Different rules may give different predictions, choose most reasonable (explain your choice in exam)

Practical Tips:

• Start with simplest rules (single feature threshold)
• For complex patterns, combine multiple features (AND/OR logic)
• Rules should be interpretable (interpretability is advantage of expert rules)

Label Encoding works by:

• Converting categories to integers (e.g., Red=1, Blue=2, Green=3)
• Problem: Algorithms might interpret "3 > 1" as Green being "greater" than Red (introduces false ordinal relationship)

One-Hot Encoding works by:

• Creating a binary column for each category (Is_Red, Is_Blue, Is_Green, etc.)
• Each row has exactly one column as 1, rest as 0

Usage Rules:

Conclusion: Use One-Hot for unordered categorical data (cities, colors, countries), use Label for ordered categorical data (education level, income brackets).

Min-Max Normalization:

• Formula: $x' = \frac{x - x_{min}}{x_{max} - x_{min}}$
• Result Range: [0, 1]
• Characteristics: Preserves original distribution shape
• Use Cases:
  • Neural networks (require inputs in fixed range)
  • Image processing (pixel values 255 → 0-1)
  • Known distribution without outliers

Z-Score Standardization:

• Formula: $z = \frac{x - \mu}{\sigma}$ (where μ is mean, σ is standard deviation)
• Result Range: Typically [-3, 3] (no fixed range)
• Characteristics: Mean of 0, standard deviation of 1
• Use Cases:
  • More robust when outliers exist
  • SVM, Logistic Regression algorithms
  • Comparing features of different scales

Observation:

• Min-Max: Outlier 100 mapped to 1, others squeezed in 0-0.03 range (information loss)
• Z-Score: Outlier ~2.67, others around -0.7 (better preserves distribution)

Conclusion: Prefer Z-Score when outliers exist!

Formulas:

$$\text{Precision} = \frac{TP}{TP + FP} = \frac{80}{80 + 10} = \frac{80}{90} \approx 0.889$$

$$\text{Recall} = \frac{TP}{TP + FN} = \frac{80}{80 + 20} = \frac{80}{100} = 0.800$$

$$\text{F1-Score} = 2 \times \frac{Precision \times Recall}{Precision + Recall} = 2 \times \frac{0.889 \times 0.800}{0.889 + 0.800} \approx 0.842$$

Interpretation:

• Precision: Of predicted positives, how many are actually positive ("don't misclassify")

  • In this example: Of 90 predicted positive, 80 are truly positive

• Recall: Of actual positives, how many are correctly predicted ("don't miss any")

  • In this example: Of 100 actual positives, found 80

• F1-Score: Harmonic mean of Precision and Recall (balanced metric)

  • Use when both are equally important

Use Cases:

• Spam detection: Precision matters (don't flag normal emails)
• Disease screening: Recall matters (don't miss patients)
• General classification: F1-Score balances both

1. Linear Kernel:

• Formula: $K(x, y) = x^T y$
• Decision Boundary: Line/Plane (linearly separable)
• Use Cases:
  • Data is linearly separable
  • Features >> Samples (high-dimensional sparse data)
  • Text classification (bag-of-words)
• Pros: Fast, interpretable
• Cons: Can't handle non-linear problems

2. RBF Kernel (Radial Basis Function, Gaussian):

• Formula: $K(x, y) = \exp(-\gamma ||x - y||^2)$
• Decision Boundary: Complex curves/surfaces (non-linear)
• Use Cases:
  • Non-linearly separable data
  • Default choice when unsure
  • Moderate sample size
• Pros: Powerful, handles complex boundaries
• Cons: Prone to overfitting, needs γ tuning

3. Polynomial Kernel:

• Formula: $K(x, y) = (x^T y + c)^d$
• Decision Boundary: Polynomial curves
• Use Cases:
  • Image processing
  • Feature interactions matter
• Pros: Controllable complexity (via d)
• Cons: Computationally expensive, parameter sensitive

Selection Guide:

1. Try Linear first → Quick baseline
2. If performance insufficient, try RBF → Most common
3. Consider Polynomial for special cases

1. n_estimators (Number of Trees):

• Meaning: Number of decision trees in the forest
• Impact:
  • Increase → More stable, better performance (diminishing returns)
  • Too few → Underfitting
  • Too many → Longer training, but rarely overfits
• Recommended: 100-500 (depends on data size)

2. max_depth (Maximum Tree Depth):

• Meaning: Maximum levels allowed in each tree
• Impact:
  • Increase → More complex, may overfit
  • Decrease → Simpler, may underfit
• Recommended:
  • None (unlimited) → Small datasets
  • 10-30 → Large datasets

3. min_samples_split (Minimum Samples to Split):

• Meaning: Minimum samples required to split a node
• Impact:
  • Increase → Simpler tree, prevents overfitting
  • Decrease → More complex, may overfit
• Recommended: 2-20

Tuning Strategy (GridSearchCV):

from sklearn.ensemble import RandomForestClassifier
from sklearn.model_selection import GridSearchCV
 
param_grid = {
    'n_estimators': [100, 200, 300],
    'max_depth': [10, 20, 30, None],
    'min_samples_split': [2, 5, 10]
}
 
rf = RandomForestClassifier(random_state=42)
grid_search = GridSearchCV(rf, param_grid, cv=5, scoring='accuracy')
# grid_search.fit(X_train, y_train)

K-Fold Cross-Validation Principle:

1. Split Data: Divide dataset into K equal-sized subsets (folds)
2. Rotate Training: Use K-1 subsets for training, 1 for validation
3. Repeat K Times: Each subset gets a chance to be the validation set
4. Average Results: Average the K validation results for final performance

Why Important:

• ✅ Maximize Data Usage: Every sample used for both training and validation
• ✅ Reduce Overfitting Risk: Avoid model performing well only on specific test set
• ✅ More Reliable Estimate: Average of multiple validations is more stable
• ✅ Good for Small Datasets: Especially useful when data is limited

Common K Values:

• K = 5 (5-fold): Balance between speed and accuracy, most common
• K = 10 (10-fold): More accurate but more computational cost
• K = N (Leave-One-Out, LOOCV): Extreme case, one sample for validation each time

Interpretation:

• 5 validation accuracies: 85%, 90%, 80%, 95%, 85%
• Mean accuracy: 87%
• 2x standard deviation (95% confidence): ±10.8%
• Conclusion: Model accuracy ~87% on this dataset, relatively stable

How to Identify:

from sklearn.model_selection import learning_curve
import matplotlib.pyplot as plt
 
train_sizes, train_scores, test_scores = learning_curve(
    model, X, y, cv=5, 
    train_sizes=np.linspace(0.1, 1.0, 10)
)
 
# Plot learning curves
plt.plot(train_sizes, train_scores.mean(axis=1), label='Training score')
plt.plot(train_sizes, test_scores.mean(axis=1), label='Test score')
plt.legend()

Diagnosis:

• Overfitting: Large gap between training and test curves
• Underfitting: Both curves low and close together

Solutions:

2. Reduce model complexity (lower tree depth, fewer features)
3. Regularization (L1/L2)
4. Dropout (neural networks)
5. Early stopping | | Underfitting | 1. Increase model complexity
6. Add more features
7. Reduce regularization
8. Train longer
9. Use more powerful model |

SVM (Support Vector Machine):

• Core Idea: Find an optimal hyperplane (line in 2D, surface in higher dimensions) that maximizes the margin between two classes
• Decision Basis: Depends on support vectors (data points closest to the hyperplane)
• Characteristics:
  • Works well with high-dimensional data and few features
  • High computational complexity (O(n²) to O(n³))
  • Requires feature normalization

Random Forest:

• Core Idea: Build multiple decision trees and use majority voting for prediction
• Decision Basis: Each tree selects split points based on feature impurity (Gini Index or Entropy)
• Characteristics:
  • Fast with parallel computing
  • Naturally handles non-linear relationships
  • Auto-normalization, easy to use
  • Handles many features and missing values well

Complete Code:

from sklearn.model_selection import train_test_split
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.25, random_state=42)
 
from sklearn.neighbors import KNeighborsClassifier
 
# [A] Instantiate model with K=7
knn = KNeighborsClassifier(n_neighbors=7)
 
# [B] Fit the model on training data
knn.fit(X_train, y_train)
 
# [C] Predict on test set
y_pred = knn.predict(X_test)
 
# Output first prediction
print(classes[y_pred[0]])

Detailed Explanation:

[A] Instantiate Model:

knn = KNeighborsClassifier(n_neighbors=7)

• KNeighborsClassifier is sklearn's KNN classifier class
• n_neighbors=7 sets K value to 7 (select 7 nearest neighbors)
• Other common parameters:
  • metric='euclidean' (default, Euclidean distance)
  • weights='uniform' (default, all neighbors equal weight)
  • weights='distance' (distance-based weighting, closer neighbors have more weight)

[B] Train Model:

knn.fit(X_train, y_train)

• fit() method trains the model
• X_train is feature matrix (shape: [samples, features])
• y_train is label vector (shape: [samples])
• Note: KNN doesn't actually "train" - it just stores training data, calculates distances during prediction

[C] Predict Results:

y_pred = knn.predict(X_test)

• predict() method predicts on test set
• X_test is test feature matrix
• y_pred is prediction vector (shape: [test samples])
• Prediction process for each sample:
  1. Calculate distance to all training samples
  2. Select K nearest neighbors
  3. Count class labels of these K neighbors, take majority vote

Complete Workflow Example:

import numpy as np
from sklearn.model_selection import train_test_split
from sklearn.neighbors import KNeighborsClassifier
from sklearn.metrics import accuracy_score, classification_report
 
# Sample data
X = np.array([[1, 2], [2, 3], [3, 4], [6, 7], [7, 8], [8, 9]])
y = np.array([0, 0, 0, 1, 1, 1])  # 0 = Not Purchase, 1 = Purchase
classes = ['Not Purchase', 'Purchase']
 
# Split data
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.25, random_state=42)
 
# [A] Instantiate
knn = KNeighborsClassifier(n_neighbors=3)  # or K=7 per requirements
 
# [B] Train
knn.fit(X_train, y_train)
 
# [C] Predict
y_pred = knn.predict(X_test)
 
# Output
print("Predictions:", y_pred)
print("First prediction:", classes[y_pred[0]])
 
# Evaluate
accuracy = accuracy_score(y_test, y_pred)
print(f"Accuracy: {accuracy:.2f}")
print("\nClassification Report:")
print(classification_report(y_test, y_pred, target_names=classes))

Key Concepts Summary:

1. sklearn standard workflow: import → instantiate → fit → predict
2. Parameter understanding: n_neighbors (K value), metric (distance measure), weights (weighting strategy)
3. Method calls: fit(X_train, y_train) for training, predict(X_test) for prediction
4. Data shapes: X must be 2D array (samples × features), y is 1D array (samples)

Common Mistakes:

• ❌ Forgetting to import KNeighborsClassifier
• ❌ Misspelling n_neighbors (not k=7)
• ❌ Wrong parameter order in fit() (should be X_train, y_train, not reversed)
• ❌ Forgetting to pass X_test to predict()

Effect of K on Decision Boundary:

Small K (e.g., K=1):

• Decision boundary is very wiggly and complex
• Model fits training data too tightly, prone to overfitting
• Very sensitive to noise
• High training accuracy, may have low test accuracy

Large K (e.g., K=100):

• Decision boundary becomes very smooth, even approaching a straight line
• Model too simple, prone to underfitting
• Not sensitive to noise
• Both training and test accuracy may be low

Optimal K Selection:

• Typically use cross-validation to find best K
• K value suggestion: 3-15 (small datasets) or $\sqrt{N}$ (N = number of samples)
• K should be odd (avoid ties, especially for binary classification)

Conclusion:

• As K increases, the decision boundary becomes smoother.

Step 1: Calculate Prior Probability

$$P(\text{Spam}) = \frac{4}{10} = 0.4$$

$$P(\text{Not Spam}) = \frac{6}{10} = 0.6$$

Step 2: Calculate Likelihood Probability

$$P(\text{"Free"} \mid \text{Spam}) = \frac{3}{4} = 0.75$$

$$P(\text{"Free"} \mid \text{Not Spam}) = \frac{1}{6} \approx 0.167$$

Step 3: Calculate Posterior Numerators

Using Bayes' theorem: $P(Spam | "Free") \propto P("Free" | Spam) \times P(Spam)$

$$\text{Spam Score} = P(\text{"Free"} \mid \text{Spam}) \times P(\text{Spam}) = 0.75 \times 0.4 = 0.30$$

$$\text{Not Spam Score} = P(\text{"Free"} \mid \text{Not Spam}) \times P(\text{Not Spam}) = 0.167 \times 0.6 \approx 0.10$$

Step 4: Make Decision

Since $0.30 > 0.10$, the model predicts this email is Spam.

Conclusion: Emails containing "Free" are more likely to be spam.

Step 1: Calculate Sample Proportions

$$p_{\text{Positive}} = \frac{6}{8} = 0.75$$

$$p_{\text{Negative}} = \frac{2}{8} = 0.25$$

Step 2: Substitute into Entropy Formula

$$\text{Entropy} = -[p_+ \log_2(p_+) + p_- \log_2(p_-)]$$

$$= -[0.75 \times \log_2(0.75) + 0.25 \times \log_2(0.25)]$$

Step 3: Use Calculator to Calculate Logarithms (Keep 4 decimals)

$$\log_2(0.75) = \frac{\log(0.75)}{\log(2)} \approx \frac{-0.1249}{0.3010} \approx -0.4150$$

$$\log_2(0.25) = \log_2(\frac{1}{4}) = -\log_2(4) = -2$$

Step 4: Calculate Final Result

Term 1: $0.75 \times (-0.4150) = -0.3112$

Term 2: $0.25 \times (-2) = -0.50$

Entropy Value:

$$\text{Entropy} = -(-0.3112 - 0.50) = -(-0.8112) = 0.8112 \approx \boxed{0.811}$$

Interpretation:

• Entropy range is 0 to 1 (for binary classification)
• Entropy 0.811 indicates a highly mixed node (close to 75%-25% distribution)
• Pure node (all positive or all negative) has entropy 0
• Node with 50%-50% split has maximum entropy of 1

Step 1: Calculate Root Entropy

Total samples: 10, where Yes = 6, No = 4

$$p_{\text{Yes}} = \frac{6}{10} = 0.6, \quad p_{\text{No}} = \frac{4}{10} = 0.4$$

$$H_{\text{total}} = -[0.6 \times \log_2(0.6) + 0.4 \times \log_2(0.4)]$$

Using calculator:

• $\log_2(0.6) = \frac{\ln(0.6)}{\ln(2)} \approx -0.737$
• $\log_2(0.4) = \frac{\ln(0.4)}{\ln(2)} \approx -1.322$

$$H_{\text{total}} = -[0.6 \times (-0.737) + 0.4 \times (-1.322)]$$ $$= -[-0.442 - 0.529] = -(-0.971) = \boxed{0.971}$$

Step 2: Calculate Weighted Entropy by Outlook Groups

Entropy of Sunny: $$H_{\text{Sunny}} = -\left[\frac{1}{4} \log_2\left(\frac{1}{4}\right) + \frac{3}{4} \log_2\left(\frac{3}{4}\right)\right]$$

• $\log_2(0.25) = -2$
• $\log_2(0.75) \approx -0.415$

$$H_{\text{Sunny}} = -[0.25 \times (-2) + 0.75 \times (-0.415)]$$ $$= -[-0.5 - 0.311] = 0.811$$

Entropy of Overcast: All Yes (pure node) $$H_{\text{Overcast}} = 0$$

Entropy of Rain: $$H_{\text{Rain}} = -\left[\frac{3}{4} \log_2\left(\frac{3}{4}\right) + \frac{1}{4} \log_2\left(\frac{1}{4}\right)\right]$$ $$= -[0.75 \times (-0.415) + 0.25 \times (-2)]$$ $$= -[-0.311 - 0.5] = 0.811$$

Weighted Average Entropy: $$H_{\text{weighted}} = \frac{4}{10} \times 0.811 + \frac{2}{10} \times 0 + \frac{4}{10} \times 0.811$$ $$= 0.324 + 0 + 0.324 = 0.648$$

Step 3: Calculate Information Gain

$$\text{IG(Outlook)} = H_{\text{total}} - H_{\text{weighted}}$$ $$= 0.971 - 0.648 = \boxed{0.323}$$

Interpretation:

• Information Gain = 0.323 means "Outlook" reduces uncertainty by 32.3%
• Higher Information Gain → stronger classification ability
• Decision trees prioritize features with highest Information Gain for splitting

Step 1: Calculate Class Probabilities

$$p_A = \frac{25}{40} = 0.625$$ $$p_B = \frac{15}{40} = 0.375$$

Step 2: Substitute into Gini Formula

$$\text{Gini} = 1 - (p_A^2 + p_B^2)$$ $$= 1 - (0.625^2 + 0.375^2)$$ $$= 1 - (0.3906 + 0.1406)$$ $$= 1 - 0.5312$$ $$= \boxed{0.4688} \approx 0.469$$

Interpretation:

• Gini range: [0, 0.5] (for binary classification)
• Gini = 0 → Pure node (all same class)
• Gini = 0.5 → Most impure (uniform distribution)
• Gini = 0.469 → High impurity, needs further splitting

Gini vs Entropy:

• Gini faster to compute (no logarithms)
• Entropy more sensitive to impurity
• Similar results in practice, Gini more common (sklearn default)

Step 1: Statistics from Training Data

• Play = Yes: 5 times
• Play = No: 3 times

Prior Probability: $$P(\text{Yes}) = \frac{5}{8} = 0.625$$ $$P(\text{No}) = \frac{3}{8} = 0.375$$

Step 2: Calculate Conditional Probabilities

For Yes class:

• Sunny & Yes: 0 times (out of 5 Yes)
• Cool & Yes: 2 times (out of 5 Yes)

$$P(\text{Sunny} | \text{Yes}) = \frac{0}{5} = 0$$ $$P(\text{Cool} | \text{Yes}) = \frac{2}{5} = 0.4$$

For No class:

• Sunny & No: 3 times (out of 3 No)
• Cool & No: 0 times (out of 3 No)

$$P(\text{Sunny} | \text{No}) = \frac{3}{3} = 1.0$$ $$P(\text{Cool} | \text{No}) = \frac{0}{3} = 0$$

Step 3: Apply Naive Bayes

$$P(\text{Yes} | \text{Sunny, Cool}) \propto P(\text{Sunny} | \text{Yes}) \times P(\text{Cool} | \text{Yes}) \times P(\text{Yes})$$ $$= 0 \times 0.4 \times 0.625 = 0$$

$$P(\text{No} | \text{Sunny, Cool}) \propto P(\text{Sunny} | \text{No}) \times P(\text{Cool} | \text{No}) \times P(\text{No})$$ $$= 1.0 \times 0 \times 0.375 = 0$$

Problem: Zero Probability Issue!

Both classes have probability 0, making classification impossible. This is because "Sunny + Cool" combination never appeared in training data.

Solution: Laplace Smoothing

Corrected formula: $$P(\text{Feature} | \text{Class}) = \frac{\text{count} + 1}{\text{total} + \text{num_features}}$$

After applying smoothing:

For Yes class (total features = 3: Sunny, Overcast, Rain): $$P(\text{Sunny} | \text{Yes}) = \frac{0 + 1}{5 + 3} = \frac{1}{8} = 0.125$$ $$P(\text{Cool} | \text{Yes}) = \frac{2 + 1}{5 + 3} = \frac{3}{8} = 0.375$$

For No class: $$P(\text{Sunny} | \text{No}) = \frac{3 + 1}{3 + 3} = \frac{4}{6} = 0.667$$ $$P(\text{Cool} | \text{No}) = \frac{0 + 1}{3 + 3} = \frac{1}{6} = 0.167$$

Recalculate:

$$\text{Yes Score} = 0.125 \times 0.375 \times 0.625 = 0.0293$$ $$\text{No Score} = 0.667 \times 0.167 \times 0.375 = 0.0418$$

Conclusion: Since $0.0418 > 0.0293$, prediction is No (Don't play).

Key Takeaways:

• Naive Bayes assumes feature independence (hence "Naive")
• Zero probability requires smoothing techniques
• Laplace smoothing is most commonly used