What is KNN?

K-Nearest Neighbors is a simple, easy-to-understand, versatile, and strong machine learning algorithm. It is used for both classification and regression problems. "Show me who your friends are, and I'll tell you who you are."

How KNN Works

1. Select K: Choose the number of neighbors (K).
2. Calculate Distance: Find the distance between the query point and all other points in the dataset (Euclidean, Manhattan).
3. Find Neighbors: Identify the K nearest neighbors.
4. Vote (Classification): Assign the class that is most common among the neighbors.
5. Average (Regression): Assign the average value of the neighbors.

Choosing K

• Small K: Sensitive to noise/outliers (Overfitting).
• Large K: Smoother decision boundary, but might miss local patterns (Underfitting).
• Rule of Thumb: K = sqrt(N), where N is the number of samples.

Pros & Cons

• Pros: Simple, No training phase (Lazy Learner).
• Cons: Computationally expensive for large datasets (needs to calculate distance to every point), Sensitive to scale (requires feature scaling).