Build Features and Distances
Recording a fruit’s length and weight gives a feature vector (length, weight). Features are observations used for a decision. Their choice and units affect the result.
Distance between two points
For A=(1,2) and B=(4,6), Euclidean distance measures a straight line; Manhattan distance sums absolute component differences.
import math
a, b = (1, 2), (4, 6)
print(math.sqrt(sum((x - y) ** 2 for x, y in zip(a, b))))
print(sum(abs(x - y) for x, y in zip(a, b)))
The outputs are 5.0 and 7. Neither is universally correct; choose a measure suited to the data and goal.
Units can dominate similarity
Lengths of 10–20 cm and weights of 100–500 g give weight a larger numerical range, potentially dominating distance. Standardization subtracts a feature’s mean and divides by its standard deviation. Fit these values on training data, then apply them unchanged to evaluation data.
Encoding unordered categories as 1, 2, and 3 invents numerical distances. If there is no meaningful order, consider another representation such as one-hot encoding.
More features do not always help. Consider noise, redundancy, cost, and sample size.
Check your understanding
Why is computing a mean from combined training and evaluation data before evaluating a model problematic?
Show explanation
The preprocessing has already used information about supposedly unseen data. This is data leakage. Treat fitted preprocessing as part of learning and fit it only on training data.