When you have features taking different range if values, you may have odd predictions.
To mitigate this, normalize the feature’s ranges to \([r_{low}, r_{high}]\):
$$ a'_i = \frac{a_i - a_{min}}{ a_{max} - a_{min}} \cdot (r_{high} - r_{low}) + r_{low} $$
Typically, the range is normalized to \([r_{low}, r_{high}] = [0, 1]\), so range normalization simplifies to:
$$ a'_i = \frac{a_i - a_{min}}{ a_{max} - a_{min}} $$
Range normalization is important in all ML algorithms. however, it may backfire if the smaller-range feature is noisy.
After normalization, the smaller-range feature will have a larger impact on the prediction. However, wouldn’t the same be said if the larger-range feature was noisier? I don’t see a pretty way out of this…
For example, if \(f_1 \in [0, 100]\) and \(f_2 \in [0, 1]\), \(f_1\) will always be penalized more than \(f_2\) when computing the distance.