Predicting Continuous Targets Using NN

Dated Oct 17, 2017; last modified on Sun, 14 Mar 2021

Return the Average Value

One possible solution is to return the average value in the neighborhood, i.e.

$$ \mathbb{M}_{k}(q) = \frac{1}{k} \sum_{i=1}^{k} t_i $$

We can improve this by using weighted \(k-NN\):

$$ \mathbb{M}_{k}(q) = \frac{ \sum_{i=1}^{k} \left( \frac{1}{dist(q, d_i)^2} \cdot t_i \right) }{ \sum_{i=1}^{k} \frac{1}{dist(q, d_i)^2} } $$

The formula looks new. However, if \(x_1\) is weighted by \(w_1\) and \(x_2\) by \(w_2\), then the weighted average is:

$$ \frac{w_1 x_1 + w_2 x_2 }{w_1 + w_2} $$

Usually, \(\sum w_i = 1\), so the denominator is frequently omitted.

Note that we don’t need to worry about \(k\) being odd, because ties don’t matter.