Processing math: 100%
Machine learning: Exercise: kernel smoothing

Kernel smoothing #

Suppose that we have a bi-infinite sequence {f(n)}nZ of real numbers, that is, a real number for each integer nZ. In fact, consider the following one: ,9,7,1,3,9,7,1,3,9,7,1,3,9,7,1,3,9,7,1,3,9,7,1,3, I would like to propose the following computation. We will form another bi-infinite sequence {h1(n)}nZ via the formula: h1(n)=13f(n1)+13f(n)+13f(n+1) Repeating the computation, this time replacing f(n) with h1(n), we get another bi-infinite sequence {h2(n)}nZ. Since we are not doing anything else, we can continue and get a sequence of bi-infinite sequences {hk}kN.

Question: What happens to the sequence {hk}kN as k? I am looking for an intuitive answer, no formal proof is required.
Solution
For simplicity, suppose that the sequence f(n) is periodic like above. Then one can prove that the sequences hk tend to the constant sequence whose value is the average value of f. On the other hand, if f(n) is unbounded, then the sequences hk may not tend to a limit.
Question: Describe what happens if instead we use a slightly different process, this time letting: h1(n)=15f(n2)+15f(n1)+15f(n)+15f(n+1)+15f(n+2) Or: h1(n)=14f(n1)+12f(n)+14f(n+1)
Solution
The limiting answer remains the same, although the rate of convergence may be different.

The above process is an example of a surprisingly useful construction. We will define and study it formally in class.