Source: http://onmyphd.com/?p=convolution

- Integrals
- Dirac delta

- What is a convolution?
- Graphical representation of a convolution
- 2D convolution
- Similarity with correlation
- Properties of a convolution

A convolution is a mathematical operation that represents a signal passing through a LTI (Linear and Time-Invariant) system or filter. If we have a signal $s(t)$ passing through a system with impulse response $h(t)$, the output is the convolution of $s(t)$ with $h(t)$. The convolution is simply the integral of the product of the two functions (in this example the functions are $s(t)$ and $h(t)$), where one is reversed. $$(s*h)(t) = \int_0^\infty s(\tau)h(t-\tau)d\tau$$ So why reverse one function? Consider the response of the system to an impulse at time $t_n$: it is $h(t - t_n)$, right? At $t = t_n$ you will have $h(0)$ and all other points of $h(t)$ will be shifted in the same way. At this point, we are convolving the systems's function with a dirac delta function: $$\delta(t-t_n)*h(t) = \int_0^\infty \delta(\tau-t_n)h(t-\tau)d\tau = h(t - t_n)$$ Only for $\tau = t_n$ the multiplication inside the integral is non-zero and it is equal to $h(t-\tau) = h(t-t_n)$. Now imagine a sequence of impulses at times $t_0$, $t_1$, and so on until $t_n$ that form a discrete signal. The impulse responses accumulate, and you will have the accumulation of all the impulse responses shifted by each impulse. $$\sum_i \int_0^\infty\delta(\tau-t_i)h(t-\tau)d\tau$$ we can put the sum inside the integral: $$ \int_0^\infty\left(\sum_i\delta(\tau-t_i)\right)h(t-\tau)d\tau$$ In the limit of a continuous signal (the combination of very close together impulses creates a signal), the summation will lead to signal $s(t)$: $$ \int_0^\infty s(\tau) h(t-\tau)d\tau$$

Graphically, we can reverse one of the functions and while sliding that function through the time axis, calculate the integral of their product for each time. Here you can try several combinations of signal and system and make your own example.

The slider controls the parameter time ($t$). You can see, for each time instance, the terms $h(t-\tau)$ and $s(\tau)$, their product and the result of the integration. You can also change the signal and impulse response with the buttons below.

Signal: | |||

System impulse response: |

The convolution can generalize to more than one dimension. Two-dimensional (2D) convolutions are also extremely useful, for example in image processing. An image is a 2D signal and can be the input to a 2D filter as well. If an image is a signal $s(x,y)$, where $(x,y)$ is the position of a pixel, and $h(x,y)$ is the kernel of a filter, the convolution is: $$ (s*h)(x,y) = \int_x \int_y s(\alpha,\beta)h(x-\alpha, y - \beta) \partial\beta \partial\alpha$$ For example, if the kernel is gaussian, the image will be smoothed because at each position $(x,y)$ we are integrating a small region of the image with an "average" filter.

Convolution and correlation are very alike. In fact, the only difference is that no function is reversed in the correlation: $$\rho_{s,h} = corr(s,h) = s(t)*h(-t) = \int_0^\infty s(\tau)h(\tau-t)d\tau$$

In what follows, I will call $s(t)$ and $h(t)$ as functions (because that is what they are). A convolution can be applied to any two functions of the same variable, for other purposes besides knowing the output of a system for a given input.

You can exchange functions: $$s*h = h*s$$ Exchange between signal and system in the plot above and see for yourself.

You can chain convolutions without any specific order: $$ f*(s*h) = (f*s)*h $$

The convolution of a sum is the same as the sum of convolutions: $$ h*(s + f) = h*s + h*f $$

A constant multiplied at any stage of the convolution will give the same result: $$\alpha (s * h) = (\alpha s) * h = s * (\alpha h)$$

The dirac delta (impulse) is to the convolution as 1 is to the multiplication. Convolving with the dirac delta returns the original function: $$ h * \delta = h $$

The conjugate of the convolution is the same as the convolution of the conjugates: $$ \overline{s*h} = \overline{s}*\overline{h} $$

The integral of the convolution is the product of the integrals of each function: $$ \int (s*h)(t)dt = \left(\int s(t)dt\right) \left(\int h(t)dt\right)$$

The derivative of the convolution is the same as the convolution of a function with the derivative of the other: $$ \frac{d s*h}{d t} = \frac{d s}{d t}*h = s*\frac{d h}{d t}$$

This one is a major property! **The Fourier transform of the convolution is the same as the product of the Fourier transform of each function**:
$$ \mathcal{F}\{s*h\} = \mathcal{F}\{s\}\mathcal{F}\{h\} $$
If we realize that the Fourier transform is an integral of the function, it is not surprising that from the Integration property this one can be obtained.