CS(5) Importance Sampling for Monte Carlo Estimation
Sampling from a distribution that matches the shape of the function
The problem
When we use Monte Carlo methods to estimate an integral, sometimes we ‘waste’ samples in regions where the function contributes very little to the total result. Imagine trying to estimate the area under a curve that has higher values in a small region and low values elsewhere.
If we pick points randomly across the entire region, many points will fall in areas where the function is close to zero, which does not help us much. To be more efficient, we want to focus on the parts of the curve that contribute the most — the parts with higher values.
Concept of Importance Sampling
In basic Monte Carlo integration, we estimate an integral by sampling points randomly from a uniform distribution. However, if the function we are integrating varies greatly over this interval, some areas might contribute more to the result than others. Importance Sampling is a technique that improves efficiency by focusing more samples on the ‘important’ parts of the function, leading to a more accurate estimate with fewer samples.
