Importance Sampling Website Links For
Importance 		 

Information About

Importance Sampling
  CATEGORIES ABOUT IMPORTANCE SAMPLING
   
monte carlo methods
   
APPAREL
BABY
BEAUTY
BOOKS
CAR TOYS
CELL PHONES
DVD'S
ELECTRONICS
GOURMET FOOD
GROCERIES
HEALTH & PERSONAL
HOME & GARDEN
JEWELRY
MUSIC
MUSIC INSTRUMENTS
OFFICE PRODUCTS
SOFTWARE
SPORTING GOODS
TOOLS & HARDWARE
TOYS
VIDEO GAMES
SHOPPING HOME
MORE SHOPPING...



The fundamental issue in implementing IS simulation is the choice of the biased distribution which encourages the important regions of the input variables. Choosing or designing a good biased distribution is the "art" of IS. The rewards for a good distribution can be huge run-time savings; the penalty for a bad distribution can be longer run times than for a general Monte Carlo simulation without any special techniques.


MATHEMATICAL APPROACH


Consider estimating by simulation the probability p_t\, of an event { X \ge t\ }, where X is a random variable with Distribution F and Probability Density Function f(x)= F'(x)\,, where prime denotes Derivative . A K-length Independent And Identically Distributed (i.i.d.) sequence X_i\, is generated from the distribution F, and the number k_t of random variables that lie above the threshold t are counted. The random variable k_t is characterized by the Binomial Distribution

:P(k_t = k)={K\choose k}p_t^k(1-p_t)^{K-k},\,\quad \quad k=0,1,\dots,K.

  • \,(for X), usually referred to as a biasing density, for the simulation experiment. This density allows the event { X \ge t\ } to occur more frequently, so the sequence lengths K gets smaller for a given Estimator variance. Alternatively, for a given K, use of the biasing density results in a variance smaller than that of [the conventional Monte Carlo estimate. From the definition of p_t\,, we can introduce f_---\, as below.



:p_t = {E} \ge t

  • (x)} f_---(x) \,dx




where

  • (\cdot)}



is a likelihood ratio and is referred to as the weighting function. The last equality in the above equation motivates the estimator




  • \, and for each sample which exceeds t\,, the estimate is incremented by the weight W\, evaluated at the sample value. The results are averaged over K\, trials. The variance of the IS estimator is easily shown to be




  • }[(X" class="copylinks" target="_blank">\ge t)^2 W^2(X) - p_t^2 \Big]


::= rac{1}{K}\Big \ge t)^2 W(X) - p_t^2 \Big]

  • \, such that the variance of the IS estimator is less than the variance of the general Monte Carlo estimate. For some biasing density function, which minimizes the variance, and under certain conditions reduces it to zero, it is called an optimal biasing density function.



CONVENTIONAL BIASING METHODS


Although there are many kinds of biasing methods, following two methods are most widely used in the applications of IS.


Scaling


Shifting probability mass into the event region { X \ge t\ } by positive scaling of the random variable X\, with a number greater than unity has the effect of increasing the variance (mean also) of the density function. This results in a heavier tail of the density, leading to an increase in the event probability. Scaling is probably one of the earliest biasing methods known and has been extensively used in practice. It is simple to implement and usually provides conservative simulation gains as compared to other methods.

In IS by scaling, the simulation density is chosen as the density function of the scaled random variable aX\,, where usually a>1 for tail probability estimation. By transformation,

  • (x)= rac{1}{a} f \bigg( rac{x}{a} \bigg)\,



and the weighting function is

: W(x)= a rac{f(x)}{f(x/a)} \,

While scaling shifts probability mass into the desired event region, it also pushes mass into the complementary region X which is undesirable. If X\, is a sum of n\, random variables, the spreading of mass takes place in an n\, dimensional space. The consequence of this is a decreasing IS gain for increasing n\,, and is called the dimensionality effect.


Translation


Another simple and effective biasing technique employs translation of the density function (and hence random variable) to place much of its probability mass in the rare event region. Translation does not suffer from a dimensionality effect and has been successfully used in several applications relating to simulation of Digital Communication systems. It often provides better simulation gains than scaling. In biasing by translation, the simulation density is given by

  • (x)= f(x-c), \quad c>0 \,



where c\, is the amount of shift and is to be chosen to minimize the variance of the IS estimator.


EFFECTS OF SYSTEM COMPLEXITY


The fundamental problem with IS is that designing good biased distributions becomes more complicated as the system complexity increases. Complex systems are the systems with long memory since complex processing of a few inputs is much easier to handle. This dimensionality or memory can cause problems in three ways:



In principle, the IS ideas remain the same in these situations, but the design becomes much harder. A successful approach to combat this problem is essentially breaking down a simulation into several smaller, more sharply defined subproblems. Then IS strategies are used to target each of the simpler subproblems. Examples of techniques to break the simulation down are conditioning and error-event simulation(EES) and regenerative simulation.


EVALUATION OF IS


In order to identify successful IS techniques, it is useful to be able to quantify the run-time savings due to the use of the IS approach. The performance measure commonly used is \sigma^2_{MC} / \sigma^2_{IS} \,, and this can be interpreted as the speed-up factor by which the IS estimator achieves the same precision as the MC estimator. This has to be computed empirically since the estimator variances are not likely to be analytically possible when their mean is intractable. Other useful concepts in quantifying an IS estimator are the variance bounds and the notion of asymptotic efficiency.


VARIANCE COST FUNCTION


Variance is only one possible Cost Function for a simulation, and other cost functions, such as the mean absolute deviation, are used in various statistical applications. Nevertheless, the variance is the primary cost function addressed in the literature, probably due to the use of variances in Confidence Interval s and in the performance measure \sigma^2_{MC} / \sigma^2_{IS} \,.

An associated issue is the fact that the ratio \sigma^2_{MC} / \sigma^2_{IS} \, overestimates the run-time savings due to IS since it does not include the extra computing time required to compute the weight function. Hence, some people evaluate the net run-time improvement by various means. Perhaps a more serious overhead to IS is the time taken to devise and program the technique and analytically derive the desired weight function.


REFERENCES


  • P. J.Smith, M.Shafi, and H. Gao, “Quick simulation: A review of importance sampling techniques in communication systems,” IEEE J.Select.Areas Commun., vol. 15, pp. 597-613, May 1997.


  • M. Ferrari, S. Bellini, “Importance Sampling simulation of turbo product codes,” ICC2001, The IEEE International Conference on Communications, vol. 9, pp. 2773-2777, June 2001.


  • Tommy Oberg, Modulation, Detection, and Coding, John Wiley & Sons, Inc., New York, 2001.


  • R. Srinivasan., Importance Sampling. New York: Springer, 2002.




SEE ALSO





EXTERNAL LINKS






EXTERNAL REFERENCES

There are 2 recently published monographs on the theory and applications of importance sampling from which interested readers can obtain more information. These are:

1) "Importance sampling - Applications in communications and detection", Rajan Srinivasan, Springer-Verlag, Berlin, 2002.

2) "Introduction to rare event simulation", James Anotonio Bucklew, Springer-Verlag, New York, 2004.