| Bayesian Theory |
Article Index for Bayesian |
Shopping Probability |
Website Links For Bayesian |
Information AboutBayesian Theory |
| CATEGORIES ABOUT BAYESIAN PROBABILITY | |
| bayesian statistics | |
| probability and statistics | |
| philosophy of mathematics | |
|
Bayesian probability is an interpretation of the probability calculus which holds that the concept of Probability can be defined as the degree to which a person (or community) believes that a Proposition is true. Bayesian theory also suggests that Bayes' Theorem can be used as a rule to infer or update the degree of belief in light of new information. HISTORY Bayesian theory and Bayesian probability are named after Thomas Bayes (1702 — 1761), who proved a special case of what is now called Bayes' Theorem . The term ''Bayesian'', however, came into use only around 1950 , and it is not clear that Bayes would have endorsed the very broad interpretation of probability that is associated with his name. Laplace proved a more general version of Bayes' theorem and used it to solve problems in celestial mechanics, medical statistics and, by some accounts, even Jurisprudence . Laplace, however, didn't consider this general theorem to be important for probability theory. He instead adhered to the Classical Definition Of Probability . The subjective theory of probability which interprets 'probability' as 'subjective degree of belief in a proposition' was proposed independently and at about the same time by Bruno De Finetti in Italy in ''Fondementi Logici dei Ragionamento Probabilistico'' (1930) and Frank Ramsey in Cambridge in ''The Foundations of Mathematics'' (1931).See p50-1, Gillies 2000 "The subjective theory of probability was discovered independently and at about the same time by Frank Ramsey in Cambridge and Bruno de Finetti in Italy." See Gillies' discussion for its explanation of how the wrong impression came about that Ramsey proposed it first. It was devised to solve the problems of the Classical Definition Of Probability and replace it. L. J. Savage expanded the idea in ''The Foundations of Statistics'' (1954). Formal attempts have been made to define and apply the intuitive notion of a "degree of belief". The most common interpretation is based on Betting : a degree of belief is reflected in the odds and stakes that the subject is willing to bet on the proposition at hand. However, the probability of the kind of spatio-temporally universal hypotheses that are so fundamental to science - such as Newton's law of inertia or his law of universal gravitation - poses a problem for the betting definition of 'degree of belief' on the ground that the only fair betting-quotient 'A betting quotient is the quantity p = k/(1+k), where k are the odds on a hypothesis you believe fair and will therefore be taken as your degree of belief it is true. 'p' is called the betting-quotient associated with the odds k. Odds can be recovered uniquely from betting-quotients by means of the reverse transformation k = p/(1-p).' Paraphrase of p76 Howson & Urbach 1993 on such universal hypothesis is zero, since a bet on its truth can never be won because its truth can never be decided. This problem of the Bayesian philosophy of probability becomes a fundamental problem for the Bayesian philosophy of science that scientific reasoning is subjective Bayesian probabilist, which thereby seeks to reduce scientific method to gambling, but some regard it as soluble.See Gillies 'Induction and Probability' Parkinson (ed) An Encyclopedia of Philosophy 1988; p263-4, Howson & Urbach 1989 But it is also noteworthy that by 1981 De Finetti himself came to reject the betting conception of probability.He said "...betting strictly speaking does not pertain to probability but to the Theory of Games". See "The role of 'Dutch Books' and 'Proper Scoring Rules' " in ''British Journal for the Philosophy of Science'' 32 1981 55-6.] On the Bayesian interpretation, the theorems of probability relate to the rationality of partial belief in the way that the theorems of logic are traditionally seen to relate to the rationality of full belief. The Bayesian approach has been explored by Harold Jeffreys , Richard T. Cox , Edwin Jaynes and I. J. Good . Other well-known proponents of Bayesian probability have included John Maynard Keynes and B.O. Koopman , and many philosophers of the 20th century. VARIETIES The terms ''subjective probability'', ''personal probability'', ''epistemic probability'' and ''logical probability'' describe some of the schools of thought which are customarily called "Bayesian". These overlap but there are differences of emphasis. Some of the people mentioned here would not call themselves Bayesians. Subjective Bayesian probability interprets 'probability' as 'the ''degree of belief'' (or ''strength of belief'') an individual has in the truth of a proposition', and is in that respect subjective. Some people who call themselves Bayesians do not accept this subjectivity. The chief exponents of this ''objectivist'' school were Edwin Thompson Jaynes and Harold Jeffreys . Perhaps the main objectivist Bayesian now living is James Berger of Duke University. Jose Bernardo and others accept some degree of subjectivity but believe a need exists for " Reference Priors " in many practical situations. Advocates of logical (or objective epistemic) probability, such as Harold Jeffreys , Rudolf Carnap , Richard Threlkeld Cox and E.T. Jaynes, hope to codify techniques whereby any two persons having the same information relevant to the truth of an uncertain proposition would calculate the same probability. Such probabilities are not relative to the person but to the epistemic situation, and thus lie somewhere between subjective and objective. However, the methods proposed are controversial. Critics challenge the claim that there are grounds for preferring one degree of belief over another in the absence of information about the facts to which those beliefs refer. Another problem is that the techniques developed so far are inadequate for dealing with realistic cases needed; see http://en.wikipedia.org/wiki/Principle_of_maximum_entropy RELATIONSHIP TO FREQUENCY PROBABILITY Bayesian probability - sometimes called ''credence'' (i.e. degree of belief) - contrasts with Frequency Probability , in which probability is derived from observed frequencies in defined distributions or proportions in populations. The theory of statistics and probability using Frequency Probability was developed by R.A. Fisher , Egon Pearson and Jerzy Neyman during the first half of the 20th century. A. N. Kolmogorov also used frequency probability to lay the mathematical foundation of probability in measure theory via the Lebesgue Integral in ''Foundations of the Theory of Probability'' (1933). Savage, Koopman, Abraham Wald and others have developed Bayesian probability since 1950. The difference between Bayesian and Frequentist interpretations of probability has important consequences in statistical practice. For example, when comparing two hypotheses using the same data, the theory of Hypothesis Test s, which is based on the frequency interpretation of probability, allows the rejection or non-rejection of one model/hypothesis (the 'null' Hypothesis ) based on the probability of mistakenly inferring that the data support the other model/hypothesis more. The probability of making such a mistake, called a Type I Error , requires the consideration of hypothetical data sets derived from the same data source that are more extreme than the data actually observed. This approach allows the inference that 'either the two hypotheses are different or the observed data are a misleading set'. In contrast, Bayesian methods condition on the data actually observed, and are therefore able to assign posterior probabilities to any number of hypotheses directly. The requirement to assign probabilities to the parameters of models representing each hypothesis is the cost of this more direct approach. APPLICATIONS Since the 1950s, Bayesian theory and Bayesian probability have been widely applied through Cox's Theorem , Jaynes' Principle Of Maximum Entropy and the Dutch Book Argument . In many applications, Bayesian methods are more general and appear to give better results than Frequency Probability . Bayes Factor s have also been applied with Occam's Razor . See Bayesian Inference and Bayes' Theorem for mathematical applications. Some regard the Scientific Method as an application of Bayesian probabilist inference because they claim Bayes's Theorem is explicitly or implicitly used to update the strength of prior scientific beliefs in the truth of Hypotheses in the light of new information from observation or Experiment . This is said to be done by the use of Bayes's Theorem to calculate a posterior probability using that evidence and is justified by the Principle of Conditionalisation that P'(h) = P(h/e), where P'(h) is the posterior probability of the hypothesis 'h' in the light of the evidence 'e', but which principle is denied by some See ''Updating Belief'', Chapter 6 of Howson & Urbach 1993, p99-114 and its references to the discussions of Bayesian Conditionalisation of Hacking 1967, Kyburg, Skyrms 1987 and Jeffrey 1965 etc. Adjusting original beliefs could mean (coming closer to) accepting or rejecting the original hypotheses. Bayesian techniques have recently been applied to filter Spam e-mail. A Bayesian spam filter uses a reference set of e-mails to define what is originally believed to be spam. After the reference has been defined, the filter then uses the characteristics in the reference to define new messages as either spam or legitimate e-mail. New e-mail messages act as new information, and if mistakes in the definitions of spam and legitimate e-mail are identified by the user, this new information updates the information in the original reference set of e-mails with the hope that future definitions are more accurate. See Bayesian Inference and Bayesian Filtering . PROBABILITIES OF PROBABILITIES One criticism levelled at the Bayesian probability interpretation has been that a single probability assignment cannot convey how well grounded the belief is—i.e., how much Evidence one has. Consider the following situations: #You have a box with white and black balls, but no knowledge as to the quantities #You have a box from which you have drawn ''n'' balls, half black and the rest white #You have a box and you know that there are the same number of white and black balls The Bayesian probability of ''the next ball drawn being black'' is 0.5 in all three cases. Keynes called this the problem of the "weight of evidence". One approach is to reflect difference in evidential support by assigning probabilities to these probabilities (so-called ''metaprobabilities'') in the following manner: :1. You have a box with white and black balls, but no knowledge as to the quantities ::Letting represent the statement that the probability of the next ball being black is , a Bayesian might assign a uniform Beta prior distribution: :: :: |
|
|