Information About

Boosting

There are several different boosting algorithms, depending on the exact mathematical form of the strength and weight. One of the most common boosting algorithms is AdaBoost . Most boosting algorithms fit into the AnyBoost framework, which shows that boosting performs Gradient Descent in Function Space .

Boosting is based on Probably Approximately Correct Learning (PAC learning), which is a branch of Computational Learning Theory .

Robert Schapire was the first to show that if a concept is Weakly PAC Learnable then it is also Strongly PAC Learnable using boosting.

Algorithmically, boosting is related to

Logistic Regression

Maximum Entropy Methods

Neural Network s

Support Vector Machine s.

REFERENCES

Robert E. Schapire and Yoram Singer. Improved Boosting Algorithms Using Confidence-Rated Predictors. Machine Learning, 37(3):297--336, 1999. http://citeseer.ist.psu.edu/schapire99improved.html

Robert E. Schapire. The Strength of Weak Learnability. Machine Learning, 5(2):197--227, 1990. http://citeseer.ist.psu.edu/schapire90strength.html

Llew Mason, Jonathan Baxter, Peter Bartlett, and Marcus Frean. Functional gradient techniques for combining hypotheses. Advances in Large Margin Classifiers, MIT Press, 1999. http://www.lsmason.com/papers/LMC-DOOMII.pdf

SEE ALSO

Bootstrap Aggregating

EXTERNAL LINKS

http://www.boosting.org/

[http://citeseer.ist.psu.edu/489339.html The boosting approach to machine learning: An overview]

The strength of weak learnability