Trending September 2023 # Types Of Boosting Algorithm With Their Working # Suggested October 2023 # Top 16 Popular

You are reading the article Types Of Boosting Algorithm With Their Working updated in September 2023 on the website Phuhoabeautyspa.com. We hope that the information we have shared is helpful to you. If you find the content interesting and meaningful, please share it with your friends and continue to follow and support us for the latest updates. Suggested October 2023 Types Of Boosting Algorithm With Their Working

What is the Boosting Algorithm?

Boosting algorithms are the special algorithms that are used to augment the existing result of the data model and help to fix the errors. They use the concept of the weak learner and strong learner conversation through the weighted average values and higher votes values for prediction. These algorithms uses decision stamp, margin maximizing classification for processing. There are three types of Algorithms available such as AdaBoost or Adaptive boosting Algorithm, Gradient, and XG Boosting algorithm. These are the machine learning algorithms that follow the process of training for predicting and fine-tuning the result.

Start Your Free Data Science Course

Hadoop, Data Science, Statistics & others

Let’s understand this concept with the help of the following example. Let’s take the example of the email. How will you recognize your email, whether it is spam or not? You can recognize it by the following conditions:

If an email contains lots of source like that means it is spam.

If an email contains only one file image, then it is spam.

If an email contains the message of “You Own a lottery of $xxxxx”, that means it is spam.

If an email contains some known source, then it is not spam.

If it contains the official domain like chúng tôi etc., that means it is not spam.

The above-mentioned rules are not that powerful to recognize the spam or not; hence these rules are called as weak learners.

To convert weak learner to strong learner, combine the prediction of the weak learner using the following methods.

Using average or weighted average.

Consider prediction has a higher vote.

Consider the above-mentioned 5 rules; there are 3 votes for spam and 2 votes for not spam. Since there is high vote spam, we consider it as spam.

How Boosting Algorithms Works?

Boosting Algorithms combines each weak learner to create one strong prediction rule. To identify the weak rule, there is a base Learning algorithm (Machine Learning). Whenever the Base algorithm is applied, it creates new prediction rules using the iteration process. After some iteration, it combines all weak rules to create one single prediction rule.

To choose the right distribution follows the below-mentioned steps:

Step 1: The base Learning algorithm combines each distribution and applies equal weight to each distribution.

Step 2: If any prediction occurs during the first base learning algorithm, then we pay high attention to that prediction error.

Step 3: Repeat step 2 until the limit of the Base Learning algorithm has been reached or high accuracy.

Types of Boosting Algorithm

Boosting algorithms uses different engines such as decision stamp, margin maximizing classification algorithm, etc. There are three types of Boosting Algorithms which are as follows:

AdaBoost (Adaptive Boosting) algorithm

Gradient Boosting algorithm

XG Boost algorithm

AdaBoost (Adaptive Boosting) Algorithm

To understand AdaBoost, please refer to the below image:

Box 1: In Box 1, for each data set, we assigned equal weights and to classify plus (+) and minus (-) sign, we apply decision stump D1, which creates a vertical line at the left-hand side of box 1. This line incorrectly predicted three-plus sign(+) as minus(-); hence we apply higher weights to these plus sign and apply another decision stump.

Box 2: In Box 2, the size of three incorrectly predicted plus signs (+) becomes bigger as compared to another. The second decision stump D2 at the right-hand side of the block predict this incorrectly predicted plus sign (+) sign as correct. But since misclassification error occurred because of the unequal weight with a minus sign (-), we assign a higher weight to a minus sign (-) and apply another decision stump.

Box 3: In Box three, because of misclassification error, three minus sign (-) has a high weight. here decision stump D3 is applied to predict this misclassification and correct it. This time to classify plus (+) and minus (-) sign horizontal line is created.

Box 4: In Box 4, decision stump D1, D2 and D3 are combined to create a new strong prediction.

Gradient Boosting Algorithm

Gradient boosting algorithm is a machine learning technique to define loss function and reduce it. It is used to solve problems of classification using prediction models. It involves the following steps:

1. Loss Function 2. Weak Learner

In gradient boosting, decision trees are used as a weak learner. A regression tree is used to give true values, which can be combined together to create correct predictions. Like in the AdaBoost algorithm, small trees with a single split are used, i.e. decision stump. Larger trees are used for large levels I,e 4-8 levels.

3. Additive Model

In this model, trees are added one at a time. existing trees remains the same. During the addition of trees, gradient descent is used to minimize the loss function.

XG Boost

XG Boost is short for Extreme Gradient Boosting. XG Boost is an upgraded implementation of the Gradient Boosting Algorithm, which is developed for high computational speed, scalability, and better performance.

XG Boost has various features, which are as follows:

Parallel Processing: XG Boost provides Parallel Processing for tree construction which uses CPU cores while training.

Cross-Validation: XG Boost enables users to run cross-validation of the boosting process at each iteration, making it easy to get the exact optimum number of boosting iterations in one run.

Cache Optimization: It provides Cache Optimization of the algorithms for higher execution speed.

Distributed Computing: For training large models, XG Boost allows Distributed Computing.