RANDOM FOREST IN MACHINE LEARNING/PYTHON/ARTIFICIAL INTELLIGENCE

Random Forest

• Types of Ensemble Methods
• Assumptions of Random Forest
• Advantages of Random Forest
• Random Forest Algorithm Working
• Applications of Random Forest
• Random Forest Regression
• Advantages of Random Forest Regression
• Disadvantages of Random Forest Regression

Random forest stands as a well-known supervised machine learning algorithm that can address both classification and regression problems within ML. Operating on ensemble learning principles, it leverages the collective intelligence of multiple classifiers to tackle intricate problems. This method harnesses the strengths of various models, improving the overall performance of the learning system. In this blog, we learn about random forest algorithms.

By amalgamating predictions from multiple decision trees, the random forest algorithm effectively mitigates overfitting while enhancing accuracy. Individual decision trees inherently possess high variance. However, when these trees are integrated into a random forest machine-learning model in parallel, the resulting variance diminishes. This reduction occurs because each decision tree is trained on a specific sample of data, ensuring that the output relies not on a single tree but on multiple trees, thereby lowering the overall variance.

In random forest models, higher numbers of trees correspond to increased accuracy while concurrently preventing overfitting tendencies, establishing a robust and more reliable model.

Image source original

Real-World Example for Random Forest

Let’s suppose we go for a hiking trip and get lost in the dense forest. Now we need to identify the type of tree we are under to navigate back to safety. But we find there are so many different trees, in such cases Random Forest, can help us it works like a team of expert tree guides that can help us to go back to safety.

A random forest isn’t a single decision tree, but it is a collection of trees, like a whole forest of knowledge. In random forests each tree votes for the type of tree it thinks it is. The final classification is based on the majority vote – the most popular choice among the trees.

When a new tree (data point) comes along, it’s passed through each decision tree in the forest. Each tree then votes for the type of tree it thinks the current tree is. The final classification is based on the majority vote – the most popular choice among the trees.

The advantage of using this method is that if one tree gets confused by an oddity, the others can compensate. Let’s take an example that one tree predicts that the current tree has bumpy bark, and it gets fooled, but other trees also have diverse knowledge that can classify the current tree correctly based on other features.

Random Forest’s strength lies in its multitude of perspectives. It’s like having many experts and each expert has their way of analyzing the data. They collaborate to make a more robust and accurate prediction, just like you’d be more confident in our hike if multiple guides agreed on the type of tree.

Types of Ensemble Methods

There are many verity of ensemble learning methods; they are:

Bagging (Bootstrap Aggregating) – In this approach, training involves using multiple models on randomly selected subsets of the training data. Following this, predictions from each model are aggregated, typically through averaging.

Boosting – This technique works in a sequence similar to a series of models, where training occurs one after the other, with each subsequent model dealing with the errors of the previous one. In this method, forecasts are combined through a weighted voting system.

Stacking – In this approach, the output from one model serves as input features for another model. Ultimately, the final prediction is derived from the second-level model.

Assumptions of Random Forest

Several decision trees are combined by a random forest technique to jointly forecast a dataset's class. Although individual trees in a forest may make incorrect predictions, the ensemble method ensures that most trees make accurate predictions. Let us now examine the two main assumptions behind the Random Forest classifier:

• For the Random Forest classifier in machine learning to make accurate predictions, it requires genuine values within the feature variables of the dataset rather than arbitrary or guessed values.
• Only when there is little to no correlation between the predictions given by the various trees can the Random Forest classifier perform successfully.

Why should we use Random Forest?

Below are some reasons or points that explain why we should use the Random Forest algorithm in machine learning:

• It takes a little amount of time to train when compared to rest algorithms.
• Its accuracy for prediction is high, and also with big datasets, it runs more efficiently.
• Furthermore, the Random Forest classifier can sustain its accuracy even in scenarios where a substantial part of the data is missing.

Random Forest Algorithm working

The two main random forest operations are the building phase and the prediction phase. During the construction phase, the algorithm builds a large number of decision trees, typically expressed as N trees. Random selections are made from a portion of the training data and feature set to create each decision tree. During the prediction stage, the algorithm generates predictions for every data point by utilizing the group of decision trees that were built during the first phase. Typically, before making a final prediction, all of the trees' projections are averaged or voted on. This rigorous process ensures that the random forest model can generate trustworthy predictions and resist variations in datasets.

First Step: Choose K randomly chosen data points from the training set to get started.

Second Step: Using the selected data subsets, decision trees are constructed in the second stage.

Third Step:  choose the number of decision trees (N) that you want to construct.

Fourth Step: Then, carry out steps 1 and 2 one more time.

Fifth Step: Get the projections from each decision tree and place the new data points in the most well-liked category.

To understand the working algorithm much better let’s look at one example.

Consider a dataset that contains several images depicting different fruits. These images are used as input for a machine-learning model that is constructed using a random forest classification technique. Using this strategy, the dataset is divided into smaller chunks, each of which is subjected to independent decision trees for analysis. Every decision tree generates a forecast when it is trained. When further data is added to the model, the Random Forest classifier predicts the result based on the output of the majority of the decision trees. This is an example of how this algorithm works.

Image source original

Applications of Random Forest

Let’s look at some applications of random forest where it is mostly used:

• Banking – it is used in the banking sector very much especially in the section of loan to check and identify the risk associated with it.
• Medicine – Using this algorithm, it becomes possible to discern disease patterns and assess the associated risks.
• Land Use – with the help of this algorithm we can identify the areas which have similar land.
• Marking – marking trends can be identified using this algorithm.
• Predicting continuous numerical values – This method can be employed to forecast various numerical outcomes such as housing prices, stock values, or the lifetime value of customers.
• Identifying risk factors – Additionally, it can identify risk factors for diseases, financial downturns, or other adverse occurrences.
• Handling high-dimensional data – because it uses decision trees inside it, therefore, it can analyze datasets that have quite a large number of features as input.
• Capturing complex relationships – Moreover, it can capture intricate connections between input features and target variables, enabling the modeling of complex relationships within the data.

What is Random Forest Regression?

Using ensemble techniques, random forest regression is one machine learning technique that combines regression and classification. Several decision trees and a procedure called Bootstrap Aggregation, or "bagging," are used in this strategy. Rather than depending just on one decision tree, a random forest combines several of them to produce the desired result.

An essential aspect of random forests is their utilization of multiple decision trees, each serving as an independent learning model. Through the Bootstrap method, sample datasets are generated for each model by randomly selecting rows and features from the original dataset. Predicting outcomes using Random Forest regression entails following standard procedures akin to other machine learning methodologies.

• Initially, we must formulate a precise question or specify the required data and identify the source from which to obtain the necessary data.
• We need to convert the data into an accessible format if it is not. 
• It's essential to identify and document all noticeable anomalies and missing data points within the dataset, as addressing these issues may be crucial for data quality and analysis purposes.
• Now we need to create a machine-learning model.
• For machine learning to work properly we need to establish a baseline model that we want to achieve.
• Following the data preprocessing steps, the next phase involves training the machine learning model using the prepared dataset.
• After training is done, we need to check its performance in unseen data or test data.
• Subsequently, it's essential to assess and compare the performance metrics between the test data and the model's predicted data.
• If the model’s performance does not achieve our expectations, we can try to improve it using tuning the hyperparameters or modeling the data with other techniques.
• At last, we interpret the data we have gained and report accordingly.

Out-of-bag score in Random Forest

The Out-of-Bag (OOB) score, also known as the Bag score, serves as a validation technique predominantly employed in bagging algorithms to assess their performance. This method involves extracting a small portion of validation data from the main dataset. Predictions are made on this specific validation subset, and the outcomes are subsequently compared with other results.

One significant benefit of the Out-of-Bag (OOB) score is its ability to evaluate the bagging algorithm's performance without using separate validation data. As a result, the OOB score provides an accurate assessment of the bagging algorithm's genuine performance.

To calculate the Out-of-Bag (OOB) score for a specific Random Forest algorithm, it is very important to set the OOB_Score parameter to "True" in the algorithm settings. This allows the algorithm to efficiently calculate and use OOB scores to evaluate its performance.

Advantages of Random Forest Regression

• We can easily use and it is less prone to be sensitive towards the training dataset compared to the decision tree.  
• It is more accurate as compared to a decision tree because it uses multiple decision trees inside it.
• It can easily handle large and complex datasets which have far more features.
• It can also easily tackle missing data problems, outliers’ detection, and noisy features.

Disadvantages of Random Forest Regression

• It can be not easy to understand.
• Subject matter experts may need to be involved for the Random Forest approach to be implemented successfully. They are essential for selecting and modifying parameters such as the number of decision trees, the maximum depth per tree, and the number of features to be taken into account at each split. A few key choices must be made to optimize the algorithm's performance and ensure accurate forecasts.
• Processing large datasets can be computationally costly.
• Overfitting can be a concern for Random Forest models when they become overly complex or contain an excessive number of decision trees. As a result, the model can perform poorly on fresh, untested data and overfit the training set.

Summary

Random Forest regression emerges as a robust solution for both continuous and classification prediction tasks, offering distinct advantages over traditional decision trees. Its ability to manage high-dimensional data, capture intricate relationships, and mitigate overfitting has propelled its widespread adoption across various domains and applications. Within a Random Forest, each constituent tree contributes its "vote" towards determining the most prevalent class in classification tasks or providing a prediction in regression scenarios. However, there's a risk of overfitting when employing excessively deep Random Forests or dealing with large and intricate datasets. Additionally, compared to individual decision trees, algorithms for Random Forest may exhibit lower interpretability due to their ensemble nature.

Python Code

here is the random forest Python example code: -