Decision Tree RStudio

A Decision Tree is a machine learning model used for both classification and regression tasks. It works by splitting the data into subsets based on the value of input features. Each internal node represents a "decision" on a feature, each branch represents the outcome of the decision, and each leaf node represents a class label (classification) or a continuous value (regression). It's a tree-like structure where each decision leads to a final prediction.

Decision Trees are popular because they are simple to understand and interpret. They mimic human decision-making processes, making them intuitive. Additionally, they require little data preprocessing and can handle both numerical and categorical data. Decision Trees are versatile, capable of solving a variety of tasks with relatively low computational cost. Moreover, they can be visualised easily, which helps in understanding the model and making decisions based on the insights gained from the tree.

We start by installing and loading the necessary packages. These libraries provide essential tools for building, evaluating, and visualising decision trees. The rpart library is used for creating the decision tree model, rpart.plot for visualising the tree, caret for splitting the data, e1071 for additional functions, and ggplot2 for creating visualisations.

We load the Iris dataset, which contains measurements of iris flowers and their species. We split the dataset into training and testing sets to ensure that we have separate data for training the model and evaluating its performance. Setting a seed ensures that the data split is reproducible.

We initialise and train the decision tree model using the rpart function. The model is trained to classify the species of Iris flowers based on their features (sepal length, sepal width, petal length, and petal width). Plotting the model provides an overview of the decision tree's structure and the rules it has learned from the training data.

We use the trained model to make predictions on the test data and evaluate its performance using a confusion matrix. Visualising the confusion matrix helps us understand the model's performance across different classes. We calculate the accuracy to measure the overall correctness of the model and display with the confusion matrix for better understanding.

Pruning is a technique used to reduce the complexity of the decision tree and prevent overfitting. By removing nodes that provide little predictive power, we create a simpler and more generalisable model. We prune the tree based on the complexity parameter (CP) that minimises cross-validation error. Plotting the pruned model summary allows us to compare it with the original tree and understand the changes made.

In this guide, we walked through the steps of creating, evaluating, pruning, and visualising a decision tree model using R and the rpart library. We loaded the Iris dataset, split it into training and testing sets, created and trained the decision tree model, made predictions, evaluated the model's performance using its accuracy, pruned the tree to avoid overfitting, and visualised the final pruned tree.