Explore Courses

Decision Tree Classification: Everything You Need to Know

Updated on 24 November, 2022

6.3K+ views

Introduction

Many analogies could be driven from nature into our real lives; trees happen to be one of the most influential of them. Trees have made their impact on a considerable area of machine learning. They cover both the essential classification and regression. When analyzing any decision, a decision tree classifier could be employed to represent the process of decision making.

So, basically, a decision tree happens to be a part of supervised machine learning where the processing of data happens by splitting the data continuously, all the while keeping in mind a particular parameter.

What are decision trees made of?

The answer to the question is straightforward. Decision trees are made of three essential things, the analogy to each one of them could be drawn to a real-life tree. All three of them are listed below:

1. Nodes: This is the place where the testing for the value happens. In a node, the value of a specific attribute is passed, and it’s checked and tested against the values to make a decision.
2. Edges/Branches: This portion corresponds to the outcome of any test result. Edges/Branches are also responsible for linking two different nodes or leaves.
3. Leaf Nodes: These are the nodes that are typically found at the terminals. Leaf nodes are responsible for predicting the outcome.

In-demand Machine Learning Skills

Get Machine Learning Certification from the World’s top Universities. Earn Masters, Executive PGP, or Advanced Certificate Programs to fast-track your career.

Decision tree classification

The decision trees can be broadly classified into two categories, namely, Classification trees and Regression trees.

1. Classification trees

Classification trees are those types of decision trees which are based on answering the “Yes” or “No” questions and using this information to come to a decision. So, a tree, which determines whether a person is fit or unfit by asking a bunch of related questions and using the answers to come to a viable solution, is a type of classification tree.

These types of trees are usually constructed by employing a process which is called binary recursive partitioning. The method of binary recursive partitioning involves splitting the data into separate modules or partitions, and then these partitions are further spliced into every branch of the decision tree classifier.

2. Regression Trees

Now, a regression type of decision tree is different from the classification type of decision tree in one aspect. The data that has been fed into the two trees are very different. The classification trees handle the data, which is discreet, while the regression decision trees handle the continuous data type. A good example of regression trees would be the house price or how long a patient will typically stay in the hospital.

How are the decision trees created?

Decision trees are created by taking the set of data that the model has to be trained on (decision trees are a part of supervised machine learning). This training dataset is to be continuously spliced into smaller data subsets. This process is complemented by the creation of an association tree that incrementally gets created side by side in the process of breaking down the data. After the machine has finished learning, the creation of a decision tree based on the training dataset that has been provided concludes, and this tree is then returned to the user.

The central idea behind using a decision tree is to separate the data into two primary regions, the region with the dense population (cluster) or the area, which are empty (or sparse) regions.

Decision Tree classification works on an elementary principle of the divide. It conquers where any new example which has been fed into the tree, after going through a series of tests, would be organized and given a class label. The algorithm of divide and conquer is discussed in details below:

Divide and conquer

It is apparent that the decision tree classifier is based and built by making use of a heuristic known as recursive partitioning, also known as the divide and conquer algorithm. It breaks down the data into smaller sets and continues to do so. Until it has determined that the data within each subset is homogenous, or if the user has defined another stopping criterion, that would put a stop to this algorithm.

How does the decision tree classifier work?

1. The divide and conquer algorithm is used to create a decision tree classifier. By making the use of the algorithm we always begin at the root of the tree and we also split the dataset to reduce the uncertainty in the final decision.
2. It happens to be an iterative process. So, we repeat this process at every node. This process is repeated until the time we don’t have the nodes of the purity we desire.
3. Generally, to avoid overfitting we set a limit of purity to be achieved. This means the final result might not be 100% pure.

Basics of the divide and conquer algorithm:

1. First comes choosing or selecting a test for the root node. Then begins the process of creating branches. The branches are designed with keeping in mind each possible outcome of the trial that has been defined.
2. Next comes the splitting of the instances of data into smaller subsets. Each branch would have its own splice, which is connected to the node.
3. This process then has to be repeated for each branch by using just the instances which come to the branch in question.
4. This recursive process should be stopped if all the instances belong to the same class.

Advantages of using decision tree classification

1. It does not require a tremendous amount of money to construct.
2. It is a swift process of classification of records that are new or unknown.
3. It can be very easily interpreted, especially if the tree is small in size.
4. The accuracy of prediction using the decision tree classifier is comparable to other methods of prediction or classification.
5. It also has the capability to exclude the features which are unimportant. This process of eliminating irrelevant features is done automatically.

Disadvantages of using the decision tree classifier

1. Overfitting the dataset is very easy in this case.
2. The boundary of the decision has a restriction. It can only be parallel to the axes, which contains the attributes.
3. Models based on decision trees often have biased-on splits that have a massive number of levels.
4. Any small changes made on the dataset can have a significant impact on the logic which governs the decision.
5. Lager trees are challenging to understand because sometimes they might feel very counter-intuitive.

Also read: Decision Trees in Machine Learning

Learn data science courses from the World’s top Universities. Earn Executive PG Programs, Advanced Certificate Programs, or Masters Programs to fast-track your career.

Conclusion

Decision trees come in handy while we are faced with problems that cannot be handled with linear solutions. From observations, it has been noted that tree-based models can easily map the non-linearity of the inputs and effectively eliminate the problem at hand. Sophisticated methods like random forest generation and gradient boosting are all based on the decision tree classifier itself.

Decision trees are a potent tool which can be used in many areas of real life such as, Biomedical Engineering, astronomy, system control, medicines, physics, etc. This effectively makes decision tree classification a critical and indispensable tool of machine learning.

1. Are Decision Trees inclined to overfit?

Decision Trees fragment the complex data into simpler forms. A Decision Tree classification tries to divide data until it can’t be further divided. A clear chart of all the possible contents is then created, which helps in further analysis. While a vast tree with numerous splices gives us a straight path, it can also generate a problem when testing the data. This excessive splicing leads to overfitting, wherein many divisions cause the tree to grow tremendously. In such cases, the predictive ability of the Decision Tree is compromised, and hence it becomes unsound. Pruning is a technique used to deal with overfitting, where the excessive subsets are removed.

2. Do Decision Trees need normalisation?

Decision Trees are the most common machine learning algorithm used for the classification and regression of data. This supervised mechanism splices data per subset into various groups until it reaches the leaf node, where it cannot be divided further. Since this data will be split into categories based on the provided attributes, it will be evenly split. It conveys that both data that went through normalisation and data that didn’t would have the same number of splits. Therefore, normalisation is not a prerequisite for decision-based tree models.

3. How to splice Decision Trees?

Decision Trees are a reliable mechanism to classify data and predict solutions. Splicing in a Decision Tree requires precision; one slight mistake can compromise the Decision Tree’s integrity. Splicing in a Decision Tree occurs using recursive partitioning. Splitting data starts with making subsets of data through the attributes assigned to it. The data is split recursively in repetition until the spliced data at each node is deemed obsolete in predicting solutions. The subset can be similar to the value of the target variable as well. Splicing has to be methodical and repetitive for good accuracy.

Director of Engineering @ upGrad. Motivated to leverage technology to solve problems. Seasoned leader for startups and fast moving orgs. Working on solving problems of scale and long term technology strategy.

See More

SUGGESTED BLOGS

898.89K+

Technology will surely kill some jobs, but not all of them

07 Jul'16

5.2K+

Keep an Eye Out for the Next Big Thing: Machine Learning

17 Oct'17

7.87K+

The Difference between Data Science, Machine Learning and Big Data!

02 Nov'17

6.14K+

Natural Language Generation: Top Things You Need to Know

30 Jan'18

8.3K+

A Beginner&#8217;s Guide To Natural Language Understanding

30 Jan'18

10.99K+

Neural Networks for Dummies: A Comprehensive Guide

06 Feb'18

20.1K+

Neural Networks: Applications in the Real World

06 Feb'18

5.81K+

Go and the Challenge to Artificial General Intelligence

14 Feb'18

11.51K+

Sentiment Analysis: What is it and Why Does it Matter?

21 Feb'18

SUPPORT

MBA

DATA SCIENCE & ANALYTICS

DOCTORATE

SOFTWARE & TECH

AI & ML

MARKETING

MANAGEMENT

LAW