Programs

# Markov Chain in Python Tutorial

## Overview

Has it ever crossed your mind how expert meteorologists make a precise prediction of the weather or how Google ranks different web pages? How they make the fascinating python applications in real world. These calculations are complex and involve several variables that are dynamic and can be solved using probability estimates.

Here lies the idea of Markov Chains; there are individual states (say, the weather conditions) where each state can randomly change into other states (rainy day can change into the sunny day), and these changes or transitions are probability-based. This article gives a brief introduction to the concept of Markov Chains and how Python Markov Chain can be utilized to code Markov Chain models in Python to solve real-world problems. If you are a beginner and would like to gain expertise in data science, check out our data science courses.

## Content Overview

• A brief introduction to the concepts of Markov Chain and Markov Property
• Mathematical and graphical expression of Markov Chain
• Python Markov Chain – coding Markov Chain examples in Python

## Introduction to Markov Chain

To use Python Markov Chain for solving practical problems, it is essential to grasp the concept of Markov Chains. In 1906, Russian mathematician Andrei Markov gave the definition of a Markov Chain – a stochastic process consisting of random variables that transition from one particular state to the next, and these transitions are based on specific assumptions and probabilistic rules.

A fundamental mathematical property called the Markov Property is the basis of the transitions of the random variables. In other words, a Markov Chain is a series of variables X1, X2, X3,…that fulfill the Markov Property.

### Principle of Markov Chain – Markov Property

A Markov Chain is based on the Markov Property. The theory of discrete-time Markov Property states that the probability of a random system changing from one particular state to the next transition state depends only on the present state and time and is independent of the preceding states.

The fact that the probable future state of a random process is independent of the sequence of states that existed before it makes the Markov Chain a memory-less process that depends only on the current state of the variable.

### The mathematical expression of the Markov Chain

In terms of a probability distribution, assume a system at time instance ‘n.’ Applying the principle of Markov property, the conditional distribution of the states at the following time instance, n+1, is independent of the states of the system at time instances 1, 2, …, n-1.

### Graphical representation of Markov Chain

Directed graphs are often used to represent a Markov Chain. In the directed graphs, the nodes indicate different likely states of the random variables while the edges denote the probability of the system moving from one state to another in the next time instance. To understand the representation, let us take the example of predicting the weather. Assume that the random variable is ‘weather,’ and it has three possible states viz. Weather = {sunny, rainy, snowy}. The Markov Chain for this scenario can be represented as: Source

In the graphical representation shown above, say the current observed state of the random variable is sunny. The probability of the random variable taking the value sunny at the next time instance is 0.8. It can also take the value snowy with a probability of 0.01, or rainy with a probability of 0.19. An important thing to note here is that the probability values existing in a state will always sum up to 1.

### Coding a Markov Chain in Python

To better understand Python Markov Chain, let us go through an instance where an example of Markov Chain is coded in Python. While solving problems in the real world, it is common practice to use a library that encodes Markov Chains efficiently. However, coding Markov Chain in Python is an excellent way to get started on Markov Chain analysis and simulation. Hence comes the utility of Python Markov Chain. Let us see how the example of weather prediction given in the previous section can be coded in Python. Begin by defining a simple class: Having defined the MarkovChain class, let us try coding the weather prediction example as a representation of how Python Markov Chain works. Source

### Parameterising Markov Chains using Transition Matrix

In the previous section, the Python code parameterised the Markov Chain using a dictionary that contained the probability values of all the likely state transitions. An alternative way of representing the transition probabilities is using a transition matrix, which is a standard, compact, and tabular representation of a Markov Chain.

In situations where there are hundreds of states, the use of the Transition Matrix is more efficient than a dictionary implementation. The Markov Chain class is modified as follows for it to accept a transition matrix: The dictionary implementation was looping over the states names. However, in case of a Transition Matrix, the probability values in the next_state method can be obtained by using NumPy indexing: Source

## Conclusion

Markov Chains are an essential mathematical tool that helps to simplify the prediction of the future state of complex stochastic processes; it solely depends on the current state of the process and views the future as independent of the past. Utilising the Markov Property, Python Markov Chain coding is an efficient way to solve practical problems that involve complex systems and dynamic variables.

Be it weather forecasting, credit rating, or typing word prediction on your mobile phone, Markov Chains have far-fetched applications in a wide variety of disciplines. Depending on the nature of the parameters and the application, there are different concepts of Markov Chains. Python Markov Chain is a logical and efficient way to implement Markov Chains by coding them in Python.

If you are curious to learn about python, data science, check out IIIT-B & upGrad’s Executive PG Programme in Data Science which is created for working professionals and offers 10+ case studies & projects, practical hands-on workshops, mentorship with industry experts, 1-on-1 with industry mentors, 400+ hours of learning and job assistance with top firms.

## What is a Markov transition matrix?

The probability of particular states changing from one to the other is contained in a transition matrix, which is a square matrix. You may use a transition matrix to conduct matrix multiplication, identify patterns, and make predictions using it. In a dynamic system, a Markov transition matrix is a square matrix that describes the probability of transitioning from one state to another. The probability of migrating from the state represented by that row to the other states is listed in each row. As a result, each row of a Markov transition matrix adds up to one. Transition matrices are used to describe how transitions between two states are produced. When occurrences are more or less likely as a result of past events, it is used.

## What is the absorbing state of the Markov chain?

According to probability theory, an absorbing Markov chain is one in which every state can reach an absorbing state. An absorbing state is one that you can't get out of after you've entered it. A Markov chain is said to be absorbing if there is at least one absorbing state present in it and it is possible to get from any state to at least one absorbing state in a limited number of steps. A transitory state is one that is not absorbed by an absorbing Markov chain.

## What are Hidden Markov Models (HMM)?

The HMM is a mathematical model in which the examined system is a Markov process with hidden or unobserved states. The Hidden Markov Model is used in machine learning and pattern recognition applications such as gesture recognition and speech recognition. In the probabilistic model, the Hidden Markov Model allows us to speak about seen or apparent events as well as hidden events. It also aids in the resolution of real-world issues such as Natural Language Processing (NLP) issues, Time Series, and many more. In HMM, two key assumptions are made. The present observation and the future state are completely dependent on the current state. 