Blog_Banner_Asset
    Homebreadcumb forward arrow iconBlogbreadcumb forward arrow iconData Sciencebreadcumb forward arrow iconRegular Expressions in Python [With Examples]: How to Implement?

Regular Expressions in Python [With Examples]: How to Implement?

Last updated:
29th Jan, 2021
Views
Read Time
7 Mins
share image icon
In this article
Chevron in toc
View All
Regular Expressions in Python [With Examples]: How to Implement?

While processing raw data from any source, extracting the right information is important so that meaningful insights can be obtained from the data. Sometimes it becomes difficult to take out the specific pattern from the data especially in the case of textual data.

The textual data consist of paragraphs of information collected via survey forms, scrapping websites, and other sources. The Channing of different string accessors with pandas functions or other custom functions can get the work done, but what if a more specific pattern needs to be obtained? Regular expressions do this job with ease.

What is a Regular Expression (RegEx)?

A regular expression is a representation of a set of characters for strings. It presents a generalized formula for a particular pattern in the strings which helps in segregating the right information from the pool of data. The expression usually consists of symbols or characters that help in forming the rule but, at first glance, it may seem weird and difficult to grasp. These symbols have associated meanings that are described here.

Learn data science courses from the World’s top Universities. Earn Executive PG Programs, Advanced Certificate Programs, or Masters Programs to fast-track your career.

Meta-characters in RegEx

  1. ‘.’: is a wildcard, matches a single character (any character, but just once)
  2. ^: denotes start of the string
  3. $: denotes the end of the string
  4. [ ]: matches one of the sets of characters within [ ]
  5. [a-z]: matches one of the range of characters a,b,…,z
  6. [^abc] : matches a character that is not a,b or c.
  7. a|b: matches either a or b, where a and b are strings
  8. () : provides scoping for operators
  9. \ : enables escape for special characters (\t, \n, \b, \.)
  10. \b: matches word boundary
  11. \d : any digit, equivalent to [0-9]
  12. \D: any non digit, equivalent to [^0-9]
  13. \s : any whitespace, equivalent to [ \t\n\r\f\v]
  14. \S : any non-whitespace, equivalent to [^\t\n\r\f\v]
  15. \w : any alphanumeric, equivalent to [a-zA-Z0-9_]
  16. \W : any non-alphanumeric, equivalent to [^a-zA-Z0-9_]
  17. ‘*’: matches zero or more occurrences
  18. ‘+’: matches one or more occurrences
  19. ‘?’: matches zero or one occurrence
  20. {n}: exactly n repetitions, n>=0
  21. {n,}: at least n repetitions
  22. {,n}: at most n repetitions
  23. {m,n}: at least m repetitions and at most n repetitions

Our learners also read: Top Python Free Courses

Examples to Understand The Workaround

Now that you are aware of the characters that make up a RegEx, let’s see how this works:

1. Email Filtering:

Suppose you want to filter out all the email ids from a long paragraph. The general format for an email is:

username@domain_name. <top_level_domain>

The username can be alphanumeric, and therefore, we can use \w to denote them but there is a possibility that the user creates an account as firstName.surname. To tackle this, we will escape the dot and create a set of characters. Next, domain_name should be only alphabetic and therefore, A-Za-z will denote that. The top-level domain is usually .com, .in, .org but depending on the use case, you can choose either the whole alphabet range or filter specific domains.

The regular expression of this will look like this:

^([a-zA-Z0-9_.]+)@([a-zA-Z0-9-]+)\.([a-zA-Z]{2,4})$

Here the start and end of the pattern are also declared as well the top-level domain can only contain 2-4 characters. The whole expression has 3 groups.

2. Dates Filtering:

The textual information you are extracting may contain the dates and no separate column is made available for you. The dates are an essential factor that helps in filtering data or time series analysis. A particular date takes the format of date/month/year, where date and month can interchange.

Also, months can be numeric as well as alphabets form and in alphabets either abbreviations or full names. It mainly depends on how many cases are present in our data and can only be achieved by hit and trial.

A simple RegEx that covers a variety of dates is shown below:

^(\d{1,2})[/-](\d{1,2})[/-](\d{2,4})$

This pattern captures the date format with a hyphen or forward slash. The date and month are confined to one or two-digit and year up to four0 digits. The respective entities are captured as groups that are optional in this case.

Also Read: Python Project Ideas and Topics

How to Implement it in Python?

The regular expressions we just built are satisfying the respective criteria we assumed and now it’s time to implement them in Python code. Python has a built-in module called re module that implements the working of these expressions. Simply,

import re

pattern = ‘^(\d{1,2})[/-](\d{1,2})[/-](\d{2,4})$’

Re module offers a wide range of functions and all of them have different use cases. Let’s look at some of the important functions:

  1. re.findall(): This function returns the list of all the matches in the test string based on the pattern passed. Consider this example:

string = ‘25-12-1999 random text here 25/12/1999’

print(re.findall(pattern, string))

Explore our Popular Data Science Online Certifications

It will return only the dates from the string in a list.

  1. re.sub(): Sub in this function stands for substitution and does the same thing. It substitutes the matches with the replacement value provided. The function takes in the pattern, string, replacement value, and optional parameter of the count. The count parameter controls how many occurrences you want to replace. By default, it replaces all of them and returns the new string.
  2. re.split(): It splits the string at the matched sites and returns the parts as separate strings in a list.
  3. re.search(): This function returns the match object that contains the match found in the string along with all the groups it captured. It can come in handy when you want to store these groups as separate columns.

To perform this:

match  = re.search(pattern, string)

match.group(1)

Group(0) returns the whole match and corresponding next numbers denote other groups.

Checkout: Python Developer Salary in India

Read our popular Data Science Articles

upGrad’s Exclusive Data Science Webinar for you –

Watch our Webinar on The Future of Consumer Data in an Open Data Economy

Top Data Science Skills You Should Learn

Conclusion

Regular expressions are a powerful way to capture patterns in textual data. It may take a bit of extra effort to hold command of the various characters but it simplifies the process of data extraction in complex use cases.

Profile

Rohit Sharma

Blog Author
Rohit Sharma is the Program Director for the UpGrad-IIIT Bangalore, PG Diploma Data Analytics Program.

Frequently Asked Questions (FAQs)

1Give some examples of Regular Expressions in Python.

The following examples illustrate the functioning or regular expressions in Python:
a. Email Filtering
The regular expressions can be efficiently used to filter emails. The regular syntax for email filtering is - ^((a-zA-Z0-9_.)+)@((a-zA-Z0-9-)+).((a-zA-Z){2,4})$
This expression is divided into three groups and tackles many cases including - when the username is alphanumeric and when it has a dot, for eg., “first.last@”. This expression will be used for top domains that contain 2-4 characters.
b. Dates Filtering
Dates can be a crucial factor while handling data filtering. The textual data that you are dealing with often contains dates. The regular expression or RegEx that extracts the data from a normal text is - ^(d{1,2})(/-)(d{1,2})(/-)(d{2,4})$
The date and the month can be up to 2 digits while the month can be up to 4 digits.

2What are the functions involved in the implementation of regular expressions in Python?

The following functions are involved in the implementation of regular expressions in Python:
1. re.findall() - This function accepts a pattern that is to be matched with the text string. It returns the strings that are a match.
2. re.sub() - Sub in “re.sub” stands for “substitution”. This method performs exactly the same function as the “re.findall()” function does.
3. re.split() - It separates the strings around the separator which is to be passed to it as its parameter. The separator could be anything.
4. re.search() - This function returns the match found in the string along with other string groups that it has captured.

3What are some special sequences used in regular expressions?

The following are some of the special sequences used in regular expressions:
1. A: Check if the string starts with the given character.
2. (Forward Slash) b: Checks if the string starts or ends with the given character. (string)/b checks for the beginning while (backward slash) b (string) checks for the end.
3. B: It is exactly opposite to the b. Checks if the string does not start with the given character.
4. d: Checks for the numerical values in the string.
5. D: Checks for any non-numerical value or character.
6. s: Checks for any whitespace character.
7. S: Checks for any non-whitespace character.
8. w: Checks for any alphanumeric character.
9. W: Checks for any non-alphanumeric character.

Explore Free Courses

Suggested Blogs

Most Common PySpark Interview Questions &#038; Answers [For Freshers &#038; Experienced]
20602
Attending a PySpark interview and wondering what are all the questions and discussions you will go through? Before attending a PySpark interview, it’s
Read More

by Rohit Sharma

05 Mar 2024

Data Science for Beginners: A Comprehensive Guide
5036
Data science is an important part of many industries today. Having worked as a data scientist for several years, I have witnessed the massive amounts
Read More

by Harish K

28 Feb 2024

6 Best Data Science Institutes in 2024 (Detailed Guide)
5113
Data science training is one of the most hyped skills in today’s world. Based on my experience as a data scientist, it’s evident that we are in
Read More

by Harish K

28 Feb 2024

Data Science Course Fees: The Roadmap to Your Analytics Career
5055
A data science course syllabus covers several basic and advanced concepts of statistics, data analytics, machine learning, and programming languages.
Read More

by Harish K

28 Feb 2024

Inheritance in Python | Python Inheritance [With Example]
17379
Python is one of the most popular programming languages. Despite a transition full of ups and downs from the Python 2 version to Python 3, the Object-
Read More

by Rohan Vats

27 Feb 2024

Data Mining Architecture: Components, Types &#038; Techniques
10660
Introduction Data mining is the process in which information that was previously unknown, which could be potentially very useful, is extracted from a
Read More

by Rohit Sharma

27 Feb 2024

6 Phases of Data Analytics Lifecycle Every Data Analyst Should Know About
79961
What is a Data Analytics Lifecycle? Data is crucial in today’s digital world. As it gets created, consumed, tested, processed, and reused, data goes
Read More

by Rohit Sharma

19 Feb 2024

Sorting in Data Structure: Categories &#038; Types [With Examples]
138282
The arrangement of data in a preferred order is called sorting in the data structure. By sorting data, it is easier to search through it quickly and e
Read More

by Rohit Sharma

19 Feb 2024

Data Science Vs Data Analytics: Difference Between Data Science and Data Analytics
68382
Summary: In this article, you will learn, Difference between Data Science and Data Analytics Job roles Skills Career perspectives Which one is right
Read More

by Rohit Sharma

19 Feb 2024

Schedule 1:1 free counsellingTalk to Career Expert
icon
footer sticky close icon