Data Frames in Python: Python In-depth Tutorial 2020

If you are a developer or coder who works in the Python programming language, you must be familiar with one of the most amazing data management libraries out there – Pandas, one of the top python libraries out there. Over the years, Pandas has emerged into a standard tool for data analysis and management using Python. Read about other important Python tools.

Pandas is undoubtedly the most versatile Python package for data science and rightly so. It provides powerful, expressive, and flexible data structures for easy data manipulation and analysis, and Data Frames in Python is one of these structures.

This is precisely our topics of discussion in this post – we’ll introduce you to the basic data format for Pandas, that is, the Pandas Data Frame.

Table of Contents

What is a Data Frame?

According to the Pandas library documentation, a Data Frame is a “two-dimensional, size-mutable, potentially heterogeneous tabular data structure with labelled axes (rows and columns)”. In simple words, a Data Frame is a data structure wherein data is aligned in a tabular fashion, that is, in rows and columns.

Image source

A Data Frame usually has the following characteristics:

It may have multiple rows and columns.
While each row represents a sample of data, each column comprises a different variable that describes the samples (rows).
The data in every column is usually the same type of data (for instance, numbers, strings, dates, etc.).
Unlike excel data sets, it avoids having missing values, so there are no gaps or empty values between rows or columns.

In a Pandas Data Frame, you can also specify the index and column names for your Data Frame. While the index indicates the difference in rows, the column names show the difference in columns.

How To Create a Data Frame In Python (Using Pandas)

Creating a Data Frame is the first step for data munging in Python. You can create a Pandas Data Frame using inputs like:

Dict
Lists
Series
Numpy “ndarray”
Another Data Frame
External files such as CS

Creating an Empty Data Frame

It is quite easy to create a basic Data Frame, a.k.a., an Empty Data Frame. Here’s an example:

Input –

Output –

Creating a Data Frame from Lists

You can create a Data Frame either using a single list or multiple lists.

Input –

Output –

Creating a Data Frame from Dict of “ndarrays” or Lists

To create a Data Frame from a dict of ndarrays, all the ndarrays must be of the same length. Also, if it is indexed, the length of the index should be equal to the length of the arrays. However, if it isn’t indexed, the index will be range(n) by default, where ‘n’ denotes the array length.

Input –

Output –

Here the values 0,1,2,3 are the default index assigned to each row using the function range(n).

What Are The Fundamental Data Frame Operations?

Now that we’ve seen three ways to create Data Frames in Python, it’s time to learn about the different operations within a Data Frame.

Selecting an index or column from a Pandas Data Frame

It is important to know how to select an index or column before can start adding, deleting, and renaming the components within a DataFrame. Suppose this is your Data Frame:

You want to access the value under index 0 in column ‘A’ – the value is 1. There are many ways to access this value, but two of the most important ones are – .loc[] and .iloc[].

Input –

Output –

So, as you can see, you can access values either by calling them by their label or by declaring their position in the index or column. While this was selecting a value from a Data Frame, how can you select rows and columns from the same?

This is how:

Input –

Output-

How To Add an Index, Row, or Column to a Pandas DataFrame

Once you learn how to access values and select columns from a Data Frame, you can learn to add index, row, or column in a Pandas Data Frame.

Adding an Index:

While creating a Data Frame, you can choose to add an input to the ‘index’ argument. This ensures that you can easily access the index you desire. If you don’t specify the index, by default, a numerically valued index that starts with 0 and continues till the last row of the DataFrame will be added to it. Although, even after the index is specified by default, you can use a column and convert it into an index by calling the set_index() function in the Data Frame.

Adding a Row:

You can add rows to a DataFrame using the append function.

Input –

Output –

You can also use .loc to insert rows in your DataFrame like so:

Input –

Output –

Adding a column

If you want to make an index the part of a Data Frame, you can take a column from the Data Frame or refer to a column that hasn’t been created yet, and assign it to the .index property like this:

Input –

Output –

For adding columns to a Data Frame, you can also use the same approach that you would use for adding an index to the Data Frame, that is, you can use the .loc[ ] or .iloc[ ] function. For example:

Input –

Output

With .loc[ ], you can add a Series to an existing DataFrame. Since a Series object is quite similar to a column of a Data Frame, it is very easy to add a Series to an existing Data Frame.

How To Reset The Index of A Data Frame?

You can reset the index of a Data Frame if it doesn’t shape out to be as you desired. You can use the .reset_index() function to do this.

Input –

Output –

How To Delete an Index, Row, or Column to a Pandas DataFrame

Deleting an index

Resetting the index of the Data Frame.
Remove the index name (if any) by using the del df.index.name function.
Remove an index along with a row.
Remove all duplicate index values by resetting the index, dropping the duplicates of the index column that has been added to the Data Frame, and reinstating the new column (devoid of a duplicate index) again as the index.

Deleting a column

For removing columns from a Data Frame, you can use the drop() function.

Input –

Output –

Deleting a row

To delete a row from a Data Frame, you can use the drop() function by using the index property to specify the index of the rows you want to delete from the DataFrame.

Input –