Also Read


Pandas - Data Science

Introduction

Data science with pandas
Image by SeekPNG

Pandas is one of the most popular and powerful data science libraries in Python. It can be considered as the stepping stone for any aspiring data scientist who prefers to code in Python. Even though the library is easy to get started, it can certainly do a wide variety of data manipulation. This makes Pandas one of the handiest data science libraries in the developer’s community. Pandas basically allow the manipulation of large datasets and data frames. It can also be considered as one of the most efficient statistical tools for mathematical computations of tabular data.

Today. we’ll cover some of the most important and recurring operations that we perform in Pandas. Make no mistake, there are tons of implementations and prospects of Pandas. Here we’ll try to cover some notable aspects only. We’ll use the analogy of Euro Cup 2020 in this tutorial. We’ll start off by creating our own minimal dataset.

4. Perform following operations using pandas

a. Creating dataframe

creating small datasets pandas | data science with pandas
Image by ActiveState

Let’s start off by creating a small sample dataset to try out various operations with Pandas. In this tutorial, we shall create a Football data frame that stores the record of 4 players each from Euro Cup 2020’s finalists – England and Italy.

import pandas as pd
# Create team data
data_england = {'Name': ['Kane', 'Sterling', 'Saka', 'Maguire'], 'Age': [27, 26, 19, 28]}
data_italy = {'Name': ['Immobile', 'Insigne', 'Chiellini', 'Chiesa'], 'Age': [31, 30, 36, 23]}


# Create Dataframe
df_england = pd.DataFrame(data_england)
df_italy = pd.DataFrame(data_italy)

The England data frame looks something like this

The Italy data frame looks something like this

b. concat()

Let’s start by concatenating our two data frames. The word “concatenate” means to “link together in series”. Now that we have created two data frames, let’s try and “concat” them.

We do this by implementing the concat() function.

frames = [df_england, df_italy]
both_teams = pd.concat(frames)
both_teams

The result looks something like this:

A similar operation could also be done using the append() function.

Try doing:

df_england.append(df_italy)

You’ll get the same result!

Now, imagine you wanted to label your original data frames with the associated countries of these players. You can do this by setting specific keys to your data frames.

Try doing:

pd.concat(frames, keys=["England", "Italy"])

And our result looks like this:

c. Setting conditions

Conditional statements basically define conditions for data frame columns. There may be situations where you have to filter out various data by applying certain column conditions (numeric or non-numeric). For eg: In an Employee data frame, you might have to list out a bunch of people whose salary is more than Rs. 50000. Also, you might want to filter the people who live in New Delhi, or whose name starts with “A”. Let’s see a hands-on example.

Imagine we want to filter experienced players from our squad. Let’s say, we want to filter those players whose age is greater than or equal to 30. In such case, try doing:

both_teams[both_teams["Age"] >= 30]

Hmm! Looks like Italians are more experienced lads.

Now, let’s try to do some string filtration. We want to filter those players whose name starts with “S”. This implementation can be done by pandas’ startswith() function. Let’s try:

both_teams[both_teams["Name"].str.startswith('S')]

Impressive!

d. Adding a new column

Let’s try adding more data to our df_england data frame.

club = ['Tottenham', 'Man City', 'Arsenal', 'Man Utd']
# 'Associated Club' is our new column name
df_england['Associated Clubs'] = club
df_england

This will add a new column ‘Associated Club’ to England’s data frame.

NameAgeAssociated Clubs
0Kane27Tottenham
1Sterling26Man City
2Saka19Arsenal
3Maguire28Man Utd

Let’s try to repeat implementing the concat function after updating the data for England.

frames = [df_england, df_italy]
both_teams = pd.concat(frames)
both_teams
NameAgeAssociated Clubs
0Kane27Tottenham
1Sterling26Man City
2Saka19Arsenal
3Maguire28Man Utd
0Immobile31NaN
1Insigne30NaN
2Chiellini36NaN
3Chiesa23NaN

Now, this is interesting! Pandas seem to have automatically appended the NaN values in the rows where ‘Associated Clubs’ weren’t explicitly mentioned. In this case, we had only updated ‘Associated Clubs’ data on England. The corresponding values for Italy were set to NaN.