Home » Python » Python Programs

Pandas dataframe remove all rows where None is the value in any column

Given a pandas dataframe, we have to remove all rows where None is the value in any column.
Submitted by Pranit Sharma, on October 19, 2022

Pandas is a special tool that allows us to perform complex manipulations of data effectively and efficiently. Inside pandas, we mostly deal with a dataset in the form of DataFrame. DataFrames are 2-dimensional data structures in pandas. DataFrames consist of rows, columns, and data.

While creating a DataFrame or importing a CSV file, there could be some NaN values in the cells. NaN values mean "Not a Number" which generally means that there are some missing values in the cell.

Problem statement

Suppose we are given a DataFrame and let us assume that when it was created, some values were inserted with None or NaN values (which were not calculated), now we need to remove these rows from this DataFrame where the NaN value is present in any column.

Removing all rows where None is the value in any column

We will use pandas.DataFrame.eq() method which will check for the string None and then we will apply pandas.DataFrame.dropna() method on this result so that it will drop all the rows where ever a None value is encountered.

Let us understand with the help of an example,

Python program to remove all rows where None is the value in any column

# Importing pandas package
import pandas as pd

# Importing numpy package
import numpy as np

# Creating a dictionary
d = {
    'a' : [1, 2, 3],
    'b':[4, 'None', 6],
    'c':['None', 7, 8],
    'd':[9, 10, 11]
}

# Creating a DataFrame
df = pd.DataFrame(d)

# Display original DataFrame
print("Original Dataframe:\n",df,"\n")

# Removing rows for None
res = df.mask(df.eq('None')).dropna()

# Display Result
print('Result:\n',res)

Output

The output of the above program is:

Python Pandas Programs »