Home » Python » Python Programs

Pandas fill missing values in dataframe from another dataframe

Given a pandas dataframe, we have to fill the missing values in a pandas DataFrame from another DataFrame.
Submitted by Pranit Sharma, on November 21, 2022

Pandas is a special tool that allows us to perform complex manipulations of data effectively and efficiently. Inside pandas, we mostly deal with a dataset in the form of DataFrame. DataFrames are 2-dimensional data structures in pandas. DataFrames consist of rows, columns, and data.

Problem statement

Suppose, we are given two DataFrames, out of which one dataframe has some nan values. We need to find a way to select the missing/nan values in dataframe and substitute them with some values from another dataframe.

Here, we are assuming that both the dataframes have some common indexes, and also both the dataframes are of the same shape and size.

Filling the missing values in dataframe from another dataframe

For that purpose, we will simply use pandas.DataFrame.isnull() method which will return all the null values and will access the location of these null values by using them inside square brackets along the dataframe and assign it with dataframe2.

Let us understand with the help of an example,

Python code to fill missing values in dataframe from another dataframe

# Importing pandas package
import pandas as pd

# Importing numpy package
import numpy as np

# Creating two dictionaries
d1 = {'0':[np.nan,5],'1':[10,np.nan]}
d2 = {'0':[20,30],'1':[40,50]}

# Creating two dataframes
df1 = pd.DataFrame(d1)
df2 = pd.DataFrame(d2)

# Display original DataFrames
print("Original DataFrame 1:\n",df1,"\n")
print("Original DataFrame 2:\n",df2,"\n")

# Filling nan of df1 with df2
df1[df1.isnull()] = df2

# Display new df
print("New DataFrame:\n",df1)

Output

The output of the above program is:

Python Pandas Programs »