When to use Category rather than Object?

Learn, when to use Category rather than Object?
Submitted by Pranit Sharma, on December 01, 2022

Pandas is a special tool that allows us to perform complex manipulations of data effectively and efficiently. Inside pandas, we mostly deal with a dataset in the form of DataFrame. DataFrames are 2-dimensional data structures in pandas. DataFrames consist of rows, columns, and data.

In this tutorial, we are going to understand the difference between the category and object data type and also, and we are going to understand in what situations we should use them.

Problem statement

Suppose we have a CSV data sets with a large number of features and we handle it with Pandas now some of the features are continuous and the rest of them are categorical.

Using Category rather than Object

This will let us into a dilemma should we use category data type for the categorical features or should we use the default object data type for them?

Now the answer to this question is:

We should use the category data type when there are a lot of partitions that we expect to exploit.
For example, if we want the aggregate size per exchange for a large table of values containing trade exchange data, then using an object is reasonable.
But since the list of possible exchanges is pretty small and because there are a lot of repetitions, we could make this faster by using a category data type.
The important point about categories is that category is a form of dynamic enumeration.

Also, we should use the categorical data type when:

A string variable with a few different values is present. Converting such a string variable to a categorical value will save some memory.

Python Pandas Programs »

Creating a new column in Pandas by using lambda function on two existing columns

How do I subtract the previous row from the current row in a pandas dataframe and apply it to every row; without using a loop?