Passing multiple arguments into Pandas Apply function
Versatility in the use of pandas is a must for python users who specialize in working with data using python. Aside from being a robust library with functionality for just about any data-related task, it boasts a large community of users ready to come to your rescue with those tricky data problems.
The apply function of pandas is one of the many useful functions I cherish in the library, as it serves as a “wild card” to provide extended capability to the library when there is no prebuilt function to deal with your problem. It’s not uncommon for me to throw a few apply functions in some of my data related projects, and most of these involve working on only one column in the dataset. However, until recently, I was poised with a problem that requires building a custom function that simultaneously requires input from multiple columns for each entry of the data. Unlike most other problems I encountered in the past, I couldn’t quickly pull up a well-written blog post or article on how to properly navigate this problem prompting my draft on this topic.
To cut the chase, I’ll use a quick demo to show how you can access multiple columns if you are in a hurry. Below is a demo dataframe created to demonstrate just this.
import pandas as pddef func(a, b):
return a * bdata = pd.DataFrame({'A': [2,3,5], 'B': [100,50,70]})data['C'] = data.apply(lambda x: func(x['A'], x['B']), axis = 1)
#ensure you add the extra argument axis = 1
The above apply function passes both columns ‘A’ and ‘B’ to the function ‘func’. One common error to look out for is omitting the “axis = 1” argument. Usually, you will be faced with a “Key Error: {column name}” error. when you omit this argument.
To break down the above code, let’s start from the first line.
Here we are importing the required library (pandas)
The function to be applied to the dataframe is first defined. Since multiple columns are employed here, the function takes more than one argument. Here in this demo, the function returns the multiplication of the values in the two columns.
In the above snippet, the demo data is created and the data has three rows and two columns (labelled ‘A’ and ‘B’).
Finally, a new column, labeled ‘C’ is created using the existing columns ‘A’ and ‘B’. It is important to note that an additional argument ‘axis = 1’ is added and the arguments are passed into the function previously defined, specifying the arguments by position.
The full excerpt of the demo code is shown below.
To further illustrate this, let’s create an hypothetical scenario using the classic titanic dataset which contains basic information on the people onboard the titanic ship. This dataset contains information such as each individuals’ age, sex, fare, cabin, number of siblings/spouse, etc. The task is to create a new column “New Var” that contains the result of multiplying the age and fare of the individual if its a male and same variable stores the result of adding the age of the individual with their fare if female.
While there might be multiple ways to achieve this, we’ll be making use of pandas apply function. We need to know the gender of each individual, as well as the age and fare of same individual over the entire dataset.
Below is the snippet of the code which executed this task smoothly. It follows a similar pattern to the demo description above.
Once again, it is always important to state the axis you are applying the function to. More often than not, this will have the value 1 to help iterate over the rows. I hope this has been helpful to simplify the use of multiple argument in the pandas apply function.
