The Risks of String Formatting Before Passing to pandas.eval()

Introduction

When working with data frames in pandas, it’s common to use the query and eval functions to perform complex operations. The query function takes a human-readable string that represents dataframe operations, while the eval function evaluates this string using Python’s dynamic syntax. However, formatting strings before passing them to pandas.eval() can introduce security risks.

In this article, we’ll explore the potential risks associated with formatting strings before passing them to pandas.eval(), and discuss how to write safer code that avoids these risks.

Background

The main issue with formatting strings before passing them to pandas.eval() is that it allows for arbitrary Python code execution. The eval function executes the string as a Python expression, which means that an attacker could inject malicious code if they control the input string.

For example, consider the following code:

import pandas as pd

def query_column_equality(df, a, b):
    if a in df.columns:
        return df.query('{} == @q'.format(a), local_dict={'q': b})
    else:
        raise KeyError(a)

df = pd.DataFrame({'a': [1, 2], 'b': [10, 20]})

query_value = 20
print(query_column_equality(df, 'b', query_value))

In this example, the string 'b' is used as a column name in the query function. However, what if an attacker were to pass in a malicious input, like this:

import pandas as pd

def query_column_equality(df, a, b):
    if a in df.columns:
        return df.query('{} == @q'.format(a), local_dict={'q': b})
    else:
        raise KeyError(a)

df = pd.DataFrame({'a': [1, 2], 'b': [10, 20]})

query_value = 20
print(query_column_equality(df, 'b', "rm -rf *"))

In this example, the attacker has injected malicious code into the query function using a string. This could potentially lead to arbitrary file deletion or other security vulnerabilities.

Alternatives to String Formatting

So what can we do instead of formatting strings before passing them to pandas.eval()? One approach is to use alternative methods that are safer and more secure.

For example, consider the following code:

import pandas as pd

def query_column_equality(df, a, b):
    if a in df.columns:
        return df.loc[df[a] == b]
    else:
        raise KeyError(a)

df = pd.DataFrame({'a': [1, 2], 'b': [10, 20]})

query_value = 20
print(query_column_equality(df, 'b', query_value))

In this example, we’re using the loc function to perform the same operation as before. However, instead of formatting a string, we’re using column indexing to access the data.

Similarly, for the isin function, we can use:

import pandas as pd

def query_column_isin(df, a, b):
    if a in df.columns:
        return df.loc[df[a].isin(b)]
    else:
        raise KeyError(a)

df = pd.DataFrame({'a': [1, 2], 'b': [10, 20]})

query_iterable = [20]
print(query_column_isin(df, 'b', query_iterable))

In this example, we’re using the isin function to perform the same operation as before. However, instead of formatting a string, we’re passing in an iterable containing the values we want to check for.

Conclusion

Formatting strings before passing them to pandas.eval() can introduce security risks due to the potential for arbitrary code execution. Instead, consider using alternative methods that are safer and more secure, such as column indexing or iterating over a list of values.

By following these guidelines, you can write safer code that avoids the risks associated with formatting strings before passing them to pandas.eval().

Last modified on 2024-06-28