Summary

RAPIDS cuDF, a GPU DataFrame library, accelerates pandas, a popular data analysis and manipulation library for Python, nearly 150 times with zero code changes. This breakthrough allows data scientists to continue using pandas as their primary tool without sacrificing performance, even as dataset sizes grow into gigabytes. The cuDF acceleration of pandas is now generally available in the latest release of RAPIDS v24.02 and is supported by NVIDIA AI Enterprise 5.0.

Accelerating Pandas with Zero Code Changes

Pandas is a top choice for data scientists due to its easy-to-use API, but it struggles with processing speed and efficiency in CPU-only systems as dataset sizes grow. RAPIDS cuDF addresses this challenge by providing a unified CPU/GPU user experience that accelerates pandas workflows without requiring code changes.

How cuDF Accelerates Pandas

cuDF accelerates pandas by running operations on the GPU when possible and falling back to the CPU (using pandas) otherwise. This seamless integration ensures that data scientists can continue using pandas as their primary tool without compromising performance.

To bring GPU acceleration into pandas workflows, users can load the cudf.pandas extension in a Jupyter notebook or use the cudf.pandas module option when running Python scripts.

# Load the cudf.pandas extension in a Jupyter notebook
load_ext cudf.pandas

import pandas as pd

# Use the cudf.pandas module option when running Python scripts
python -m cudf.pandas script.py

Key Features of cuDF Acceleration

  • Zero Code Change Acceleration: Accelerate pandas workflows without changing the code.
  • Third-Party Library Compatibility: Compatible with most third-party libraries that operate on pandas objects, accelerating pandas operations within these libraries.
  • Unified CPU/GPU Workflows: Develop, test, and run in production with a single code path, regardless of hardware.

Benchmarking Performance

The impact of larger datasets on performance is evident in the pandas portion of the DuckDB Database-like Ops Benchmark. This benchmark compares popular CPU-based DataFrame and SQL engines on common analytics tasks such as joining data together or computing statistical measures per group.

Performance Comparison

Dataset Size pandas (CPU-only) cuDF (GPU-accelerated)
5 GB Minutes 1-2 seconds

Hardware and Software Used:

  • HW: NVIDIA Grace Hopper, CPU: Intel Xeon Platinum 8480C
  • SW: pandas v2.2, RAPIDS cuDF 23.10

Conclusion from Benchmarks

The cuDF unified CPU/GPU experience turns minutes of processing into just 1 or 2 seconds with no code change required. This significant speedup is achieved by leveraging the GPU for most operations and the CPU for a small portion to ensure workflow success.

Getting Started with cuDF

To accelerate pandas code, users can follow these steps:

  1. Load the cudf.pandas extension in a Jupyter notebook or use the cudf.pandas module option when running Python scripts.
  2. Run the pandas benchmark code unchanged to achieve significant speedups using the GPU for most operations and the CPU for a small portion.

Example Use Case

For a practical demonstration, users can run the pandas portion of the DuckDB Database-like Ops Benchmark originally developed by H2O.ai. This benchmark setup compares popular CPU-based DataFrame and SQL engines on a series of common analytics tasks.

Future Directions

The latest release of RAPIDS cuDF includes unified memory and text data processing features that help data scientists work with larger and text-heavy datasets. These advancements address user feedback and further enhance the usability of cuDF in demanding workloads.

Final Thoughts

With cuDF, data scientists can now accelerate their pandas workflows with ease, achieving significant speedups without the need for code changes. This breakthrough is a testament to the power of GPU acceleration in data science and analytics pipelines.

Conclusion

RAPIDS cuDF offers a groundbreaking solution for data scientists by accelerating pandas nearly 150 times with zero code changes. This feature is now generally available in the latest release of RAPIDS v24.02 and is supported by NVIDIA AI Enterprise 5.0. By leveraging the power of GPU acceleration, data scientists can continue using pandas as their primary tool without sacrificing performance, even as dataset sizes grow into gigabytes.