Finishing the Analysis with NVIDIA Nsight Compute, Part 3

Summary

This article delves into the process of analysis-driven optimization using NVIDIA Nsight Compute, a powerful tool for improving the performance of GPU kernels. It guides readers through a step-by-step approach to identify and address performance limiters, leading to significant improvements in code efficiency.

Improving Code Performance with Analysis-Driven Optimization

Analysis-driven optimization (ADO) is a methodical approach to enhancing the performance of GPU kernels. It involves using tools like NVIDIA Nsight Compute to identify the most critical performance limiters and then iteratively addressing these issues. This process ensures that efforts are focused on the most impactful optimizations, leading to substantial improvements in code efficiency.

Understanding Analysis-Driven Optimization

ADO is based on the principle of identifying and addressing the most significant performance limiters in a cyclical process. This involves using a tool to pinpoint the current most important limiter, making code changes to address it, and then using the tool again to assess the impact of these changes and identify the next area for improvement. This process continues until further optimization is unlikely to yield significant performance improvements.

Using NVIDIA Nsight Compute for ADO

NVIDIA Nsight Compute is a primary tool for CUDA kernel-level performance analysis. It provides detailed performance metrics and API debugging via a user interface and command-line tool. Here are some key features and steps to leverage Nsight Compute for ADO:

1. Guided Analysis: Nsight Compute offers guided analysis that identifies common performance limiters and provides valuable optimization advice. This feature helps users focus on the most critical issues without needing extensive hardware architecture expertise.

2. Correlating Source Code with Performance Metrics: Nsight Compute allows users to correlate performance metrics down to individual lines of code. This includes connecting assembly (SASS) with PTX and higher-level code, such as CUDA C/C++, Fortran, OpenACC, or Python. Heat-map visualizations highlight areas with high metric values, making it easier to locate problematic code sections.

3. Interactive Profiling: Interactive profiling enables live sessions where application states can be viewed dynamically, and full control of the target is preserved. This feature allows users to step through API calls, inspect resources, or experiment with different kernel configurations to make performance comparisons.

4. CUDA Graphs: Nsight Compute supports the exploration and export of CUDA graphs to understand how nodes are connected and profile individual nodes or the entire graph with detailed hardware metrics.

Practical Application of ADO with Nsight Compute

To illustrate the practical application of ADO with Nsight Compute, let’s consider a code example that involves two major phases: averaging a set of vectors and performing a matrix-vector multiply on the average vector.

1. Initial Profiling: Start by profiling the code using Nsight Compute to identify the most significant performance limiter. This might involve analyzing GPU throughput, warp state statistics, and source code correlation.

2. Iterative Optimization: Based on the initial profiling results, make targeted code changes to address the identified performance limiter. Use Nsight Compute again to assess the impact of these changes and identify the next area for improvement.

3. Baseline Comparisons: Use baseline comparisons to understand the effects of changes to the workload. This feature in Nsight Compute enables efficient feedback directly in the tool.

Example Walkthrough

Let’s walk through an example to demonstrate how ADO with Nsight Compute can lead to significant performance improvements.

Step 1: Initial Profiling

• Code Analysis: The code to be optimized involves averaging a set of vectors and performing a matrix-vector multiply on the average vector.
• Profiling Results: Initial profiling with Nsight Compute reveals low GPU throughput and identifies memory access patterns as the primary performance limiter.

Step 2: Iterative Optimization

• Code Changes: Based on the profiling results, the code is modified to improve memory access patterns, such as optimizing data layout and reducing memory transfers.
• Re-profiling: Nsight Compute is used again to assess the impact of these changes, revealing improved GPU throughput but identifying warp stalls as the next performance limiter.

Step 3: Further Optimization

• Addressing Warp Stalls: The code is further modified to reduce warp stalls, such as by improving thread block configuration and reducing branch divergence.
• Final Profiling: A final round of profiling with Nsight Compute confirms significant improvements in GPU throughput and overall performance.

#Table: Key Features of NVIDIA Nsight Compute

Table: Example Optimization Steps

Conclusion

Analysis-driven optimization with NVIDIA Nsight Compute is a powerful approach to improving the performance of GPU kernels. By iteratively identifying and addressing the most significant performance limiters, developers can achieve substantial improvements in code efficiency. This methodical approach ensures that optimization efforts are focused on the most impactful areas, leading to better overall performance.