GPU-Centric RDMA Data Paths with NVIDIA DOCA GPUNetIO

NVIDIA DOCA GPUNetIO can be used to place the RDMA data path under GPU control when an application already processes data on the GPU. In the DOCA 2.7 functionality described here, CUDA kernels can use extended GPUNetIO APIs to communicate with an NIC over RoCE or InfiniBand transport. The initial RDMA setup and peer-information exchange remain CPU-managed, while GPU code issues RDMA operations during the data phase.

When a GPU-centric RDMA path fits

This approach is relevant to network applications in which data processing occurs on the GPU and repeated CPU interaction is part of the latency-sensitive path. DOCA GPUNetIO is a library in the NVIDIA DOCA SDK designed for real-time inline GPU packet processing. It combines GPUDirect RDMA and GPUDirect Async to support applications in which CUDA kernels communicate directly with the NIC for packet transmission and reception, excluding the CPU from the critical data path.

For RDMA use cases, the DOCA 2.7 API extensions described in the source add direct support from GPU CUDA kernels for RoCE and InfiniBand transport. This changes the placement of the operational data path, not the complete connection lifecycle. Teams should assess it where GPU-resident data and GPU-based processing make a GPU-controlled network path materially relevant to application design.

Architecture path: CPU setup, GPU data operations

RDMA enables direct remote memory access between the memory of two computers without involving either operating system in the operation. The source describes three basic stages for applications that exchange data between RDMA peers:

1. Local configuration: Each peer creates RDMA queues and memory buffers to be shared with the other peer.
2. Out-of-band exchange: The peers exchange queue and remotely accessible memory-buffer information through an OOB mechanism, such as Linux sockets.
3. Data path: The peers use remote memory addresses for RDMA read, write, send, and receive operations.

With the described GPUNetIO RDMA capability, stages one and two remain on the CPU because they are not part of the GPU data path. In stage three, CUDA kernels manage the RDMA application data path. A GPU can manage multiple RDMA queues in parallel; the source gives the example of separate CUDA blocks issuing operations on different RDMA queues.

Implementation checkpoints

Start by mapping the application against the three RDMA stages. Confirm which queues, buffers, remote addresses, and OOB exchange mechanisms are required before moving data-path operations into CUDA kernels. The source describes DOCA RDMA as handling the three-stage communication flow over InfiniBand or RoCE with CPU execution; GPUNetIO RDMA changes the execution location for the data path.

Next, validate that the target workflow can issue and complete the intended RDMA operations from GPU code. In the example approach described, a CUDA kernel publishes a list of RDMA writes in parallel, with CUDA threads participating in each block. After thread synchronization, one thread commands the NIC to execute the writes, waits for completion, and flushes the queue before the next iteration.

Finally, evaluate behavior with a workload that reflects the application rather than relying only on a microbenchmark. Check functional correctness, queue behavior, completion handling, buffer registration, and the selected RoCE or InfiniBand transport in dated official NVIDIA documentation and the complete deployment BOM.

Performance evaluation and evidence limits

The source discusses a DOCA 2.7 GPUNetIO RDMA client-server code example that demonstrates API use and evaluates correctness. It also describes reproducing an RDMA write bandwidth microbenchmark associated with the perftest suite. In a typical ib_write_bw flow, the CPU posts a list of same-size RDMA write requests, rings the NIC, and polls the completion queue before proceeding to the next iteration. With its CUDA option enabled, the data to be written resides in GPU memory, while the network communication remains CPU-controlled.

The GPUNetIO comparison instead uses DOCA RDMA on the CPU as the control path for client-server establishment and DOCA GPUNetIO RDMA inside a CUDA kernel to post writes. The source explicitly notes that this is not an exact like-for-like comparison: perftest uses GPUDirect RDMA for transfer with CPU-controlled networking, whereas the GPUNetIO approach uses GPUDirect RDMA and GPUDirect Async for GPU-controlled networking and transfer. It states that the objective is performance comparable to the IB Verbs baseline, not a universal performance claim.

Do not infer throughput, latency, CPU-use, or compatibility results for a specific environment from this description. Verify supported hardware, software versions, APIs, transport configuration, and measured application behavior in dated official product documentation and a project test.

FAQ

Does GPUNetIO move every RDMA task to the GPU?

No. In the architecture described, local creation of RDMA queues and buffers, plus the out-of-band exchange of peer information, remain CPU tasks. GPUNetIO RDMA places the third-stage data path, including RDMA operations from CUDA kernels, on the GPU.

Is GPUDirect RDMA the same as the RDMA protocol?

No. The source distinguishes the two. RDMA refers to the protocol for remote direct memory access between computers. GPUDirect RDMA is an NVIDIA technology that allows an NIC to access GPU memory directly, bypassing CPU memory copies and operating-system routines.

Conclusion

DOCA GPUNetIO RDMA provides an architecture path for GPU-based applications that need CUDA kernels to manage RoCE or InfiniBand RDMA data operations. Keep CPU-based setup and peer exchange in the design, then validate the GPU data path against the exact software stack, hardware configuration, and workload before making deployment decisions.

After reviewing GPU-Centric RDMA Data Paths with NVIDIA DOCA GPUNetIO, continue with NVIDIA products and networking solutions for related evaluation paths.