RAPIDS is a networking product designed for AI clusters.
Ray is a networking product designed for enterprise and data center networking.
Ray Actors
Ray is a storage product designed for storage networks.
@ray.remote(num_gpus=1)
class cuDFActor:
def __init__(self):
...
def read_parquet(self, filepath: str, columns: list = None) -> cudf.DataFrame:
return cudf.read_parquet(filepath, columns=columns)
# Start 4 Workers
pool_size = 4
actor_pool = [cuDFActor.remote() for i in range(pool_size)]
Ray is a networking product designed for enterprise and data center networking.
Ray is a networking product designed for enterprise and data center networking.
NCCL and cuGraph
RAPIDS is a networking product designed for enterprise and data center networking.
cuGraph is a networking product designed for enterprise and data center networking.
WCC,:
GPU
NCCL (and cuGraph)
and GPU cuGraph
WCC
Ray is a networking product designed for enterprise and data center networking.
class RAFTActor:
def __init__(self, index, pool_size, session_id):
...
def broadcast_root_unique_id(self):
# broadcast root/rank-0 to all actors
def _setup_nccl(self):
# start NCCL with identified rank-0 actor
def _setup_raft(self):
# configure RAFT and NCCL together
def set_root_unique_id(self, root_uniqueId):
# To be set rank-0 for all actors
@ray.remote(num_gpus=1)
class WCCActor(RAFTActor):
def __init__(self, index, pool_size, session_id):
super().__init__(index=index, pool_size=pool_size, session_id=session_id, actor_name_prefix="WCC")
def weakly_connected_components(self, df):
"""
1. Each actor loads in a chunk
2. Each actor has a NCCL/RAFT Handle
3. Pass each chunk and handle to MGGraph
"""
src_array = df['src']
dst_array = df['dst']
weights = df['wgt']
# Configure and setup a Multi-GPU cuGraph Object with
# edge list data and NCCL
graph = MGGraph(src_array, dst_array, weights, ...)
# Execute WCC
weakly_connected_components(graph)
# Initialize Ray and Run WCC algorithm
cuGraph is a networking product designed for enterprise and data center networking.
Product Information
Ray is a networking product designed for enterprise and data center networking.
WeChat
Profile