all_reduce
all_gather
Just had the same problem and debugged it. You need to put
torch.cuda.set_device(rank)
before dist.init_process_group()
거인의 어깨에 올라서서 더 넓은 세상을 바라보라 - 아이작 뉴턴
all_reduce
all_gather
Just had the same problem and debugged it. You need to put
torch.cuda.set_device(rank)
before dist.init_process_group()