torch-tensor-parallelism
This skill provides guidance for implementing tensor parallelism in PyTorch, specifically column-parallel and row-parallel linear layers. Use when implementing distributed neural network layers that split weights/activations across multiple ranks, working with torch.distributed for model parallelism, or implementing ColumnParallelLinear and RowParallelLinear classes.
Installation and usage
This skill provides guidance for implementing tensor parallelism in PyTorch, specifically column-parallel and row-parallel linear layers. Use when implementing distributed neural network layers that split weights/activations across multiple ranks, working with torch.distributed for model parallelism, or implementing ColumnParallelLinear and RowParallelLinear classes.
설치 후 터미널에서 다음 명령을 실행하여 이 스킬을 사용할 수 있습니다:
skills use torch-tensor-parallelism