76 lines
1.3 KiB
Markdown
76 lines
1.3 KiB
Markdown
# FullyShardedDataParallel
|
|
|
|
```{eval-rst}
|
|
.. automodule:: torch.distributed.fsdp
|
|
```
|
|
|
|
```{eval-rst}
|
|
.. autoclass:: torch.distributed.fsdp.FullyShardedDataParallel
|
|
:members:
|
|
```
|
|
|
|
```{eval-rst}
|
|
.. autoclass:: torch.distributed.fsdp.BackwardPrefetch
|
|
:members:
|
|
```
|
|
|
|
```{eval-rst}
|
|
.. autoclass:: torch.distributed.fsdp.ShardingStrategy
|
|
:members:
|
|
```
|
|
|
|
```{eval-rst}
|
|
.. autoclass:: torch.distributed.fsdp.MixedPrecision
|
|
:members:
|
|
```
|
|
|
|
```{eval-rst}
|
|
.. autoclass:: torch.distributed.fsdp.CPUOffload
|
|
:members:
|
|
```
|
|
|
|
```{eval-rst}
|
|
.. autoclass:: torch.distributed.fsdp.StateDictConfig
|
|
:members:
|
|
```
|
|
|
|
```{eval-rst}
|
|
.. autoclass:: torch.distributed.fsdp.FullStateDictConfig
|
|
:members:
|
|
```
|
|
|
|
```{eval-rst}
|
|
.. autoclass:: torch.distributed.fsdp.ShardedStateDictConfig
|
|
:members:
|
|
```
|
|
|
|
```{eval-rst}
|
|
.. autoclass:: torch.distributed.fsdp.LocalStateDictConfig
|
|
:members:
|
|
```
|
|
|
|
```{eval-rst}
|
|
.. autoclass:: torch.distributed.fsdp.OptimStateDictConfig
|
|
:members:
|
|
```
|
|
|
|
```{eval-rst}
|
|
.. autoclass:: torch.distributed.fsdp.FullOptimStateDictConfig
|
|
:members:
|
|
```
|
|
|
|
```{eval-rst}
|
|
.. autoclass:: torch.distributed.fsdp.ShardedOptimStateDictConfig
|
|
:members:
|
|
```
|
|
|
|
```{eval-rst}
|
|
.. autoclass:: torch.distributed.fsdp.LocalOptimStateDictConfig
|
|
:members:
|
|
```
|
|
|
|
```{eval-rst}
|
|
.. autoclass:: torch.distributed.fsdp.StateDictSettings
|
|
:members:
|
|
```
|