Skip to content

Commit 26820af

Browse files
chore: update submodules (#187)
Co-authored-by: ydcjeff <[email protected]>
1 parent 62fdc34 commit 26820af

File tree

3 files changed

+6
-6
lines changed

3 files changed

+6
-6
lines changed

src/tutorials/intermediate/01-cifar10-distributed.md

+3-3
Original file line numberDiff line numberDiff line change
@@ -31,7 +31,7 @@ The type of distributed training we will use is called data parallelism in which
3131
>
3232
> -- <cite>[Distributed Deep Learning 101: Introduction](https://towardsdatascience.com/distributed-deep-learning-101-introduction-ebfc1bcd59d9)</cite>
3333
34-
PyTorch provides a [torch.nn.parallel.DistributedDataParallel](https://pytorch.org/docs/stable/generated/torch.nn.parallel.DistributedDataParallel.html) API for this task however the implementation that supports different backends + configurations is tedious. In this example, we will see how to can enable data distributed training which is adaptable to various backends in just a few lines of code alongwith:
34+
PyTorch provides a [torch.nn.parallel.DistributedDataParallel](https://pytorch.org/docs/stable/generated/torch.nn.parallel.DistributedDataParallel.html) API for this task however the implementation that supports different backends + configurations is tedious. In this example, we will see how to enable data distributed training which is adaptable to various backends in just a few lines of code alongwith:
3535
* Computing training and validation metrics
3636
* Setup logging (and connecting with ClearML)
3737
* Saving the best model weights
@@ -229,7 +229,7 @@ def get_model(config):
229229

230230
### Optimizer
231231

232-
Then we can setup the optimizer using hyperameters from `config` and pass it through [`auto_optim()`](https://pytorch.org/ignite/generated/ignite.distributed.auto.auto_optim.html#ignite.distributed.auto.auto_optim).
232+
Then we can setup the optimizer using hyperparameters from `config` and pass it through [`auto_optim()`](https://pytorch.org/ignite/generated/ignite.distributed.auto.auto_optim.html#ignite.distributed.auto.auto_optim).
233233

234234

235235
```python
@@ -651,7 +651,7 @@ torchrun --nproc_per_node=2 main.py run --backend="nccl"
651651

652652

653653

654-
### Run with internal spawining (`torch.multiprocessing.spawn`)
654+
### Run with internal spawning (`torch.multiprocessing.spawn`)
655655

656656
```
657657
python -u main.py run --backend="nccl" --nproc_per_node=2

src/tutorials/intermediate/02-Machine_Translation_using_PyTorch_Ignite.md

+2-2
Original file line numberDiff line numberDiff line change
@@ -40,7 +40,7 @@ This notebook uses Models, Dataset and Tokenizers from Huggingface, hence they c
4040
## Common Configuration
4141
We maintain a config dictionary which can be extended or changed to store parameters required during training. We can refer back to this code when we will use these parameters later.
4242

43-
In this example we are using ``t5-small``, which has 60M parameters. The way t5 models work is they taske an input with the a task-specific prefix. This prefix (like "Translate English to German") will let our model know which task it needs to perform. For more details refer to the original paper [here](https://arxiv.org/abs/1910.10683).
43+
In this example we are using ``t5-small``, which has 60M parameters. The way t5 models work is they take an input with a task-specific prefix. This prefix (like "Translate English to German") will let our model know which task it needs to perform. For more details refer to the original paper [here](https://arxiv.org/abs/1910.10683).
4444

4545

4646
Here we train on less number of iterations per step and on a limited dataset, this can be modified using the ``train_dataset_length`` and ``epoch_length`` config.
@@ -254,7 +254,7 @@ The forward pass is wrapped in the autocast context manager for mixed precision
254254
Gradient accumulation is implemented as batch size of 1 would lead to noisy updates otherwise. Check the ``accumulation_steps`` variable in config to define the number of steps to accumulate the gradient.
255255

256256
#### Trainer Handlers
257-
Handlers can be defined and attached directly to the trainer engine. Here we also make use of a special function : `setup_common_training_handlers` which has a lot of the commonly used, useful handlers (like `save_every_iters`, `clear_cuda_cache` etc) already defined. To know more about this function, refer to the docs [here](https://pytorch.org/ignite/contrib/engines.html#ignite.contrib.engines.common.setup_common_training_handlers).
257+
Handlers can be defined and attached directly to the trainer engine. Here we also make use of a special function : `setup_common_training_handlers` which has a lot of the commonly used, useful handlers (like `save_every_iters`, `clear_cuda_cache`, etc) already defined. To know more about this function, refer to the docs [here](https://pytorch.org/ignite/contrib/engines.html#ignite.contrib.engines.common.setup_common_training_handlers).
258258

259259

260260
```python

0 commit comments

Comments
 (0)