Elektrooptik

Eintrag vom: 24.09.2013.

Performance optimization is crucial for efficient deep learning model training and inference. This tutorial covers a comprehensive set of techniques to accelerate PyTorch workloads across different hardware configurations and use cases.

 

These techniques are available to you whether you are training your model with Trainer or writing a pure PyTorch loop in which case you can configure these optimizations with ? Accelerate. If these methods do not result in sufficient gains you can explore the following options:

 

For a neural network training a model is typically the most computationally expensive part of your code and so that?s where GPUs are normally utilised. To run a training loop in this way requires that two things are passed to the GPU: (i) the model itself and (ii) the training data.

 

We will use the numba.jit decorator for the function we want to compute over the GPU. The decorator has several parameters but we will work with only the target parameter.

 

Both CUDA and OpenCL are compatible with Python and allow the user to implement highly parallel processes on a GPU without needing to explicitly specify the parallelisation i.e. they do the optimisation for you.

 

This guide focuses on training large models efficiently on a single GPU. These approaches are still valid if you have access to a machine with multiple GPUs but you will also have access to additional methods outlined in the multi-GPU section.

 

Here's a layer-by-layer deep dive of the training stack from your Python code in PyTorch down to the GPU?s smallest execution units.

 

We?ll start with the core concepts of GPU programming show you how accessible it has become thanks to Python and then address the critical next step: how to move from running code on a single GPU to deploying it efficiently on the powerful multi-GPU clusters that power real-world AI.