ForBo7 // Salman Naqvi - A No Nonsense Guide on how to use an M-Series Mac GPU with PyTorch

[giscus[bot]]

ForBo7 // Salman Naqvi - A No Nonsense Guide on how to use an M-Series Mac GPU with PyTorch

Squeezing out that extra performance.

https://forbo7.github.io/forblog/posts/8_how_to_use_apple_gpu_with_pytorch.html

[toastersocks]

Thank you for this. Very straight-forward and clear.
One small nit: the Tensor.to() function does not mutate the original tensor but returns a new copy of the tensor with the new device so the last line would be something like: t = t.to(device)
Also, I could be doing something wrong, but all three of the 'relevant links:' seem to just point back at this article rather than the the addresses they appear to link to.

[ForBo7]

Thanks for the correction! I've edited the guide to mention that a copy is made. I've fixed the links too.

[erikpb]

Simply setting default_device(torch.device("mps")) after the PyTorch import seems to do the trick as well.
All tensors created after that will use the Mac M1 GPU.

[toastersocks]

While trying to do this:

device = torch.device('mps')
torch.default_device(device)

I'm getting the error module 'torch' has no attribute 'default_device' Googling for the docs,I can't find anything about the default_device() function. What version of PyTorch are you using?

[ForBo7]

After googling myself for a bit, found out it's a fastai function: https://docs.fast.ai/torch_core.html#default_device

However, I'm not quite sure whether it sets mps as the default device. While running the function without any input, it does state that the default device is 'mps'. However, tensors created after don't state device='mps:0' as I would expect.

>>> from fastai.tabular.all import *
>>> default_device()
device(type='cpu')
>>> default_device('mps')
device(type='mps')
>>> default_device()
device(type='mps')
>>> t = tensor([1, 2, 3])
>>> t
tensor([1, 2, 3])

[erikpb]

I was indeed using this in a fastai context. And when using the fastai wrappers, it was definitely using the Mac's GPU as I could observe that in Activity Monitor. So maybe it only works with the fastai wrappers.

[ForBo7]

Just tried training a tabular model twice, once without changing the device, and once with using default_device('mps'). Checked the GPU usage in Activity Monitor for both, and the GPU is indeed being used with the latter. Interesting though that tensors and models don't state they're set to MPS.

I think that this should work with non-fastai wrappers too, after viewing the source for this function:

def default_device(use=-1):
    "Return or set default device; `use_cuda`: -1 - CUDA/mps if available; True - error if not available; False - CPU"
    if use == -1: use = defaults.use_cuda
    else: defaults.use_cuda=use
    if use is None:
        if torch.cuda.is_available() or _has_mps(): use = True
    if use:
        if torch.cuda.is_available(): return torch.device(torch.cuda.current_device())
        if _has_mps(): return torch.device('mps')
    return torch.device('cpu')

[dana-ludwig]

When I first read about the M1 chip set, I thought it was only useful for deep learning inference and not training. Is it equally good for both now? I have an M1 (Macbook Pro) that I bought recent, so this is pretty exciting!

ps - do they overheat easily? The Macbook doesn't look like it is optimized for cooling :-)

[ForBo7]

As far as I know, the M-Series chips are good for inference, and as for training, their performance is getting better. Nvidia GPUs are still way better, since ML libraries are not fully optimized for these new chipsets yet. These new chips do have potential, they're just not fully utilized yet.

And these new Macs don't overheat easily! The new chips are really efficient and run very cool.

[dana-ludwig]

That's encouraging that the issues are in the libraries; I just bought this cool laptop and I would hate to have to upgrade hardware to make full use of it! BTW, the battery seems to last a long time too (not running ML code yet).
Dana

[ForBo7]

Yeah, just have to wait some time for optimizations.

And yup, battery life is great! I can make it last 2–3 days.