[SYCL][CUDA] Multiple queues on the same device.

[Ryefalk]

As of right now multiple queues create multiple contexts. Is this intended to always be the case or will this be moved to streams over time? If I have understood correctly openCL queues are akin to CUDA streams so that's why I'm asking. If it will continue to be contexts what would be some ways to have them utilize the GPU in a good way. MPS maybe, or something else. But it feels wierd having to do a workaround for a core feature.

[vrpascuzzi]

I don't think this is a workaround, rather it's standard in OpenCL to share contexts. For now, I take the context from the first queue I instantiate and use pass it to create other queues via it's available devices. In my case, I need to create queues from a shared context in order for the different queues to access memory (allocated via USM) between them.

[mmoadeli]

As you pointed out, in SYCL one context is created for each queue. However, shared context among multiple queues may be realised by constructing the queues with one context, or using Default Context in DPC++ extension which offers multiple queues to share the same context by default.

To address better utilisation of the GPUs within a single application process, DPC++ offers creating multiple streams in each queue to improve concurrent execution of copy and compute operation. Moreover, if you want to improve utilisation of a single GPU when using multiple application processes, the MPS potential can be exploited in post-Volta architectures using DPC++ CUDA backend.

Regarding the statement that “openCL queues are akin to CUDA streams”, it may not be the case, as OpenCL supports out-of-order queues.