Add Python bindings for SYCL IPC memory via dpctl by zxue2 · Pull Request #2331 · IntelPython/dpctl

zxue2 · 2026-06-30T06:20:59Z

Add inter-process communication (IPC) support for SYCL USM memory, enabling zero-copy GPU memory sharing across processes. It wraps sycl::ext::oneapi::experimental::ipc::memory (get/open/close).

C API (libsyclinterface):

dpctl_sycl_ipc_memory_interface.h: declares DPCTLIPCMem_GetHandle, DPCTLIPCMem_OpenHandle, DPCTLIPCMem_CloseHandle, DPCTLIPCMem_FreeHandleData
dpctl_sycl_ipc_memory_interface.cpp: implements the C API by calling the SYCL experimental ipc::memory functions; auto-discovered by CMake.

Cython declarations:

_backend.pxd: extern block for the 3 new C functions

Python subpackage (dpctl.ipc):

IPCMemoryHandle: Cython extension class wrapping IPC memory export/import init(usm_memory): calls get() + put(), stores handle as bytes to_bytes(): serializable payload for cross-process transport open(handle_bytes, device, nbytes): returns MemoryUSMDevice close_mapping(usm_memory): explicitly closes an IPC mapping

Test In Docker (oneAPI 2026.0 installed):

python -c "
import dpctl
from dpctl.memory import MemoryUSMDevice
from dpctl.ipc import IPCMemoryHandle
d = dpctl.SyclDevice('level_zero:gpu')
q = dpctl.SyclQueue(d)
mem = MemoryUSMDevice(4096, queue=q)
h = IPCMemoryHandle(mem)
raw = h.to_bytes()
print(f'IPC handle: {len(raw)} bytes')
mem2 = IPCMemoryHandle.open(raw, d, nbytes=4096)
print(f'Opened: {mem2.nbytes} bytes on {mem2.sycl_device.name}')
IPCMemoryHandle.close_mapping(mem2)
print('close OK')
"

IPC handle: 120 bytes
Opened: 4096 bytes on Intel(R) Arc(TM) Pro B60 Graphics
close OK

coveralls · 2026-07-02T13:35:46Z

coverage: 75.485% (-0.2%) from 75.677% — zxue2:enable_ipc_mem into IntelPython:master

zxue2 · 2026-07-07T13:52:37Z

also add fix about "Remove unnecessary context parameters"

ndgrigorian · 2026-07-07T20:30:26Z

@zxue2
can we also add the IPC functions, IPCMemoryHandle, and MemoryIPCDevice to the docs? There is an RST for the memory, it should be straightfoward, if formatting is bad we can follow up

ndgrigorian

aside from above changes LGTM

zxue2 · 2026-07-08T02:23:35Z

@zxue2 can we also add the IPC functions, IPCMemoryHandle, and MemoryIPCDevice to the docs? There is an RST for the memory, it should be straightfoward, if formatting is bad we can follow up

Please kindly check "public IPC class and update the docs"

ndgrigorian

LGTM if no more comments from @antonwolfy

Add inter-process communication (IPC) support for SYCL USM memory, enabling zero-copy GPU memory sharing across processes. It wraps sycl::ext::oneapi::experimental::ipc::memory (get/open/close/put). C API (libsyclinterface): - dpctl_sycl_ipc_memory_interface.h: declares DPCTLIPCMem_GetHandle, DPCTLIPCMem_OpenHandle, DPCTLIPCMem_CloseHandle, DPCTLIPCMem_FreeHandleData - dpctl_sycl_ipc_memory_interface.cpp: implements the C API by calling the SYCL experimental ipc::memory functions; auto-discovered by CMake. Cython declarations: - _backend.pxd: extern block for the 4 new C functions Python subpackage (dpctl.ipc): - IPCMemoryHandle: Cython extension class wrapping IPC memory export/import __init__(usm_memory): calls get() + put(), stores handle as bytes to_bytes(): serializable payload for cross-process transport open(handle_bytes, device, nbytes): returns MemoryUSMDevice close_mapping(usm_memory): explicitly closes an IPC mapping Signed-off-by: Zhan Xue <zhan.xue@intel.com>

add has_aspect_ext_oneapi_ipc_memory checks in __init__ and open() Signed-off-by: Zhan Xue <zhan.xue@intel.com>

Signed-off-by: Zhan Xue <zhan.xue@intel.com>

create_ipc_from_usm_pointer_size_qref(): Wrap the MemoryIPCDevice construction in try/except. On failure, null base._memory_ptr to signal that __dealloc__ already closed the handle. On success, null base._memory_ptr to prevent base's own dealloc from touching it. Also null _memory_ptr in the SyclQueueCreationError path so the caller knows it still owns the handle. IPCMemoryHandle.open(): Split the single try/except into two: - SyclQueue creation failure: handle not yet transferred, close it. - create_ipc_from_usm_pointer_size_qref raises ValueError: failed before construction (queue copy), handle not closed, close it. - Any other exception from construction: __dealloc__ already closed the handle, do NOT close again. Signed-off-by: Zhan Xue <zhan.xue@intel.com>

Signed-off-by: Zhan Xue <zhan.xue@intel.com>

antonwolfy

Thank you @zxue2, no more comments from me

Signed-off-by: Zhan Xue <zhan.xue@intel.com>

zxue2 requested review from antonwolfy, ndgrigorian and vlad-perevezentsev as code owners June 30, 2026 06:21