grsecurity forums

Hi, I'm trying to use Nvidia CUDA on a system with a grsec patched kernel. I successfully installed the nvidia proprietary drivers using the suggested patch nvidia-drivers-367.35-pax.patch.

I am able to load the modules and run opengl applications but I can't run any CUDA application; trying to execute some of the samples delivered with cuda result in an out of memory error:

Code: Select all

./deviceQuery Starting...

 CUDA Device Query (Runtime API) version (CUDART static linking)

cudaGetDeviceCount returned 2
-> out of memory
Result = FAIL


Or using pycuda:

Code: Select all

>>> import pycuda.driver as cuda
>>> import pycuda.autoinit
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/lib/python3.5/site-packages/pycuda/autoinit.py", line 5, in <module>
    cuda.init()
pycuda._driver.MemoryError: cuInit failed: out of memory


I don't see any errors running dmesg and all required nvidia modules are loaded. On the same system everything works using the unpatched kernel and drivers.
This is my current configuration:

Code: Select all

#
# Grsecurity
#
CONFIG_PAX_PER_CPU_PGD=y
CONFIG_TASK_SIZE_MAX_SHIFT=42
CONFIG_PAX_USERCOPY_SLABS=y
CONFIG_GRKERNSEC=y
# CONFIG_GRKERNSEC_CONFIG_AUTO is not set
CONFIG_GRKERNSEC_CONFIG_CUSTOM=y
CONFIG_GRKERNSEC_TPE_TRUSTED_GID=200
CONFIG_GRKERNSEC_SYMLINKOWN_GID=33

#
# Customize Configuration
#

#
# PaX
#
CONFIG_PAX=y

#
# PaX Control
#
CONFIG_PAX_SOFTMODE=y
# CONFIG_PAX_EI_PAX is not set
# CONFIG_PAX_PT_PAX_FLAGS is not set
CONFIG_PAX_XATTR_PAX_FLAGS=y
# CONFIG_PAX_NO_ACL_FLAGS is not set
CONFIG_PAX_HAVE_ACL_FLAGS=y
# CONFIG_PAX_HOOK_ACL_FLAGS is not set

#
# Non-executable pages
#
CONFIG_PAX_NOEXEC=y
CONFIG_PAX_PAGEEXEC=y
CONFIG_PAX_EMUTRAMP=y
CONFIG_PAX_MPROTECT=y
# CONFIG_PAX_MPROTECT_COMPAT is not set
# CONFIG_PAX_ELFRELOCS is not set
CONFIG_PAX_KERNEXEC=y
CONFIG_PAX_KERNEXEC_PLUGIN=y
# CONFIG_PAX_KERNEXEC_PLUGIN_METHOD_NONE is not set
CONFIG_PAX_KERNEXEC_PLUGIN_METHOD_BTS=y
# CONFIG_PAX_KERNEXEC_PLUGIN_METHOD_OR is not set

#
# Address Space Layout Randomization
#
CONFIG_PAX_ASLR=y
CONFIG_PAX_RANDKSTACK=y
CONFIG_PAX_RANDUSTACK=y
CONFIG_PAX_RANDMMAP=y

#
# Miscellaneous hardening features
#
# CONFIG_PAX_MEMORY_SANITIZE is not set
# CONFIG_PAX_MEMORY_STACKLEAK is not set
CONFIG_PAX_MEMORY_STRUCTLEAK=y
# CONFIG_PAX_MEMORY_UDEREF is not set
CONFIG_PAX_REFCOUNT=y
CONFIG_PAX_CONSTIFY_PLUGIN=y
CONFIG_PAX_USERCOPY=y
# CONFIG_PAX_USERCOPY_DEBUG is not set
CONFIG_PAX_SIZE_OVERFLOW=y
# CONFIG_PAX_LATENT_ENTROPY is not set
# CONFIG_PAX_RAP is not set

#
# Memory Protections
#
CONFIG_GRKERNSEC_KMEM=y
CONFIG_GRKERNSEC_IO=y
CONFIG_GRKERNSEC_BPF_HARDEN=y
CONFIG_GRKERNSEC_PERF_HARDEN=y
CONFIG_GRKERNSEC_RAND_THREADSTACK=y
CONFIG_GRKERNSEC_PROC_MEMMAP=y
CONFIG_GRKERNSEC_KSTACKOVERFLOW=y
CONFIG_GRKERNSEC_BRUTE=y
CONFIG_GRKERNSEC_MODHARDEN=y
CONFIG_GRKERNSEC_HIDESYM=y
# CONFIG_GRKERNSEC_RANDSTRUCT is not set
CONFIG_GRKERNSEC_KERN_LOCKOUT=y

#
# Role Based Access Control Options
#
CONFIG_GRKERNSEC_NO_RBAC=y
CONFIG_GRKERNSEC_ACL_HIDEKERN=y
CONFIG_GRKERNSEC_ACL_MAXTRIES=4
CONFIG_GRKERNSEC_ACL_TIMEOUT=30

#
# Filesystem Protections
#
# CONFIG_GRKERNSEC_PROC is not set
# CONFIG_GRKERNSEC_LINK is not set
CONFIG_GRKERNSEC_SYMLINKOWN=y
CONFIG_GRKERNSEC_FIFO=y
# CONFIG_GRKERNSEC_SYSFS_RESTRICT is not set
CONFIG_GRKERNSEC_ROFS=y
CONFIG_GRKERNSEC_DEVICE_SIDECHANNEL=y
CONFIG_GRKERNSEC_CHROOT=y
CONFIG_GRKERNSEC_CHROOT_MOUNT=y
CONFIG_GRKERNSEC_CHROOT_DOUBLE=y
CONFIG_GRKERNSEC_CHROOT_PIVOT=y
CONFIG_GRKERNSEC_CHROOT_CHDIR=y
CONFIG_GRKERNSEC_CHROOT_CHMOD=y
CONFIG_GRKERNSEC_CHROOT_FCHDIR=y
CONFIG_GRKERNSEC_CHROOT_MKNOD=y
CONFIG_GRKERNSEC_CHROOT_SHMAT=y
CONFIG_GRKERNSEC_CHROOT_UNIX=y
CONFIG_GRKERNSEC_CHROOT_FINDTASK=y
CONFIG_GRKERNSEC_CHROOT_NICE=y
CONFIG_GRKERNSEC_CHROOT_SYSCTL=y
CONFIG_GRKERNSEC_CHROOT_RENAME=y
CONFIG_GRKERNSEC_CHROOT_CAPS=y
CONFIG_GRKERNSEC_CHROOT_INITRD=y

#
# Kernel Auditing
#
CONFIG_GRKERNSEC_AUDIT_GROUP=y
CONFIG_GRKERNSEC_AUDIT_GID=201
# CONFIG_GRKERNSEC_EXECLOG is not set
CONFIG_GRKERNSEC_RESLOG=y
# CONFIG_GRKERNSEC_CHROOT_EXECLOG is not set
CONFIG_GRKERNSEC_AUDIT_PTRACE=y
# CONFIG_GRKERNSEC_AUDIT_CHDIR is not set
# CONFIG_GRKERNSEC_AUDIT_MOUNT is not set
CONFIG_GRKERNSEC_SIGNAL=y
CONFIG_GRKERNSEC_FORKFAIL=y
CONFIG_GRKERNSEC_TIME=y
# CONFIG_GRKERNSEC_PROC_IPADDR is not set
CONFIG_GRKERNSEC_RWXMAP_LOG=y

#
# Executable Protections
#
CONFIG_GRKERNSEC_DMESG=y
CONFIG_GRKERNSEC_HARDEN_PTRACE=y
CONFIG_GRKERNSEC_PTRACE_READEXEC=y
CONFIG_GRKERNSEC_SETXID=y
CONFIG_GRKERNSEC_HARDEN_IPC=y
CONFIG_GRKERNSEC_HARDEN_TTY=y
CONFIG_GRKERNSEC_TPE=y
CONFIG_GRKERNSEC_TPE_ALL=y
CONFIG_GRKERNSEC_TPE_INVERT=y
CONFIG_GRKERNSEC_TPE_GID=200


Is it possible to fix/workaround this issue?

my guess would be that MPROTECT prevents runtime code generation in some CUDA code, so try to disable that on the affected binary, probably the python interpreter. note that it's not ideal to disable MPROTECT on such a widely used program, might be safer to make a copy of it and use that for your CUDA endeavours.

worked, thanks, but I also had to disable RANDMMAP.