increasing file-nr with kernel 3.4.7 and 3.6.5 + grsec 2.9.1

[rha]

Hi again

With kernels 3.4.7 and 3.6.5 (grsec patch 2.9.1), the first value of /proc/sys/fs/file-nr is increasing rapidly and will never decrease.
Within 30 days it hits the max value of 400'000, the kernel log then starts to show several of these: "VFS: file-max limit 413725 reached", forcing processes to fail.

We tried to reproduce this artificially with a perl script that opens file descriptors without closing them before exit, without luck.
The number of open file handles (lsof) over all processes does not explain this huge collection of allocated file descriptors.
It does not appear on all types of hardware.
One example where file-nr remains within normal range: Intel(R) Xeon(R) CPU E31220 @ 3.10GHz (quadcore)
One bad example, where file-nr reaches the limit: Intel(R) Core(TM)2 Duo CPU E7400 @ 2.80GHz

Removing the grsec patch solves the issue on both kernels.

Cheers,
Roman

[spender]

Can you reproduce it when applying only the PaX patch (from http://grsecurity.net/~paxguy1/)? Is the userland and kernel configuration exactly the same for both machines you list? Do you see any other grsec-related kernel logs?

-Brad

[rha]

Using http://grsecurity.net/~paxguy1/pax-linux-3.4.8-test32.patch on kernel 3.4.7, the file descriptor count behaves well.
Kernel and installed software were the same on both machines.

[spender]

Are you using NFS (or anything else unusual, as you're the first to ever report this problem)? Are there any grsec logs in the kernel? Was there any earlier patch of grsecurity you used that did not exhibit this problem? Can you also email [email protected] with the contents of your /proc/slabinfo (read as root) at the time when the file-nr limit is reached? You could also patch in grsecurity but disable CONFIG_GRKERNSEC to see if the problem was from an upstream patch that was backported.

-Brad

[rha]

You can find slabinfo and dmesg output in my email.

We are not using NFS.

The only grsec logs in the kernel occur after the file-nr limit has been reached:

Code: Select all

grsec: denied resource overstep by requesting 8392704 for RLIMIT_STACK against limit 8388608 for /opt/OSAGslog/sbin/syslog-ng[syslog-ng:30131] uid/euid:137/137 gid/egid:137/137, parent /sbin/init[init:1] uid/euid:0/0 gid/egid:0/0

syslog-ng[30131]: segfault at bf266d0c ip b7643467 sp bf266d10 error 6 in libc-2.13.so[b7606000+158000]
grsec: Segmentation fault occurred at bf266d0c in /opt/OSAGslog/sbin/syslog-ng[syslog-ng:30131] uid/euid:137/137 gid/egid:137/137, parent /sbin/init[init:1] uid/euid:0/0 gid/egid:0/0

The previous kernel we used was 2.6.32.52 with grsecurity-2.2.2-2.6.32.52-201201031758.patch where we did not have the issue.

While disabling CONFIG_GRKERNSEC on kernel 3.4.7, the issue disappears.

[spender]

Can you compile a kernel with sysctl support so that you can use the same kernel image to bisect config settings to see which is the culprit? I would first try to disable "consistent_setxid" and if that doesn't resolve the issue for that boot, then proceed with bisecting. The only thing I have to go on so far is that your cred_jar slab has close to the file-nr max number of objects (though the cred struct itself does nothing with files). Could you mail me your kernel .config as well?

Thanks,
-Brad

[rha]

Speaking of bisect, is there a repository of all grsec patches so we could look for the version that broke things?
How do you developers keep track of all the changes over time?

[spender]

It will be faster to bisect via sysctl than compiling a new kernel each time, since you've already narrowed it down to a grsecurity feature. If you want to though, you can try the latest patch for the 2.6.32.60 kernel to further determine if it's a problem arising from a forward port.

-Brad

[spender]

Looking at your config, start with disabling all chroot options via sysctl.

-Brad

[rha]

Kernel 2.6.32.60 with latest grsec 2.9.1 works fine. GRSEC kernel config was exactly the same as used on kernel 3.4.7.

Kernel 3.4.7 with all GRSEC features switched off by sysctl (/proc) still has the issue of file descriptor leakage.

Code: Select all

/proc/sys/kernel/grsecurity/chroot_caps:   0
/proc/sys/kernel/grsecurity/chroot_deny_chmod:   0
/proc/sys/kernel/grsecurity/chroot_deny_chroot:   0
/proc/sys/kernel/grsecurity/chroot_deny_fchdir:   0
/proc/sys/kernel/grsecurity/chroot_deny_mknod:   0
/proc/sys/kernel/grsecurity/chroot_deny_mount:   0
/proc/sys/kernel/grsecurity/chroot_deny_pivot:   0
/proc/sys/kernel/grsecurity/chroot_deny_shmat:   0
/proc/sys/kernel/grsecurity/chroot_deny_sysctl:   0
/proc/sys/kernel/grsecurity/chroot_deny_unix:   0
/proc/sys/kernel/grsecurity/chroot_enforce_chdir:   0
/proc/sys/kernel/grsecurity/chroot_findtask:   0
/proc/sys/kernel/grsecurity/chroot_restrict_nice:   0
/proc/sys/kernel/grsecurity/fifo_restrictions:   0
/proc/sys/kernel/grsecurity/linking_restrictions:   0
/proc/sys/kernel/grsecurity/resource_logging:   0
/proc/sys/kernel/grsecurity/signal_logging:   0

(I removed grsec_lock(=0) from the output, because it would prevent us from switching features and has no use for these tests.)

[spender]

Do you have a sense of whether the file-nr increases as new (unique) binaries/scripts are executed? I'll be mailing you a patch to test.

-Brad

[rha]

With the following script I can not make the number of allocated file descriptors increase faster:

Code: Select all

#!/bin/bash

rounds=1000

get_filenr() {
   filenr=`awk '{print $1}' < /proc/sys/fs/file-nr`
}

get_filenr
start_filenr=$filenr
echo "start file-nr: $filenr"

files=""
i=0
while [ $i -lt $rounds ]; do
   bin_name=`mktemp bash-$i-XXXXXXXXXX`
   # copy bash executable so it has a new inode
   install /bin/bash $bin_name
   # use different exit codes
   code=$(($i % 200))
   # fork & exec
   ./$bin_name -r -c "exit $code"
   echo -n "$? "
   # allow cleanup
   files="$bin_name $files"
   let i++
done
echo ""
rm $files

get_filenr
echo "stop file-nr: $filenr (rounds: $rounds, delta: $(($filenr-$start_filenr)))"


I am still testing your patch.

[rha]

Your patch solves the issue on kernels 3.2.33 and 3.4.7. I applied the same patch on top of those kernels after applying their corresponding grsec 2.9.1 patch.

For the 3.6.5 kernel I am still working on verification of the results. I have a hard time reproducing the issue with kernel 3.6.5 (without your patch).

[spender]

Great! I have confidence that it resolved the issue for 3.6 as well Thank you for your patience in seeing this through to a fix. It will be included in the next patches.

-Brad