PAX: size overflow detected in function try_merge_map

[mathias]

Running Linux 4.3.3 with grsecurity-3.1-4.3.3-201512162141.patch. While running a heavy I/O load (rsync), I see this error:

Code: Select all

[136981.023187] PAX: size overflow detected in function try_merge_map fs/btrfs/extent_map.c:238 cicus.109_102 max, count: 13, decl: block_len; num: 0; context: extent_map;
[136981.023216] CPU: 2 PID: 20881 Comm: apache2 Not tainted 4.3.3-grsec-201512162141 #1
[136981.023219] Hardware name: Supermicro X10SRA/X10SRA, BIOS 1.0a 11/27/2014
[136981.023222]  0000000000000000 0964917aa9f60d05 0000000000000000 ffffffffa04d6fe3
[136981.023227]  ffffffff812f879f ffffffffa04d6ff9 ffffffff811a9951 ffff88004bc70530
[136981.023231]  ffffc9001336b668 ffff880815cbac80 ffff880815ccd1e8 ffffffffa04498eb
[136981.023235] Call Trace:
[136981.023265]  [<ffffffffa04d6fe3>] ? __func__.45613+0x1a993/0x34d50 [btrfs]
[136981.023271]  [<ffffffff812f879f>] ? dump_stack+0x40/0x61
[136981.023285]  [<ffffffffa04d6ff9>] ? __func__.45613+0x1a9a9/0x34d50 [btrfs]
[136981.023290]  [<ffffffff811a9951>] ? report_size_overflow+0x31/0x40
[136981.023316]  [<ffffffffa04498eb>] ? try_merge_map+0x1fb/0x310 [btrfs]
[136981.023339]  [<ffffffffa0449c6b>] ? add_extent_mapping+0x12b/0x1b0 [btrfs]
[136981.023361]  [<ffffffffa042fe77>] ? btrfs_get_extent+0x6f7/0xdb0 [btrfs]
[136981.023384]  [<ffffffffa0454d84>] ? __do_readpage+0xc04/0xda0 [btrfs]
[136981.023405]  [<ffffffffa0450167>] ? insert_state+0x97/0x120 [btrfs]
[136981.023426]  [<ffffffffa042f780>] ? btrfs_set_bit_hook+0x250/0x250 [btrfs]
[136981.023432]  [<ffffffff81133004>] ? __alloc_pages_nodemask+0x1b4/0xa00
[136981.023452]  [<ffffffffa045510b>] ? __extent_read_full_page+0x1eb/0x220 [btrfs]
[136981.023470]  [<ffffffffa042f780>] ? btrfs_set_bit_hook+0x250/0x250 [btrfs]
[136981.023486]  [<ffffffffa042f780>] ? btrfs_set_bit_hook+0x250/0x250 [btrfs]
[136981.023506]  [<ffffffffa04568f9>] ? extent_read_full_page+0x59/0xa0 [btrfs]
[136981.023523]  [<ffffffffa042c570>] ? btrfs_endio_direct_write+0x110/0x110 [btrfs]
[136981.023527]  [<ffffffff8112a428>] ? do_read_cache_page+0x88/0x1b0
[136981.023543]  [<ffffffffa042c570>] ? btrfs_endio_direct_write+0x110/0x110 [btrfs]
[136981.023547]  [<ffffffff811b05c6>] ? page_getlink.isra.36.constprop.39+0x26/0x90
[136981.023550]  [<ffffffff811b0660>] ? page_follow_link_light+0x30/0x70
[136981.023554]  [<ffffffff811b3df7>] ? link_path_walk+0x5b7/0x610
[136981.023557]  [<ffffffff811b3f90>] ? path_lookupat+0x90/0x170
[136981.023560]  [<ffffffff811b65d4>] ? filename_lookup+0xd4/0x1c0
[136981.023566]  [<ffffffff8157956d>] ? unix_find_other+0x4d/0x2b0
[136981.023571]  [<ffffffff8157a453>] ? unix_stream_connect+0x103/0x500
[136981.023579]  [<ffffffff81497ae7>] ? SYSC_connect+0x107/0x140
[136981.023587]  [<ffffffff815e98ed>] ? entry_SYSCALL_64_fastpath+0x16/0x87
[136981.023591]  [<ffffffff815e991d>] ? entry_SYSCALL_64_fastpath+0x46/0x87

[ephox]

This is a known upstream bug. We are waiting for the fix. You can find the ticket here:
https://bugs.archlinux.org/task/47173

[ephox]

Could you please test this patch?
https://projects.archlinux.org/svntogit ... inux-grsec

[mathias]

ephox, I applied the patch you provided and it seems to have resolved the problem. I came up with a small test case of running rsync and dd concurrently which would reliably trigger the overflow error. When running the kernel with the btrfs patch, I was unable to trigger the error after multiple attempts.

Could you please test this patch?
https://projects.archlinux.org/svntogit ... inux-grsec

[Dwokfur]

I also hit this overflow (4.2.7-hardened), but I'm not sure whether there are any official fix out?
https://lkml.org/lkml/2015/11/27/207
https://bugs.gentoo.org/show_bug.cgi?id=567046
https://projects.archlinux.org/svntogit ... inux-grsec

[mathias]

As another data point, I was running 4.2.6-grsec-201511172005 before the 4.3.3 patch came out, and I never hit this issue when running my rsync tasks. I've since returned to the 4.2.6 kernel until some of the issues with the 4.3.3 patch get ironed out.

I also hit this overflow (4.2.7-hardened), but I'm not sure whether there are any official fix out?
https://lkml.org/lkml/2015/11/27/207
https://bugs.gentoo.org/show_bug.cgi?id=567046
https://projects.archlinux.org/svntogit ... inux-grsec

[Dwokfur]

I haven't experienced this problem for awhile, but the bug seems to be back using 4.7.10-hardened-r2:
PAX: size overflow detected in function try_merge_map fs/btrfs/extent_map.c:242 cicus.93_94 max, count: 5, decl: mod_len; num: 0; context: extent_map;
Is it a regression upstreams? Cicus numbers differ and there's no decl. Popped up during compiling gimp.

[ephox]

PAX: size overflow detected in function try_merge_map fs/btrfs/extent_map.c:242 cicus.93_94 max, count: 5, decl: mod_len; num: 0; context: extent_map;

Hi,
Could you please apply this patch and send me the result from dmesg?

Code: Select all

--- fs/btrfs/extent_map.c.orig  2016-11-14 00:19:54.877823835 +0100
+++ fs/btrfs/extent_map.c       2016-11-14 00:22:21.469817299 +0100
@@ -239,6 +239,7 @@
                            em->block_start != EXTENT_MAP_INLINE)
                                em->block_len += merge->block_len;
                        em->block_start = merge->block_start;
+                       printk(KERN_ERR "PAX em->mod_len: %llx em->mod_start: %llx merge->mod_start: %llx em->block_start: %llx em->block_len: %llx\n", em->mod_len, em->mod_start, merge->mod_start, em->block_start, em->block_len);
                        em->mod_len = (em->mod_len + em->mod_start) - merge->mod_start;
                        em->mod_start = merge->mod_start;
                        em->generation = max(em->generation, merge->generation);


[Dwokfur]

PAX: size overflow detected in function try_merge_map fs/btrfs/extent_map.c:242 cicus.93_94 max, count: 5, decl: mod_len; num: 0; context: extent_map;

Hi,
Could you please apply this patch and send me the result from dmesg?

Hi Ephox,

I've applied your patch and running the previous and my current kernel in report-only-size-overflow mode. So far there have been no occasion of another try_merge_map size overflow I described lately. I can see bursts of messages a couple of times each day. Like this:

Code: Select all

Nov 18 21:21:01 hostname kernel: PAX em->mod_len: 80000 em->mod_start: e00000 merge->mod_start: b00000 em->block_start: 18d1c5000 em->block_len: 380000


There are dozens of these right after eachother.
The prior occurrence had been triggered after a long list of package removals and installs by compiling a package.
I couldn't successfully reproduce the overflow so far. Unfortunately it seems to me it will be harder to reproduce than I initially thought.
I keep the patch applied and watch out for an overflow.

BR: Dw.

[Dwokfur]

I couldn't successfully reproduce the overflow so far. Unfortunately it seems to me it will be harder to reproduce than I initially thought.
I keep the patch applied and watch out for an overflow.

BR: Dw.

I'm reverting back to ext4 from btrfs due to other problems unrealted to this problem. I couldn't reproduce the issue so far. BR: Dw.