Kernel panic/PAX ?

[Virsh]

Hello!
Sorry if I write not in that section.
All I've managed to catch.

Code: Select all

[ 1584.738781] PAX: size overflow detected in function skb_headers_offset_update net/core/skbuff.c:1051 cicus.698_38 min, count: 10, decl: network_header; num: 0; context: sk_buff;
[ 1584.738929] Kernel panic - not syncing: Aiee, killing interrupt handler!
[ 1584.738963] CPU: 2 PID: 0 Comm: swapper/2 Not tainted 4.7.4-grsec #1
[ 1584.738989] Hardware name:
[ 1584.739033]  0000000000000046 0aa23a283aa7b298 ffff88022f303978 ffffffff814c2f8b
[ 1584.739066]  0000000000000020 0aa23a283aa7b298 ffffffff81d105a0 0000000000000009
[ 1584.739100]  ffff88022f303a10 ffffffff811c23b2 ffff88022f303a20 0000000000000008
[ 1584.739131] Call Trace:
[ 1584.739141]  <IRQ>  [<ffffffff814c2f8b>] dump_stack+0x60/0xb5
[ 1584.739169]  [<ffffffff811c23b2>] panic+0xe6/0x290
[ 1584.739189]  [<ffffffff810e6939>] ? vprintk_default+0x29/0x60
[ 1584.739210]  [<ffffffff81083e1e>] do_exit+0x90e/0xb90
[ 1584.739230]  [<ffffffff81023ed2>] ? show_stack_log_lvl+0x102/0x180
[ 1584.739253]  [<ffffffff8108415e>] do_group_exit+0x5e/0xf0
[ 1584.739272]  [<ffffffff8126c8bf>] report_size_overflow+0x7f/0x90
[ 1584.739294]  [<ffffffff8187e520>] skb_headers_offset_update+0x140/0x1d0
[ 1584.739318]  [<ffffffff81881245>] skb_copy_expand+0x115/0x1e0
[ 1584.739352]  [<ffffffffc050063f>] ieee80211_rx_handlers+0x15ef/0x2600 [mac80211]
[ 1584.739379]  [<ffffffff814e1955>] ? find_next_bit+0x15/0x40
[ 1584.739400]  [<ffffffff814c2d4f>] ? cpumask_next_and+0x2f/0x60
[ 1584.739430]  [<ffffffffc0501f02>] ieee80211_prepare_and_rx_handle+0x632/0x1600 [mac80211]
[ 1584.739459]  [<ffffffff8188af98>] ? __build_skb+0x48/0x240
[ 1584.739490]  [<ffffffffc050351a>] ieee80211_rx_napi+0x64a/0xca0 [mac80211]
[ 1584.739516]  [<ffffffff815002af>] ? swiotlb_tbl_sync_single+0x7f/0xa0
[ 1584.739541]  [<ffffffffc0d20b1c>] ath_rx_tasklet+0xb2c/0xe80 [ath9k]
[ 1584.739564]  [<ffffffff815006d0>] ? swiotlb_tbl_unmap_single+0x130/0x130
[ 1584.739590]  [<ffffffffc0d1da9e>] ath9k_tasklet+0xee/0x2b0 [ath9k]
[ 1584.739613]  [<ffffffff81085659>] tasklet_action+0x209/0x230
[ 1584.739634]  [<ffffffff819bdb1e>] __do_softirq+0x11e/0x2d4
[ 1584.739655]  [<ffffffff81085c43>] irq_exit+0x93/0xb0
[ 1584.739673]  [<ffffffff819bd774>] do_IRQ+0x54/0x110
[ 1584.739694]  [<ffffffff819bbc4b>] common_interrupt+0x8b/0x8b
[ 1584.739714]  <EOI>  [<ffffffff8181b029>] ? cpuidle_enter_state+0x129/0x2b0
[ 1584.739744]  [<ffffffff8181b217>] cpuidle_enter+0x17/0x30
[ 1584.739764]  [<ffffffff810cd8d3>] call_cpuidle+0x23/0x50
[ 1584.739783]  [<ffffffff810cdd87>] cpu_startup_entry+0x2a7/0x350
[ 1584.739805]  [<ffffffff81046acf>] start_secondary+0x24f/0x2f0
[ 1584.739860] Kernel Offset: disabled
[ 1584.745922] ---[ end Kernel panic - not syncing: Aiee, killing interrupt handler!

What can be done to roll back the kernel? The panic occurs with a specific access point.

[PaX Team]

this looks like the same issue as viewtopic.php?f=3&t=4448, can you follow the instructions there and help us find out the actual runtime values involved? also as a temporary workaround you can boot with pax_size_overflow_report_only to disable the reaction mechanism (it'll affect all size overflow reports though).

[Virsh]

this looks like the same issue as viewtopic.php?f=3&t=4448, can you follow the instructions there and help us find out the actual runtime values involved? also as a temporary workaround you can boot with pax_size_overflow_report_only to disable the reaction mechanism (it'll affect all size overflow reports though).

I found bug report https://bugs.gentoo.org/show_bug.cgi?id=584378 .
Write what you need to do a more detailed instruction. I'll try. How to boot from pax_size_overflow_report_only?

[PaX Team]

you should apply the patch i posted at https://bugs.gentoo.org/show_bug.cgi?id=584378#c1 and then show us the resulting logs (there will be "PAX: network_header...." messsages before the size overflow report, we'll need them too). pax_size_overflow_report_only is a kernel command line parameter, it's described in Documentation/kernel-parameters.txt.

[Virsh]

you should apply the patch i posted at https://bugs.gentoo.org/show_bug.cgi?id=584378#c1 and then show us the resulting logs (there will be "PAX: network_header...." messsages before the size overflow report, we'll need them too). pax_size_overflow_report_only is a kernel command line parameter, it's described in Documentation/kernel-parameters.txt.

patch skbuff.c patchtest

patching file skbuff.c
Hunk #1 FAILED at 973.
1 out of 1 hunk FAILED -- saving rejects to file skbuff.c.rej

patch --ignore-whitespace skbuff.c patchtest

Code: Select all

patching file skbuff.c
Hunk #1 succeeded at 1048 (offset 75 lines).

Is this normal? To compile the kernel?

[PaX Team]

use patch -l as the diff is whitespace damaged, or just apply it by hand, it's just a single line to add.

[Virsh]

use patch -l as the diff is whitespace damaged, or just apply it by hand, it's just a single line to add.

I boot new kernel
dmesg

Code: Select all

PAX:network_header:0 off:0

At the time of kernel panic, I don't see anything new. Caught via netconsole
What did I do wrong?
Here are the reports:
http://pastebin.com/0N0bnwQz
http://pastebin.com/Tds4uFwx
------------------------------
Need full dmesg? But I don't know how to get it.

[PaX Team]

pass pax_size_overflow_report_only to the kernel and it won't panic then you can grab dmesg easily.

[Virsh]

pass pax_size_overflow_report_only to the kernel and it won't panic then you can grab dmesg easily.

Ок.
http://pastebin.com/KALQTQ3K

[PaX Team]

thanks, the offending values are:

Code: Select all

[  307.843964] PAX: network_header:0 off:ffffffc4
[  307.843968] PAX: size overflow detected in function skb_headers_offset_update net/core/skbuff.c:1052 cicus.735_41 min, count: 22, decl: network_header; num: 0; context: sk_buff;


that is, the kernel is trying to reduce skb->network_header when it is already 0 which i believe is non-sensical and thus an upstream problem. as i already suggested in the gentoo bugzilla, you should report this to [email protected] and work with upstream developers to figure out what's going on.

[Virsh]

I send mail, my mail - gmail. Delivery to the following recipient failed permanently.
Gmail banned vger.kernel.org ? =)

[PaX Team]

it's probably because kernel mailing lists don't accept html email, you'll have to send plain text (also CC emese/spender/me please).

[Virsh]

Ok. I send mail via mutt...

[Virsh]

it's probably because kernel mailing lists don't accept html email, you'll have to send plain text (also CC emese/spender/me please).

Hello ! Can you say anything has changed regarding this bug?
P.s. In my experience this occurs on routers Dlink.

[PaX Team]

i haven't heard back anything so i have still no idea what to do with this...