[10276.079844] rcu: INFO: rcu_preempt detected stalls on CPUs/tasks: [10276.080438] rcu: 1-...!: (1 GPs behind) idle=bf6c/1/0x4000000000000000 softirq=232343/232345 fqs=8 [10276.081148] rcu: (detected by 3, t=9080037 jiffies, g=565849, q=98 ncpus=4) [10276.081699] Sending NMI from CPU 3 to CPUs 1: [10276.082068] NMI backtrace for cpu 1 [10276.082079] CPU: 1 PID: 22 Comm: ksoftirqd/1 Not tainted 6.8.0-rc3-virtme #1 [10276.082083] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS rel-1.16.3-0-ga6ed6b701f0a-prebuilt.qemu.org 04/01/2014 [10276.082090] RIP: 0010:queued_spin_lock_slowpath+0x36/0x310 [10276.082104] Code: 54 55 48 89 fd 53 66 90 ba 01 00 00 00 8b 45 00 85 c0 75 14 f0 0f b1 55 00 85 c0 75 f0 5b 5d 41 5c 41 5d c3 cc cc cc cc f3 90 e1 81 fe 00 01 00 00 74 52 40 30 f6 85 f6 75 75 f0 0f ba 6d 00 [10276.082107] RSP: 0000:ffffb29e400d35c8 EFLAGS: 00000202 [10276.082111] RAX: 0000000000000001 RBX: ffff9e43c5d40c98 RCX: 0000000000000001 [10276.082113] RDX: 0000000000000001 RSI: 0000000000000001 RDI: ffff9e43c5d40c98 [10276.082114] RBP: ffff9e43c5d40c98 R08: ffffb29e400d3388 R09: 0000000000000020 [10276.082116] R10: ffffb29e400d3478 R11: 0000000000000001 R12: 0000000000000000 [10276.082118] R13: ffff9e43c5d40c00 R14: ffff9e43c5d40c98 R15: ffff9e43c256e4cc [10276.082122] FS: 0000000000000000(0000) GS:ffff9e43fec80000(0000) knlGS:0000000000000000 [10276.082124] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [10276.082126] CR2: 000055f2d0684000 CR3: 0000000003e5c003 CR4: 0000000000770ef0 [10276.082128] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 [10276.082129] DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400 [10276.082131] PKRU: 55555554 [10276.082132] Call Trace: [10276.082142] [10276.082151] ? nmi_cpu_backtrace+0xa2/0x110 [10276.082157] ? nmi_cpu_backtrace_handler+0x11/0x20 [10276.082163] ? nmi_handle+0xd0/0x270 [10276.082168] ? default_do_nmi+0x41/0x100 [10276.082173] ? exc_nmi+0xf3/0x120 [10276.082176] ? end_repeat_nmi+0xf/0x60 [10276.082185] ? queued_spin_lock_slowpath+0x36/0x310 [10276.082188] ? queued_spin_lock_slowpath+0x36/0x310 [10276.082191] ? queued_spin_lock_slowpath+0x36/0x310 [10276.082195] [10276.082195] [10276.082197] do_raw_spin_lock+0x9b/0xa0 [10276.082205] tcp_v4_rcv+0xdb5/0xef0 [10276.082215] ip_protocol_deliver_rcu+0x4b/0x1e0 [10276.082221] ip_local_deliver_finish+0xaf/0x180 [10276.082223] __netif_receive_skb_one_core+0x8b/0xa0 [10276.082228] netif_receive_skb+0x87/0x2c0 [10276.082234] tcf_mirred_to_dev+0x1f6/0x410 [act_mirred] [10276.082244] tcf_mirred_act+0xee/0x560 [act_mirred] [10276.082251] ? tcf_skbedit_act+0x1d6/0x410 [act_skbedit] [10276.082259] tcf_action_exec+0xac/0x230 [10276.082266] fl_classify+0x20e/0x260 [cls_flower] [10276.082278] ? __bfs+0xf3/0x240 [10276.082282] ? check_irq_usage+0x19c/0xc40 [10276.082285] ? find_held_lock+0x2b/0x80 [10276.082289] ? fib_rules_lookup+0x175/0x3b0 [10276.082295] ? check_path.constprop.0+0x24/0x50 [10276.082297] ? check_noncircular+0x81/0x150 [10276.082304] ? __lock_acquire+0xa6e/0x1750 [10276.082311] tcf_classify+0x17a/0x540 [10276.082318] tc_run+0x9f/0x150 [10276.082327] __dev_queue_xmit+0x311/0x1180 [10276.082333] ? eth_header+0x2a/0xc0 [10276.082340] ip_finish_output2+0x22d/0xa20 [10276.082344] ? ip_skb_dst_mtu+0x51/0x1c0 [10276.082348] ? __ip_queue_xmit+0x1d5/0x700 [10276.082351] __ip_queue_xmit+0x1d5/0x700 [10276.082356] __tcp_transmit_skb+0xb65/0xd30 [10276.082362] ? mark_held_locks+0x49/0x80 [10276.082367] tcp_write_xmit+0x492/0x1400 [10276.082372] ? __pfx_tcp_write_timer+0x10/0x10 [10276.082375] tcp_send_loss_probe+0x183/0x260 [10276.082379] tcp_write_timer+0xb2/0x120 [10276.082382] ? __pfx_tcp_write_timer+0x10/0x10 [10276.082384] call_timer_fn+0xa5/0x220 [10276.082390] __run_timers.part.0+0x1f8/0x2c0 [10276.082393] ? rcu_core+0x5ed/0xeb0 [10276.082400] run_timer_softirq+0x2a/0x50 [10276.082402] __do_softirq+0xc8/0x37a [10276.082406] ? __pfx_smpboot_thread_fn+0x10/0x10 [10276.082413] run_ksoftirqd+0x2e/0x40 [10276.082418] smpboot_thread_fn+0xd9/0x1d0 [10276.082422] kthread+0xda/0x110 [10276.082426] ? __pfx_kthread+0x10/0x10 [10276.082429] ret_from_fork+0x31/0x50 [10276.082434] ? __pfx_kthread+0x10/0x10 [10276.082437] ret_from_fork_asm+0x1b/0x30 [10276.082447] [10276.083052] rcu: rcu_preempt kthread starved for 9080003 jiffies! g565849 f0x0 RCU_GP_WAIT_FQS(5) ->state=0x0 ->cpu=0 [10276.263531] rcu: Unless rcu_preempt kthread gets sufficient CPU time, OOM is now expected behavior. [10276.264248] rcu: RCU grace-period kthread stack dump: [10276.264633] task:rcu_preempt state:R running task stack:14760 pid:17 tgid:17 ppid:2 flags:0x00004000 [10276.265479] Call Trace: [10276.265684] [10276.265873] __schedule+0x33e/0xc70 [10276.266175] ? __pfx_rcu_gp_kthread+0x10/0x10 [10276.266530] schedule+0x49/0x110 [10276.266798] schedule_timeout+0xc8/0x1b0 [10276.267128] ? __pfx_process_timeout+0x10/0x10 [10276.267503] ? mt_destroy_walk+0x130/0x540 [10276.267846] rcu_gp_fqs_loop+0x145/0x830 [10276.268179] rcu_gp_kthread+0x202/0x2e0 [10276.268505] kthread+0xda/0x110 [10276.268775] ? __pfx_kthread+0x10/0x10 [10276.269092] ret_from_fork+0x31/0x50 [10276.269395] ? __pfx_kthread+0x10/0x10 [10276.269703] ret_from_fork_asm+0x1b/0x30 [10276.270041] [10276.310362] rcu: Stack dump where RCU GP kthread last ran: [10276.310793] Sending NMI from CPU 3 to CPUs 0: [10276.311166] NMI backtrace for cpu 0 [10276.311173] CPU: 0 PID: 0 Comm: swapper/0 Not tainted 6.8.0-rc3-virtme #1 [10276.311177] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS rel-1.16.3-0-ga6ed6b701f0a-prebuilt.qemu.org 04/01/2014 [10276.311180] RIP: 0010:queued_spin_lock_slowpath+0x36/0x310 [10276.311191] Code: 54 55 48 89 fd 53 66 90 ba 01 00 00 00 8b 45 00 85 c0 75 14 f0 0f b1 55 00 85 c0 75 f0 5b 5d 41 5c 41 5d c3 cc cc cc cc f3 90 e1 81 fe 00 01 00 00 74 52 40 30 f6 85 f6 75 75 f0 0f ba 6d 00 [10276.311194] RSP: 0018:ffffb29e40003748 EFLAGS: 00000202 [10276.311197] RAX: 0000000000000001 RBX: ffff9e43c5d40098 RCX: 0000000000000001 [10276.311199] RDX: 0000000000000001 RSI: 0000000000000001 RDI: ffff9e43c5d40098 [10276.311200] RBP: ffff9e43c5d40098 R08: ffffb29e40003508 R09: 0000000000000020 [10276.311202] R10: ffffb29e400035f8 R11: 0000000000000001 R12: 0000000000000000 [10276.311203] R13: ffff9e43c5d40000 R14: ffff9e43c5d40098 R15: ffff9e43c256d08c [10276.311207] FS: 0000000000000000(0000) GS:ffff9e43fec00000(0000) knlGS:0000000000000000 [10276.311209] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [10276.311211] CR2: 00007ff4c98b2000 CR3: 0000000002b7c003 CR4: 0000000000770ef0 [10276.311213] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 [10276.311214] DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400 [10276.311215] PKRU: 55555554 [10276.311216] Call Trace: [10276.311221] [10276.311226] ? nmi_cpu_backtrace+0xa2/0x110 [10276.311233] ? nmi_cpu_backtrace_handler+0x11/0x20 [10276.311239] ? nmi_handle+0xd0/0x270 [10276.311244] ? default_do_nmi+0x41/0x100 [10276.311248] ? exc_nmi+0xf3/0x120 [10276.311251] ? end_repeat_nmi+0xf/0x60 [10276.311259] ? queued_spin_lock_slowpath+0x36/0x310 [10276.311262] ? queued_spin_lock_slowpath+0x36/0x310 [10276.311265] ? queued_spin_lock_slowpath+0x36/0x310 [10276.311268] [10276.311269] [10276.311271] do_raw_spin_lock+0x9b/0xa0 [10276.311278] tcp_v4_rcv+0xdb5/0xef0 [10276.311288] ip_protocol_deliver_rcu+0x4b/0x1e0 [10276.311294] ip_local_deliver_finish+0xaf/0x180 [10276.311297] __netif_receive_skb_one_core+0x8b/0xa0 [10276.311302] netif_receive_skb+0x87/0x2c0 [10276.311308] tcf_mirred_to_dev+0x1f6/0x410 [act_mirred] [10276.311317] tcf_mirred_act+0xee/0x560 [act_mirred] [10276.311325] ? tcf_skbedit_act+0x1d6/0x410 [act_skbedit] [10276.311331] tcf_action_exec+0xac/0x230 [10276.311339] fl_classify+0x20e/0x260 [cls_flower] [10276.311351] ? __bfs+0xf3/0x240 [10276.311355] ? check_irq_usage+0x19c/0xc40 [10276.311359] ? find_held_lock+0x2b/0x80 [10276.311364] ? check_path.constprop.0+0x24/0x50 [10276.311367] ? check_noncircular+0x81/0x150 [10276.311371] ? update_load_avg+0x6d/0x740 [10276.311378] ? __lock_acquire+0xa6e/0x1750 [10276.311384] tcf_classify+0x17a/0x540 [10276.311390] tc_run+0x9f/0x150 [10276.311398] __dev_queue_xmit+0x311/0x1180 [10276.311403] ? mark_held_locks+0x49/0x80 [10276.311406] ? eth_header+0x2a/0xc0 [10276.311412] ip_finish_output2+0x22d/0xa20 [10276.311416] ? ip_skb_dst_mtu+0x51/0x1c0 [10276.311420] ? __ip_queue_xmit+0x1d5/0x700 [10276.311423] __ip_queue_xmit+0x1d5/0x700 [10276.311427] __tcp_transmit_skb+0xb65/0xd30 [10276.311434] ? __alloc_skb+0xdc/0x1a0 [10276.311439] ? __pfx_tcp_delack_timer+0x10/0x10 [10276.311442] tcp_delack_timer_handler+0xa3/0xf0 [10276.311445] tcp_delack_timer+0x5a/0x140 [10276.311447] ? __pfx_tcp_delack_timer+0x10/0x10 [10276.311449] call_timer_fn+0xa5/0x220 [10276.311456] __run_timers.part.0+0x1f8/0x2c0 [10276.311460] ? kvm_clock_get_cycles+0x18/0x30 [10276.311468] run_timer_softirq+0x2a/0x50 [10276.311470] __do_softirq+0xc8/0x37a [10276.311475] irq_exit_rcu+0x7f/0xa0 [10276.311479] sysvec_apic_timer_interrupt+0x78/0x90 [10276.311484] [10276.311485] [10276.311487] asm_sysvec_apic_timer_interrupt+0x1a/0x20 [10276.311490] RIP: 0010:default_idle+0xf/0x20 [10276.311493] Code: 4c 01 c7 4c 29 c2 e9 72 ff ff ff 90 90 90 90 90 90 90 90 90 90 90 90 90 90 90 90 f3 0f 1e fa 66 90 0f 00 2d 13 ae 3f 00 fb f4 c3 cc cc cc cc 66 66 2e 0f 1f 84 00 00 00 00 00 90 90 90 90 90 [10276.311496] RSP: 0018:ffffffff8bc03e88 EFLAGS: 00000242 [10276.311498] RAX: 000000000408f73b RBX: ffffffff8bc12a00 RCX: 4000000000000000 [10276.311500] RDX: 0000000000000000 RSI: 0000000000000000 RDI: ffffffff8a3a8863 [10276.311501] RBP: 0000000000000000 R08: 0000000000000001 R09: 0000000000000001 [10276.311502] R10: 0000000000000000 R11: 0000000000000001 R12: 0000000000000000 [10276.311504] R13: 0000000000000000 R14: ffffffff8bc12080 R15: 0000000000013d50 [10276.311508] ? do_idle+0x1e3/0x220 [10276.311515] default_idle_call+0x83/0x180 [10276.311517] do_idle+0x1e3/0x220 [10276.311520] ? finish_task_switch.isra.0+0xf7/0x330 [10276.311526] cpu_startup_entry+0x2a/0x30 [10276.311529] rest_init+0xe9/0x180 [10276.311532] arch_call_rest_init+0xe/0x30 [10276.311538] start_kernel+0x5b2/0x690 [10276.311543] x86_64_start_reservations+0x18/0x30 [10276.311547] x86_64_start_kernel+0xc6/0xe0 [10276.311550] secondary_startup_64_no_verify+0x16d/0x17b [10276.311560] [10354.319822] rcu: INFO: rcu_preempt detected stalls on CPUs/tasks: [10354.320451] rcu: 1-...!: (1 GPs behind) idle=bf6c/1/0x4000000000000000 softirq=232343/232345 fqs=8 [10354.321233] rcu: (detected by 3, t=9158277 jiffies, g=565849, q=98 ncpus=4) [10354.321871] Sending NMI from CPU 3 to CPUs 1: [10354.322277] NMI backtrace for cpu 1 [10354.322284] CPU: 1 PID: 22 Comm: ksoftirqd/1 Not tainted 6.8.0-rc3-virtme #1 [10354.322288] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS rel-1.16.3-0-ga6ed6b701f0a-prebuilt.qemu.org 04/01/2014 [10354.322291] RIP: 0010:queued_spin_lock_slowpath+0x36/0x310 [10354.322313] Code: 54 55 48 89 fd 53 66 90 ba 01 00 00 00 8b 45 00 85 c0 75 14 f0 0f b1 55 00 85 c0 75 f0 5b 5d 41 5c 41 5d c3 cc cc cc cc f3 90 e1 81 fe 00 01 00 00 74 52 40 30 f6 85 f6 75 75 f0 0f ba 6d 00 [10354.322316] RSP: 0000:ffffb29e400d35c8 EFLAGS: 00000202 [10354.322319] RAX: 0000000000000001 RBX: ffff9e43c5d40c98 RCX: 0000000000000001 [10354.322321] RDX: 0000000000000001 RSI: 0000000000000001 RDI: ffff9e43c5d40c98 [10354.322322] RBP: ffff9e43c5d40c98 R08: ffffb29e400d3388 R09: 0000000000000020 [10354.322324] R10: ffffb29e400d3478 R11: 0000000000000001 R12: 0000000000000000 [10354.322326] R13: ffff9e43c5d40c00 R14: ffff9e43c5d40c98 R15: ffff9e43c256e4cc [10354.322330] FS: 0000000000000000(0000) GS:ffff9e43fec80000(0000) knlGS:0000000000000000 [10354.322332] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [10354.322334] CR2: 000055f2d0684000 CR3: 0000000003e5c003 CR4: 0000000000770ef0 [10354.322336] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 [10354.322337] DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400 [10354.322339] PKRU: 55555554 [10354.322340] Call Trace: [10354.322344] [10354.322349] ? nmi_cpu_backtrace+0xa2/0x110 [10354.322356] ? nmi_cpu_backtrace_handler+0x11/0x20 [10354.322363] ? nmi_handle+0xd0/0x270 [10354.322368] ? default_do_nmi+0x41/0x100 [10354.322373] ? exc_nmi+0xf3/0x120 [10354.322376] ? end_repeat_nmi+0xf/0x60 [10354.322385] ? queued_spin_lock_slowpath+0x36/0x310 [10354.322389] ? queued_spin_lock_slowpath+0x36/0x310 [10354.322392] ? queued_spin_lock_slowpath+0x36/0x310 [10354.322395] [10354.322396] [10354.322398] do_raw_spin_lock+0x9b/0xa0 [10354.322405] tcp_v4_rcv+0xdb5/0xef0 [10354.322413] ip_protocol_deliver_rcu+0x4b/0x1e0 [10354.322419] ip_local_deliver_finish+0xaf/0x180 [10354.322422] __netif_receive_skb_one_core+0x8b/0xa0 [10354.322427] netif_receive_skb+0x87/0x2c0 [10354.322433] tcf_mirred_to_dev+0x1f6/0x410 [act_mirred] [10354.322446] tcf_mirred_act+0xee/0x560 [act_mirred] [10354.322454] ? tcf_skbedit_act+0x1d6/0x410 [act_skbedit] [10354.322461] tcf_action_exec+0xac/0x230 [10354.322468] fl_classify+0x20e/0x260 [cls_flower] [10354.322481] ? __bfs+0xf3/0x240 [10354.322485] ? check_irq_usage+0x19c/0xc40 [10354.322489] ? find_held_lock+0x2b/0x80 [10354.322492] ? fib_rules_lookup+0x175/0x3b0 [10354.322498] ? check_path.constprop.0+0x24/0x50 [10354.322501] ? check_noncircular+0x81/0x150 [10354.322506] ? __lock_acquire+0xa6e/0x1750 [10354.322512] tcf_classify+0x17a/0x540 [10354.322517] tc_run+0x9f/0x150 [10354.322524] __dev_queue_xmit+0x311/0x1180 [10354.322529] ? eth_header+0x2a/0xc0 [10354.322535] ip_finish_output2+0x22d/0xa20 [10354.322538] ? ip_skb_dst_mtu+0x51/0x1c0 [10354.322542] ? __ip_queue_xmit+0x1d5/0x700 [10354.322544] __ip_queue_xmit+0x1d5/0x700 [10354.322549] __tcp_transmit_skb+0xb65/0xd30 [10354.322556] ? mark_held_locks+0x49/0x80 [10354.322561] tcp_write_xmit+0x492/0x1400 [10354.322566] ? __pfx_tcp_write_timer+0x10/0x10 [10354.322569] tcp_send_loss_probe+0x183/0x260 [10354.322573] tcp_write_timer+0xb2/0x120 [10354.322576] ? __pfx_tcp_write_timer+0x10/0x10 [10354.322578] call_timer_fn+0xa5/0x220 [10354.322584] __run_timers.part.0+0x1f8/0x2c0 [10354.322587] ? rcu_core+0x5ed/0xeb0 [10354.322594] run_timer_softirq+0x2a/0x50 [10354.322596] __do_softirq+0xc8/0x37a [10354.322599] ? __pfx_smpboot_thread_fn+0x10/0x10 [10354.322606] run_ksoftirqd+0x2e/0x40 [10354.322611] smpboot_thread_fn+0xd9/0x1d0 [10354.322615] kthread+0xda/0x110 [10354.322619] ? __pfx_kthread+0x10/0x10 [10354.322622] ret_from_fork+0x31/0x50 [10354.322627] ? __pfx_kthread+0x10/0x10 [10354.322630] ret_from_fork_asm+0x1b/0x30 [10354.322640] [10354.323269] rcu: rcu_preempt kthread starved for 9158243 jiffies! g565849 f0x2 RCU_GP_WAIT_FQS(5) ->state=0x0 ->cpu=0 [10354.444021] rcu: Unless rcu_preempt kthread gets sufficient CPU time, OOM is now expected behavior. [10354.444760] rcu: RCU grace-period kthread stack dump: [10354.445125] task:rcu_preempt state:R running task stack:14760 pid:17 tgid:17 ppid:2 flags:0x00004000 [10354.445964] Call Trace: [10354.446183] [10354.446378] __schedule+0x33e/0xc70 [10354.446699] ? __pfx_rcu_gp_kthread+0x10/0x10 [10354.447071] schedule+0x49/0x110 [10354.447358] schedule_timeout+0xc8/0x1b0 [10354.447701] ? __pfx_process_timeout+0x10/0x10 [10354.448086] ? mt_destroy_walk+0x130/0x540 [10354.448437] rcu_gp_fqs_loop+0x145/0x830 [10354.448776] rcu_gp_kthread+0x202/0x2e0 [10354.449111] kthread+0xda/0x110 [10354.449400] ? __pfx_kthread+0x10/0x10 [10354.449723] ret_from_fork+0x31/0x50 [10354.490142] ? __pfx_kthread+0x10/0x10 [10354.490454] ret_from_fork_asm+0x1b/0x30 [10354.490796] [10354.490993] rcu: Stack dump where RCU GP kthread last ran: [10354.491446] Sending NMI from CPU 3 to CPUs 0: [10354.521352] NMI backtrace for cpu 0 [10354.521360] CPU: 0 PID: 0 Comm: swapper/0 Not tainted 6.8.0-rc3-virtme #1 [10354.521364] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS rel-1.16.3-0-ga6ed6b701f0a-prebuilt.qemu.org 04/01/2014 [10354.521367] RIP: 0010:queued_spin_lock_slowpath+0x36/0x310 [10354.521378] Code: 54 55 48 89 fd 53 66 90 ba 01 00 00 00 8b 45 00 85 c0 75 14 f0 0f b1 55 00 85 c0 75 f0 5b 5d 41 5c 41 5d c3 cc cc cc cc f3 90 e1 81 fe 00 01 00 00 74 52 40 30 f6 85 f6 75 75 f0 0f ba 6d 00 [10354.521381] RSP: 0018:ffffb29e40003748 EFLAGS: 00000202 [10354.521384] RAX: 0000000000000001 RBX: ffff9e43c5d40098 RCX: 0000000000000001 [10354.521386] RDX: 0000000000000001 RSI: 0000000000000001 RDI: ffff9e43c5d40098 [10354.521387] RBP: ffff9e43c5d40098 R08: ffffb29e40003508 R09: 0000000000000020 [10354.521389] R10: ffffb29e400035f8 R11: 0000000000000001 R12: 0000000000000000 [10354.521390] R13: ffff9e43c5d40000 R14: ffff9e43c5d40098 R15: ffff9e43c256d08c [10354.521396] FS: 0000000000000000(0000) GS:ffff9e43fec00000(0000) knlGS:0000000000000000 [10354.521398] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [10354.521400] CR2: 00007ff4c98b2000 CR3: 0000000002b7c003 CR4: 0000000000770ef0 [10354.521402] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 [10354.521403] DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400 [10354.521404] PKRU: 55555554 [10354.521406] Call Trace: [10354.521412] [10354.521418] ? nmi_cpu_backtrace+0xa2/0x110 [10354.521424] ? nmi_cpu_backtrace_handler+0x11/0x20 [10354.521430] ? nmi_handle+0xd0/0x270 [10354.521435] ? default_do_nmi+0x41/0x100 [10354.521440] ? exc_nmi+0xf3/0x120 [10354.521443] ? end_repeat_nmi+0xf/0x60 [10354.521453] ? queued_spin_lock_slowpath+0x36/0x310 [10354.521457] ? queued_spin_lock_slowpath+0x36/0x310 [10354.521460] ? queued_spin_lock_slowpath+0x36/0x310 [10354.521463] [10354.521464] [10354.521466] do_raw_spin_lock+0x9b/0xa0 [10354.521474] tcp_v4_rcv+0xdb5/0xef0 [10354.521484] ip_protocol_deliver_rcu+0x4b/0x1e0 [10354.521490] ip_local_deliver_finish+0xaf/0x180 [10354.521493] __netif_receive_skb_one_core+0x8b/0xa0 [10354.521498] netif_receive_skb+0x87/0x2c0 [10354.521504] tcf_mirred_to_dev+0x1f6/0x410 [act_mirred] [10354.521513] tcf_mirred_act+0xee/0x560 [act_mirred] [10354.521521] ? tcf_skbedit_act+0x1d6/0x410 [act_skbedit] [10354.521529] tcf_action_exec+0xac/0x230 [10354.521537] fl_classify+0x20e/0x260 [cls_flower] [10354.521548] ? __bfs+0xf3/0x240 [10354.521553] ? check_irq_usage+0x19c/0xc40 [10354.521556] ? find_held_lock+0x2b/0x80 [10354.521561] ? check_path.constprop.0+0x24/0x50 [10354.521564] ? check_noncircular+0x81/0x150 [10354.521567] ? update_load_avg+0x6d/0x740 [10354.521573] ? __lock_acquire+0xa6e/0x1750 [10354.521580] tcf_classify+0x17a/0x540 [10354.521586] tc_run+0x9f/0x150 [10354.521593] __dev_queue_xmit+0x311/0x1180 [10354.521598] ? mark_held_locks+0x49/0x80 [10354.521602] ? eth_header+0x2a/0xc0 [10354.521608] ip_finish_output2+0x22d/0xa20 [10354.521611] ? ip_skb_dst_mtu+0x51/0x1c0 [10354.521616] ? __ip_queue_xmit+0x1d5/0x700 [10354.521618] __ip_queue_xmit+0x1d5/0x700 [10354.521623] __tcp_transmit_skb+0xb65/0xd30 [10354.521630] ? __alloc_skb+0xdc/0x1a0 [10354.521634] ? __pfx_tcp_delack_timer+0x10/0x10 [10354.521637] tcp_delack_timer_handler+0xa3/0xf0 [10354.521640] tcp_delack_timer+0x5a/0x140 [10354.521642] ? __pfx_tcp_delack_timer+0x10/0x10 [10354.521644] call_timer_fn+0xa5/0x220 [10354.521650] __run_timers.part.0+0x1f8/0x2c0 [10354.521655] ? kvm_clock_get_cycles+0x18/0x30 [10354.521663] run_timer_softirq+0x2a/0x50 [10354.521665] __do_softirq+0xc8/0x37a [10354.521669] irq_exit_rcu+0x7f/0xa0 [10354.521674] sysvec_apic_timer_interrupt+0x78/0x90 [10354.521679] [10354.521679] [10354.521681] asm_sysvec_apic_timer_interrupt+0x1a/0x20 [10354.521685] RIP: 0010:default_idle+0xf/0x20 [10354.521688] Code: 4c 01 c7 4c 29 c2 e9 72 ff ff ff 90 90 90 90 90 90 90 90 90 90 90 90 90 90 90 90 f3 0f 1e fa 66 90 0f 00 2d 13 ae 3f 00 fb f4 c3 cc cc cc cc 66 66 2e 0f 1f 84 00 00 00 00 00 90 90 90 90 90 [10354.521690] RSP: 0018:ffffffff8bc03e88 EFLAGS: 00000242 [10354.521693] RAX: 000000000408f73b RBX: ffffffff8bc12a00 RCX: 4000000000000000 [10354.521695] RDX: 0000000000000000 RSI: 0000000000000000 RDI: ffffffff8a3a8863 [10354.521696] RBP: 0000000000000000 R08: 0000000000000001 R09: 0000000000000001 [10354.521697] R10: 0000000000000000 R11: 0000000000000001 R12: 0000000000000000 [10354.521698] R13: 0000000000000000 R14: ffffffff8bc12080 R15: 0000000000013d50 [10354.521703] ? do_idle+0x1e3/0x220 [10354.521709] default_idle_call+0x83/0x180 [10354.521711] do_idle+0x1e3/0x220 [10354.521714] ? finish_task_switch.isra.0+0xf7/0x330 [10354.521719] cpu_startup_entry+0x2a/0x30 [10354.521722] rest_init+0xe9/0x180 [10354.521725] arch_call_rest_init+0xe/0x30 [10354.521732] start_kernel+0x5b2/0x690 [10354.521736] x86_64_start_reservations+0x18/0x30 [10354.521740] x86_64_start_kernel+0xc6/0xe0 [10354.521743] secondary_startup_64_no_verify+0x16d/0x17b [10354.521752] [10432.538827] rcu: INFO: rcu_preempt detected stalls on CPUs/tasks: [10432.539392] rcu: 1-...!: (1 GPs behind) idle=bf6c/1/0x4000000000000000 softirq=232343/232345 fqs=8 [10432.540108] rcu: (detected by 3, t=9236495 jiffies, g=565849, q=98 ncpus=4) [10432.540682] Sending NMI from CPU 3 to CPUs 1: [10432.554814] NMI backtrace for cpu 1 [10432.554823] CPU: 1 PID: 22 Comm: ksoftirqd/1 Not tainted 6.8.0-rc3-virtme #1 [10432.554827] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS rel-1.16.3-0-ga6ed6b701f0a-prebuilt.qemu.org 04/01/2014 [10432.554830] RIP: 0010:queued_spin_lock_slowpath+0x36/0x310 [10432.554842] Code: 54 55 48 89 fd 53 66 90 ba 01 00 00 00 8b 45 00 85 c0 75 14 f0 0f b1 55 00 85 c0 75 f0 5b 5d 41 5c 41 5d c3 cc cc cc cc f3 90 e1 81 fe 00 01 00 00 74 52 40 30 f6 85 f6 75 75 f0 0f ba 6d 00 [10432.554844] RSP: 0000:ffffb29e400d35c8 EFLAGS: 00000202 [10432.554847] RAX: 0000000000000001 RBX: ffff9e43c5d40c98 RCX: 0000000000000001 [10432.554849] RDX: 0000000000000001 RSI: 0000000000000001 RDI: ffff9e43c5d40c98 [10432.554851] RBP: ffff9e43c5d40c98 R08: ffffb29e400d3388 R09: 0000000000000020 [10432.554852] R10: ffffb29e400d3478 R11: 0000000000000001 R12: 0000000000000000 [10432.554854] R13: ffff9e43c5d40c00 R14: ffff9e43c5d40c98 R15: ffff9e43c256e4cc [10432.554860] FS: 0000000000000000(0000) GS:ffff9e43fec80000(0000) knlGS:0000000000000000 [10432.554862] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [10432.554864] CR2: 000055f2d0684000 CR3: 0000000003e5c003 CR4: 0000000000770ef0 [10432.554866] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 [10432.554867] DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400 [10432.554869] PKRU: 55555554 [10432.554870] Call Trace: [10432.554874] [10432.554880] ? nmi_cpu_backtrace+0xa2/0x110 [10432.554887] ? nmi_cpu_backtrace_handler+0x11/0x20 [10432.554894] ? nmi_handle+0xd0/0x270 [10432.554899] ? default_do_nmi+0x41/0x100 [10432.554903] ? exc_nmi+0xf3/0x120 [10432.554906] ? end_repeat_nmi+0xf/0x60 [10432.554915] ? queued_spin_lock_slowpath+0x36/0x310 [10432.554918] ? queued_spin_lock_slowpath+0x36/0x310 [10432.554922] ? queued_spin_lock_slowpath+0x36/0x310 [10432.554925] [10432.554926] [10432.554928] do_raw_spin_lock+0x9b/0xa0 [10432.554935] tcp_v4_rcv+0xdb5/0xef0 [10432.554946] ip_protocol_deliver_rcu+0x4b/0x1e0 [10432.554952] ip_local_deliver_finish+0xaf/0x180 [10432.554954] __netif_receive_skb_one_core+0x8b/0xa0 [10432.554959] netif_receive_skb+0x87/0x2c0 [10432.554965] tcf_mirred_to_dev+0x1f6/0x410 [act_mirred] [10432.554974] tcf_mirred_act+0xee/0x560 [act_mirred] [10432.554981] ? tcf_skbedit_act+0x1d6/0x410 [act_skbedit] [10432.554989] tcf_action_exec+0xac/0x230 [10432.554996] fl_classify+0x20e/0x260 [cls_flower] [10432.555009] ? __bfs+0xf3/0x240 [10432.555013] ? check_irq_usage+0x19c/0xc40 [10432.555017] ? find_held_lock+0x2b/0x80 [10432.555020] ? fib_rules_lookup+0x175/0x3b0 [10432.555026] ? check_path.constprop.0+0x24/0x50 [10432.555029] ? check_noncircular+0x81/0x150 [10432.555035] ? __lock_acquire+0xa6e/0x1750 [10432.555041] tcf_classify+0x17a/0x540 [10432.555048] tc_run+0x9f/0x150 [10432.555057] __dev_queue_xmit+0x311/0x1180 [10432.555062] ? eth_header+0x2a/0xc0 [10432.555069] ip_finish_output2+0x22d/0xa20 [10432.555072] ? ip_skb_dst_mtu+0x51/0x1c0 [10432.555077] ? __ip_queue_xmit+0x1d5/0x700 [10432.555079] __ip_queue_xmit+0x1d5/0x700 [10432.555084] __tcp_transmit_skb+0xb65/0xd30 [10432.555091] ? mark_held_locks+0x49/0x80 [10432.555095] tcp_write_xmit+0x492/0x1400 [10432.555101] ? __pfx_tcp_write_timer+0x10/0x10 [10432.555104] tcp_send_loss_probe+0x183/0x260 [10432.555108] tcp_write_timer+0xb2/0x120 [10432.555111] ? __pfx_tcp_write_timer+0x10/0x10 [10432.555113] call_timer_fn+0xa5/0x220 [10432.555120] __run_timers.part.0+0x1f8/0x2c0 [10432.555123] ? rcu_core+0x5ed/0xeb0 [10432.555130] run_timer_softirq+0x2a/0x50 [10432.555132] __do_softirq+0xc8/0x37a [10432.555137] ? __pfx_smpboot_thread_fn+0x10/0x10 [10432.555143] run_ksoftirqd+0x2e/0x40 [10432.555149] smpboot_thread_fn+0xd9/0x1d0 [10432.555153] kthread+0xda/0x110 [10432.555157] ? __pfx_kthread+0x10/0x10 [10432.555160] ret_from_fork+0x31/0x50 [10432.555165] ? __pfx_kthread+0x10/0x10 [10432.555167] ret_from_fork_asm+0x1b/0x30 [10432.555178] [10432.559590] rcu: rcu_preempt kthread starved for 9236475 jiffies! g565849 f0x2 RCU_GP_WAIT_FQS(5) ->state=0x0 ->cpu=0 [10432.857420] rcu: Unless rcu_preempt kthread gets sufficient CPU time, OOM is now expected behavior. [10432.858140] rcu: RCU grace-period kthread stack dump: [10432.858530] task:rcu_preempt state:R running task stack:14760 pid:17 tgid:17 ppid:2 flags:0x00004000 [10432.859276] Call Trace: [10432.889660] [10432.889849] __schedule+0x33e/0xc70 [10432.890147] ? __pfx_rcu_gp_kthread+0x10/0x10 [10432.890503] schedule+0x49/0x110 [10432.890771] schedule_timeout+0xc8/0x1b0 [10432.891090] ? __pfx_process_timeout+0x10/0x10 [10432.891416] ? mt_destroy_walk+0x130/0x540 [10432.891741] rcu_gp_fqs_loop+0x145/0x830 [10432.892062] rcu_gp_kthread+0x202/0x2e0 [10432.892387] kthread+0xda/0x110 [10432.892661] ? __pfx_kthread+0x10/0x10 [10432.892992] ret_from_fork+0x31/0x50 [10432.893303] ? __pfx_kthread+0x10/0x10 [10432.893628] ret_from_fork_asm+0x1b/0x30 [10432.893976] [10432.894171] rcu: Stack dump where RCU GP kthread last ran: [10432.894621] Sending NMI from CPU 3 to CPUs 0: [10432.899589] NMI backtrace for cpu 0 [10432.899597] CPU: 0 PID: 0 Comm: swapper/0 Not tainted 6.8.0-rc3-virtme #1 [10432.899602] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS rel-1.16.3-0-ga6ed6b701f0a-prebuilt.qemu.org 04/01/2014 [10432.899605] RIP: 0010:queued_spin_lock_slowpath+0x36/0x310 [10432.899616] Code: 54 55 48 89 fd 53 66 90 ba 01 00 00 00 8b 45 00 85 c0 75 14 f0 0f b1 55 00 85 c0 75 f0 5b 5d 41 5c 41 5d c3 cc cc cc cc f3 90 e1 81 fe 00 01 00 00 74 52 40 30 f6 85 f6 75 75 f0 0f ba 6d 00 [10432.899619] RSP: 0018:ffffb29e40003748 EFLAGS: 00000202 [10432.899623] RAX: 0000000000000001 RBX: ffff9e43c5d40098 RCX: 0000000000000001 [10432.899625] RDX: 0000000000000001 RSI: 0000000000000001 RDI: ffff9e43c5d40098 [10432.899627] RBP: ffff9e43c5d40098 R08: ffffb29e40003508 R09: 0000000000000020 [10432.899628] R10: ffffb29e400035f8 R11: 0000000000000001 R12: 0000000000000000 [10432.899630] R13: ffff9e43c5d40000 R14: ffff9e43c5d40098 R15: ffff9e43c256d08c [10432.899635] FS: 0000000000000000(0000) GS:ffff9e43fec00000(0000) knlGS:0000000000000000 [10432.899637] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [10432.899639] CR2: 00007ff4c98b2000 CR3: 0000000002b7c003 CR4: 0000000000770ef0 [10432.899641] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 [10432.899642] DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400 [10432.899644] PKRU: 55555554 [10432.899646] Call Trace: [10432.899650] [10432.899656] ? nmi_cpu_backtrace+0xa2/0x110 [10432.899663] ? nmi_cpu_backtrace_handler+0x11/0x20 [10432.899670] ? nmi_handle+0xd0/0x270 [10432.899675] ? default_do_nmi+0x41/0x100 [10432.899681] ? exc_nmi+0xf3/0x120 [10432.899683] ? end_repeat_nmi+0xf/0x60 [10432.899693] ? queued_spin_lock_slowpath+0x36/0x310 [10432.899697] ? queued_spin_lock_slowpath+0x36/0x310 [10432.899700] ? queued_spin_lock_slowpath+0x36/0x310 [10432.899703] [10432.899704] [10432.899705] do_raw_spin_lock+0x9b/0xa0 [10432.899712] tcp_v4_rcv+0xdb5/0xef0 [10432.899721] ip_protocol_deliver_rcu+0x4b/0x1e0 [10432.899727] ip_local_deliver_finish+0xaf/0x180 [10432.899729] __netif_receive_skb_one_core+0x8b/0xa0 [10432.899734] netif_receive_skb+0x87/0x2c0 [10432.899740] tcf_mirred_to_dev+0x1f6/0x410 [act_mirred] [10432.899749] tcf_mirred_act+0xee/0x560 [act_mirred] [10432.899757] ? tcf_skbedit_act+0x1d6/0x410 [act_skbedit] [10432.899764] tcf_action_exec+0xac/0x230 [10432.899772] fl_classify+0x20e/0x260 [cls_flower] [10432.899784] ? __bfs+0xf3/0x240 [10432.899789] ? check_irq_usage+0x19c/0xc40 [10432.899793] ? find_held_lock+0x2b/0x80 [10432.899799] ? check_path.constprop.0+0x24/0x50 [10432.899802] ? check_noncircular+0x81/0x150 [10432.899805] ? update_load_avg+0x6d/0x740 [10432.899813] ? __lock_acquire+0xa6e/0x1750 [10432.899820] tcf_classify+0x17a/0x540 [10432.899826] tc_run+0x9f/0x150 [10432.899835] __dev_queue_xmit+0x311/0x1180 [10432.899840] ? mark_held_locks+0x49/0x80 [10432.899843] ? eth_header+0x2a/0xc0 [10432.899849] ip_finish_output2+0x22d/0xa20 [10432.899853] ? ip_skb_dst_mtu+0x51/0x1c0 [10432.899857] ? __ip_queue_xmit+0x1d5/0x700 [10432.899860] __ip_queue_xmit+0x1d5/0x700 [10432.899865] __tcp_transmit_skb+0xb65/0xd30 [10432.899872] ? __alloc_skb+0xdc/0x1a0 [10432.899877] ? __pfx_tcp_delack_timer+0x10/0x10 [10432.899880] tcp_delack_timer_handler+0xa3/0xf0 [10432.899883] tcp_delack_timer+0x5a/0x140 [10432.899886] ? __pfx_tcp_delack_timer+0x10/0x10 [10432.899888] call_timer_fn+0xa5/0x220 [10432.899894] __run_timers.part.0+0x1f8/0x2c0 [10432.899898] ? kvm_clock_get_cycles+0x18/0x30 [10432.899907] run_timer_softirq+0x2a/0x50 [10432.899909] __do_softirq+0xc8/0x37a [10432.899914] irq_exit_rcu+0x7f/0xa0 [10432.899919] sysvec_apic_timer_interrupt+0x78/0x90 [10432.899925] [10432.899925] [10432.899927] asm_sysvec_apic_timer_interrupt+0x1a/0x20 [10432.899931] RIP: 0010:default_idle+0xf/0x20 [10432.899935] Code: 4c 01 c7 4c 29 c2 e9 72 ff ff ff 90 90 90 90 90 90 90 90 90 90 90 90 90 90 90 90 f3 0f 1e fa 66 90 0f 00 2d 13 ae 3f 00 fb f4 c3 cc cc cc cc 66 66 2e 0f 1f 84 00 00 00 00 00 90 90 90 90 90 [10432.899937] RSP: 0018:ffffffff8bc03e88 EFLAGS: 00000242 [10432.899940] RAX: 000000000408f73b RBX: ffffffff8bc12a00 RCX: 4000000000000000 [10432.899942] RDX: 0000000000000000 RSI: 0000000000000000 RDI: ffffffff8a3a8863 [10432.899943] RBP: 0000000000000000 R08: 0000000000000001 R09: 0000000000000001 [10432.899944] R10: 0000000000000000 R11: 0000000000000001 R12: 0000000000000000 [10432.899946] R13: 0000000000000000 R14: ffffffff8bc12080 R15: 0000000000013d50 [10432.899951] ? do_idle+0x1e3/0x220 [10432.899957] default_idle_call+0x83/0x180 [10432.899959] do_idle+0x1e3/0x220 [10432.899961] ? finish_task_switch.isra.0+0xf7/0x330 [10432.899966] cpu_startup_entry+0x2a/0x30 [10432.899969] rest_init+0xe9/0x180 [10432.899972] arch_call_rest_init+0xe/0x30 [10432.899978] start_kernel+0x5b2/0x690 [10432.899982] x86_64_start_reservations+0x18/0x30 [10432.899986] x86_64_start_kernel+0xc6/0xe0 [10432.899988] secondary_startup_64_no_verify+0x16d/0x17b [10432.899997] [10510.963825] rcu: INFO: rcu_preempt detected stalls on CPUs/tasks: [10510.964408] rcu: 1-...!: (1 GPs behind) idle=bf6c/1/0x4000000000000000 softirq=232343/232345 fqs=8 [10510.965170] rcu: (detected by 3, t=9314920 jiffies, g=565849, q=98 ncpus=4) [10510.965775] Sending NMI from CPU 3 to CPUs 1: [10510.967887] NMI backtrace for cpu 1 [10510.967908] CPU: 1 PID: 22 Comm: ksoftirqd/1 Not tainted 6.8.0-rc3-virtme #1 [10510.967913] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS rel-1.16.3-0-ga6ed6b701f0a-prebuilt.qemu.org 04/01/2014 [10510.967918] RIP: 0010:queued_spin_lock_slowpath+0x36/0x310 [10510.967933] Code: 54 55 48 89 fd 53 66 90 ba 01 00 00 00 8b 45 00 85 c0 75 14 f0 0f b1 55 00 85 c0 75 f0 5b 5d 41 5c 41 5d c3 cc cc cc cc f3 90 e1 81 fe 00 01 00 00 74 52 40 30 f6 85 f6 75 75 f0 0f ba 6d 00 [10510.967936] RSP: 0000:ffffb29e400d35c8 EFLAGS: 00000202 [10510.967939] RAX: 0000000000000001 RBX: ffff9e43c5d40c98 RCX: 0000000000000001 [10510.967941] RDX: 0000000000000001 RSI: 0000000000000001 RDI: ffff9e43c5d40c98 [10510.967943] RBP: ffff9e43c5d40c98 R08: ffffb29e400d3388 R09: 0000000000000020 [10510.967945] R10: ffffb29e400d3478 R11: 0000000000000001 R12: 0000000000000000 [10510.967947] R13: ffff9e43c5d40c00 R14: ffff9e43c5d40c98 R15: ffff9e43c256e4cc [10510.967951] FS: 0000000000000000(0000) GS:ffff9e43fec80000(0000) knlGS:0000000000000000 [10510.967953] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [10510.967955] CR2: 000055f2d0684000 CR3: 0000000003e5c003 CR4: 0000000000770ef0 [10510.967957] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 [10510.967958] DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400 [10510.967959] PKRU: 55555554 [10510.967960] Call Trace: [10510.967968] [10510.967982] ? nmi_cpu_backtrace+0xa2/0x110 [10510.967992] ? nmi_cpu_backtrace_handler+0x11/0x20 [10510.967999] ? nmi_handle+0xd0/0x270 [10510.968003] ? default_do_nmi+0x41/0x100 [10510.968008] ? exc_nmi+0xf3/0x120 [10510.968011] ? end_repeat_nmi+0xf/0x60 [10510.968020] ? queued_spin_lock_slowpath+0x36/0x310 [10510.968024] ? queued_spin_lock_slowpath+0x36/0x310 [10510.968027] ? queued_spin_lock_slowpath+0x36/0x310 [10510.968031] [10510.968031] [10510.968034] do_raw_spin_lock+0x9b/0xa0 [10510.968041] tcp_v4_rcv+0xdb5/0xef0 [10510.968052] ip_protocol_deliver_rcu+0x4b/0x1e0 [10510.968058] ip_local_deliver_finish+0xaf/0x180 [10510.968061] __netif_receive_skb_one_core+0x8b/0xa0 [10510.968066] netif_receive_skb+0x87/0x2c0 [10510.968076] tcf_mirred_to_dev+0x1f6/0x410 [act_mirred] [10510.968089] tcf_mirred_act+0xee/0x560 [act_mirred] [10510.968097] ? tcf_skbedit_act+0x1d6/0x410 [act_skbedit] [10510.968105] tcf_action_exec+0xac/0x230 [10510.968113] fl_classify+0x20e/0x260 [cls_flower] [10510.968126] ? __bfs+0xf3/0x240 [10510.968130] ? check_irq_usage+0x19c/0xc40 [10510.968133] ? find_held_lock+0x2b/0x80 [10510.968136] ? fib_rules_lookup+0x175/0x3b0 [10510.968141] ? check_path.constprop.0+0x24/0x50 [10510.968143] ? check_noncircular+0x81/0x150 [10510.968149] ? __lock_acquire+0xa6e/0x1750 [10510.968156] tcf_classify+0x17a/0x540 [10510.968163] tc_run+0x9f/0x150 [10510.968172] __dev_queue_xmit+0x311/0x1180 [10510.968177] ? eth_header+0x2a/0xc0 [10510.968184] ip_finish_output2+0x22d/0xa20 [10510.968188] ? ip_skb_dst_mtu+0x51/0x1c0 [10510.968193] ? __ip_queue_xmit+0x1d5/0x700 [10510.968195] __ip_queue_xmit+0x1d5/0x700 [10510.968200] __tcp_transmit_skb+0xb65/0xd30 [10510.968206] ? mark_held_locks+0x49/0x80 [10510.968211] tcp_write_xmit+0x492/0x1400 [10510.968217] ? __pfx_tcp_write_timer+0x10/0x10 [10510.968220] tcp_send_loss_probe+0x183/0x260 [10510.968224] tcp_write_timer+0xb2/0x120 [10510.968227] ? __pfx_tcp_write_timer+0x10/0x10 [10510.968229] call_timer_fn+0xa5/0x220 [10510.968236] __run_timers.part.0+0x1f8/0x2c0 [10510.968239] ? rcu_core+0x5ed/0xeb0 [10510.968246] run_timer_softirq+0x2a/0x50 [10510.968248] __do_softirq+0xc8/0x37a [10510.968252] ? __pfx_smpboot_thread_fn+0x10/0x10 [10510.968259] run_ksoftirqd+0x2e/0x40 [10510.968265] smpboot_thread_fn+0xd9/0x1d0 [10510.968269] kthread+0xda/0x110 [10510.968273] ? __pfx_kthread+0x10/0x10 [10510.968276] ret_from_fork+0x31/0x50 [10510.968281] ? __pfx_kthread+0x10/0x10 [10510.968283] ret_from_fork_asm+0x1b/0x30 [10510.968293] [10510.969175] rcu: rcu_preempt kthread starved for 9314888 jiffies! g565849 f0x2 RCU_GP_WAIT_FQS(5) ->state=0x0 ->cpu=0 [10511.155807] rcu: Unless rcu_preempt kthread gets sufficient CPU time, OOM is now expected behavior. [10511.156558] rcu: RCU grace-period kthread stack dump: [10511.156975] task:rcu_preempt state:R running task stack:14760 pid:17 tgid:17 ppid:2 flags:0x00004000 [10511.157844] Call Trace: [10511.158055] [10511.158247] __schedule+0x33e/0xc70 [10511.158555] ? __pfx_rcu_gp_kthread+0x10/0x10 [10511.158924] schedule+0x49/0x110 [10511.199402] schedule_timeout+0xc8/0x1b0 [10511.199708] ? __pfx_process_timeout+0x10/0x10 [10511.200056] ? mt_destroy_walk+0x130/0x540 [10511.200392] rcu_gp_fqs_loop+0x145/0x830 [10511.200703] rcu_gp_kthread+0x202/0x2e0 [10511.200994] kthread+0xda/0x110 [10511.201233] ? __pfx_kthread+0x10/0x10 [10511.201509] ret_from_fork+0x31/0x50 [10511.201774] ? __pfx_kthread+0x10/0x10 [10511.202060] ret_from_fork_asm+0x1b/0x30 [10511.202394] [10511.202579] rcu: Stack dump where RCU GP kthread last ran: [10511.203006] Sending NMI from CPU 3 to CPUs 0: [10511.229189] NMI backtrace for cpu 0 [10511.229198] CPU: 0 PID: 0 Comm: swapper/0 Not tainted 6.8.0-rc3-virtme #1 [10511.229202] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS rel-1.16.3-0-ga6ed6b701f0a-prebuilt.qemu.org 04/01/2014 [10511.229205] RIP: 0010:queued_spin_lock_slowpath+0x36/0x310 [10511.229217] Code: 54 55 48 89 fd 53 66 90 ba 01 00 00 00 8b 45 00 85 c0 75 14 f0 0f b1 55 00 85 c0 75 f0 5b 5d 41 5c 41 5d c3 cc cc cc cc f3 90 e1 81 fe 00 01 00 00 74 52 40 30 f6 85 f6 75 75 f0 0f ba 6d 00 [10511.229220] RSP: 0018:ffffb29e40003748 EFLAGS: 00000202 [10511.229223] RAX: 0000000000000001 RBX: ffff9e43c5d40098 RCX: 0000000000000001 [10511.229225] RDX: 0000000000000001 RSI: 0000000000000001 RDI: ffff9e43c5d40098 [10511.229226] RBP: ffff9e43c5d40098 R08: ffffb29e40003508 R09: 0000000000000020 [10511.229228] R10: ffffb29e400035f8 R11: 0000000000000001 R12: 0000000000000000 [10511.229229] R13: ffff9e43c5d40000 R14: ffff9e43c5d40098 R15: ffff9e43c256d08c [10511.229235] FS: 0000000000000000(0000) GS:ffff9e43fec00000(0000) knlGS:0000000000000000 [10511.229237] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [10511.229238] CR2: 00007ff4c98b2000 CR3: 0000000002b7c003 CR4: 0000000000770ef0 [10511.229240] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 [10511.229241] DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400 [10511.229243] PKRU: 55555554 [10511.229244] Call Trace: [10511.229248] [10511.229254] ? nmi_cpu_backtrace+0xa2/0x110 [10511.229260] ? nmi_cpu_backtrace_handler+0x11/0x20 [10511.229267] ? nmi_handle+0xd0/0x270 [10511.229272] ? default_do_nmi+0x41/0x100 [10511.229276] ? exc_nmi+0xf3/0x120 [10511.229279] ? end_repeat_nmi+0xf/0x60 [10511.229288] ? queued_spin_lock_slowpath+0x36/0x310 [10511.229291] ? queued_spin_lock_slowpath+0x36/0x310 [10511.229294] ? queued_spin_lock_slowpath+0x36/0x310 [10511.229297] [10511.229297] [10511.229299] do_raw_spin_lock+0x9b/0xa0 [10511.229306] tcp_v4_rcv+0xdb5/0xef0 [10511.229317] ip_protocol_deliver_rcu+0x4b/0x1e0 [10511.229322] ip_local_deliver_finish+0xaf/0x180 [10511.229325] __netif_receive_skb_one_core+0x8b/0xa0 [10511.229330] netif_receive_skb+0x87/0x2c0 [10511.229336] tcf_mirred_to_dev+0x1f6/0x410 [act_mirred] [10511.229345] tcf_mirred_act+0xee/0x560 [act_mirred] [10511.229353] ? tcf_skbedit_act+0x1d6/0x410 [act_skbedit] [10511.229360] tcf_action_exec+0xac/0x230 [10511.229368] fl_classify+0x20e/0x260 [cls_flower] [10511.229379] ? __bfs+0xf3/0x240 [10511.229383] ? check_irq_usage+0x19c/0xc40 [10511.229386] ? find_held_lock+0x2b/0x80 [10511.229391] ? check_path.constprop.0+0x24/0x50 [10511.229394] ? check_noncircular+0x81/0x150 [10511.229397] ? update_load_avg+0x6d/0x740 [10511.229403] ? __lock_acquire+0xa6e/0x1750 [10511.229409] tcf_classify+0x17a/0x540 [10511.229415] tc_run+0x9f/0x150 [10511.229425] __dev_queue_xmit+0x311/0x1180 [10511.229430] ? mark_held_locks+0x49/0x80 [10511.229432] ? eth_header+0x2a/0xc0 [10511.229438] ip_finish_output2+0x22d/0xa20 [10511.229442] ? ip_skb_dst_mtu+0x51/0x1c0 [10511.229446] ? __ip_queue_xmit+0x1d5/0x700 [10511.229448] __ip_queue_xmit+0x1d5/0x700 [10511.229453] __tcp_transmit_skb+0xb65/0xd30 [10511.229460] ? __alloc_skb+0xdc/0x1a0 [10511.229466] ? __pfx_tcp_delack_timer+0x10/0x10 [10511.229469] tcp_delack_timer_handler+0xa3/0xf0 [10511.229471] tcp_delack_timer+0x5a/0x140 [10511.229473] ? __pfx_tcp_delack_timer+0x10/0x10 [10511.229476] call_timer_fn+0xa5/0x220 [10511.229482] __run_timers.part.0+0x1f8/0x2c0 [10511.229487] ? kvm_clock_get_cycles+0x18/0x30 [10511.229495] run_timer_softirq+0x2a/0x50 [10511.229496] __do_softirq+0xc8/0x37a [10511.229501] irq_exit_rcu+0x7f/0xa0 [10511.229507] sysvec_apic_timer_interrupt+0x78/0x90 [10511.229511] [10511.229512] [10511.229514] asm_sysvec_apic_timer_interrupt+0x1a/0x20 [10511.229517] RIP: 0010:default_idle+0xf/0x20 [10511.229521] Code: 4c 01 c7 4c 29 c2 e9 72 ff ff ff 90 90 90 90 90 90 90 90 90 90 90 90 90 90 90 90 f3 0f 1e fa 66 90 0f 00 2d 13 ae 3f 00 fb f4 c3 cc cc cc cc 66 66 2e 0f 1f 84 00 00 00 00 00 90 90 90 90 90 [10511.229524] RSP: 0018:ffffffff8bc03e88 EFLAGS: 00000242 [10511.229526] RAX: 000000000408f73b RBX: ffffffff8bc12a00 RCX: 4000000000000000 [10511.229528] RDX: 0000000000000000 RSI: 0000000000000000 RDI: ffffffff8a3a8863 [10511.229529] RBP: 0000000000000000 R08: 0000000000000001 R09: 0000000000000001 [10511.229531] R10: 0000000000000000 R11: 0000000000000001 R12: 0000000000000000 [10511.229532] R13: 0000000000000000 R14: ffffffff8bc12080 R15: 0000000000013d50 [10511.229536] ? do_idle+0x1e3/0x220 [10511.229543] default_idle_call+0x83/0x180 [10511.229545] do_idle+0x1e3/0x220 [10511.229548] ? finish_task_switch.isra.0+0xf7/0x330 [10511.229553] cpu_startup_entry+0x2a/0x30 [10511.229556] rest_init+0xe9/0x180 [10511.229559] arch_call_rest_init+0xe/0x30 [10511.229567] start_kernel+0x5b2/0x690 [10511.229571] x86_64_start_reservations+0x18/0x30 [10511.229575] x86_64_start_kernel+0xc6/0xe0 [10511.229578] secondary_startup_64_no_verify+0x16d/0x17b [10511.229588] [10589.271830] rcu: INFO: rcu_preempt self-detected stall on CPU [10589.272358] rcu: 1-...!: (1 GPs behind) idle=bf6c/1/0x4000000000000000 softirq=232343/232345 fqs=8 [10589.273108] rcu: (t=9393228 jiffies g=565849 q=98 ncpus=4) [10589.273601] rcu: rcu_preempt kthread starved for 9393192 jiffies! g565849 f0x2 RCU_GP_WAIT_FQS(5) ->state=0x0 ->cpu=0 [10589.274463] rcu: Unless rcu_preempt kthread gets sufficient CPU time, OOM is now expected behavior. [10589.275183] rcu: RCU grace-period kthread stack dump: [10589.275595] task:rcu_preempt state:R running task stack:14760 pid:17 tgid:17 ppid:2 flags:0x00004000 [10589.276538] Call Trace: [10589.276779] [10589.276998] __schedule+0x33e/0xc70 [10589.277365] ? __pfx_rcu_gp_kthread+0x10/0x10 [10589.277772] schedule+0x49/0x110 [10589.278078] schedule_timeout+0xc8/0x1b0 [10589.278439] ? __pfx_process_timeout+0x10/0x10 [10589.349052] ? mt_destroy_walk+0x130/0x540 [10589.349418] rcu_gp_fqs_loop+0x145/0x830 [10589.349771] rcu_gp_kthread+0x202/0x2e0 [10589.350122] kthread+0xda/0x110 [10589.350411] ? __pfx_kthread+0x10/0x10 [10589.350744] ret_from_fork+0x31/0x50 [10589.351038] ? __pfx_kthread+0x10/0x10 [10589.351349] ret_from_fork_asm+0x1b/0x30 [10589.351691] [10589.351882] rcu: Stack dump where RCU GP kthread last ran: [10589.352338] Sending NMI from CPU 1 to CPUs 0: [10589.358715] NMI backtrace for cpu 0 [10589.358722] CPU: 0 PID: 0 Comm: swapper/0 Not tainted 6.8.0-rc3-virtme #1 [10589.358725] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS rel-1.16.3-0-ga6ed6b701f0a-prebuilt.qemu.org 04/01/2014 [10589.358727] RIP: 0010:queued_spin_lock_slowpath+0x36/0x310 [10589.358736] Code: 54 55 48 89 fd 53 66 90 ba 01 00 00 00 8b 45 00 85 c0 75 14 f0 0f b1 55 00 85 c0 75 f0 5b 5d 41 5c 41 5d c3 cc cc cc cc f3 90 e1 81 fe 00 01 00 00 74 52 40 30 f6 85 f6 75 75 f0 0f ba 6d 00 [10589.358738] RSP: 0018:ffffb29e40003748 EFLAGS: 00000202 [10589.358741] RAX: 0000000000000001 RBX: ffff9e43c5d40098 RCX: 0000000000000001 [10589.358742] RDX: 0000000000000001 RSI: 0000000000000001 RDI: ffff9e43c5d40098 [10589.358743] RBP: ffff9e43c5d40098 R08: ffffb29e40003508 R09: 0000000000000020 [10589.358745] R10: ffffb29e400035f8 R11: 0000000000000001 R12: 0000000000000000 [10589.358746] R13: ffff9e43c5d40000 R14: ffff9e43c5d40098 R15: ffff9e43c256d08c [10589.358750] FS: 0000000000000000(0000) GS:ffff9e43fec00000(0000) knlGS:0000000000000000 [10589.358752] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [10589.358753] CR2: 00007ff4c98b2000 CR3: 0000000002b7c003 CR4: 0000000000770ef0 [10589.358755] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 [10589.358756] DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400 [10589.358757] PKRU: 55555554 [10589.358758] Call Trace: [10589.358762] [10589.358767] ? nmi_cpu_backtrace+0xa2/0x110 [10589.358773] ? nmi_cpu_backtrace_handler+0x11/0x20 [10589.358779] ? nmi_handle+0xd0/0x270 [10589.358783] ? default_do_nmi+0x41/0x100 [10589.358788] ? exc_nmi+0xf3/0x120 [10589.358789] ? end_repeat_nmi+0xf/0x60 [10589.358797] ? queued_spin_lock_slowpath+0x36/0x310 [10589.358800] ? queued_spin_lock_slowpath+0x36/0x310 [10589.358803] ? queued_spin_lock_slowpath+0x36/0x310 [10589.358816] [10589.358817] [10589.358818] do_raw_spin_lock+0x9b/0xa0 [10589.358824] tcp_v4_rcv+0xdb5/0xef0 [10589.358832] ip_protocol_deliver_rcu+0x4b/0x1e0 [10589.358837] ip_local_deliver_finish+0xaf/0x180 [10589.358839] __netif_receive_skb_one_core+0x8b/0xa0 [10589.358844] netif_receive_skb+0x87/0x2c0 [10589.358848] tcf_mirred_to_dev+0x1f6/0x410 [act_mirred] [10589.358857] tcf_mirred_act+0xee/0x560 [act_mirred] [10589.358864] ? tcf_skbedit_act+0x1d6/0x410 [act_skbedit] [10589.358871] tcf_action_exec+0xac/0x230 [10589.358878] fl_classify+0x20e/0x260 [cls_flower] [10589.358888] ? __bfs+0xf3/0x240 [10589.358892] ? check_irq_usage+0x19c/0xc40 [10589.358895] ? find_held_lock+0x2b/0x80 [10589.358899] ? check_path.constprop.0+0x24/0x50 [10589.358901] ? check_noncircular+0x81/0x150 [10589.358904] ? update_load_avg+0x6d/0x740 [10589.358912] ? __lock_acquire+0xa6e/0x1750 [10589.358917] tcf_classify+0x17a/0x540 [10589.358923] tc_run+0x9f/0x150 [10589.358930] __dev_queue_xmit+0x311/0x1180 [10589.358935] ? mark_held_locks+0x49/0x80 [10589.358938] ? eth_header+0x2a/0xc0 [10589.358943] ip_finish_output2+0x22d/0xa20 [10589.358947] ? ip_skb_dst_mtu+0x51/0x1c0 [10589.358951] ? __ip_queue_xmit+0x1d5/0x700 [10589.358953] __ip_queue_xmit+0x1d5/0x700 [10589.358957] __tcp_transmit_skb+0xb65/0xd30 [10589.358962] ? __alloc_skb+0xdc/0x1a0 [10589.358968] ? __pfx_tcp_delack_timer+0x10/0x10 [10589.358970] tcp_delack_timer_handler+0xa3/0xf0 [10589.358973] tcp_delack_timer+0x5a/0x140 [10589.358975] ? __pfx_tcp_delack_timer+0x10/0x10 [10589.358977] call_timer_fn+0xa5/0x220 [10589.358982] __run_timers.part.0+0x1f8/0x2c0 [10589.358986] ? kvm_clock_get_cycles+0x18/0x30 [10589.358994] run_timer_softirq+0x2a/0x50 [10589.358996] __do_softirq+0xc8/0x37a [10589.359000] irq_exit_rcu+0x7f/0xa0 [10589.359004] sysvec_apic_timer_interrupt+0x78/0x90 [10589.359008] [10589.359009] [10589.359010] asm_sysvec_apic_timer_interrupt+0x1a/0x20 [10589.359014] RIP: 0010:default_idle+0xf/0x20 [10589.359017] Code: 4c 01 c7 4c 29 c2 e9 72 ff ff ff 90 90 90 90 90 90 90 90 90 90 90 90 90 90 90 90 f3 0f 1e fa 66 90 0f 00 2d 13 ae 3f 00 fb f4 c3 cc cc cc cc 66 66 2e 0f 1f 84 00 00 00 00 00 90 90 90 90 90 [10589.359020] RSP: 0018:ffffffff8bc03e88 EFLAGS: 00000242 [10589.359022] RAX: 000000000408f73b RBX: ffffffff8bc12a00 RCX: 4000000000000000 [10589.359024] RDX: 0000000000000000 RSI: 0000000000000000 RDI: ffffffff8a3a8863 [10589.359025] RBP: 0000000000000000 R08: 0000000000000001 R09: 0000000000000001 [10589.359026] R10: 0000000000000000 R11: 0000000000000001 R12: 0000000000000000 [10589.359028] R13: 0000000000000000 R14: ffffffff8bc12080 R15: 0000000000013d50 [10589.359032] ? do_idle+0x1e3/0x220 [10589.359041] default_idle_call+0x83/0x180 [10589.359043] do_idle+0x1e3/0x220 [10589.359046] ? finish_task_switch.isra.0+0xf7/0x330 [10589.359052] cpu_startup_entry+0x2a/0x30 [10589.359055] rest_init+0xe9/0x180 [10589.359058] arch_call_rest_init+0xe/0x30 [10589.359066] start_kernel+0x5b2/0x690 [10589.359070] x86_64_start_reservations+0x18/0x30 [10589.359076] x86_64_start_kernel+0xc6/0xe0 [10589.359078] secondary_startup_64_no_verify+0x16d/0x17b [10589.359087] [10589.418684] CPU: 1 PID: 22 Comm: ksoftirqd/1 Not tainted 6.8.0-rc3-virtme #1 [10589.633040] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS rel-1.16.3-0-ga6ed6b701f0a-prebuilt.qemu.org 04/01/2014 [10589.633996] RIP: 0010:queued_spin_lock_slowpath+0x36/0x310 [10589.634464] Code: 54 55 48 89 fd 53 66 90 ba 01 00 00 00 8b 45 00 85 c0 75 14 f0 0f b1 55 00 85 c0 75 f0 5b 5d 41 5c 41 5d c3 cc cc cc cc f3 90 e1 81 fe 00 01 00 00 74 52 40 30 f6 85 f6 75 75 f0 0f ba 6d 00 [10589.635946] RSP: 0000:ffffb29e400d35c8 EFLAGS: 00000202 [10589.636393] RAX: 0000000000000001 RBX: ffff9e43c5d40c98 RCX: 0000000000000001 [10589.637005] RDX: 0000000000000001 RSI: 0000000000000001 RDI: ffff9e43c5d40c98 [10589.637608] RBP: ffff9e43c5d40c98 R08: ffffb29e400d3388 R09: 0000000000000020 [10589.638212] R10: ffffb29e400d3478 R11: 0000000000000001 R12: 0000000000000000 [10589.686568] R13: ffff9e43c5d40c00 R14: ffff9e43c5d40c98 R15: ffff9e43c256e4cc [10589.687167] FS: 0000000000000000(0000) GS:ffff9e43fec80000(0000) knlGS:0000000000000000 [10589.687835] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [10589.688319] CR2: 000055f2d0684000 CR3: 0000000003e5c003 CR4: 0000000000770ef0 [10589.688939] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 [10589.689571] DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400 [10589.690200] PKRU: 55555554 [10589.690453] Call Trace: [10589.690683] [10589.690871] ? rcu_dump_cpu_stacks+0xfa/0x180 [10589.691256] ? rcu_sched_clock_irq+0x7bb/0x13d0 [10589.691679] ? lock_is_held_type+0x9f/0x120 [10589.692050] ? lock_is_held_type+0x9f/0x120 [10589.692435] ? update_process_times+0x5f/0x90 [10589.692812] ? __pfx_tick_nohz_highres_handler+0x10/0x10 [10589.693259] ? tick_sched_handle+0x33/0x50 [10589.693612] ? tick_nohz_highres_handler+0x71/0x90 [10589.694023] ? __hrtimer_run_queues+0x86/0x390 [10589.694402] ? hrtimer_interrupt+0x100/0x240 [10589.694799] ? __sysvec_apic_timer_interrupt+0x66/0x1a0 [10589.695252] ? sysvec_apic_timer_interrupt+0x73/0x90 [10589.695683] [10589.695877] [10589.696075] ? asm_sysvec_apic_timer_interrupt+0x1a/0x20 [10589.696562] ? queued_spin_lock_slowpath+0x36/0x310 [10589.696981] do_raw_spin_lock+0x9b/0xa0 [10589.697314] tcp_v4_rcv+0xdb5/0xef0 [10589.697623] ip_protocol_deliver_rcu+0x4b/0x1e0 [10589.698014] ip_local_deliver_finish+0xaf/0x180 [10589.698401] __netif_receive_skb_one_core+0x8b/0xa0 [10589.760311] netif_receive_skb+0x87/0x2c0 [10589.760669] tcf_mirred_to_dev+0x1f6/0x410 [act_mirred] [10589.761121] tcf_mirred_act+0xee/0x560 [act_mirred] [10589.761546] ? tcf_skbedit_act+0x1d6/0x410 [act_skbedit] [10589.761999] tcf_action_exec+0xac/0x230 [10589.762360] fl_classify+0x20e/0x260 [cls_flower] [10589.762766] ? __bfs+0xf3/0x240 [10589.763044] ? check_irq_usage+0x19c/0xc40 [10589.763384] ? find_held_lock+0x2b/0x80 [10589.763727] ? fib_rules_lookup+0x175/0x3b0 [10589.764078] ? check_path.constprop.0+0x24/0x50 [10589.764449] ? check_noncircular+0x81/0x150 [10589.764801] ? __lock_acquire+0xa6e/0x1750 [10589.765150] tcf_classify+0x17a/0x540 [10589.765467] tc_run+0x9f/0x150 [10589.765732] __dev_queue_xmit+0x311/0x1180 [10589.766074] ? eth_header+0x2a/0xc0 [10589.766369] ip_finish_output2+0x22d/0xa20 [10589.766731] ? ip_skb_dst_mtu+0x51/0x1c0 [10589.767056] ? __ip_queue_xmit+0x1d5/0x700 [10589.767391] __ip_queue_xmit+0x1d5/0x700 [10589.767752] __tcp_transmit_skb+0xb65/0xd30 [10589.768118] ? mark_held_locks+0x49/0x80 [10589.768451] tcp_write_xmit+0x492/0x1400 [10589.818932] ? __pfx_tcp_write_timer+0x10/0x10 [10589.819339] tcp_send_loss_probe+0x183/0x260 [10589.819745] tcp_write_timer+0xb2/0x120 [10589.820117] ? __pfx_tcp_write_timer+0x10/0x10 [10589.820527] call_timer_fn+0xa5/0x220 [10589.820856] __run_timers.part.0+0x1f8/0x2c0 [10589.821224] ? rcu_core+0x5ed/0xeb0 [10589.821535] run_timer_softirq+0x2a/0x50 [10589.821870] __do_softirq+0xc8/0x37a [10589.822182] ? __pfx_smpboot_thread_fn+0x10/0x10 [10589.822574] run_ksoftirqd+0x2e/0x40 [10589.822882] smpboot_thread_fn+0xd9/0x1d0 [10589.823229] kthread+0xda/0x110 [10589.823502] ? __pfx_kthread+0x10/0x10 [10589.823822] ret_from_fork+0x31/0x50 [10589.824129] ? __pfx_kthread+0x10/0x10 [10589.824451] ret_from_fork_asm+0x1b/0x30 [10589.824794] [10667.825800] rcu: INFO: rcu_preempt self-detected stall on CPU [10667.826150] rcu: 1-...!: (1 GPs behind) idle=bf6c/1/0x4000000000000000 softirq=232343/232345 fqs=8 [10667.826611] rcu: (t=9471782 jiffies g=565849 q=98 ncpus=4) [10667.826903] rcu: rcu_preempt kthread starved for 9471746 jiffies! g565849 f0x2 RCU_GP_WAIT_FQS(5) ->state=0x0 ->cpu=0 [10667.827435] rcu: Unless rcu_preempt kthread gets sufficient CPU time, OOM is now expected behavior. [10667.827896] rcu: RCU grace-period kthread stack dump: [10667.828157] task:rcu_preempt state:R running task stack:14760 pid:17 tgid:17 ppid:2 flags:0x00004000 [10667.828725] Call Trace: [10667.828859] [10667.828980] __schedule+0x33e/0xc70 [10667.829175] ? __pfx_rcu_gp_kthread+0x10/0x10 [10667.829407] schedule+0x49/0x110 [10667.829582] schedule_timeout+0xc8/0x1b0 [10667.829791] ? __pfx_process_timeout+0x10/0x10 [10667.830027] ? mt_destroy_walk+0x130/0x540 [10667.830246] rcu_gp_fqs_loop+0x145/0x830 [10667.830457] rcu_gp_kthread+0x202/0x2e0 [10667.830660] kthread+0xda/0x110 [10667.830831] ? __pfx_kthread+0x10/0x10 [10667.831030] ret_from_fork+0x31/0x50 [10667.831222] ? __pfx_kthread+0x10/0x10 [10667.831420] ret_from_fork_asm+0x1b/0x30 [10667.831632] [10667.831757] rcu: Stack dump where RCU GP kthread last ran: [10667.832040] Sending NMI from CPU 1 to CPUs 0: [10667.832274] NMI backtrace for cpu 0 [10667.832279] CPU: 0 PID: 0 Comm: swapper/0 Not tainted 6.8.0-rc3-virtme #1 [10667.832281] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS rel-1.16.3-0-ga6ed6b701f0a-prebuilt.qemu.org 04/01/2014 [10667.832283] RIP: 0010:queued_spin_lock_slowpath+0x36/0x310 [10667.832289] Code: 54 55 48 89 fd 53 66 90 ba 01 00 00 00 8b 45 00 85 c0 75 14 f0 0f b1 55 00 85 c0 75 f0 5b 5d 41 5c 41 5d c3 cc cc cc cc f3 90 e1 81 fe 00 01 00 00 74 52 40 30 f6 85 f6 75 75 f0 0f ba 6d 00 [10667.832290] RSP: 0018:ffffb29e40003748 EFLAGS: 00000202 [10667.832292] RAX: 0000000000000001 RBX: ffff9e43c5d40098 RCX: 0000000000000001 [10667.832294] RDX: 0000000000000001 RSI: 0000000000000001 RDI: ffff9e43c5d40098 [10667.832295] RBP: ffff9e43c5d40098 R08: ffffb29e40003508 R09: 0000000000000020 [10667.832296] R10: ffffb29e400035f8 R11: 0000000000000001 R12: 0000000000000000 [10667.832297] R13: ffff9e43c5d40000 R14: ffff9e43c5d40098 R15: ffff9e43c256d08c [10667.832299] FS: 0000000000000000(0000) GS:ffff9e43fec00000(0000) knlGS:0000000000000000 [10667.832301] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [10667.832302] CR2: 00007ff4c98b2000 CR3: 0000000002b7c003 CR4: 0000000000770ef0 [10667.832303] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 [10667.832303] DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400 [10667.832304] PKRU: 55555554 [10667.832305] Call Trace: [10667.832307] [10667.832309] ? nmi_cpu_backtrace+0xa2/0x110 [10667.832313] ? nmi_cpu_backtrace_handler+0x11/0x20 [10667.832317] ? nmi_handle+0xd0/0x270 [10667.832320] ? default_do_nmi+0x41/0x100 [10667.832323] ? exc_nmi+0xf3/0x120 [10667.832325] ? end_repeat_nmi+0xf/0x60 [10667.832331] ? queued_spin_lock_slowpath+0x36/0x310 [10667.832334] ? queued_spin_lock_slowpath+0x36/0x310 [10667.832336] ? queued_spin_lock_slowpath+0x36/0x310 [10667.832338] [10667.832338] [10667.832339] do_raw_spin_lock+0x9b/0xa0 [10667.832345] tcp_v4_rcv+0xdb5/0xef0 [10667.832352] ip_protocol_deliver_rcu+0x4b/0x1e0 [10667.832356] ip_local_deliver_finish+0xaf/0x180 [10667.832357] __netif_receive_skb_one_core+0x8b/0xa0 [10667.832361] netif_receive_skb+0x87/0x2c0 [10667.832365] tcf_mirred_to_dev+0x1f6/0x410 [act_mirred] [10667.832372] tcf_mirred_act+0xee/0x560 [act_mirred] [10667.832377] ? tcf_skbedit_act+0x1d6/0x410 [act_skbedit] [10667.832383] tcf_action_exec+0xac/0x230 [10667.832388] fl_classify+0x20e/0x260 [cls_flower] [10667.832397] ? __bfs+0xf3/0x240 [10667.832400] ? check_irq_usage+0x19c/0xc40 [10667.832402] ? find_held_lock+0x2b/0x80 [10667.832405] ? check_path.constprop.0+0x24/0x50 [10667.832407] ? check_noncircular+0x81/0x150 [10667.832409] ? update_load_avg+0x6d/0x740 [10667.832415] ? __lock_acquire+0xa6e/0x1750 [10667.832419] tcf_classify+0x17a/0x540 [10667.832423] tc_run+0x9f/0x150 [10667.832430] __dev_queue_xmit+0x311/0x1180 [10667.832434] ? mark_held_locks+0x49/0x80 [10667.832435] ? eth_header+0x2a/0xc0 [10667.832440] ip_finish_output2+0x22d/0xa20 [10667.832443] ? ip_skb_dst_mtu+0x51/0x1c0 [10667.832445] ? __ip_queue_xmit+0x1d5/0x700 [10667.832447] __ip_queue_xmit+0x1d5/0x700 [10667.832450] __tcp_transmit_skb+0xb65/0xd30 [10667.832455] ? __alloc_skb+0xdc/0x1a0 [10667.832458] ? __pfx_tcp_delack_timer+0x10/0x10 [10667.832460] tcp_delack_timer_handler+0xa3/0xf0 [10667.832462] tcp_delack_timer+0x5a/0x140 [10667.832463] ? __pfx_tcp_delack_timer+0x10/0x10 [10667.832464] call_timer_fn+0xa5/0x220 [10667.832467] __run_timers.part.0+0x1f8/0x2c0 [10667.832470] ? kvm_clock_get_cycles+0x18/0x30 [10667.832474] run_timer_softirq+0x2a/0x50 [10667.832475] __do_softirq+0xc8/0x37a [10667.832478] irq_exit_rcu+0x7f/0xa0 [10667.832480] sysvec_apic_timer_interrupt+0x78/0x90 [10667.832483] [10667.832484] [10667.832484] asm_sysvec_apic_timer_interrupt+0x1a/0x20 [10667.832487] RIP: 0010:default_idle+0xf/0x20 [10667.832489] Code: 4c 01 c7 4c 29 c2 e9 72 ff ff ff 90 90 90 90 90 90 90 90 90 90 90 90 90 90 90 90 f3 0f 1e fa 66 90 0f 00 2d 13 ae 3f 00 fb f4 c3 cc cc cc cc 66 66 2e 0f 1f 84 00 00 00 00 00 90 90 90 90 90 [10667.832490] RSP: 0018:ffffffff8bc03e88 EFLAGS: 00000242 [10667.832491] RAX: 000000000408f73b RBX: ffffffff8bc12a00 RCX: 4000000000000000 [10667.832492] RDX: 0000000000000000 RSI: 0000000000000000 RDI: ffffffff8a3a8863 [10667.832493] RBP: 0000000000000000 R08: 0000000000000001 R09: 0000000000000001 [10667.832493] R10: 0000000000000000 R11: 0000000000000001 R12: 0000000000000000 [10667.832494] R13: 0000000000000000 R14: ffffffff8bc12080 R15: 0000000000013d50 [10667.832496] ? do_idle+0x1e3/0x220 [10667.832501] default_idle_call+0x83/0x180 [10667.832503] do_idle+0x1e3/0x220 [10667.832505] ? finish_task_switch.isra.0+0xf7/0x330 [10667.832508] cpu_startup_entry+0x2a/0x30 [10667.832511] rest_init+0xe9/0x180 [10667.832512] arch_call_rest_init+0xe/0x30 [10667.832518] start_kernel+0x5b2/0x690 [10667.832521] x86_64_start_reservations+0x18/0x30 [10667.832524] x86_64_start_kernel+0xc6/0xe0 [10667.832525] secondary_startup_64_no_verify+0x16d/0x17b [10667.832531] [10667.833271] CPU: 1 PID: 22 Comm: ksoftirqd/1 Not tainted 6.8.0-rc3-virtme #1 [10667.856725] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS rel-1.16.3-0-ga6ed6b701f0a-prebuilt.qemu.org 04/01/2014 [10667.857305] RIP: 0010:queued_spin_lock_slowpath+0x36/0x310 [10667.857589] Code: 54 55 48 89 fd 53 66 90 ba 01 00 00 00 8b 45 00 85 c0 75 14 f0 0f b1 55 00 85 c0 75 f0 5b 5d 41 5c 41 5d c3 cc cc cc cc f3 90 e1 81 fe 00 01 00 00 74 52 40 30 f6 85 f6 75 75 f0 0f ba 6d 00 [10667.858530] RSP: 0000:ffffb29e400d35c8 EFLAGS: 00000202 [10667.858805] RAX: 0000000000000001 RBX: ffff9e43c5d40c98 RCX: 0000000000000001 [10667.859169] RDX: 0000000000000001 RSI: 0000000000000001 RDI: ffff9e43c5d40c98 [10667.859532] RBP: ffff9e43c5d40c98 R08: ffffb29e400d3388 R09: 0000000000000020 [10667.859898] R10: ffffb29e400d3478 R11: 0000000000000001 R12: 0000000000000000 [10667.860263] R13: ffff9e43c5d40c00 R14: ffff9e43c5d40c98 R15: ffff9e43c256e4cc [10667.860630] FS: 0000000000000000(0000) GS:ffff9e43fec80000(0000) knlGS:0000000000000000 [10667.861041] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [10667.861339] CR2: 000055f2d0684000 CR3: 0000000003e5c003 CR4: 0000000000770ef0 [10667.861705] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 [10667.862069] DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400 [10667.862435] PKRU: 55555554 [10667.862580] Call Trace: [10667.862726] [10667.862839] ? rcu_dump_cpu_stacks+0xfa/0x180 [10667.863075] ? rcu_sched_clock_irq+0x7bb/0x13d0 [10667.863315] ? lock_is_held_type+0x9f/0x120 [10667.863534] ? lock_is_held_type+0x9f/0x120 [10667.863757] ? update_process_times+0x5f/0x90 [10667.863985] ? __pfx_tick_nohz_highres_handler+0x10/0x10 [10667.864263] ? tick_sched_handle+0x33/0x50 [10667.864477] ? tick_nohz_highres_handler+0x71/0x90 [10667.864729] ? __hrtimer_run_queues+0x86/0x390 [10667.864965] ? hrtimer_interrupt+0x100/0x240 [10667.865192] ? __sysvec_apic_timer_interrupt+0x66/0x1a0 [10667.865463] ? sysvec_apic_timer_interrupt+0x73/0x90 [10667.865724] [10667.865840] [10667.865957] ? asm_sysvec_apic_timer_interrupt+0x1a/0x20 [10667.866241] ? queued_spin_lock_slowpath+0x36/0x310 [10667.866498] do_raw_spin_lock+0x9b/0xa0 [10667.866702] tcp_v4_rcv+0xdb5/0xef0 [10667.866895] ip_protocol_deliver_rcu+0x4b/0x1e0 [10667.867132] ip_local_deliver_finish+0xaf/0x180 [10667.867372] __netif_receive_skb_one_core+0x8b/0xa0 [10667.867626] netif_receive_skb+0x87/0x2c0 [10667.867839] tcf_mirred_to_dev+0x1f6/0x410 [act_mirred] [10667.868117] tcf_mirred_act+0xee/0x560 [act_mirred] [10667.868378] ? tcf_skbedit_act+0x1d6/0x410 [act_skbedit] [10667.868657] tcf_action_exec+0xac/0x230 [10667.868863] fl_classify+0x20e/0x260 [cls_flower] [10667.869116] ? __bfs+0xf3/0x240 [10667.869288] ? check_irq_usage+0x19c/0xc40 [10667.869503] ? find_held_lock+0x2b/0x80 [10667.869706] ? fib_rules_lookup+0x175/0x3b0 [10667.869931] ? check_path.constprop.0+0x24/0x50 [10667.870168] ? check_noncircular+0x81/0x150 [10667.870391] ? __lock_acquire+0xa6e/0x1750 [10667.870608] tcf_classify+0x17a/0x540 [10667.870805] tc_run+0x9f/0x150 [10667.870977] __dev_queue_xmit+0x311/0x1180 [10667.871192] ? eth_header+0x2a/0xc0 [10667.871380] ip_finish_output2+0x22d/0xa20 [10667.871598] ? ip_skb_dst_mtu+0x51/0x1c0 [10667.871806] ? __ip_queue_xmit+0x1d5/0x700 [10667.872019] __ip_queue_xmit+0x1d5/0x700 [10667.872230] __tcp_transmit_skb+0xb65/0xd30 [10667.872452] ? mark_held_locks+0x49/0x80 [10667.872660] tcp_write_xmit+0x492/0x1400 [10667.872871] ? __pfx_tcp_write_timer+0x10/0x10 [10667.873103] tcp_send_loss_probe+0x183/0x260 [10667.873329] tcp_write_timer+0xb2/0x120 [10667.873532] ? __pfx_tcp_write_timer+0x10/0x10 [10667.873765] call_timer_fn+0xa5/0x220 [10667.873962] __run_timers.part.0+0x1f8/0x2c0 [10667.874186] ? rcu_core+0x5ed/0xeb0 [10667.874374] run_timer_softirq+0x2a/0x50 [10667.874580] __do_softirq+0xc8/0x37a [10667.874772] ? __pfx_smpboot_thread_fn+0x10/0x10 [10667.875017] run_ksoftirqd+0x2e/0x40 [10667.875209] smpboot_thread_fn+0xd9/0x1d0 [10667.875422] kthread+0xda/0x110 [10667.875589] ? __pfx_kthread+0x10/0x10 [10667.875787] ret_from_fork+0x31/0x50 [10667.875987] ? __pfx_kthread+0x10/0x10 [10667.876187] ret_from_fork_asm+0x1b/0x30 [10667.876401] [10745.878800] rcu: INFO: rcu_preempt self-detected stall on CPU [10745.879178] rcu: 1-...!: (1 GPs behind) idle=bf6c/1/0x4000000000000000 softirq=232343/232345 fqs=8 [10745.879690] rcu: (t=9549835 jiffies g=565849 q=98 ncpus=4) [10745.880013] rcu: rcu_preempt kthread starved for 9549799 jiffies! g565849 f0x2 RCU_GP_WAIT_FQS(5) ->state=0x0 ->cpu=0 [10745.880613] rcu: Unless rcu_preempt kthread gets sufficient CPU time, OOM is now expected behavior. [10745.881130] rcu: RCU grace-period kthread stack dump: [10745.881432] task:rcu_preempt state:R running task stack:14760 pid:17 tgid:17 ppid:2 flags:0x00004000 [10745.882067] Call Trace: [10745.882215] [10745.882348] __schedule+0x33e/0xc70 [10745.882566] ? __pfx_rcu_gp_kthread+0x10/0x10 [10745.882828] schedule+0x49/0x110 [10745.883029] schedule_timeout+0xc8/0x1b0 [10745.883281] ? __pfx_process_timeout+0x10/0x10 [10745.883544] ? mt_destroy_walk+0x130/0x540 [10745.883786] rcu_gp_fqs_loop+0x145/0x830 [10745.884023] rcu_gp_kthread+0x202/0x2e0 [10745.884249] kthread+0xda/0x110 [10745.884448] ? __pfx_kthread+0x10/0x10 [10745.884662] ret_from_fork+0x31/0x50 [10745.884872] ? __pfx_kthread+0x10/0x10 [10745.885111] ret_from_fork_asm+0x1b/0x30 [10745.885348] [10745.885481] rcu: Stack dump where RCU GP kthread last ran: [10745.885796] Sending NMI from CPU 1 to CPUs 0: [10745.886073] NMI backtrace for cpu 0 [10745.886078] CPU: 0 PID: 0 Comm: swapper/0 Not tainted 6.8.0-rc3-virtme #1 [10745.886080] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS rel-1.16.3-0-ga6ed6b701f0a-prebuilt.qemu.org 04/01/2014 [10745.886082] RIP: 0010:queued_spin_lock_slowpath+0x36/0x310 [10745.886088] Code: 54 55 48 89 fd 53 66 90 ba 01 00 00 00 8b 45 00 85 c0 75 14 f0 0f b1 55 00 85 c0 75 f0 5b 5d 41 5c 41 5d c3 cc cc cc cc f3 90 e1 81 fe 00 01 00 00 74 52 40 30 f6 85 f6 75 75 f0 0f ba 6d 00 [10745.886089] RSP: 0018:ffffb29e40003748 EFLAGS: 00000202 [10745.886091] RAX: 0000000000000001 RBX: ffff9e43c5d40098 RCX: 0000000000000001 [10745.886092] RDX: 0000000000000001 RSI: 0000000000000001 RDI: ffff9e43c5d40098 [10745.886093] RBP: ffff9e43c5d40098 R08: ffffb29e40003508 R09: 0000000000000020 [10745.886094] R10: ffffb29e400035f8 R11: 0000000000000001 R12: 0000000000000000 [10745.886095] R13: ffff9e43c5d40000 R14: ffff9e43c5d40098 R15: ffff9e43c256d08c [10745.886098] FS: 0000000000000000(0000) GS:ffff9e43fec00000(0000) knlGS:0000000000000000 [10745.886099] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [10745.886100] CR2: 00007ff4c98b2000 CR3: 0000000002b7c003 CR4: 0000000000770ef0 [10745.886101] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 [10745.886102] DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400 [10745.886103] PKRU: 55555554 [10745.886103] Call Trace: [10745.886105] [10745.886107] ? nmi_cpu_backtrace+0xa2/0x110 [10745.886110] ? nmi_cpu_backtrace_handler+0x11/0x20 [10745.886115] ? nmi_handle+0xd0/0x270 [10745.886117] ? default_do_nmi+0x41/0x100 [10745.886121] ? exc_nmi+0xf3/0x120 [10745.886122] ? end_repeat_nmi+0xf/0x60 [10745.886128] ? queued_spin_lock_slowpath+0x36/0x310 [10745.886130] ? queued_spin_lock_slowpath+0x36/0x310 [10745.886132] ? queued_spin_lock_slowpath+0x36/0x310 [10745.886134] [10745.886134] [10745.886135] do_raw_spin_lock+0x9b/0xa0 [10745.886141] tcp_v4_rcv+0xdb5/0xef0 [10745.886148] ip_protocol_deliver_rcu+0x4b/0x1e0 [10745.886152] ip_local_deliver_finish+0xaf/0x180 [10745.886154] __netif_receive_skb_one_core+0x8b/0xa0 [10745.886158] netif_receive_skb+0x87/0x2c0 [10745.886162] tcf_mirred_to_dev+0x1f6/0x410 [act_mirred] [10745.886169] tcf_mirred_act+0xee/0x560 [act_mirred] [10745.886175] ? tcf_skbedit_act+0x1d6/0x410 [act_skbedit] [10745.886180] tcf_action_exec+0xac/0x230 [10745.886185] fl_classify+0x20e/0x260 [cls_flower] [10745.886193] ? __bfs+0xf3/0x240 [10745.886196] ? check_irq_usage+0x19c/0xc40 [10745.886198] ? find_held_lock+0x2b/0x80 [10745.886201] ? check_path.constprop.0+0x24/0x50 [10745.886203] ? check_noncircular+0x81/0x150 [10745.886206] ? update_load_avg+0x6d/0x740 [10745.886211] ? __lock_acquire+0xa6e/0x1750 [10745.886215] tcf_classify+0x17a/0x540 [10745.886220] tc_run+0x9f/0x150 [10745.886226] __dev_queue_xmit+0x311/0x1180 [10745.886230] ? mark_held_locks+0x49/0x80 [10745.886232] ? eth_header+0x2a/0xc0 [10745.886236] ip_finish_output2+0x22d/0xa20 [10745.886239] ? ip_skb_dst_mtu+0x51/0x1c0 [10745.886242] ? __ip_queue_xmit+0x1d5/0x700 [10745.886244] __ip_queue_xmit+0x1d5/0x700 [10745.886246] __tcp_transmit_skb+0xb65/0xd30 [10745.886251] ? __alloc_skb+0xdc/0x1a0 [10745.886254] ? __pfx_tcp_delack_timer+0x10/0x10 [10745.886256] tcp_delack_timer_handler+0xa3/0xf0 [10745.886258] tcp_delack_timer+0x5a/0x140 [10745.886259] ? __pfx_tcp_delack_timer+0x10/0x10 [10745.886260] call_timer_fn+0xa5/0x220 [10745.886264] __run_timers.part.0+0x1f8/0x2c0 [10745.886266] ? kvm_clock_get_cycles+0x18/0x30 [10745.886270] run_timer_softirq+0x2a/0x50 [10745.886271] __do_softirq+0xc8/0x37a [10745.886274] irq_exit_rcu+0x7f/0xa0 [10745.886278] sysvec_apic_timer_interrupt+0x78/0x90 [10745.886281] [10745.886281] [10745.886282] asm_sysvec_apic_timer_interrupt+0x1a/0x20 [10745.886285] RIP: 0010:default_idle+0xf/0x20 [10745.886287] Code: 4c 01 c7 4c 29 c2 e9 72 ff ff ff 90 90 90 90 90 90 90 90 90 90 90 90 90 90 90 90 f3 0f 1e fa 66 90 0f 00 2d 13 ae 3f 00 fb f4 c3 cc cc cc cc 66 66 2e 0f 1f 84 00 00 00 00 00 90 90 90 90 90 [10745.886289] RSP: 0018:ffffffff8bc03e88 EFLAGS: 00000242 [10745.886290] RAX: 000000000408f73b RBX: ffffffff8bc12a00 RCX: 4000000000000000 [10745.886291] RDX: 0000000000000000 RSI: 0000000000000000 RDI: ffffffff8a3a8863 [10745.886292] RBP: 0000000000000000 R08: 0000000000000001 R09: 0000000000000001 [10745.886292] R10: 0000000000000000 R11: 0000000000000001 R12: 0000000000000000 [10745.886293] R13: 0000000000000000 R14: ffffffff8bc12080 R15: 0000000000013d50 [10745.886295] ? do_idle+0x1e3/0x220 [10745.886300] default_idle_call+0x83/0x180 [10745.886301] do_idle+0x1e3/0x220 [10745.886303] ? finish_task_switch.isra.0+0xf7/0x330 [10745.886307] cpu_startup_entry+0x2a/0x30 [10745.886309] rest_init+0xe9/0x180 [10745.886311] arch_call_rest_init+0xe/0x30 [10745.886316] start_kernel+0x5b2/0x690 [10745.886319] x86_64_start_reservations+0x18/0x30 [10745.886322] x86_64_start_kernel+0xc6/0xe0 [10745.886324] secondary_startup_64_no_verify+0x16d/0x17b [10745.886329] [10745.887068] CPU: 1 PID: 22 Comm: ksoftirqd/1 Not tainted 6.8.0-rc3-virtme #1 [10745.913579] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS rel-1.16.3-0-ga6ed6b701f0a-prebuilt.qemu.org 04/01/2014 [10745.914216] RIP: 0010:queued_spin_lock_slowpath+0x36/0x310 [10745.914529] Code: 54 55 48 89 fd 53 66 90 ba 01 00 00 00 8b 45 00 85 c0 75 14 f0 0f b1 55 00 85 c0 75 f0 5b 5d 41 5c 41 5d c3 cc cc cc cc f3 90 e1 81 fe 00 01 00 00 74 52 40 30 f6 85 f6 75 75 f0 0f ba 6d 00 [10745.915547] RSP: 0000:ffffb29e400d35c8 EFLAGS: 00000202 [10745.915846] RAX: 0000000000000001 RBX: ffff9e43c5d40c98 RCX: 0000000000000001 [10745.916243] RDX: 0000000000000001 RSI: 0000000000000001 RDI: ffff9e43c5d40c98 [10745.916640] RBP: ffff9e43c5d40c98 R08: ffffb29e400d3388 R09: 0000000000000020 [10745.917041] R10: ffffb29e400d3478 R11: 0000000000000001 R12: 0000000000000000 [10745.917442] R13: ffff9e43c5d40c00 R14: ffff9e43c5d40c98 R15: ffff9e43c256e4cc [10745.917855] FS: 0000000000000000(0000) GS:ffff9e43fec80000(0000) knlGS:0000000000000000 [10745.918305] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [10745.918630] CR2: 000055f2d0684000 CR3: 0000000003e5c003 CR4: 0000000000770ef0 [10745.919028] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 [10745.919429] DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400 [10745.919831] PKRU: 55555554 [10745.919986] Call Trace: [10745.920132] [10745.920258] ? rcu_dump_cpu_stacks+0xfa/0x180 [10745.920516] ? rcu_sched_clock_irq+0x7bb/0x13d0 [10745.920780] ? lock_is_held_type+0x9f/0x120 [10745.921020] ? lock_is_held_type+0x9f/0x120 [10745.921260] ? update_process_times+0x5f/0x90 [10745.921513] ? __pfx_tick_nohz_highres_handler+0x10/0x10 [10745.921820] ? tick_sched_handle+0x33/0x50 [10745.922053] ? tick_nohz_highres_handler+0x71/0x90 [10745.922327] ? __hrtimer_run_queues+0x86/0x390 [10745.922588] ? hrtimer_interrupt+0x100/0x240 [10745.922833] ? __sysvec_apic_timer_interrupt+0x66/0x1a0 [10745.923127] ? sysvec_apic_timer_interrupt+0x73/0x90 [10745.923424] [10745.923553] [10745.923679] ? asm_sysvec_apic_timer_interrupt+0x1a/0x20 [10745.923982] ? queued_spin_lock_slowpath+0x36/0x310 [10745.924261] do_raw_spin_lock+0x9b/0xa0 [10745.924488] tcp_v4_rcv+0xdb5/0xef0 [10745.924693] ip_protocol_deliver_rcu+0x4b/0x1e0 [10745.924954] ip_local_deliver_finish+0xaf/0x180 [10745.925215] __netif_receive_skb_one_core+0x8b/0xa0 [10745.925491] netif_receive_skb+0x87/0x2c0 [10745.925720] tcf_mirred_to_dev+0x1f6/0x410 [act_mirred] [10745.926027] tcf_mirred_act+0xee/0x560 [act_mirred] [10745.926309] ? tcf_skbedit_act+0x1d6/0x410 [act_skbedit] [10745.926639] tcf_action_exec+0xac/0x230 [10745.926863] fl_classify+0x20e/0x260 [cls_flower] [10745.927143] ? __bfs+0xf3/0x240 [10745.927333] ? check_irq_usage+0x19c/0xc40 [10745.927570] ? find_held_lock+0x2b/0x80 [10745.927803] ? fib_rules_lookup+0x175/0x3b0 [10745.928045] ? check_path.constprop.0+0x24/0x50 [10745.928300] ? check_noncircular+0x81/0x150 [10745.928544] ? __lock_acquire+0xa6e/0x1750 [10745.928781] tcf_classify+0x17a/0x540 [10745.928993] tc_run+0x9f/0x150 [10745.929177] __dev_queue_xmit+0x311/0x1180 [10745.929413] ? eth_header+0x2a/0xc0 [10745.929618] ip_finish_output2+0x22d/0xa20 [10745.929858] ? ip_skb_dst_mtu+0x51/0x1c0 [10745.930086] ? __ip_queue_xmit+0x1d5/0x700 [10745.930319] __ip_queue_xmit+0x1d5/0x700 [10745.930553] __tcp_transmit_skb+0xb65/0xd30 [10745.930791] ? mark_held_locks+0x49/0x80 [10745.931019] tcp_write_xmit+0x492/0x1400 [10745.931248] ? __pfx_tcp_write_timer+0x10/0x10 [10745.931505] tcp_send_loss_probe+0x183/0x260 [10745.931749] tcp_write_timer+0xb2/0x120 [10745.931970] ? __pfx_tcp_write_timer+0x10/0x10 [10745.932224] call_timer_fn+0xa5/0x220 [10745.932440] __run_timers.part.0+0x1f8/0x2c0 [10745.932689] ? rcu_core+0x5ed/0xeb0 [10745.932893] run_timer_softirq+0x2a/0x50 [10745.933115] __do_softirq+0xc8/0x37a [10745.933325] ? __pfx_smpboot_thread_fn+0x10/0x10 [10745.933589] run_ksoftirqd+0x2e/0x40 [10745.933796] smpboot_thread_fn+0xd9/0x1d0 [10745.934026] kthread+0xda/0x110 [10745.934210] ? __pfx_kthread+0x10/0x10 [10745.934426] ret_from_fork+0x31/0x50 [10745.934634] ? __pfx_kthread+0x10/0x10 [10745.934846] ret_from_fork_asm+0x1b/0x30 [10745.935082]