[ 73.903746][ T617] GACT probability NOT on [ 75.315297][ C1] [ 75.315447][ C1] ======================================================== [ 75.315738][ C1] WARNING: possible irq lock inversion dependency detected [ 75.316015][ C1] 6.9.0-rc1-virtme #1 Not tainted [ 75.316201][ C1] -------------------------------------------------------- [ 75.316487][ C1] mausezahn/630 just changed the state of lock: [ 75.316735][ C1] ffff888003d8b098 (k-slock-AF_INET){+.-.}-{2:2}, at: ipv4_sk_update_pmtu+0xb3/0xb60 [ 75.317093][ C1] but this lock took another, SOFTIRQ-unsafe lock in the past: [ 75.317381][ C1] (&p->alloc_lock){+.+.}-{2:2} [ 75.317386][ C1] [ 75.317386][ C1] [ 75.317386][ C1] and interrupts could create inverse lock ordering between them. [ 75.317386][ C1] [ 75.318092][ C1] [ 75.318092][ C1] other info that might help us debug this: [ 75.318377][ C1] Possible interrupt unsafe locking scenario: [ 75.318377][ C1] [ 75.318659][ C1] CPU0 CPU1 [ 75.318854][ C1] ---- ---- [ 75.319063][ C1] lock(&p->alloc_lock); [ 75.319212][ C1] local_irq_disable(); [ 75.319451][ C1] lock(k-slock-AF_INET); [ 75.319692][ C1] lock(&p->alloc_lock); [ 75.319929][ C1] [ 75.320071][ C1] lock(k-slock-AF_INET); [ 75.320247][ C1] [ 75.320247][ C1] *** DEADLOCK *** [ 75.320247][ C1] [ 75.320533][ C1] 2 locks held by mausezahn/630: [ 75.320733][ C1] #0: ffffffffae5522e0 (rcu_read_lock){....}-{1:2}, at: process_backlog+0x1f8/0x670 [ 75.321068][ C1] #1: ffffffffae5522e0 (rcu_read_lock){....}-{1:2}, at: ip_local_deliver_finish+0x209/0x4b0 [ 75.321453][ C1] [ 75.321453][ C1] the shortest dependencies between 2nd lock and 1st lock: [ 75.321806][ C1] -> (&p->alloc_lock){+.+.}-{2:2} { [ 75.322009][ C1] HARDIRQ-ON-W at: [ 75.322167][ C1] __lock_acquire+0x797/0x1570 [ 75.322421][ C1] lock_acquire.part.0+0xeb/0x330 [ 75.322645][ C1] _raw_spin_lock+0x30/0x40 [ 75.322889][ C1] set_mems_allowed+0x1d/0x210 [ 75.323121][ C1] kernel_init_freeable+0x72/0x310 [ 75.323364][ C1] kernel_init+0x20/0x200 [ 75.323614][ C1] ret_from_fork+0x31/0x70 [ 75.323839][ C1] ret_from_fork_asm+0x1a/0x30 [ 75.324075][ C1] SOFTIRQ-ON-W at: [ 75.324215][ C1] __lock_acquire+0x797/0x1570 [ 75.324473][ C1] lock_acquire.part.0+0xeb/0x330 [ 75.324858][ C1] _raw_spin_lock+0x30/0x40 [ 75.325110][ C1] set_mems_allowed+0x1d/0x210 [ 75.325362][ C1] kernel_init_freeable+0x72/0x310 [ 75.325594][ C1] kernel_init+0x20/0x200 [ 75.325830][ C1] ret_from_fork+0x31/0x70 [ 75.326072][ C1] ret_from_fork_asm+0x1a/0x30 [ 75.326298][ C1] INITIAL USE at: [ 75.326437][ C1] __lock_acquire+0x797/0x1570 [ 75.326669][ C1] lock_acquire.part.0+0xeb/0x330 [ 75.326918][ C1] _raw_spin_lock+0x30/0x40 [ 75.327149][ C1] set_mems_allowed+0x1d/0x210 [ 75.327376][ C1] kernel_init_freeable+0x72/0x310 [ 75.327625][ C1] kernel_init+0x20/0x200 [ 75.327846][ C1] ret_from_fork+0x31/0x70 [ 75.328082][ C1] ret_from_fork_asm+0x1a/0x30 [ 75.328308][ C1] } [ 75.328411][ C1] ... key at: [] __key.387+0x0/0x40 [ 75.328692][ C1] ... acquired at: [ 75.328833][ C1] __lock_acquire+0xaf0/0x1570 [ 75.329033][ C1] lock_acquire.part.0+0xeb/0x330 [ 75.329209][ C1] _raw_spin_lock+0x30/0x40 [ 75.329403][ C1] __get_task_comm+0x27/0x70 [ 75.329592][ C1] ref_tracker_alloc+0x2ee/0x490 [ 75.329781][ C1] dst_init+0x84/0x4a0 [ 75.329912][ C1] dst_alloc+0x97/0x150 [ 75.330046][ C1] rt_dst_alloc+0x31/0x3b0 [ 75.330228][ C1] __mkroute_output+0x451/0x1070 [ 75.330412][ C1] ip_route_output_key_hash+0x15d/0x2b0 [ 75.330596][ C1] icmp_route_lookup.constprop.0+0x321/0xb30 [ 75.330822][ C1] __icmp_send+0x8e8/0x1910 [ 75.331003][ C1] ip_fragment.constprop.0+0x183/0x240 [ 75.331186][ C1] ip_output+0x16b/0x4f0 [ 75.331374][ C1] iptunnel_xmit+0x578/0x930 [ 75.331565][ C1] vxlan_xmit_one+0x2a6d/0x4570 [vxlan] [ 75.331782][ C1] vxlan_xmit+0x855/0xf40 [vxlan] [ 75.331995][ C1] dev_hard_start_xmit+0x10e/0x360 [ 75.332186][ C1] __dev_queue_xmit+0xe69/0x16d0 [ 75.332374][ C1] packet_snd+0xd08/0x1840 [ 75.332556][ C1] __sys_sendto+0x37d/0x410 [ 75.332754][ C1] __x64_sys_sendto+0xe0/0x1c0 [ 75.332958][ C1] do_syscall_64+0xc6/0x1e0 [ 75.333153][ C1] entry_SYSCALL_64_after_hwframe+0x6d/0x75 [ 75.333380][ C1] [ 75.333491][ C1] -> (k-slock-AF_INET){+.-.}-{2:2} { [ 75.333686][ C1] HARDIRQ-ON-W at: [ 75.333835][ C1] __lock_acquire+0x797/0x1570 [ 75.334066][ C1] lock_acquire.part.0+0xeb/0x330 [ 75.334323][ C1] _raw_spin_trylock+0x4b/0x70 [ 75.334553][ C1] icmp_reply+0x2e6/0xa60 [ 75.334779][ C1] icmp_echo.part.0+0x182/0x220 [ 75.335076][ C1] icmp_echo+0xcc/0x190 [ 75.335298][ C1] icmp_rcv+0x791/0xec0 [ 75.335547][ C1] ip_protocol_deliver_rcu+0x2e5/0x360 [ 75.335785][ C1] ip_local_deliver_finish+0x2c2/0x4b0 [ 75.336009][ C1] ip_local_deliver+0x19d/0x480 [ 75.336244][ C1] ip_rcv+0x564/0x740 [ 75.336447][ C1] __netif_receive_skb_one_core+0x166/0x1b0 [ 75.336720][ C1] process_backlog+0x103/0x670 [ 75.336952][ C1] __napi_poll.constprop.0+0xa2/0x460 [ 75.337190][ C1] net_rx_action+0x49f/0xc00 [ 75.337423][ C1] __do_softirq+0x1f8/0x5df [ 75.337647][ C1] do_softirq+0x4d/0xa0 [ 75.337881][ C1] __local_bh_enable_ip+0xf6/0x120 [ 75.338106][ C1] raw_sendmsg+0xb33/0x1950 [ 75.338342][ C1] __sys_sendto+0x332/0x410 [ 75.338581][ C1] __x64_sys_sendto+0xe0/0x1c0 [ 75.338914][ C1] do_syscall_64+0xc6/0x1e0 [ 75.339138][ C1] entry_SYSCALL_64_after_hwframe+0x6d/0x75 [ 75.339411][ C1] IN-SOFTIRQ-W at: [ 75.339564][ C1] __lock_acquire+0x797/0x1570 [ 75.339889][ C1] lock_acquire.part.0+0xeb/0x330 [ 75.340132][ C1] _raw_spin_lock+0x30/0x40 [ 75.340374][ C1] ipv4_sk_update_pmtu+0xb3/0xb60 [ 75.340710][ C1] __udp4_lib_err+0x7f6/0xa50 [ 75.340955][ C1] icmp_unreach+0x2c1/0xba0 [ 75.341180][ C1] icmp_rcv+0x791/0xec0 [ 75.341506][ C1] ip_protocol_deliver_rcu+0x2e5/0x360 [ 75.341746][ C1] ip_local_deliver_finish+0x2c2/0x4b0 [ 75.341983][ C1] ip_local_deliver+0x19d/0x480 [ 75.342228][ C1] ip_rcv+0x564/0x740 [ 75.342435][ C1] __netif_receive_skb_one_core+0x166/0x1b0 [ 75.342725][ C1] process_backlog+0x103/0x670 [ 75.342960][ C1] __napi_poll.constprop.0+0xa2/0x460 [ 75.343273][ C1] net_rx_action+0x49f/0xc00 [ 75.343496][ C1] __do_softirq+0x1f8/0x5df [ 75.343732][ C1] do_softirq+0x4d/0xa0 [ 75.343951][ C1] __local_bh_enable_ip+0xf6/0x120 [ 75.344270][ C1] __dev_queue_xmit+0x6ac/0x16d0 [ 75.344511][ C1] packet_snd+0xd08/0x1840 [ 75.344747][ C1] __sys_sendto+0x37d/0x410 [ 75.345099][ C1] __x64_sys_sendto+0xe0/0x1c0 [ 75.345334][ C1] do_syscall_64+0xc6/0x1e0 [ 75.345562][ C1] entry_SYSCALL_64_after_hwframe+0x6d/0x75 [ 75.345946][ C1] INITIAL USE at: [ 75.346096][ C1] __lock_acquire+0x797/0x1570 [ 75.346328][ C1] lock_acquire.part.0+0xeb/0x330 [ 75.346561][ C1] _raw_spin_trylock+0x4b/0x70 [ 75.346881][ C1] icmp_reply+0x2e6/0xa60 [ 75.347129][ C1] icmp_echo.part.0+0x182/0x220 [ 75.347361][ C1] icmp_echo+0xcc/0x190 [ 75.347565][ C1] icmp_rcv+0x791/0xec0 [ 75.347926][ C1] ip_protocol_deliver_rcu+0x2e5/0x360 [ 75.348160][ C1] ip_local_deliver_finish+0x2c2/0x4b0 [ 75.348391][ C1] ip_local_deliver+0x19d/0x480 [ 75.348717][ C1] ip_rcv+0x564/0x740 [ 75.348894][ C1] __netif_receive_skb_one_core+0x166/0x1b0 [ 75.349161][ C1] process_backlog+0x103/0x670 [ 75.349387][ C1] __napi_poll.constprop.0+0xa2/0x460 [ 75.349741][ C1] net_rx_action+0x49f/0xc00 [ 75.349963][ C1] __do_softirq+0x1f8/0x5df [ 75.350207][ C1] do_softirq+0x4d/0xa0 [ 75.350402][ C1] __local_bh_enable_ip+0xf6/0x120 [ 75.350631][ C1] raw_sendmsg+0xb33/0x1950 [ 75.350872][ C1] __sys_sendto+0x332/0x410 [ 75.351109][ C1] __x64_sys_sendto+0xe0/0x1c0 [ 75.351447][ C1] do_syscall_64+0xc6/0x1e0 [ 75.351685][ C1] entry_SYSCALL_64_after_hwframe+0x6d/0x75 [ 75.351959][ C1] } [ 75.352144][ C1] ... key at: [] af_family_kern_slock_keys+0x20/0x300 [ 75.352482][ C1] ... acquired at: [ 75.352620][ C1] mark_lock+0x28d/0x3e0 [ 75.352857][ C1] mark_usage+0xd9/0x2a0 [ 75.353269][ C1] __lock_acquire+0x797/0x1570 [ 75.353539][ C1] lock_acquire.part.0+0xeb/0x330 [ 75.353803][ C1] _raw_spin_lock+0x30/0x40 [ 75.354078][ C1] ipv4_sk_update_pmtu+0xb3/0xb60 [ 75.354515][ C1] __udp4_lib_err+0x7f6/0xa50 [ 75.354798][ C1] icmp_unreach+0x2c1/0xba0 [ 75.355085][ C1] icmp_rcv+0x791/0xec0 [ 75.355294][ C1] ip_protocol_deliver_rcu+0x2e5/0x360 [ 75.355566][ C1] ip_local_deliver_finish+0x2c2/0x4b0 [ 75.355853][ C1] ip_local_deliver+0x19d/0x480 [ 75.356132][ C1] ip_rcv+0x564/0x740 [ 75.356337][ C1] __netif_receive_skb_one_core+0x166/0x1b0 [ 75.356673][ C1] process_backlog+0x103/0x670 [ 75.356991][ C1] __napi_poll.constprop.0+0xa2/0x460 [ 75.357179][ C1] net_rx_action+0x49f/0xc00 [ 75.357372][ C1] __do_softirq+0x1f8/0x5df [ 75.357571][ C1] do_softirq+0x4d/0xa0 [ 75.357711][ C1] __local_bh_enable_ip+0xf6/0x120 [ 75.357980][ C1] __dev_queue_xmit+0x6ac/0x16d0 [ 75.358157][ C1] packet_snd+0xd08/0x1840 [ 75.358345][ C1] __sys_sendto+0x37d/0x410 [ 75.358536][ C1] __x64_sys_sendto+0xe0/0x1c0 [ 75.358799][ C1] do_syscall_64+0xc6/0x1e0 [ 75.358993][ C1] entry_SYSCALL_64_after_hwframe+0x6d/0x75 [ 75.359229][ C1] [ 75.359330][ C1] [ 75.359330][ C1] stack backtrace: [ 75.359655][ C1] CPU: 1 PID: 630 Comm: mausezahn Not tainted 6.9.0-rc1-virtme #1 [ 75.359940][ C1] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS rel-1.16.3-0-ga6ed6b701f0a-prebuilt.qemu.org 04/01/2014 [ 75.360352][ C1] Call Trace: [ 75.360490][ C1] [ 75.360589][ C1] dump_stack_lvl+0x82/0xd0 [ 75.360789][ C1] print_irq_inversion_bug.part.0+0x3d9/0x570 [ 75.361024][ C1] ? kernel_text_address+0xce/0xe0 [ 75.361300][ C1] ? __pfx_print_irq_inversion_bug.part.0+0x10/0x10 [ 75.361553][ C1] ? __pfx_usage_skip+0x10/0x10 [ 75.361735][ C1] ? __pfx_usage_match+0x10/0x10 [ 75.361921][ C1] ? arch_stack_walk+0xa2/0xf0 [ 75.362198][ C1] mark_lock_irq+0x4cd/0xa10 [ 75.362399][ C1] ? __pfx_mark_lock_irq+0x10/0x10 [ 75.362590][ C1] ? __pfx_stack_trace_save+0x10/0x10 [ 75.362788][ C1] ? save_trace+0x8f/0x5b0 [ 75.362974][ C1] mark_lock+0x28d/0x3e0 [ 75.363294][ C1] ? __lock_unpin_lock+0x137/0x2b0 [ 75.363473][ C1] mark_usage+0xd9/0x2a0 [ 75.363646][ C1] __lock_acquire+0x797/0x1570 [ 75.363837][ C1] ? do_raw_spin_unlock+0x58/0x220 [ 75.364120][ C1] lock_acquire.part.0+0xeb/0x330 [ 75.364309][ C1] ? ipv4_sk_update_pmtu+0xb3/0xb60 [ 75.364497][ C1] ? __pfx_lock_acquire.part.0+0x10/0x10 [ 75.364683][ C1] ? trace_lock_acquire+0x135/0x1c0 [ 75.364887][ C1] ? ipv4_sk_update_pmtu+0xb3/0xb60 [ 75.365169][ C1] ? lock_acquire+0x32/0xc0 [ 75.365350][ C1] ? ipv4_sk_update_pmtu+0xb3/0xb60 [ 75.365543][ C1] _raw_spin_lock+0x30/0x40 [ 75.365729][ C1] ? ipv4_sk_update_pmtu+0xb3/0xb60 [ 75.366017][ C1] ipv4_sk_update_pmtu+0xb3/0xb60 [ 75.366214][ C1] ? vxlan_err_lookup+0x23f/0x420 [vxlan] [ 75.366407][ C1] ? __pfx_ipv4_sk_update_pmtu+0x10/0x10 [ 75.366609][ C1] ? __udp4_lib_err_encap+0x59c/0x930 [ 75.366896][ C1] __udp4_lib_err+0x7f6/0xa50 [ 75.367097][ C1] icmp_unreach+0x2c1/0xba0 [ 75.367274][ C1] icmp_rcv+0x791/0xec0 [ 75.367429][ C1] ip_protocol_deliver_rcu+0x2e5/0x360 [ 75.367622][ C1] ip_local_deliver_finish+0x2c2/0x4b0 [ 75.367903][ C1] ip_local_deliver+0x19d/0x480 [ 75.368093][ C1] ? __pfx_ip_local_deliver+0x10/0x10 [ 75.368292][ C1] ? ip_rcv_finish_core.constprop.0+0x1ad/0x1300 [ 75.368531][ C1] ip_rcv+0x564/0x740 [ 75.368764][ C1] ? __pfx_ip_rcv+0x10/0x10 [ 75.368956][ C1] ? lock_acquire.part.0+0xeb/0x330 [ 75.369151][ C1] ? process_backlog+0x1f8/0x670 [ 75.369347][ C1] ? __pfx_ip_rcv+0x10/0x10 [ 75.369652][ C1] __netif_receive_skb_one_core+0x166/0x1b0 [ 75.369877][ C1] ? __pfx___netif_receive_skb_one_core+0x10/0x10 [ 75.370104][ C1] ? process_backlog+0x1f8/0x670 [ 75.370276][ C1] ? lock_acquire+0x32/0xc0 [ 75.370448][ C1] ? process_backlog+0x1f8/0x670 [ 75.370657][ C1] process_backlog+0x103/0x670 [ 75.370848][ C1] ? find_held_lock+0x2c/0x110 [ 75.371039][ C1] __napi_poll.constprop.0+0xa2/0x460 [ 75.371313][ C1] net_rx_action+0x49f/0xc00 [ 75.371507][ C1] ? __pfx_net_rx_action+0x10/0x10 [ 75.371692][ C1] ? ktime_get+0xb3/0x190 [ 75.371831][ C1] ? hrtimer_interrupt+0x311/0x770 [ 75.372024][ C1] ? hrtimer_interrupt+0x31d/0x770 [ 75.372306][ C1] __do_softirq+0x1f8/0x5df [ 75.372494][ C1] ? __dev_queue_xmit+0x697/0x16d0 [ 75.372683][ C1] do_softirq+0x4d/0xa0 [ 75.372821][ C1] [ 75.372916][ C1] [ 75.373097][ C1] __local_bh_enable_ip+0xf6/0x120 [ 75.373276][ C1] ? __dev_queue_xmit+0x697/0x16d0 [ 75.373454][ C1] __dev_queue_xmit+0x6ac/0x16d0 [ 75.373647][ C1] ? packet_parse_headers+0x231/0x9d0 [ 75.373924][ C1] ? packet_parse_headers+0x231/0x9d0 [ 75.374116][ C1] ? __pfx_sock_alloc_send_pskb+0x10/0x10 [ 75.374299][ C1] ? __pfx___dev_queue_xmit+0x10/0x10 [ 75.374490][ C1] ? dev_get_by_index+0x177/0x270 [ 75.374691][ C1] packet_snd+0xd08/0x1840 [ 75.374997][ C1] ? __pfx___lock_release+0x10/0x10 [ 75.375192][ C1] ? __pfx_packet_snd+0x10/0x10 [ 75.375376][ C1] ? __might_fault+0xc3/0x170 [ 75.375566][ C1] ? trace_raw_output_mmap_lock_acquire_returned+0x60/0x1e0 [ 75.375924][ C1] ? __might_fault+0x11b/0x170 [ 75.376109][ C1] __sys_sendto+0x37d/0x410 [ 75.376314][ C1] ? __pfx___sys_sendto+0x10/0x10 [ 75.376515][ C1] ? sock_ioctl+0x3bc/0x590 [ 75.376795][ C1] ? __up_read+0x1a7/0x780 [ 75.376984][ C1] ? __pfx___up_read+0x10/0x10 [ 75.377169][ C1] ? do_user_addr_fault+0x4b8/0xbe0 [ 75.377365][ C1] __x64_sys_sendto+0xe0/0x1c0 [ 75.377658][ C1] ? lockdep_hardirqs_on_prepare.part.0+0x1af/0x370 [ 75.377899][ C1] do_syscall_64+0xc6/0x1e0 [ 75.378087][ C1] entry_SYSCALL_64_after_hwframe+0x6d/0x75 [ 75.378341][ C1] RIP: 0033:0x7fe090f7c85a [ 75.378531][ C1] Code: d8 64 89 02 48 c7 c0 ff ff ff ff eb b8 0f 1f 00 f3 0f 1e fa 41 89 ca 64 8b 04 25 18 00 00 00 85 c0 75 15 b8 2c 00 00 00 0f 05 <48> 3d 00 f0 ff ff 77 7e c3 0f 1f 44 00 00 41 54 48 83 ec 30 44 89 [ 75.379277][ C1] RSP: 002b:00007ffc8ead0c28 EFLAGS: 00000246 ORIG_RAX: 000000000000002c [ 75.379582][ C1] RAX: ffffffffffffffda RBX: 0000000018c239d0 RCX: 00007fe090f7c85a [ 75.379865][ C1] RDX: 00000000000003c5 RSI: 0000000018c24022 RDI: 0000000000000005 [ 75.380231][ C1] RBP: 0000000018c24022 R08: 00007ffc8ead0c30 R09: 0000000000000014 [ 75.380527][ C1] R10: 0000000000000000 R11: 0000000000000246 R12: 0000000000000000 [ 75.380824][ C1] R13: 00000000000003c5 R14: 00007ffc8ead0c30 R15: 0000000000000000 [ 75.381192][ C1]