[ 29.127337][ T299] gre: GRE over IPv4 demultiplexor driver [ 29.148527][ T299] ip_gre: GRE over IPv4 tunneling driver [ 29.929150][ T309] GACT probability NOT on [ 30.193503][ T311] ip6_gre: GRE over IPv6 tunneling driver [ 30.356475][ T50] ip6_tunnel: h3-gt6 xmit: Local address not yet configured! [ 30.458343][ T7] ip6_tunnel: h3-gt6 xmit: Local address not yet configured! [ 31.402017][ T10] ip6_tunnel: h3-gt6 xmit: Local address not yet configured! [ 31.402734][ T10] ip6_tunnel: h3-gt6 xmit: Local address not yet configured! [ 31.519052][ T325] br1: port 1(veth1) entered blocking state [ 31.519429][ T325] br1: port 1(veth1) entered disabled state [ 31.519723][ T325] veth1: entered allmulticast mode [ 31.521586][ T325] veth1: entered promiscuous mode [ 31.638188][ T51] br1: port 1(veth1) entered blocking state [ 31.638628][ T51] br1: port 1(veth1) entered forwarding state [ 31.754691][ T327] br1: port 2(veth2) entered blocking state [ 31.754930][ T327] br1: port 2(veth2) entered disabled state [ 31.755194][ T327] veth2: entered allmulticast mode [ 31.756969][ T327] veth2: entered promiscuous mode [ 31.878701][ T49] br1: port 2(veth2) entered blocking state [ 31.879023][ T49] br1: port 2(veth2) entered forwarding state [ 32.425985][ T10] ip6_tunnel: h3-gt6 xmit: Local address not yet configured! [ 32.426890][ T10] ip6_tunnel: h3-gt6 xmit: Local address not yet configured! [ 32.516406][ T7] ip6_tunnel: gt6 xmit: Local address not yet configured! [ 32.713286][ T7] ip6_tunnel: gt6 xmit: Local address not yet configured! [ 33.193329][ T10] ip6_tunnel: gt6 xmit: Local address not yet configured! [ 33.193715][ T7] ip6_tunnel: h3-gt6 xmit: Local address not yet configured! [ 36.457398][ C1] ip6_tnl_xmit_ctl: 2 callbacks suppressed [ 36.458001][ C1] ip6_tunnel: h3-gt6 xmit: Local address not yet configured! [ 40.084665][ T366] Mirror/redirect action on [ 44.329446][ C2] ip6_tunnel: h3-gt6 xmit: Local address not yet configured! [ 46.889254][ C3] [ 46.889406][ C3] ======================================================== [ 46.889717][ C3] WARNING: possible irq lock inversion dependency detected [ 46.890035][ C3] 6.9.0-rc1-virtme #1 Not tainted [ 46.890246][ C3] -------------------------------------------------------- [ 46.890557][ C3] swapper/3/0 just changed the state of lock: [ 46.890804][ C3] ffff8880056f2c38 (&br->lock){+.-.}-{2:2}, at: br_forward_delay_timer_expired+0x48/0x440 [bridge] [ 46.891332][ C3] but this lock took another, SOFTIRQ-unsafe lock in the past: [ 46.891644][ C3] (&p->alloc_lock){+.+.}-{2:2} [ 46.891652][ C3] [ 46.891652][ C3] [ 46.891652][ C3] and interrupts could create inverse lock ordering between them. [ 46.891652][ C3] [ 46.892397][ C3] [ 46.892397][ C3] other info that might help us debug this: [ 46.892698][ C3] Chain exists of: [ 46.892698][ C3] &br->lock --> lweventlist_lock --> &p->alloc_lock [ 46.892698][ C3] [ 46.893168][ C3] Possible interrupt unsafe locking scenario: [ 46.893168][ C3] [ 46.893488][ C3] CPU0 CPU1 [ 46.893697][ C3] ---- ---- [ 46.893905][ C3] lock(&p->alloc_lock); [ 46.894067][ C3] local_irq_disable(); [ 46.894305][ C3] lock(&br->lock); [ 46.894560][ C3] lock(lweventlist_lock); [ 46.894821][ C3] [ 46.894987][ C3] lock(&br->lock); [ 46.895153][ C3] [ 46.895153][ C3] *** DEADLOCK *** [ 46.895153][ C3] [ 46.895440][ C3] 1 lock held by swapper/3/0: [ 46.895633][ C3] #0: ffffc90000288d68 ((&p->forward_delay_timer)){+.-.}-{0:0}, at: call_timer_fn+0xe8/0x230 [ 46.896066][ C3] [ 46.896066][ C3] the shortest dependencies between 2nd lock and 1st lock: [ 46.896426][ C3] -> (&p->alloc_lock){+.+.}-{2:2} { [ 46.896648][ C3] HARDIRQ-ON-W at: [ 46.896813][ C3] __lock_acquire+0x797/0x1570 [ 46.897076][ C3] lock_acquire.part.0+0xeb/0x330 [ 46.897344][ C3] _raw_spin_lock+0x30/0x40 [ 46.897618][ C3] set_mems_allowed+0x1d/0x210 [ 46.897888][ C3] kernel_init_freeable+0x72/0x310 [ 46.898170][ C3] kernel_init+0x20/0x200 [ 46.898445][ C3] ret_from_fork+0x31/0x70 [ 46.898718][ C3] ret_from_fork_asm+0x1a/0x30 [ 46.898995][ C3] SOFTIRQ-ON-W at: [ 46.899158][ C3] __lock_acquire+0x797/0x1570 [ 46.899422][ C3] lock_acquire.part.0+0xeb/0x330 [ 46.899696][ C3] _raw_spin_lock+0x30/0x40 [ 46.899976][ C3] set_mems_allowed+0x1d/0x210 [ 46.900278][ C3] kernel_init_freeable+0x72/0x310 [ 46.900555][ C3] kernel_init+0x20/0x200 [ 46.900829][ C3] ret_from_fork+0x31/0x70 [ 46.901103][ C3] ret_from_fork_asm+0x1a/0x30 [ 46.901373][ C3] INITIAL USE at: [ 46.901544][ C3] __lock_acquire+0x797/0x1570 [ 46.901816][ C3] lock_acquire.part.0+0xeb/0x330 [ 46.902088][ C3] _raw_spin_lock+0x30/0x40 [ 46.902362][ C3] set_mems_allowed+0x1d/0x210 [ 46.902630][ C3] kernel_init_freeable+0x72/0x310 [ 46.902904][ C3] kernel_init+0x20/0x200 [ 46.903184][ C3] ret_from_fork+0x31/0x70 [ 46.903455][ C3] ret_from_fork_asm+0x1a/0x30 [ 46.903725][ C3] } [ 46.903839][ C3] ... key at: [] __key.387+0x0/0x40 [ 46.904167][ C3] ... acquired at: [ 46.904328][ C3] __lock_acquire+0xaf0/0x1570 [ 46.904560][ C3] lock_acquire.part.0+0xeb/0x330 [ 46.904787][ C3] _raw_spin_lock+0x30/0x40 [ 46.905000][ C3] __get_task_comm+0x27/0x70 [ 46.905219][ C3] ref_tracker_alloc+0x2ee/0x490 [ 46.905435][ C3] linkwatch_fire_event+0x196/0x200 [ 46.905652][ C3] veth_newlink+0x367/0x830 [veth] [ 46.905882][ C3] rtnl_newlink_create+0x341/0x850 [ 46.906103][ C3] __rtnl_newlink+0xac9/0xd80 [ 46.906317][ C3] rtnl_newlink+0x63/0xa0 [ 46.906541][ C3] rtnetlink_rcv_msg+0x2fb/0xbf0 [ 46.906757][ C3] netlink_rcv_skb+0x130/0x360 [ 46.906977][ C3] netlink_unicast+0x449/0x710 [ 46.907192][ C3] netlink_sendmsg+0x723/0xbe0 [ 46.907407][ C3] ____sys_sendmsg+0x7b2/0xa10 [ 46.907631][ C3] ___sys_sendmsg+0xee/0x170 [ 46.907851][ C3] __sys_sendmsg+0xcd/0x170 [ 46.908072][ C3] do_syscall_64+0xc6/0x1e0 [ 46.908288][ C3] entry_SYSCALL_64_after_hwframe+0x6d/0x75 [ 46.908562][ C3] [ 46.908673][ C3] -> (lweventlist_lock){....}-{2:2} { [ 46.908900][ C3] INITIAL USE at: [ 46.909090][ C3] __lock_acquire+0x797/0x1570 [ 46.909366][ C3] lock_acquire.part.0+0xeb/0x330 [ 46.909636][ C3] _raw_spin_lock_irqsave+0x3f/0x60 [ 46.909911][ C3] linkwatch_sync_dev+0x29/0x1f0 [ 46.910190][ C3] rtnl_getlink+0x4c9/0x970 [ 46.910458][ C3] rtnetlink_rcv_msg+0x2fb/0xbf0 [ 46.910723][ C3] netlink_rcv_skb+0x130/0x360 [ 46.910993][ C3] netlink_unicast+0x449/0x710 [ 46.911263][ C3] netlink_sendmsg+0x723/0xbe0 [ 46.911529][ C3] __sys_sendto+0x37d/0x410 [ 46.911822][ C3] __x64_sys_sendto+0xe0/0x1c0 [ 46.912094][ C3] do_syscall_64+0xc6/0x1e0 [ 46.912362][ C3] entry_SYSCALL_64_after_hwframe+0x6d/0x75 [ 46.912689][ C3] } [ 46.912801][ C3] ... key at: [] lweventlist_lock+0x18/0x60 [ 46.913126][ C3] ... acquired at: [ 46.913285][ C3] __lock_acquire+0xaf0/0x1570 [ 46.913510][ C3] lock_acquire.part.0+0xeb/0x330 [ 46.913725][ C3] _raw_spin_lock_irqsave+0x3f/0x60 [ 46.913944][ C3] linkwatch_fire_event+0x74/0x200 [ 46.914162][ C3] br_stp_change_bridge_id+0x25f/0x360 [bridge] [ 46.914509][ C3] br_set_mac_address+0x1f3/0x260 [bridge] [ 46.914846][ C3] dev_set_mac_address+0x28a/0x410 [ 46.915062][ C3] dev_set_mac_address_user+0x31/0x50 [ 46.915291][ C3] do_setlink+0x17b4/0x21e0 [ 46.915520][ C3] rtnl_setlink+0x1f7/0x310 [ 46.915748][ C3] rtnetlink_rcv_msg+0x2fb/0xbf0 [ 46.915965][ C3] netlink_rcv_skb+0x130/0x360 [ 46.916180][ C3] netlink_unicast+0x449/0x710 [ 46.916395][ C3] netlink_sendmsg+0x723/0xbe0 [ 46.916613][ C3] __sys_sendto+0x37d/0x410 [ 46.916829][ C3] __x64_sys_sendto+0xe0/0x1c0 [ 46.917044][ C3] do_syscall_64+0xc6/0x1e0 [ 46.917264][ C3] entry_SYSCALL_64_after_hwframe+0x6d/0x75 [ 46.917530][ C3] [ 46.917640][ C3] -> (&br->lock){+.-.}-{2:2} { [ 46.917864][ C3] HARDIRQ-ON-W at: [ 46.918026][ C3] __lock_acquire+0x797/0x1570 [ 46.918304][ C3] lock_acquire.part.0+0xeb/0x330 [ 46.918572][ C3] _raw_spin_lock_bh+0x38/0x50 [ 46.918845][ C3] recalculate_group_addr+0x55/0x130 [bridge] [ 46.919262][ C3] br_vlan_filter_toggle+0x77/0x130 [bridge] [ 46.919654][ C3] br_changelink+0x36c/0x1360 [bridge] [ 46.919994][ C3] br_dev_newlink+0xa5/0x100 [bridge] [ 46.920330][ C3] rtnl_newlink_create+0x341/0x850 [ 46.920605][ C3] __rtnl_newlink+0xac9/0xd80 [ 46.920884][ C3] rtnl_newlink+0x63/0xa0 [ 46.921156][ C3] rtnetlink_rcv_msg+0x2fb/0xbf0 [ 46.921430][ C3] netlink_rcv_skb+0x130/0x360 [ 46.921703][ C3] netlink_unicast+0x449/0x710 [ 46.921971][ C3] netlink_sendmsg+0x723/0xbe0 [ 46.922241][ C3] ____sys_sendmsg+0x7b2/0xa10 [ 46.922515][ C3] ___sys_sendmsg+0xee/0x170 [ 46.922797][ C3] __sys_sendmsg+0xcd/0x170 [ 46.923066][ C3] do_syscall_64+0xc6/0x1e0 [ 46.923330][ C3] entry_SYSCALL_64_after_hwframe+0x6d/0x75 [ 46.923652][ C3] IN-SOFTIRQ-W at: [ 46.923815][ C3] __lock_acquire+0x797/0x1570 [ 46.924086][ C3] lock_acquire.part.0+0xeb/0x330 [ 46.924356][ C3] _raw_spin_lock+0x30/0x40 [ 46.924637][ C3] br_forward_delay_timer_expired+0x48/0x440 [bridge] [ 46.925031][ C3] call_timer_fn+0x13b/0x230 [ 46.925307][ C3] __run_timers+0x545/0x810 [ 46.925572][ C3] run_timer_softirq+0xe8/0x1b0 [ 46.925845][ C3] __do_softirq+0x1f8/0x5df [ 46.926118][ C3] irq_exit_rcu+0x97/0xc0 [ 46.926400][ C3] sysvec_apic_timer_interrupt+0x75/0x80 [ 46.926721][ C3] asm_sysvec_apic_timer_interrupt+0x1a/0x20 [ 46.927190][ C3] default_idle+0xf/0x20 [ 46.927459][ C3] default_idle_call+0x6d/0xb0 [ 46.927727][ C3] cpuidle_idle_call+0x1f4/0x280 [ 46.927998][ C3] do_idle+0xf9/0x160 [ 46.928228][ C3] cpu_startup_entry+0x54/0x60 [ 46.928499][ C3] start_secondary+0x21c/0x2b0 [ 46.928771][ C3] common_startup_64+0x12c/0x138 [ 46.929194][ C3] INITIAL USE at: [ 46.929358][ C3] __lock_acquire+0x797/0x1570 [ 46.929628][ C3] lock_acquire.part.0+0xeb/0x330 [ 46.929897][ C3] _raw_spin_lock_bh+0x38/0x50 [ 46.930177][ C3] recalculate_group_addr+0x55/0x130 [bridge] [ 46.930569][ C3] br_vlan_filter_toggle+0x77/0x130 [bridge] [ 46.930963][ C3] br_changelink+0x36c/0x1360 [bridge] [ 46.931442][ C3] br_dev_newlink+0xa5/0x100 [bridge] [ 46.931779][ C3] rtnl_newlink_create+0x341/0x850 [ 46.932051][ C3] __rtnl_newlink+0xac9/0xd80 [ 46.932331][ C3] rtnl_newlink+0x63/0xa0 [ 46.932883][ C3] rtnetlink_rcv_msg+0x2fb/0xbf0 [ 46.933150][ C3] netlink_rcv_skb+0x130/0x360 [ 46.933418][ C3] netlink_unicast+0x449/0x710 [ 46.933831][ C3] netlink_sendmsg+0x723/0xbe0 [ 46.934106][ C3] ____sys_sendmsg+0x7b2/0xa10 [ 46.934373][ C3] ___sys_sendmsg+0xee/0x170 [ 46.934774][ C3] __sys_sendmsg+0xcd/0x170 [ 46.935039][ C3] do_syscall_64+0xc6/0x1e0 [ 46.935303][ C3] entry_SYSCALL_64_after_hwframe+0x6d/0x75 [ 46.935614][ C3] } [ 46.935852][ C3] ... key at: [] __key.6+0x0/0xfffffffffffe5ac0 [bridge] [ 46.936279][ C3] ... acquired at: [ 46.936433][ C3] mark_lock+0x28d/0x3e0 [ 46.936648][ C3] mark_usage+0xd9/0x2a0 [ 46.936993][ C3] __lock_acquire+0x797/0x1570 [ 46.937204][ C3] lock_acquire.part.0+0xeb/0x330 [ 46.937415][ C3] _raw_spin_lock+0x30/0x40 [ 46.937634][ C3] br_forward_delay_timer_expired+0x48/0x440 [bridge] [ 46.938113][ C3] call_timer_fn+0x13b/0x230 [ 46.938323][ C3] __run_timers+0x545/0x810 [ 46.938537][ C3] run_timer_softirq+0xe8/0x1b0 [ 46.938749][ C3] __do_softirq+0x1f8/0x5df [ 46.939103][ C3] irq_exit_rcu+0x97/0xc0 [ 46.939313][ C3] sysvec_apic_timer_interrupt+0x75/0x80 [ 46.939578][ C3] asm_sysvec_apic_timer_interrupt+0x1a/0x20 [ 46.939838][ C3] default_idle+0xf/0x20 [ 46.940164][ C3] default_idle_call+0x6d/0xb0 [ 46.940369][ C3] cpuidle_idle_call+0x1f4/0x280 [ 46.940581][ C3] do_idle+0xf9/0x160 [ 46.940745][ C3] cpu_startup_entry+0x54/0x60 [ 46.941087][ C3] start_secondary+0x21c/0x2b0 [ 46.941300][ C3] common_startup_64+0x12c/0x138 [ 46.941514][ C3] [ 46.941618][ C3] [ 46.941618][ C3] stack backtrace: [ 46.941875][ C3] CPU: 3 PID: 0 Comm: swapper/3 Not tainted 6.9.0-rc1-virtme #1 [ 46.942317][ C3] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS rel-1.16.3-0-ga6ed6b701f0a-prebuilt.qemu.org 04/01/2014 [ 46.942788][ C3] Call Trace: [ 46.942955][ C3] [ 46.943202][ C3] dump_stack_lvl+0x82/0xd0 [ 46.943412][ C3] print_irq_inversion_bug.part.0+0x3d9/0x570 [ 46.943662][ C3] ? common_startup_64+0x12c/0x138 [ 46.943878][ C3] ? __pfx_print_irq_inversion_bug.part.0+0x10/0x10 [ 46.944139][ C3] ? __pfx_usage_skip+0x10/0x10 [ 46.944349][ C3] ? __pfx_usage_match+0x10/0x10 [ 46.944569][ C3] ? arch_stack_walk+0x8c/0xf0 [ 46.944774][ C3] mark_lock_irq+0x4cd/0xa10 [ 46.945120][ C3] ? __pfx_mark_lock_irq+0x10/0x10 [ 46.945333][ C3] ? stack_trace_save+0x94/0xd0 [ 46.945554][ C3] ? save_trace+0x8f/0x5b0 [ 46.945770][ C3] mark_lock+0x28d/0x3e0 [ 46.945944][ C3] mark_usage+0xd9/0x2a0 [ 46.946249][ C3] __lock_acquire+0x797/0x1570 [ 46.946463][ C3] ? trace_rcu_dyntick+0x112/0x180 [ 46.946690][ C3] ? __pfx_br_forward_delay_timer_expired+0x10/0x10 [bridge] [ 46.947076][ C3] lock_acquire.part.0+0xeb/0x330 [ 46.947291][ C3] ? br_forward_delay_timer_expired+0x48/0x440 [bridge] [ 46.947624][ C3] ? __pfx_lock_acquire.part.0+0x10/0x10 [ 46.947839][ C3] ? trace_lock_acquire+0x135/0x1c0 [ 46.948055][ C3] ? br_forward_delay_timer_expired+0x48/0x440 [bridge] [ 46.948384][ C3] ? lock_acquire+0x32/0xc0 [ 46.948608][ C3] ? br_forward_delay_timer_expired+0x48/0x440 [bridge] [ 46.948943][ C3] _raw_spin_lock+0x30/0x40 [ 46.949311][ C3] ? br_forward_delay_timer_expired+0x48/0x440 [bridge] [ 46.949639][ C3] br_forward_delay_timer_expired+0x48/0x440 [bridge] [ 46.950015][ C3] ? __pfx_br_forward_delay_timer_expired+0x10/0x10 [bridge] [ 46.950400][ C3] call_timer_fn+0x13b/0x230 [ 46.950622][ C3] ? call_timer_fn+0xe8/0x230 [ 46.950835][ C3] ? call_timer_fn+0xe8/0x230 [ 46.951051][ C3] ? __pfx_call_timer_fn+0x10/0x10 [ 46.951266][ C3] ? mark_lock+0x38/0x3e0 [ 46.951579][ C3] __run_timers+0x545/0x810 [ 46.951798][ C3] ? __pfx_br_forward_delay_timer_expired+0x10/0x10 [bridge] [ 46.952191][ C3] ? __pfx___run_timers+0x10/0x10 [ 46.952409][ C3] ? do_raw_spin_lock+0x131/0x270 [ 46.952766][ C3] ? __pfx_do_raw_spin_lock+0x10/0x10 [ 46.952991][ C3] ? lock_acquire+0x32/0xc0 [ 46.953205][ C3] ? run_timer_softirq+0xe0/0x1b0 [ 46.953421][ C3] run_timer_softirq+0xe8/0x1b0 [ 46.953778][ C3] __do_softirq+0x1f8/0x5df [ 46.953996][ C3] irq_exit_rcu+0x97/0xc0 [ 46.954161][ C3] sysvec_apic_timer_interrupt+0x75/0x80 [ 46.954377][ C3] [ 46.954488][ C3] [ 46.954601][ C3] asm_sysvec_apic_timer_interrupt+0x1a/0x20 [ 46.955017][ C3] RIP: 0010:default_idle+0xf/0x20 [ 46.955242][ C3] Code: 4c 01 c7 4c 29 c2 e9 72 ff ff ff 90 90 90 90 90 90 90 90 90 90 90 90 90 90 90 90 f3 0f 1e fa 66 90 0f 00 2d 93 20 35 00 fb f4 c3 cc cc cc cc 66 66 2e 0f 1f 84 00 00 00 00 00 90 90 90 90 90 [ 46.956141][ C3] RSP: 0018:ffffc9000016fdf8 EFLAGS: 00000242 [ 46.956421][ C3] RAX: 000000000008a8c1 RBX: 1ffff9200002dfc1 RCX: ffffffff9310f9e5 [ 46.956753][ C3] RDX: 0000000000000000 RSI: 0000000000000000 RDI: ffffffff90aaa4a4 [ 46.957219][ C3] RBP: 0000000000000000 R08: 0000000000000001 R09: ffffed1006c3709c [ 46.957535][ C3] R10: ffff8880361b84e3 R11: ffff8880361bdc80 R12: 0000000000000000 [ 46.958000][ C3] R13: ffff888001b88040 R14: dffffc0000000000 R15: 0000000000000000 [ 46.958326][ C3] ? ct_kernel_exit.constprop.0+0xc5/0xf0 [ 46.958546][ C3] ? cpuidle_idle_call+0x1f4/0x280 [ 46.958764][ C3] default_idle_call+0x6d/0xb0 [ 46.959162][ C3] cpuidle_idle_call+0x1f4/0x280 [ 46.959307][ C3] ? __pfx_cpuidle_idle_call+0x10/0x10 [ 46.959445][ C3] ? tsc_verify_tsc_adjust+0x5e/0x2b0 [ 46.959582][ C3] do_idle+0xf9/0x160 [ 46.959686][ C3] cpu_startup_entry+0x54/0x60 [ 46.959897][ C3] start_secondary+0x21c/0x2b0 [ 46.960038][ C3] ? __pfx_start_secondary+0x10/0x10 [ 46.960183][ C3] common_startup_64+0x12c/0x138 [ 46.960321][ C3] [ 59.689310][ C0] ip6_tunnel: h3-gt6 xmit: Local address not yet configured! [ 90.409554][ C1] ip6_tunnel: h3-gt6 xmit: Local address not yet configured! [ 112.129331][ T734] br1: port 1(veth1) entered disabled state [ 112.219072][ T735] br1: port 2(veth2) entered disabled state [ 112.317043][ T736] veth2: left allmulticast mode [ 112.317229][ T736] veth2: left promiscuous mode [ 112.317482][ T736] br1: port 2(veth2) entered disabled state [ 112.318417][ T736] veth1: left allmulticast mode [ 112.318563][ T736] veth1: left promiscuous mode [ 112.318823][ T736] br1: port 1(veth1) entered disabled state