[ 24.226982][ T238] ip (238) used greatest stack depth: 23168 bytes left [ 33.066593][ T290] veth1: entered promiscuous mode [ 57.403848][ T372] veth1: entered allmulticast mode [ 63.734128][ T387] veth1: left allmulticast mode [ 73.140957][ T457] veth1: left promiscuous mode [ 76.800042][ T488] br0: port 1(veth1) entered blocking state [ 76.801053][ T488] br0: port 1(veth1) entered disabled state [ 76.801499][ T488] veth1: entered allmulticast mode [ 76.805526][ T488] veth1: entered promiscuous mode [ 76.956196][ T50] br0: port 1(veth1) entered blocking state [ 76.956882][ T50] br0: port 1(veth1) entered forwarding state [ 77.565778][ T495] br0: port 1(veth1) entered disabled state [ 77.568266][ T495] br0: port 1(veth1) entered blocking state [ 77.568649][ T495] br0: port 1(veth1) entered forwarding state [ 78.304367][ T502] br0: entered promiscuous mode [ 93.005889][ C2] [ 93.006013][ C2] ======================================================== [ 93.006262][ C2] WARNING: possible irq lock inversion dependency detected [ 93.006486][ C2] 6.9.0-rc1-virtme #1 Not tainted [ 93.006636][ C2] -------------------------------------------------------- [ 93.006860][ C2] swapper/2/0 just changed the state of lock: [ 93.007046][ C2] ffff888007f3ac38 (&br->lock){+.-.}-{2:2}, at: br_forward_delay_timer_expired+0x48/0x440 [bridge] [ 93.007429][ C2] but this lock took another, SOFTIRQ-unsafe lock in the past: [ 93.007654][ C2] (&p->alloc_lock){+.+.}-{2:2} [ 93.007658][ C2] [ 93.007658][ C2] [ 93.007658][ C2] and interrupts could create inverse lock ordering between them. [ 93.007658][ C2] [ 93.008201][ C2] [ 93.008201][ C2] other info that might help us debug this: [ 93.008422][ C2] Chain exists of: [ 93.008422][ C2] &br->lock --> lweventlist_lock --> &p->alloc_lock [ 93.008422][ C2] [ 93.008761][ C2] Possible interrupt unsafe locking scenario: [ 93.008761][ C2] [ 93.008987][ C2] CPU0 CPU1 [ 93.009138][ C2] ---- ---- [ 93.009287][ C2] lock(&p->alloc_lock); [ 93.009405][ C2] local_irq_disable(); [ 93.009586][ C2] lock(&br->lock); [ 93.009771][ C2] lock(lweventlist_lock); [ 93.009957][ C2] [ 93.010071][ C2] lock(&br->lock); [ 93.010185][ C2] [ 93.010185][ C2] *** DEADLOCK *** [ 93.010185][ C2] [ 93.010404][ C2] 1 lock held by swapper/2/0: [ 93.010562][ C2] #0: ffffc90000230b68 ((&p->forward_delay_timer)){+.-.}-{0:0}, at: call_timer_fn+0xe8/0x230 [ 93.010871][ C2] [ 93.010871][ C2] the shortest dependencies between 2nd lock and 1st lock: [ 93.011127][ C2] -> (&p->alloc_lock){+.+.}-{2:2} { [ 93.011284][ C2] HARDIRQ-ON-W at: [ 93.011400][ C2] __lock_acquire+0x797/0x1570 [ 93.011592][ C2] lock_acquire.part.0+0xeb/0x330 [ 93.011787][ C2] _raw_spin_lock+0x30/0x40 [ 93.011978][ C2] set_mems_allowed+0x1d/0x210 [ 93.012167][ C2] kernel_init_freeable+0x72/0x310 [ 93.012360][ C2] kernel_init+0x20/0x200 [ 93.012556][ C2] ret_from_fork+0x31/0x70 [ 93.012750][ C2] ret_from_fork_asm+0x1a/0x30 [ 93.012950][ C2] SOFTIRQ-ON-W at: [ 93.013067][ C2] __lock_acquire+0x797/0x1570 [ 93.013254][ C2] lock_acquire.part.0+0xeb/0x330 [ 93.013444][ C2] _raw_spin_lock+0x30/0x40 [ 93.013633][ C2] set_mems_allowed+0x1d/0x210 [ 93.013821][ C2] kernel_init_freeable+0x72/0x310 [ 93.014012][ C2] kernel_init+0x20/0x200 [ 93.014222][ C2] ret_from_fork+0x31/0x70 [ 93.014418][ C2] ret_from_fork_asm+0x1a/0x30 [ 93.014607][ C2] INITIAL USE at: [ 93.014721][ C2] __lock_acquire+0x797/0x1570 [ 93.014910][ C2] lock_acquire.part.0+0xeb/0x330 [ 93.015109][ C2] _raw_spin_lock+0x30/0x40 [ 93.015300][ C2] set_mems_allowed+0x1d/0x210 [ 93.015490][ C2] kernel_init_freeable+0x72/0x310 [ 93.015675][ C2] kernel_init+0x20/0x200 [ 93.015849][ C2] ret_from_fork+0x31/0x70 [ 93.016021][ C2] ret_from_fork_asm+0x1a/0x30 [ 93.016195][ C2] } [ 93.016274][ C2] ... key at: [] __key.387+0x0/0x40 [ 93.016485][ C2] ... acquired at: [ 93.016590][ C2] __lock_acquire+0xaf0/0x1570 [ 93.016735][ C2] lock_acquire.part.0+0xeb/0x330 [ 93.016876][ C2] _raw_spin_lock+0x30/0x40 [ 93.017013][ C2] __get_task_comm+0x27/0x70 [ 93.017153][ C2] ref_tracker_alloc+0x2ee/0x490 [ 93.017295][ C2] linkwatch_fire_event+0x196/0x200 [ 93.017434][ C2] veth_newlink+0x367/0x830 [veth] [ 93.017582][ C2] rtnl_newlink_create+0x341/0x850 [ 93.017721][ C2] __rtnl_newlink+0xac9/0xd80 [ 93.017862][ C2] rtnl_newlink+0x63/0xa0 [ 93.018004][ C2] rtnetlink_rcv_msg+0x2fb/0xbf0 [ 93.018158][ C2] netlink_rcv_skb+0x130/0x360 [ 93.018310][ C2] netlink_unicast+0x449/0x710 [ 93.018456][ C2] netlink_sendmsg+0x723/0xbe0 [ 93.018599][ C2] ____sys_sendmsg+0x7b2/0xa10 [ 93.018834][ C2] ___sys_sendmsg+0xee/0x170 [ 93.018977][ C2] __sys_sendmsg+0xcd/0x170 [ 93.019116][ C2] do_syscall_64+0xc6/0x1e0 [ 93.019258][ C2] entry_SYSCALL_64_after_hwframe+0x6d/0x75 [ 93.019433][ C2] [ 93.019505][ C2] -> (lweventlist_lock){....}-{2:2} { [ 93.019648][ C2] INITIAL USE at: [ 93.019755][ C2] __lock_acquire+0x797/0x1570 [ 93.019929][ C2] lock_acquire.part.0+0xeb/0x330 [ 93.020186][ C2] _raw_spin_lock_irqsave+0x3f/0x60 [ 93.020360][ C2] linkwatch_sync_dev+0x29/0x1f0 [ 93.020533][ C2] rtnl_getlink+0x4c9/0x970 [ 93.020794][ C2] rtnetlink_rcv_msg+0x2fb/0xbf0 [ 93.020973][ C2] netlink_rcv_skb+0x130/0x360 [ 93.021145][ C2] netlink_unicast+0x449/0x710 [ 93.021318][ C2] netlink_sendmsg+0x723/0xbe0 [ 93.021493][ C2] __sys_sendto+0x37d/0x410 [ 93.021666][ C2] __x64_sys_sendto+0xe0/0x1c0 [ 93.021842][ C2] do_syscall_64+0xc6/0x1e0 [ 93.022117][ C2] entry_SYSCALL_64_after_hwframe+0x6d/0x75 [ 93.022334][ C2] } [ 93.022409][ C2] ... key at: [] lweventlist_lock+0x18/0x60 [ 93.022632][ C2] ... acquired at: [ 93.022823][ C2] __lock_acquire+0xaf0/0x1570 [ 93.023059][ C2] lock_acquire.part.0+0xeb/0x330 [ 93.023202][ C2] _raw_spin_lock_irqsave+0x3f/0x60 [ 93.023348][ C2] linkwatch_fire_event+0x74/0x200 [ 93.023488][ C2] br_stp_change_bridge_id+0x25f/0x360 [bridge] [ 93.023706][ C2] br_set_mac_address+0x1f3/0x260 [bridge] [ 93.023910][ C2] dev_set_mac_address+0x28a/0x410 [ 93.024051][ C2] dev_set_mac_address_user+0x31/0x50 [ 93.024270][ C2] do_setlink+0x17b4/0x21e0 [ 93.024409][ C2] rtnl_setlink+0x1f7/0x310 [ 93.024545][ C2] rtnetlink_rcv_msg+0x2fb/0xbf0 [ 93.024687][ C2] netlink_rcv_skb+0x130/0x360 [ 93.024988][ C2] netlink_unicast+0x449/0x710 [ 93.025128][ C2] netlink_sendmsg+0x723/0xbe0 [ 93.025267][ C2] __sys_sendto+0x37d/0x410 [ 93.025405][ C2] __x64_sys_sendto+0xe0/0x1c0 [ 93.025546][ C2] do_syscall_64+0xc6/0x1e0 [ 93.025762][ C2] entry_SYSCALL_64_after_hwframe+0x6d/0x75 [ 93.025935][ C2] [ 93.026005][ C2] -> (&br->lock){+.-.}-{2:2} { [ 93.026153][ C2] HARDIRQ-ON-W at: [ 93.026340][ C2] __lock_acquire+0x797/0x1570 [ 93.026515][ C2] lock_acquire.part.0+0xeb/0x330 [ 93.026689][ C2] _raw_spin_lock_bh+0x38/0x50 [ 93.026865][ C2] br_set_mac_address+0xd2/0x260 [bridge] [ 93.027190][ C2] dev_set_mac_address+0x28a/0x410 [ 93.027363][ C2] dev_set_mac_address_user+0x31/0x50 [ 93.027548][ C2] do_setlink+0x17b4/0x21e0 [ 93.027804][ C2] rtnl_setlink+0x1f7/0x310 [ 93.027977][ C2] rtnetlink_rcv_msg+0x2fb/0xbf0 [ 93.028149][ C2] netlink_rcv_skb+0x130/0x360 [ 93.028406][ C2] netlink_unicast+0x449/0x710 [ 93.028580][ C2] netlink_sendmsg+0x723/0xbe0 [ 93.028773][ C2] __sys_sendto+0x37d/0x410 [ 93.028960][ C2] __x64_sys_sendto+0xe0/0x1c0 [ 93.029232][ C2] do_syscall_64+0xc6/0x1e0 [ 93.029420][ C2] entry_SYSCALL_64_after_hwframe+0x6d/0x75 [ 93.029645][ C2] IN-SOFTIRQ-W at: [ 93.029845][ C2] __lock_acquire+0x797/0x1570 [ 93.030035][ C2] lock_acquire.part.0+0xeb/0x330 [ 93.030226][ C2] _raw_spin_lock+0x30/0x40 [ 93.030414][ C2] br_forward_delay_timer_expired+0x48/0x440 [bridge] [ 93.030677][ C2] call_timer_fn+0x13b/0x230 [ 93.030869][ C2] __run_timers+0x545/0x810 [ 93.031056][ C2] timer_expire_remote+0x9b/0xe0 [ 93.031328][ C2] tmigr_handle_remote_cpu+0x270/0x430 [ 93.031524][ C2] tmigr_handle_remote_up+0x19f/0x2e0 [ 93.031712][ C2] tmigr_handle_remote+0x21d/0x3a0 [ 93.031983][ C2] __do_softirq+0x1f8/0x5df [ 93.032174][ C2] irq_exit_rcu+0x97/0xc0 [ 93.032363][ C2] sysvec_apic_timer_interrupt+0x75/0x80 [ 93.032588][ C2] asm_sysvec_apic_timer_interrupt+0x1a/0x20 [ 93.032910][ C2] default_idle+0xf/0x20 [ 93.033100][ C2] default_idle_call+0x6d/0xb0 [ 93.033306][ C2] cpuidle_idle_call+0x1f4/0x280 [ 93.033581][ C2] do_idle+0xf9/0x160 [ 93.033733][ C2] cpu_startup_entry+0x54/0x60 [ 93.033921][ C2] start_secondary+0x21c/0x2b0 [ 93.034194][ C2] common_startup_64+0x12c/0x138 [ 93.034388][ C2] INITIAL USE at: [ 93.034501][ C2] __lock_acquire+0x797/0x1570 [ 93.034688][ C2] lock_acquire.part.0+0xeb/0x330 [ 93.034959][ C2] _raw_spin_lock_bh+0x38/0x50 [ 93.035146][ C2] br_set_mac_address+0xd2/0x260 [bridge] [ 93.035406][ C2] dev_set_mac_address+0x28a/0x410 [ 93.035679][ C2] dev_set_mac_address_user+0x31/0x50 [ 93.035867][ C2] do_setlink+0x17b4/0x21e0 [ 93.036056][ C2] rtnl_setlink+0x1f7/0x310 [ 93.036242][ C2] rtnetlink_rcv_msg+0x2fb/0xbf0 [ 93.036592][ C2] netlink_rcv_skb+0x130/0x360 [ 93.036780][ C2] netlink_unicast+0x449/0x710 [ 93.036974][ C2] netlink_sendmsg+0x723/0xbe0 [ 93.037247][ C2] __sys_sendto+0x37d/0x410 [ 93.037439][ C2] __x64_sys_sendto+0xe0/0x1c0 [ 93.037630][ C2] do_syscall_64+0xc6/0x1e0 [ 93.037818][ C2] entry_SYSCALL_64_after_hwframe+0x6d/0x75 [ 93.038126][ C2] } [ 93.038202][ C2] ... key at: [] __key.6+0x0/0xfffffffffffe5ac0 [bridge] [ 93.038499][ C2] ... acquired at: [ 93.038690][ C2] mark_lock+0x28d/0x3e0 [ 93.038840][ C2] mark_usage+0xd9/0x2a0 [ 93.038992][ C2] __lock_acquire+0x797/0x1570 [ 93.039144][ C2] lock_acquire.part.0+0xeb/0x330 [ 93.039296][ C2] _raw_spin_lock+0x30/0x40 [ 93.039526][ C2] br_forward_delay_timer_expired+0x48/0x440 [bridge] [ 93.039751][ C2] call_timer_fn+0x13b/0x230 [ 93.039906][ C2] __run_timers+0x545/0x810 [ 93.040057][ C2] timer_expire_remote+0x9b/0xe0 [ 93.040288][ C2] tmigr_handle_remote_cpu+0x270/0x430 [ 93.040437][ C2] tmigr_handle_remote_up+0x19f/0x2e0 [ 93.040587][ C2] tmigr_handle_remote+0x21d/0x3a0 [ 93.040738][ C2] __do_softirq+0x1f8/0x5df [ 93.040976][ C2] irq_exit_rcu+0x97/0xc0 [ 93.041129][ C2] sysvec_apic_timer_interrupt+0x75/0x80 [ 93.041318][ C2] asm_sysvec_apic_timer_interrupt+0x1a/0x20 [ 93.041508][ C2] default_idle+0xf/0x20 [ 93.041740][ C2] default_idle_call+0x6d/0xb0 [ 93.041891][ C2] cpuidle_idle_call+0x1f4/0x280 [ 93.042040][ C2] do_idle+0xf9/0x160 [ 93.042154][ C2] cpu_startup_entry+0x54/0x60 [ 93.042387][ C2] start_secondary+0x21c/0x2b0 [ 93.042542][ C2] common_startup_64+0x12c/0x138 [ 93.042701][ C2] [ 93.042779][ C2] [ 93.042779][ C2] stack backtrace: [ 93.042977][ C2] CPU: 2 PID: 0 Comm: swapper/2 Not tainted 6.9.0-rc1-virtme #1 [ 93.043288][ C2] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS rel-1.16.3-0-ga6ed6b701f0a-prebuilt.qemu.org 04/01/2014 [ 93.043627][ C2] Call Trace: [ 93.043833][ C2] [ 93.043914][ C2] dump_stack_lvl+0x82/0xd0 [ 93.044074][ C2] print_irq_inversion_bug.part.0+0x3d9/0x570 [ 93.044267][ C2] ? common_startup_64+0x12c/0x138 [ 93.044424][ C2] ? __pfx_print_irq_inversion_bug.part.0+0x10/0x10 [ 93.044613][ C2] ? __pfx_usage_skip+0x10/0x10 [ 93.044761][ C2] ? __pfx_usage_match+0x10/0x10 [ 93.044911][ C2] ? arch_stack_walk+0x8c/0xf0 [ 93.045065][ C2] mark_lock_irq+0x4cd/0xa10 [ 93.045301][ C2] ? __pfx_mark_lock_irq+0x10/0x10 [ 93.045457][ C2] ? stack_trace_save+0x94/0xd0 [ 93.045612][ C2] ? find_held_lock+0x2c/0x110 [ 93.045765][ C2] ? save_trace+0x8f/0x5b0 [ 93.045994][ C2] ? kvm_sched_clock_read+0x11/0x20 [ 93.046145][ C2] mark_lock+0x28d/0x3e0 [ 93.046260][ C2] mark_usage+0xd9/0x2a0 [ 93.046375][ C2] __lock_acquire+0x797/0x1570 [ 93.046525][ C2] ? trace_rcu_dyntick+0x112/0x180 [ 93.046761][ C2] ? __pfx_br_forward_delay_timer_expired+0x10/0x10 [bridge] [ 93.047028][ C2] lock_acquire.part.0+0xeb/0x330 [ 93.047179][ C2] ? br_forward_delay_timer_expired+0x48/0x440 [bridge] [ 93.047486][ C2] ? __pfx_lock_acquire.part.0+0x10/0x10 [ 93.047637][ C2] ? trace_lock_acquire+0x135/0x1c0 [ 93.047790][ C2] ? br_forward_delay_timer_expired+0x48/0x440 [bridge] [ 93.048013][ C2] ? lock_acquire+0x32/0xc0 [ 93.048325][ C2] ? br_forward_delay_timer_expired+0x48/0x440 [bridge] [ 93.048552][ C2] _raw_spin_lock+0x30/0x40 [ 93.048705][ C2] ? br_forward_delay_timer_expired+0x48/0x440 [bridge] [ 93.048928][ C2] br_forward_delay_timer_expired+0x48/0x440 [bridge] [ 93.049233][ C2] ? __pfx_br_forward_delay_timer_expired+0x10/0x10 [bridge] [ 93.049495][ C2] call_timer_fn+0x13b/0x230 [ 93.049649][ C2] ? call_timer_fn+0xe8/0x230 [ 93.049804][ C2] ? call_timer_fn+0xe8/0x230 [ 93.050037][ C2] ? __pfx_call_timer_fn+0x10/0x10 [ 93.050188][ C2] ? mark_lock+0x38/0x3e0 [ 93.050309][ C2] __run_timers+0x545/0x810 [ 93.050462][ C2] ? __pfx_br_forward_delay_timer_expired+0x10/0x10 [bridge] [ 93.050810][ C2] ? __pfx___run_timers+0x10/0x10 [ 93.050962][ C2] ? __lock_release+0x103/0x460 [ 93.051119][ C2] ? do_raw_spin_lock+0x131/0x270 [ 93.051275][ C2] ? __pfx_do_raw_spin_lock+0x10/0x10 [ 93.051510][ C2] ? lock_acquire+0x32/0xc0 [ 93.051663][ C2] ? timer_expire_remote+0x93/0xe0 [ 93.051817][ C2] timer_expire_remote+0x9b/0xe0 [ 93.051968][ C2] tmigr_handle_remote_cpu+0x270/0x430 [ 93.052201][ C2] ? __pfx_tmigr_handle_remote_cpu+0x10/0x10 [ 93.052392][ C2] tmigr_handle_remote_up+0x19f/0x2e0 [ 93.052543][ C2] tmigr_handle_remote+0x21d/0x3a0 [ 93.052697][ C2] ? __pfx_tmigr_handle_remote+0x10/0x10 [ 93.052939][ C2] __do_softirq+0x1f8/0x5df [ 93.053092][ C2] irq_exit_rcu+0x97/0xc0 [ 93.053205][ C2] sysvec_apic_timer_interrupt+0x75/0x80 [ 93.053356][ C2] [ 93.053439][ C2] [ 93.053516][ C2] asm_sysvec_apic_timer_interrupt+0x1a/0x20 [ 93.053786][ C2] RIP: 0010:default_idle+0xf/0x20 [ 93.053943][ C2] Code: 4c 01 c7 4c 29 c2 e9 72 ff ff ff 90 90 90 90 90 90 90 90 90 90 90 90 90 90 90 90 f3 0f 1e fa 66 90 0f 00 2d a3 30 35 00 fb f4 c3 cc cc cc cc 66 66 2e 0f 1f 84 00 00 00 00 00 90 90 90 90 90 [ 93.054567][ C2] RSP: 0018:ffffc9000015fdf8 EFLAGS: 00000246 [ 93.054760][ C2] RAX: 00000000001bdce1 RBX: 1ffff9200002bfc1 RCX: ffffffffbbb0e9d5 [ 93.054987][ C2] RDX: 0000000000000000 RSI: 0000000000000000 RDI: ffffffffb94aa4a4 [ 93.055301][ C2] RBP: 0000000000000000 R08: 0000000000000001 R09: ffffed1006c2709c [ 93.055525][ C2] R10: ffff8880361384e3 R11: ffff88803613dc80 R12: 0000000000000000 [ 93.055837][ C2] R13: ffff888001b7c5c0 R14: dffffc0000000000 R15: 0000000000000000 [ 93.056065][ C2] ? ct_kernel_exit.constprop.0+0xc5/0xf0 [ 93.056218][ C2] ? cpuidle_idle_call+0x1f4/0x280 [ 93.056376][ C2] default_idle_call+0x6d/0xb0 [ 93.056529][ C2] cpuidle_idle_call+0x1f4/0x280 [ 93.056679][ C2] ? __pfx_cpuidle_idle_call+0x10/0x10 [ 93.056828][ C2] ? tsc_verify_tsc_adjust+0x5e/0x2b0 [ 93.056979][ C2] do_idle+0xf9/0x160 [ 93.057092][ C2] cpu_startup_entry+0x54/0x60 [ 93.057323][ C2] start_secondary+0x21c/0x2b0 [ 93.057471][ C2] ? __pfx_start_secondary+0x10/0x10 [ 93.057623][ C2] common_startup_64+0x12c/0x138 [ 93.057781][ C2] [ 102.507485][ T584] br0: entered allmulticast mode [ 108.733996][ T599] br0: left allmulticast mode [ 116.906822][ T669] br0: left promiscuous mode [ 117.230000][ T674] br0: port 1(veth1) entered disabled state [ 117.817701][ T679] veth1: left allmulticast mode [ 117.818531][ T679] veth1: left promiscuous mode [ 117.819058][ T679] br0: port 1(veth1) entered disabled state