[ 33.082734][ T651] 8021q: 802.1Q VLAN Support v1.8 [ 35.799175][ T686] br0: port 1(veth1) entered blocking state [ 35.799879][ T686] br0: port 1(veth1) entered disabled state [ 35.800371][ T686] veth1: entered allmulticast mode [ 35.804222][ T686] veth1: entered promiscuous mode [ 35.915591][ T39] br0: port 1(veth1) entered blocking state [ 35.916025][ T39] br0: port 1(veth1) entered forwarding state [ 36.130526][ T689] br0: port 2(veth2) entered blocking state [ 36.130897][ T689] br0: port 2(veth2) entered disabled state [ 36.131164][ T689] veth2: entered allmulticast mode [ 36.134050][ T689] veth2: entered promiscuous mode [ 36.233739][ T37] br0: port 2(veth2) entered blocking state [ 36.234050][ T37] br0: port 2(veth2) entered forwarding state [ 107.203313][ T2472] br0: port 2(veth2) entered disabled state [ 107.278356][ T2473] veth2: left allmulticast mode [ 107.278650][ T2473] veth2: left promiscuous mode [ 107.279060][ T2473] br0: port 2(veth2) entered disabled state [ 107.288772][ T2473] ip (2473) used greatest stack depth: 23632 bytes left [ 107.359870][ T2474] br0: port 1(veth1) entered disabled state [ 107.441587][ T2475] veth1: left allmulticast mode [ 107.441874][ T2475] veth1: left promiscuous mode [ 107.442216][ T2475] br0: port 1(veth1) entered disabled state [ 108.235496][ T2493] br0: port 1(veth1) entered blocking state [ 108.235802][ T2493] br0: port 1(veth1) entered disabled state [ 108.236100][ T2493] veth1: entered allmulticast mode [ 108.239023][ T2493] veth1: entered promiscuous mode [ 108.321260][ T37] br0: port 1(veth1) entered blocking state [ 108.321577][ T37] br0: port 1(veth1) entered forwarding state [ 108.647011][ T2498] br0: port 2(veth2) entered blocking state [ 108.647809][ T2498] br0: port 2(veth2) entered disabled state [ 108.648105][ T2498] veth2: entered allmulticast mode [ 108.650124][ T2498] veth2: entered promiscuous mode [ 108.732787][ T37] br0: port 2(veth2) entered blocking state [ 108.733103][ T37] br0: port 2(veth2) entered forwarding state [ 179.900941][ T4290] br0: port 2(veth2) entered disabled state [ 179.975749][ T4291] veth2: left allmulticast mode [ 179.976034][ T4291] veth2: left promiscuous mode [ 179.976393][ T4291] br0: port 2(veth2) entered disabled state [ 180.063641][ T4292] br0: port 1(veth1) entered disabled state [ 180.139712][ T4293] veth1: left allmulticast mode [ 180.140431][ T4293] veth1: left promiscuous mode [ 180.140840][ T4293] br0: port 1(veth1) entered disabled state [ 180.911958][ T4311] br0: port 1(veth1) entered blocking state [ 180.912259][ T4311] br0: port 1(veth1) entered disabled state [ 180.912813][ T4311] veth1: entered allmulticast mode [ 180.914797][ T4311] veth1: entered promiscuous mode [ 180.989577][ T39] br0: port 1(veth1) entered blocking state [ 180.989876][ T39] br0: port 1(veth1) entered forwarding state [ 181.314910][ T4316] br0: port 2(veth2) entered blocking state [ 181.315304][ T4316] br0: port 2(veth2) entered disabled state [ 181.315618][ T4316] veth2: entered allmulticast mode [ 181.317614][ T4316] veth2: entered promiscuous mode [ 181.393098][ T299] br0: port 2(veth2) entered blocking state [ 181.393525][ T299] br0: port 2(veth2) entered forwarding state [ 329.937720][ T8166] br0: port 2(veth2) entered disabled state [ 330.018436][ T8167] veth2: left allmulticast mode [ 330.018714][ T8167] veth2: left promiscuous mode [ 330.019063][ T8167] br0: port 2(veth2) entered disabled state [ 330.102532][ T8168] br0: port 1(veth1) entered disabled state [ 330.185192][ T8169] veth1: left allmulticast mode [ 330.185497][ T8169] veth1: left promiscuous mode [ 330.185848][ T8169] br0: port 1(veth1) entered disabled state [ 331.350154][ T8190] [ 331.350356][ T8190] ============================================ [ 331.350674][ T8190] WARNING: possible recursive locking detected [ 331.350983][ T8190] 6.13.0-rc5-virtme #1 Not tainted [ 331.351230][ T8190] -------------------------------------------- [ 331.351617][ T8190] ip/8190 is trying to acquire lock: [ 331.351783][ T8190] ffffffff9c3d6630 ((netdev_chain).rwsem){++++}-{4:4}, at: blocking_notifier_call_chain+0x50/0x90 [ 331.352118][ T8190] [ 331.352118][ T8190] but task is already holding lock: [ 331.352361][ T8190] ffffffff9c3d6630 ((netdev_chain).rwsem){++++}-{4:4}, at: blocking_notifier_call_chain+0x50/0x90 [ 331.352689][ T8190] [ 331.352689][ T8190] other info that might help us debug this: [ 331.352926][ T8190] Possible unsafe locking scenario: [ 331.352926][ T8190] [ 331.353162][ T8190] CPU0 [ 331.353283][ T8190] ---- [ 331.353414][ T8190] lock((netdev_chain).rwsem); [ 331.353584][ T8190] lock((netdev_chain).rwsem); [ 331.353757][ T8190] [ 331.353757][ T8190] *** DEADLOCK *** [ 331.353757][ T8190] [ 331.354012][ T8190] May be due to missing lock nesting notation [ 331.354012][ T8190] [ 331.354257][ T8190] 2 locks held by ip/8190: [ 331.354427][ T8190] #0: ffffffff9c3e7a48 (rtnl_mutex){+.+.}-{4:4}, at: rtnl_dellink+0x228/0xa40 [ 331.354735][ T8190] #1: ffffffff9c3d6630 ((netdev_chain).rwsem){++++}-{4:4}, at: blocking_notifier_call_chain+0x50/0x90 [ 331.355070][ T8190] [ 331.355070][ T8190] stack backtrace: [ 331.355366][ T8190] CPU: 3 UID: 0 PID: 8190 Comm: ip Not tainted 6.13.0-rc5-virtme #1 [ 331.355611][ T8190] Hardware name: Bochs Bochs, BIOS Bochs 01/01/2011 [ 331.355814][ T8190] Call Trace: [ 331.355942][ T8190] [ 331.356036][ T8190] dump_stack_lvl+0x82/0xd0 [ 331.356213][ T8190] print_deadlock_bug+0x40a/0x650 [ 331.356378][ T8190] validate_chain+0x5bf/0xae0 [ 331.356541][ T8190] ? __pfx_validate_chain+0x10/0x10 [ 331.356704][ T8190] ? hlock_class+0x4e/0x130 [ 331.356867][ T8190] ? mark_lock+0x38/0x3e0 [ 331.356994][ T8190] __lock_acquire+0xb9a/0x1680 [ 331.357157][ T8190] lock_acquire.part.0+0xeb/0x330 [ 331.357324][ T8190] ? blocking_notifier_call_chain+0x50/0x90 [ 331.357528][ T8190] ? netconsole_process_cleanups_core+0x2e/0x3d0 [ 331.357735][ T8190] ? __pfx_lock_acquire.part.0+0x10/0x10 [ 331.357897][ T8190] ? trace_lock_acquire+0x14c/0x1f0 [ 331.358063][ T8190] ? lock_acquire+0x32/0xc0 [ 331.358229][ T8190] ? blocking_notifier_call_chain+0x50/0x90 [ 331.358432][ T8190] down_read+0x9f/0x340 [ 331.358562][ T8190] ? blocking_notifier_call_chain+0x50/0x90 [ 331.358764][ T8190] ? __pfx_down_read+0x10/0x10 [ 331.358926][ T8190] ? __lock_release+0x103/0x460 [ 331.359091][ T8190] ? netconsole_netdev_event+0x2c1/0x620 [ 331.359255][ T8190] ? __pfx___lock_release+0x10/0x10 [ 331.359423][ T8190] blocking_notifier_call_chain+0x50/0x90 [ 331.359585][ T8190] __netdev_upper_dev_unlink+0xd9/0x220 [ 331.359758][ T8190] ? __pfx___netdev_upper_dev_unlink+0x10/0x10 [ 331.359958][ T8190] ? __mutex_unlock_slowpath+0x14a/0x3d0 [ 331.360122][ T8190] ? __pfx___mutex_unlock_slowpath+0x10/0x10 [ 331.360325][ T8190] ? rtnl_is_locked+0x15/0x20 [ 331.360490][ T8190] netdev_upper_dev_unlink+0x71/0xa0 [ 331.360651][ T8190] ? __pfx_netdev_upper_dev_unlink+0x10/0x10 [ 331.360857][ T8190] ? netconsole_netdev_event+0x2c1/0x620 [ 331.361022][ T8190] ? mutex_is_locked+0x1c/0x60 [ 331.361192][ T8190] ? rtnl_is_locked+0x15/0x20 [ 331.361368][ T8190] ? netdev_master_upper_dev_get+0x13/0x120 [ 331.361574][ T8190] vrf_device_event+0x78/0xe0 [vrf] [ 331.361740][ T8190] notifier_call_chain+0xcd/0x150 [ 331.361901][ T8190] blocking_notifier_call_chain+0x66/0x90 [ 331.362070][ T8190] unregister_netdevice_many_notify+0xa7b/0x1580 [ 331.362276][ T8190] ? hlock_class+0x4e/0x130 [ 331.362439][ T8190] ? mark_lock+0x38/0x3e0 [ 331.362563][ T8190] ? __pfx_unregister_netdevice_many_notify+0x10/0x10 [ 331.362765][ T8190] ? lockdep_hardirqs_on_prepare+0x275/0x410 [ 331.362969][ T8190] ? kfree+0xf3/0x330 [ 331.363093][ T8190] ? vlan_vid_del+0x310/0x5e0 [ 331.363265][ T8190] ? vlan_vid_del+0x310/0x5e0 [ 331.363433][ T8190] rtnl_dellink+0x348/0xa40 [ 331.363597][ T8190] ? __pfx_rtnl_dellink+0x10/0x10 [ 331.363776][ T8190] ? find_held_lock+0x2c/0x110 [ 331.363938][ T8190] ? __lock_release+0x103/0x460 [ 331.364101][ T8190] ? rtnetlink_rcv_msg+0x6ef/0xc10 [ 331.364262][ T8190] ? __pfx___lock_release+0x10/0x10 [ 331.364424][ T8190] ? __pfx_lock_acquire.part.0+0x10/0x10 [ 331.364592][ T8190] ? rtnetlink_rcv_msg+0x6ef/0xc10 [ 331.364754][ T8190] ? __pfx_rtnl_dellink+0x10/0x10 [ 331.364913][ T8190] rtnetlink_rcv_msg+0x712/0xc10 [ 331.365073][ T8190] ? __pfx_rtnetlink_rcv_msg+0x10/0x10 [ 331.365232][ T8190] ? hlock_class+0x4e/0x130 [ 331.365391][ T8190] ? mark_lock+0x38/0x3e0 [ 331.365516][ T8190] ? __lock_acquire+0xb9a/0x1680 [ 331.365686][ T8190] netlink_rcv_skb+0x130/0x360 [ 331.365847][ T8190] ? __pfx_rtnetlink_rcv_msg+0x10/0x10 [ 331.366012][ T8190] ? __pfx_netlink_rcv_skb+0x10/0x10 [ 331.366176][ T8190] ? netlink_deliver_tap+0x13e/0x340 [ 331.366343][ T8190] ? netlink_deliver_tap+0xc3/0x340 [ 331.366507][ T8190] netlink_unicast+0x44b/0x710 [ 331.366674][ T8190] ? __pfx_netlink_unicast+0x10/0x10 [ 331.366842][ T8190] ? find_held_lock+0x2c/0x110 [ 331.367005][ T8190] netlink_sendmsg+0x723/0xbe0 [ 331.367171][ T8190] ? __pfx_netlink_sendmsg+0x10/0x10 [ 331.367337][ T8190] ____sys_sendmsg+0x7ac/0xa10 [ 331.367503][ T8190] ? __pfx_____sys_sendmsg+0x10/0x10 [ 331.367664][ T8190] ? __pfx_copy_msghdr_from_user+0x10/0x10 [ 331.367869][ T8190] ___sys_sendmsg+0xee/0x170 [ 331.368038][ T8190] ? __pfx____sys_sendmsg+0x10/0x10 [ 331.368200][ T8190] ? kasan_save_stack+0x34/0x50 [ 331.368364][ T8190] ? kasan_save_stack+0x24/0x50 [ 331.368530][ T8190] ? __kasan_record_aux_stack+0x8e/0xa0 [ 331.368696][ T8190] ? __call_rcu_common.constprop.0+0xa1/0x4b0 [ 331.368901][ T8190] ? __x64_sys_close+0x7c/0xd0 [ 331.369061][ T8190] ? do_syscall_64+0xc1/0x1d0 [ 331.369230][ T8190] ? entry_SYSCALL_64_after_hwframe+0x77/0x7f [ 331.369433][ T8190] ? __lock_acquire+0xb9a/0x1680 [ 331.369596][ T8190] ? find_held_lock+0x2c/0x110 [ 331.369758][ T8190] ? __lock_release+0x103/0x460 [ 331.369920][ T8190] ? __virt_addr_valid+0x22b/0x430 [ 331.370083][ T8190] ? __pfx___lock_release+0x10/0x10 [ 331.370244][ T8190] ? __pfx_lock_acquire.part.0+0x10/0x10 [ 331.370414][ T8190] __sys_sendmsg+0x109/0x1a0 [ 331.370581][ T8190] ? __pfx___sys_sendmsg+0x10/0x10 [ 331.370745][ T8190] ? __pfx_slab_free_after_rcu_debug+0x10/0x10 [ 331.370952][ T8190] do_syscall_64+0xc1/0x1d0 [ 331.371117][ T8190] entry_SYSCALL_64_after_hwframe+0x77/0x7f [ 331.371371][ T8190] RIP: 0033:0x7f3f9468a9a7 [ 331.371586][ T8190] Code: 0a 00 f7 d8 64 89 02 48 c7 c0 ff ff ff ff eb b9 0f 1f 00 f3 0f 1e fa 64 8b 04 25 18 00 00 00 85 c0 75 10 b8 2e 00 00 00 0f 05 <48> 3d 00 f0 ff ff 77 51 c3 48 83 ec 28 89 54 24 1c 48 89 74 24 10 [ 331.372222][ T8190] RSP: 002b:00007ffc90bf1848 EFLAGS: 00000246 ORIG_RAX: 000000000000002e [ 331.372469][ T8190] RAX: ffffffffffffffda RBX: 00007ffc90bf1f70 RCX: 00007f3f9468a9a7 [ 331.372714][ T8190] RDX: 0000000000000000 RSI: 00007ffc90bf18b0 RDI: 0000000000000005 [ 331.372963][ T8190] RBP: 0000000000000002 R08: 0000000000000003 R09: 0000000000000078 [ 331.373209][ T8190] R10: 00007f3f94548ef8 R11: 0000000000000246 R12: 0000000000000002 [ 331.373465][ T8190] R13: 00000000677955d9 R14: 0000000000498600 R15: 0000000000000000 [ 331.373709][ T8190]