[ 151.421972][ T1071] 8021q: 802.1Q VLAN Support v1.8 [ 155.703057][ T1176] br10: port 1(veth1.10) entered blocking state [ 155.703552][ T1176] br10: port 1(veth1.10) entered disabled state [ 155.703868][ T1176] veth1.10: entered allmulticast mode [ 155.704080][ T1176] veth1: entered allmulticast mode [ 155.706184][ T1176] veth1.10: entered promiscuous mode [ 155.706418][ T1176] veth1: entered promiscuous mode [ 156.180849][ T1193] br11: port 1(veth1.11) entered blocking state [ 156.181898][ T1193] br11: port 1(veth1.11) entered disabled state [ 156.182239][ T1193] veth1.11: entered allmulticast mode [ 156.184332][ T1193] veth1.11: entered promiscuous mode [ 157.032405][ T1223] br10: port 2(veth2.10) entered blocking state [ 157.032808][ T1223] br10: port 2(veth2.10) entered disabled state [ 157.033230][ T1223] veth2.10: entered allmulticast mode [ 157.033847][ T1223] veth2: entered allmulticast mode [ 157.037237][ T1223] veth2.10: entered promiscuous mode [ 157.037492][ T1223] veth2: entered promiscuous mode [ 157.548561][ T1240] br11: port 2(veth2.11) entered blocking state [ 157.548867][ T1240] br11: port 2(veth2.11) entered disabled state [ 157.549281][ T1240] veth2.11: entered allmulticast mode [ 157.551614][ T1240] veth2.11: entered promiscuous mode [ 158.017107][ T1252] br10: port 2(veth2.10) entered blocking state [ 158.017758][ T1252] br10: port 2(veth2.10) entered forwarding state [ 158.019015][ T1252] br10: port 1(veth1.10) entered blocking state [ 158.019341][ T1252] br10: port 1(veth1.10) entered forwarding state [ 158.161326][ T1257] br11: port 2(veth2.11) entered blocking state [ 158.161691][ T1257] br11: port 2(veth2.11) entered forwarding state [ 158.162200][ T1257] br11: port 1(veth1.11) entered blocking state [ 158.162517][ T1257] br11: port 1(veth1.11) entered forwarding state [ 239.173931][ T1600] br11: port 2(veth2.11) entered disabled state [ 239.174899][ T1600] br11: port 1(veth1.11) entered disabled state [ 239.275687][ T1602] br10: port 2(veth2.10) entered disabled state [ 239.276761][ T1602] br10: port 1(veth1.10) entered disabled state [ 239.469226][ T1606] veth2.11: left allmulticast mode [ 239.469467][ T1606] veth2.11: left promiscuous mode [ 239.469801][ T1606] br11: port 2(veth2.11) entered disabled state [ 239.720885][ T1610] veth2: left allmulticast mode [ 239.721712][ T1610] veth2: left promiscuous mode [ 239.827090][ T1614] veth2.10: left allmulticast mode [ 239.827382][ T1614] veth2.10: left promiscuous mode [ 239.827732][ T1614] br10: port 2(veth2.10) entered disabled state [ 240.335037][ T1624] veth1.11: left allmulticast mode [ 240.335268][ T1624] veth1.11: left promiscuous mode [ 240.335583][ T1624] br11: port 1(veth1.11) entered disabled state [ 240.564649][ T1628] veth1: left allmulticast mode [ 240.565102][ T1628] veth1: left promiscuous mode [ 240.677111][ T1630] veth1.10: left allmulticast mode [ 240.677380][ T1630] veth1.10: left promiscuous mode [ 240.677708][ T1630] br10: port 1(veth1.10) entered disabled state [ 241.514747][ T1644] [ 241.514911][ T1644] ============================================ [ 241.515102][ T1644] WARNING: possible recursive locking detected [ 241.515280][ T1644] 6.13.0-rc5-virtme #1 Not tainted [ 241.515426][ T1644] -------------------------------------------- [ 241.515608][ T1644] ip/1644 is trying to acquire lock: [ 241.515757][ T1644] ffffffffa75d6630 ((netdev_chain).rwsem){++++}-{4:4}, at: blocking_notifier_call_chain+0x50/0x90 [ 241.516067][ T1644] [ 241.516067][ T1644] but task is already holding lock: [ 241.516278][ T1644] ffffffffa75d6630 ((netdev_chain).rwsem){++++}-{4:4}, at: blocking_notifier_call_chain+0x50/0x90 [ 241.516571][ T1644] [ 241.516571][ T1644] other info that might help us debug this: [ 241.516784][ T1644] Possible unsafe locking scenario: [ 241.516784][ T1644] [ 241.517004][ T1644] CPU0 [ 241.517120][ T1644] ---- [ 241.517230][ T1644] lock((netdev_chain).rwsem); [ 241.517380][ T1644] lock((netdev_chain).rwsem); [ 241.517529][ T1644] [ 241.517529][ T1644] *** DEADLOCK *** [ 241.517529][ T1644] [ 241.517747][ T1644] May be due to missing lock nesting notation [ 241.517747][ T1644] [ 241.517959][ T1644] 2 locks held by ip/1644: [ 241.518122][ T1644] #0: ffffffffa75e7a48 (rtnl_mutex){+.+.}-{4:4}, at: rtnl_dellink+0x228/0xa40 [ 241.518387][ T1644] #1: ffffffffa75d6630 ((netdev_chain).rwsem){++++}-{4:4}, at: blocking_notifier_call_chain+0x50/0x90 [ 241.518681][ T1644] [ 241.518681][ T1644] stack backtrace: [ 241.518868][ T1644] CPU: 3 UID: 0 PID: 1644 Comm: ip Not tainted 6.13.0-rc5-virtme #1 [ 241.519105][ T1644] Hardware name: Bochs Bochs, BIOS Bochs 01/01/2011 [ 241.519298][ T1644] Call Trace: [ 241.519410][ T1644] [ 241.519485][ T1644] dump_stack_lvl+0x82/0xd0 [ 241.519636][ T1644] print_deadlock_bug+0x40a/0x650 [ 241.519783][ T1644] validate_chain+0x5bf/0xae0 [ 241.519933][ T1644] ? __pfx_validate_chain+0x10/0x10 [ 241.520097][ T1644] ? hlock_class+0x4e/0x130 [ 241.520248][ T1644] ? mark_lock+0x38/0x3e0 [ 241.520371][ T1644] __lock_acquire+0xb9a/0x1680 [ 241.520524][ T1644] lock_acquire.part.0+0xeb/0x330 [ 241.520679][ T1644] ? blocking_notifier_call_chain+0x50/0x90 [ 241.520859][ T1644] ? netconsole_process_cleanups_core+0x2e/0x3d0 [ 241.521043][ T1644] ? __pfx_lock_acquire.part.0+0x10/0x10 [ 241.521188][ T1644] ? trace_lock_acquire+0x14c/0x1f0 [ 241.521348][ T1644] ? lock_acquire+0x32/0xc0 [ 241.521496][ T1644] ? blocking_notifier_call_chain+0x50/0x90 [ 241.521679][ T1644] down_read+0x9f/0x340 [ 241.521790][ T1644] ? blocking_notifier_call_chain+0x50/0x90 [ 241.521989][ T1644] ? __pfx_down_read+0x10/0x10 [ 241.522137][ T1644] ? __lock_release+0x103/0x460 [ 241.522297][ T1644] ? netconsole_netdev_event+0x2c1/0x620 [ 241.522492][ T1644] ? __pfx___lock_release+0x10/0x10 [ 241.522651][ T1644] blocking_notifier_call_chain+0x50/0x90 [ 241.522797][ T1644] __netdev_upper_dev_unlink+0xd9/0x220 [ 241.522945][ T1644] ? __pfx___netdev_upper_dev_unlink+0x10/0x10 [ 241.523146][ T1644] ? __mutex_unlock_slowpath+0x14a/0x3d0 [ 241.523294][ T1644] ? __pfx___mutex_unlock_slowpath+0x10/0x10 [ 241.523481][ T1644] ? rtnl_is_locked+0x15/0x20 [ 241.523638][ T1644] netdev_upper_dev_unlink+0x71/0xa0 [ 241.523795][ T1644] ? __pfx_netdev_upper_dev_unlink+0x10/0x10 [ 241.524019][ T1644] ? netconsole_netdev_event+0x2c1/0x620 [ 241.524185][ T1644] ? mutex_is_locked+0x1c/0x60 [ 241.524346][ T1644] ? rtnl_is_locked+0x15/0x20 [ 241.524505][ T1644] ? netdev_master_upper_dev_get+0x13/0x120 [ 241.524687][ T1644] vrf_device_event+0x78/0xe0 [vrf] [ 241.524842][ T1644] notifier_call_chain+0xcd/0x150 [ 241.524991][ T1644] blocking_notifier_call_chain+0x66/0x90 [ 241.525150][ T1644] unregister_netdevice_many_notify+0xa7b/0x1580 [ 241.525351][ T1644] ? hlock_class+0x4e/0x130 [ 241.525515][ T1644] ? mark_lock+0x38/0x3e0 [ 241.525639][ T1644] ? __pfx_unregister_netdevice_many_notify+0x10/0x10 [ 241.525839][ T1644] ? lockdep_hardirqs_on_prepare+0x275/0x410 [ 241.526051][ T1644] ? kfree+0xf3/0x330 [ 241.526185][ T1644] ? vlan_vid_del+0x310/0x5e0 [ 241.526354][ T1644] ? vlan_vid_del+0x310/0x5e0 [ 241.526540][ T1644] rtnl_dellink+0x348/0xa40 [ 241.526717][ T1644] ? __pfx_rtnl_dellink+0x10/0x10 [ 241.526909][ T1644] ? find_held_lock+0x2c/0x110 [ 241.527090][ T1644] ? __lock_release+0x103/0x460 [ 241.527263][ T1644] ? rtnetlink_rcv_msg+0x6ef/0xc10 [ 241.527446][ T1644] ? __pfx___lock_release+0x10/0x10 [ 241.527618][ T1644] ? __pfx_lock_acquire.part.0+0x10/0x10 [ 241.527786][ T1644] ? rtnetlink_rcv_msg+0x6ef/0xc10 [ 241.527950][ T1644] ? __pfx_rtnl_dellink+0x10/0x10 [ 241.528116][ T1644] rtnetlink_rcv_msg+0x712/0xc10 [ 241.528280][ T1644] ? __pfx_rtnetlink_rcv_msg+0x10/0x10 [ 241.528448][ T1644] ? hlock_class+0x4e/0x130 [ 241.528623][ T1644] ? mark_lock+0x38/0x3e0 [ 241.528748][ T1644] ? __lock_acquire+0xb9a/0x1680 [ 241.528928][ T1644] netlink_rcv_skb+0x130/0x360 [ 241.529115][ T1644] ? __pfx_rtnetlink_rcv_msg+0x10/0x10 [ 241.529288][ T1644] ? __pfx_netlink_rcv_skb+0x10/0x10 [ 241.529456][ T1644] ? netlink_deliver_tap+0x13e/0x340 [ 241.529634][ T1644] ? netlink_deliver_tap+0xc3/0x340 [ 241.529812][ T1644] netlink_unicast+0x44b/0x710 [ 241.529989][ T1644] ? __pfx_netlink_unicast+0x10/0x10 [ 241.530161][ T1644] ? find_held_lock+0x2c/0x110 [ 241.530341][ T1644] netlink_sendmsg+0x723/0xbe0 [ 241.530516][ T1644] ? __pfx_netlink_sendmsg+0x10/0x10 [ 241.530694][ T1644] ____sys_sendmsg+0x7ac/0xa10 [ 241.530865][ T1644] ? __pfx_____sys_sendmsg+0x10/0x10 [ 241.531019][ T1644] ? __pfx_copy_msghdr_from_user+0x10/0x10 [ 241.531226][ T1644] ___sys_sendmsg+0xee/0x170 [ 241.531382][ T1644] ? __pfx____sys_sendmsg+0x10/0x10 [ 241.531547][ T1644] ? kasan_save_stack+0x34/0x50 [ 241.531706][ T1644] ? kasan_save_stack+0x24/0x50 [ 241.531864][ T1644] ? __kasan_record_aux_stack+0x8e/0xa0 [ 241.532012][ T1644] ? __call_rcu_common.constprop.0+0xa1/0x4b0 [ 241.532218][ T1644] ? __x64_sys_close+0x7c/0xd0 [ 241.532388][ T1644] ? do_syscall_64+0xc1/0x1d0 [ 241.532562][ T1644] ? entry_SYSCALL_64_after_hwframe+0x77/0x7f [ 241.532785][ T1644] ? __lock_acquire+0xb9a/0x1680 [ 241.532939][ T1644] ? find_held_lock+0x2c/0x110 [ 241.533103][ T1644] ? __lock_release+0x103/0x460 [ 241.533258][ T1644] ? __virt_addr_valid+0x22b/0x430 [ 241.533434][ T1644] ? __pfx___lock_release+0x10/0x10 [ 241.533606][ T1644] ? __pfx_lock_acquire.part.0+0x10/0x10 [ 241.533776][ T1644] __sys_sendmsg+0x109/0x1a0 [ 241.533937][ T1644] ? __pfx___sys_sendmsg+0x10/0x10 [ 241.534110][ T1644] ? __pfx_slab_free_after_rcu_debug+0x10/0x10 [ 241.534324][ T1644] do_syscall_64+0xc1/0x1d0 [ 241.534502][ T1644] entry_SYSCALL_64_after_hwframe+0x77/0x7f [ 241.534708][ T1644] RIP: 0033:0x7f6b4fe809a7 [ 241.534888][ T1644] Code: 0a 00 f7 d8 64 89 02 48 c7 c0 ff ff ff ff eb b9 0f 1f 00 f3 0f 1e fa 64 8b 04 25 18 00 00 00 85 c0 75 10 b8 2e 00 00 00 0f 05 <48> 3d 00 f0 ff ff 77 51 c3 48 83 ec 28 89 54 24 1c 48 89 74 24 10 [ 241.535475][ T1644] RSP: 002b:00007ffcb0bfff28 EFLAGS: 00000246 ORIG_RAX: 000000000000002e [ 241.535725][ T1644] RAX: ffffffffffffffda RBX: 00007ffcb0c00650 RCX: 00007f6b4fe809a7 [ 241.535974][ T1644] RDX: 0000000000000000 RSI: 00007ffcb0bfff90 RDI: 0000000000000005 [ 241.536229][ T1644] RBP: 0000000000000002 R08: 0000000000000003 R09: 0000000000000078 [ 241.536482][ T1644] R10: 00007f6b4fd3eef8 R11: 0000000000000246 R12: 0000000000000002 [ 241.536724][ T1644] R13: 00000000677952ed R14: 0000000000498600 R15: 0000000000000000 [ 241.536993][ T1644]