[ 14.280047][ T305] 8021q: 802.1Q VLAN Support v1.8 [ 19.841427][ T435] br10: port 1(veth1.10) entered blocking state [ 19.842088][ T435] br10: port 1(veth1.10) entered disabled state [ 19.843230][ T435] veth1.10: entered allmulticast mode [ 19.843545][ T435] veth1: entered allmulticast mode [ 19.846800][ T435] veth1.10: entered promiscuous mode [ 19.847051][ T435] veth1: entered promiscuous mode [ 19.857275][ T435] ip (435) used greatest stack depth: 24288 bytes left [ 19.994602][ T440] br10: port 2(veth2.10) entered blocking state [ 19.994899][ T440] br10: port 2(veth2.10) entered disabled state [ 19.995569][ T440] veth2.10: entered allmulticast mode [ 19.995773][ T440] veth2: entered allmulticast mode [ 19.997834][ T440] veth2.10: entered promiscuous mode [ 19.998041][ T440] veth2: entered promiscuous mode [ 20.143750][ T445] br10: port 2(veth2.10) entered blocking state [ 20.144234][ T445] br10: port 2(veth2.10) entered forwarding state [ 20.145029][ T445] br10: port 1(veth1.10) entered blocking state [ 20.145367][ T445] br10: port 1(veth1.10) entered forwarding state [ 21.339005][ T481] br11: port 1(veth1.11) entered blocking state [ 21.339344][ T481] br11: port 1(veth1.11) entered disabled state [ 21.339653][ T481] veth1.11: entered allmulticast mode [ 21.341759][ T481] veth1.11: entered promiscuous mode [ 21.487604][ T486] br11: port 2(veth2.11) entered blocking state [ 21.487905][ T486] br11: port 2(veth2.11) entered disabled state [ 21.488683][ T486] veth2.11: entered allmulticast mode [ 21.490622][ T486] veth2.11: entered promiscuous mode [ 21.626539][ T491] br11: port 2(veth2.11) entered blocking state [ 21.627038][ T491] br11: port 2(veth2.11) entered forwarding state [ 21.627849][ T491] br11: port 1(veth1.11) entered blocking state [ 21.628288][ T491] br11: port 1(veth1.11) entered forwarding state [ 22.724478][ T527] br12: port 1(veth1.12) entered blocking state [ 22.724773][ T527] br12: port 1(veth1.12) entered disabled state [ 22.725107][ T527] veth1.12: entered allmulticast mode [ 22.727410][ T527] veth1.12: entered promiscuous mode [ 22.885163][ T532] br12: port 2(veth2.12) entered blocking state [ 22.885486][ T532] br12: port 2(veth2.12) entered disabled state [ 22.885789][ T532] veth2.12: entered allmulticast mode [ 22.888427][ T532] veth2.12: entered promiscuous mode [ 23.010512][ T537] br12: port 2(veth2.12) entered blocking state [ 23.010823][ T537] br12: port 2(veth2.12) entered forwarding state [ 23.011639][ T537] br12: port 1(veth1.12) entered blocking state [ 23.011931][ T537] br12: port 1(veth1.12) entered forwarding state [ 441.598088][ T1968] br12: port 2(veth2.12) entered disabled state [ 441.730241][ T1970] br12: port 1(veth1.12) entered disabled state [ 441.936099][ T1974] veth2.12: left allmulticast mode [ 441.937318][ T1974] veth2.12: left promiscuous mode [ 441.937650][ T1974] br12: port 2(veth2.12) entered disabled state [ 442.073163][ T1976] veth1.12: left allmulticast mode [ 442.073419][ T1976] veth1.12: left promiscuous mode [ 442.073766][ T1976] br12: port 1(veth1.12) entered disabled state [ 442.691807][ T1986] br11: port 2(veth2.11) entered disabled state [ 442.819072][ T1988] br11: port 1(veth1.11) entered disabled state [ 443.029224][ T1992] veth2.11: left allmulticast mode [ 443.029500][ T1992] veth2.11: left promiscuous mode [ 443.029847][ T1992] br11: port 2(veth2.11) entered disabled state [ 443.123440][ T1994] veth1.11: left allmulticast mode [ 443.123752][ T1994] veth1.11: left promiscuous mode [ 443.124264][ T1994] br11: port 1(veth1.11) entered disabled state [ 443.705797][ T2002] veth2: left allmulticast mode [ 443.706092][ T2002] veth2: left promiscuous mode [ 443.707831][ T2002] br10: port 2(veth2.10) entered disabled state [ 443.804822][ T2004] veth1: left allmulticast mode [ 443.805070][ T2004] veth1: left promiscuous mode [ 443.806816][ T2004] br10: port 1(veth1.10) entered disabled state [ 444.051868][ T2008] veth2.10: left allmulticast mode [ 444.052224][ T2008] veth2.10: left promiscuous mode [ 444.052617][ T2008] br10: port 2(veth2.10) entered disabled state [ 444.180996][ T2010] veth1.10: left allmulticast mode [ 444.181423][ T2010] veth1.10: left promiscuous mode [ 444.181900][ T2010] br10: port 1(veth1.10) entered disabled state [ 445.185452][ T2027] [ 445.185644][ T2027] ============================================ [ 445.185852][ T2027] WARNING: possible recursive locking detected [ 445.186067][ T2027] 6.13.0-rc5-virtme #1 Not tainted [ 445.186257][ T2027] -------------------------------------------- [ 445.186461][ T2027] ip/2027 is trying to acquire lock: [ 445.186625][ T2027] ffffffffa55d6630 ((netdev_chain).rwsem){++++}-{4:4}, at: blocking_notifier_call_chain+0x50/0x90 [ 445.186954][ T2027] [ 445.186954][ T2027] but task is already holding lock: [ 445.187189][ T2027] ffffffffa55d6630 ((netdev_chain).rwsem){++++}-{4:4}, at: blocking_notifier_call_chain+0x50/0x90 [ 445.187612][ T2027] [ 445.187612][ T2027] other info that might help us debug this: [ 445.187910][ T2027] Possible unsafe locking scenario: [ 445.187910][ T2027] [ 445.188147][ T2027] CPU0 [ 445.188275][ T2027] ---- [ 445.188384][ T2027] lock((netdev_chain).rwsem); [ 445.188574][ T2027] lock((netdev_chain).rwsem); [ 445.188732][ T2027] [ 445.188732][ T2027] *** DEADLOCK *** [ 445.188732][ T2027] [ 445.189003][ T2027] May be due to missing lock nesting notation [ 445.189003][ T2027] [ 445.189261][ T2027] 2 locks held by ip/2027: [ 445.189408][ T2027] #0: ffffffffa55e7a48 (rtnl_mutex){+.+.}-{4:4}, at: rtnl_dellink+0x228/0xa40 [ 445.189696][ T2027] #1: ffffffffa55d6630 ((netdev_chain).rwsem){++++}-{4:4}, at: blocking_notifier_call_chain+0x50/0x90 [ 445.190043][ T2027] [ 445.190043][ T2027] stack backtrace: [ 445.190264][ T2027] CPU: 0 UID: 0 PID: 2027 Comm: ip Not tainted 6.13.0-rc5-virtme #1 [ 445.190520][ T2027] Hardware name: Bochs Bochs, BIOS Bochs 01/01/2011 [ 445.190733][ T2027] Call Trace: [ 445.190861][ T2027] [ 445.190966][ T2027] dump_stack_lvl+0x82/0xd0 [ 445.191165][ T2027] print_deadlock_bug+0x40a/0x650 [ 445.191334][ T2027] validate_chain+0x5bf/0xae0 [ 445.191544][ T2027] ? __pfx_validate_chain+0x10/0x10 [ 445.191707][ T2027] ? hlock_class+0x4e/0x130 [ 445.191894][ T2027] ? mark_lock+0x38/0x3e0 [ 445.192025][ T2027] __lock_acquire+0xb9a/0x1680 [ 445.192217][ T2027] lock_acquire.part.0+0xeb/0x330 [ 445.192385][ T2027] ? blocking_notifier_call_chain+0x50/0x90 [ 445.192616][ T2027] ? netconsole_process_cleanups_core+0x2e/0x3d0 [ 445.192854][ T2027] ? __pfx_lock_acquire.part.0+0x10/0x10 [ 445.193019][ T2027] ? trace_lock_acquire+0x14c/0x1f0 [ 445.193223][ T2027] ? lock_acquire+0x32/0xc0 [ 445.193394][ T2027] ? blocking_notifier_call_chain+0x50/0x90 [ 445.193623][ T2027] down_read+0x9f/0x340 [ 445.193737][ T2027] ? blocking_notifier_call_chain+0x50/0x90 [ 445.193933][ T2027] ? __pfx_down_read+0x10/0x10 [ 445.194162][ T2027] ? __lock_release+0x103/0x460 [ 445.194407][ T2027] ? netconsole_netdev_event+0x2c1/0x620 [ 445.194628][ T2027] ? __pfx___lock_release+0x10/0x10 [ 445.194823][ T2027] blocking_notifier_call_chain+0x50/0x90 [ 445.195026][ T2027] __netdev_upper_dev_unlink+0xd9/0x220 [ 445.195220][ T2027] ? __pfx___netdev_upper_dev_unlink+0x10/0x10 [ 445.195429][ T2027] ? __mutex_unlock_slowpath+0x14a/0x3d0 [ 445.195580][ T2027] ? __pfx___mutex_unlock_slowpath+0x10/0x10 [ 445.195773][ T2027] ? rtnl_is_locked+0x15/0x20 [ 445.195929][ T2027] netdev_upper_dev_unlink+0x71/0xa0 [ 445.196095][ T2027] ? __pfx_netdev_upper_dev_unlink+0x10/0x10 [ 445.196305][ T2027] ? netconsole_netdev_event+0x2c1/0x620 [ 445.196468][ T2027] ? mutex_is_locked+0x1c/0x60 [ 445.196633][ T2027] ? rtnl_is_locked+0x15/0x20 [ 445.196789][ T2027] ? netdev_master_upper_dev_get+0x13/0x120 [ 445.197018][ T2027] vrf_device_event+0x78/0xe0 [vrf] [ 445.197186][ T2027] notifier_call_chain+0xcd/0x150 [ 445.197340][ T2027] blocking_notifier_call_chain+0x66/0x90 [ 445.197510][ T2027] unregister_netdevice_many_notify+0xa7b/0x1580 [ 445.197698][ T2027] ? hlock_class+0x4e/0x130 [ 445.197873][ T2027] ? mark_lock+0x38/0x3e0 [ 445.198017][ T2027] ? __pfx_unregister_netdevice_many_notify+0x10/0x10 [ 445.198220][ T2027] ? lockdep_hardirqs_on_prepare+0x275/0x410 [ 445.198418][ T2027] ? kfree+0xf3/0x330 [ 445.198539][ T2027] ? vlan_vid_del+0x310/0x5e0 [ 445.198690][ T2027] ? vlan_vid_del+0x310/0x5e0 [ 445.198838][ T2027] rtnl_dellink+0x348/0xa40 [ 445.199010][ T2027] ? __pfx_rtnl_dellink+0x10/0x10 [ 445.199172][ T2027] ? find_held_lock+0x2c/0x110 [ 445.199350][ T2027] ? __lock_release+0x103/0x460 [ 445.199547][ T2027] ? rtnetlink_rcv_msg+0x6ef/0xc10 [ 445.199706][ T2027] ? __pfx___lock_release+0x10/0x10 [ 445.199902][ T2027] ? __pfx_lock_acquire.part.0+0x10/0x10 [ 445.200068][ T2027] ? rtnetlink_rcv_msg+0x6ef/0xc10 [ 445.200223][ T2027] ? __pfx_rtnl_dellink+0x10/0x10 [ 445.200393][ T2027] rtnetlink_rcv_msg+0x712/0xc10 [ 445.200568][ T2027] ? __pfx_rtnetlink_rcv_msg+0x10/0x10 [ 445.200740][ T2027] ? hlock_class+0x4e/0x130 [ 445.200894][ T2027] ? mark_lock+0x38/0x3e0 [ 445.201039][ T2027] ? __lock_acquire+0xb9a/0x1680 [ 445.201277][ T2027] netlink_rcv_skb+0x130/0x360 [ 445.201507][ T2027] ? __pfx_rtnetlink_rcv_msg+0x10/0x10 [ 445.201666][ T2027] ? __pfx_netlink_rcv_skb+0x10/0x10 [ 445.201836][ T2027] ? netlink_deliver_tap+0x13e/0x340 [ 445.202006][ T2027] ? netlink_deliver_tap+0xc3/0x340 [ 445.202225][ T2027] netlink_unicast+0x44b/0x710 [ 445.202380][ T2027] ? __pfx_netlink_unicast+0x10/0x10 [ 445.202545][ T2027] ? find_held_lock+0x2c/0x110 [ 445.202718][ T2027] netlink_sendmsg+0x723/0xbe0 [ 445.202882][ T2027] ? __pfx_netlink_sendmsg+0x10/0x10 [ 445.203055][ T2027] ____sys_sendmsg+0x7ac/0xa10 [ 445.203225][ T2027] ? __pfx_____sys_sendmsg+0x10/0x10 [ 445.203381][ T2027] ? __pfx_copy_msghdr_from_user+0x10/0x10 [ 445.203591][ T2027] ___sys_sendmsg+0xee/0x170 [ 445.203747][ T2027] ? __pfx____sys_sendmsg+0x10/0x10 [ 445.203895][ T2027] ? kasan_save_stack+0x34/0x50 [ 445.204082][ T2027] ? kasan_save_stack+0x24/0x50 [ 445.204254][ T2027] ? __kasan_record_aux_stack+0x8e/0xa0 [ 445.204412][ T2027] ? __call_rcu_common.constprop.0+0xa1/0x4b0 [ 445.204619][ T2027] ? __x64_sys_close+0x7c/0xd0 [ 445.204809][ T2027] ? do_syscall_64+0xc1/0x1d0 [ 445.204995][ T2027] ? entry_SYSCALL_64_after_hwframe+0x77/0x7f [ 445.205204][ T2027] ? __lock_acquire+0xb9a/0x1680 [ 445.205384][ T2027] ? find_held_lock+0x2c/0x110 [ 445.205566][ T2027] ? __lock_release+0x103/0x460 [ 445.205727][ T2027] ? __virt_addr_valid+0x22b/0x430 [ 445.205941][ T2027] ? __pfx___lock_release+0x10/0x10 [ 445.206173][ T2027] ? __pfx_lock_acquire.part.0+0x10/0x10 [ 445.206349][ T2027] __sys_sendmsg+0x109/0x1a0 [ 445.206509][ T2027] ? __pfx___sys_sendmsg+0x10/0x10 [ 445.206693][ T2027] ? __pfx_slab_free_after_rcu_debug+0x10/0x10 [ 445.206900][ T2027] do_syscall_64+0xc1/0x1d0 [ 445.207117][ T2027] entry_SYSCALL_64_after_hwframe+0x77/0x7f [ 445.207324][ T2027] RIP: 0033:0x7fdc138289a7 [ 445.207489][ T2027] Code: 0a 00 f7 d8 64 89 02 48 c7 c0 ff ff ff ff eb b9 0f 1f 00 f3 0f 1e fa 64 8b 04 25 18 00 00 00 85 c0 75 10 b8 2e 00 00 00 0f 05 <48> 3d 00 f0 ff ff 77 51 c3 48 83 ec 28 89 54 24 1c 48 89 74 24 10 [ 445.208091][ T2027] RSP: 002b:00007ffdcbac20d8 EFLAGS: 00000246 ORIG_RAX: 000000000000002e [ 445.208333][ T2027] RAX: ffffffffffffffda RBX: 00007ffdcbac2800 RCX: 00007fdc138289a7 [ 445.208612][ T2027] RDX: 0000000000000000 RSI: 00007ffdcbac2140 RDI: 0000000000000005 [ 445.208899][ T2027] RBP: 0000000000000002 R08: 0000000000000003 R09: 0000000000000078 [ 445.209211][ T2027] R10: 00007fdc136e6ef8 R11: 0000000000000246 R12: 0000000000000002 [ 445.209479][ T2027] R13: 000000006778fcf5 R14: 0000000000498600 R15: 0000000000000000 [ 445.209757][ T2027] [ 445.233599][ T2027] ip (2027) used greatest stack depth: 23840 bytes left