[ 11.570459][ T250] ip (250) used greatest stack depth: 23720 bytes left [ 14.613410][ T303] 8021q: 802.1Q VLAN Support v1.8 [ 17.918743][ T345] br1: port 1(vx100) entered blocking state [ 17.919176][ T345] br1: port 1(vx100) entered disabled state [ 17.919537][ T345] vx100: entered allmulticast mode [ 17.921520][ T345] vx100: entered promiscuous mode [ 17.922475][ T345] br1: port 1(vx100) entered blocking state [ 17.922833][ T345] br1: port 1(vx100) entered forwarding state [ 18.085341][ T347] br1: port 2(veth1) entered blocking state [ 18.085759][ T347] br1: port 2(veth1) entered disabled state [ 18.086315][ T347] veth1: entered allmulticast mode [ 18.088404][ T347] veth1: entered promiscuous mode [ 18.169616][ T37] br1: port 2(veth1) entered blocking state [ 18.170054][ T37] br1: port 2(veth1) entered forwarding state [ 18.531636][ T353] br2: port 1(vx200) entered blocking state [ 18.531955][ T353] br2: port 1(vx200) entered disabled state [ 18.532247][ T353] vx200: entered allmulticast mode [ 18.534320][ T353] vx200: entered promiscuous mode [ 18.536301][ T353] br2: port 1(vx200) entered blocking state [ 18.536601][ T353] br2: port 1(vx200) entered forwarding state [ 18.788452][ T356] br2: port 2(veth2.20) entered blocking state [ 18.788781][ T356] br2: port 2(veth2.20) entered disabled state [ 18.789079][ T356] veth2.20: entered allmulticast mode [ 18.791046][ T356] veth2.20: entered promiscuous mode [ 18.887771][ T358] veth2: entered allmulticast mode [ 18.888062][ T358] veth2: entered promiscuous mode [ 18.889380][ T358] br2: port 2(veth2.20) entered blocking state [ 18.889845][ T358] br2: port 2(veth2.20) entered forwarding state [ 22.239637][ T402] br3: port 1(w1) entered blocking state [ 22.239954][ T402] br3: port 1(w1) entered disabled state [ 22.240276][ T402] w1: entered allmulticast mode [ 22.243374][ T402] w1: entered promiscuous mode [ 22.733912][ T408] br3: port 2(vx100) entered blocking state [ 22.734204][ T408] br3: port 2(vx100) entered disabled state [ 22.734473][ T408] vx100: entered allmulticast mode [ 22.736400][ T408] vx100: entered promiscuous mode [ 22.737411][ T408] br3: port 2(vx100) entered blocking state [ 22.737695][ T408] br3: port 2(vx100) entered forwarding state [ 23.434175][ T40] br3: port 1(w1) entered blocking state [ 23.434620][ T40] br3: port 1(w1) entered forwarding state [ 26.232493][ T456] br3: port 1(w1) entered blocking state [ 26.233216][ T456] br3: port 1(w1) entered disabled state [ 26.233444][ T456] w1: entered allmulticast mode [ 26.235348][ T456] w1: entered promiscuous mode [ 26.758755][ T462] br3: port 2(vx200) entered blocking state [ 26.759170][ T462] br3: port 2(vx200) entered disabled state [ 26.759617][ T462] vx200: entered allmulticast mode [ 26.762768][ T462] vx200: entered promiscuous mode [ 26.764016][ T462] br3: port 2(vx200) entered blocking state [ 26.764383][ T462] br3: port 2(vx200) entered forwarding state [ 27.447972][ T37] br3: port 1(w1) entered blocking state [ 27.448586][ T37] br3: port 1(w1) entered forwarding state [ 29.348102][ T501] br3: port 3(w1.20) entered blocking state [ 29.348417][ T501] br3: port 3(w1.20) entered disabled state [ 29.348815][ T501] w1.20: entered allmulticast mode [ 29.350770][ T501] w1.20: entered promiscuous mode [ 29.990023][ T514] br3: port 3(w1.20) entered blocking state [ 29.990476][ T514] br3: port 3(w1.20) entered forwarding state [ 40.063362][ T585] ip (585) used greatest stack depth: 23472 bytes left [ 40.133881][ T67] w1.20: left allmulticast mode [ 40.134409][ T67] w1.20: left promiscuous mode [ 40.135639][ T67] br3: port 3(w1.20) entered disabled state [ 40.140115][ T67] vx200: left allmulticast mode [ 40.140363][ T67] vx200: left promiscuous mode [ 40.140725][ T67] br3: port 2(vx200) entered disabled state [ 40.142425][ T67] w1: left allmulticast mode [ 40.142843][ T67] w1: left promiscuous mode [ 40.143284][ T67] br3: port 1(w1) entered disabled state [ 40.600767][ T67] vx100: left allmulticast mode [ 40.600991][ T67] vx100: left promiscuous mode [ 40.601299][ T67] br3: port 2(vx100) entered disabled state [ 40.602655][ T67] w1: left allmulticast mode [ 40.602836][ T67] w1: left promiscuous mode [ 40.603127][ T67] br3: port 1(w1) entered disabled state [ 42.548223][ T612] vx200: left allmulticast mode [ 42.548499][ T612] vx200: left promiscuous mode [ 42.550000][ T612] br2: port 1(vx200) entered disabled state [ 42.827969][ T615] veth2: left allmulticast mode [ 42.828350][ T615] veth2: left promiscuous mode [ 42.831178][ T615] br2: port 2(veth2.20) entered disabled state [ 42.857351][ T615] veth2.20 (unregistering): left allmulticast mode [ 42.858034][ T615] veth2.20 (unregistering): left promiscuous mode [ 42.858447][ T615] br2: port 2(veth2.20) entered disabled state [ 42.859933][ T615] [ 42.860085][ T615] ============================================ [ 42.860392][ T615] WARNING: possible recursive locking detected [ 42.860704][ T615] 6.13.0-rc5-virtme #1 Not tainted [ 42.860946][ T615] -------------------------------------------- [ 42.861262][ T615] ip/615 is trying to acquire lock: [ 42.861505][ T615] ffffffffbb7d6630 ((netdev_chain).rwsem){++++}-{4:4}, at: blocking_notifier_call_chain+0x50/0x90 [ 42.862042][ T615] [ 42.862042][ T615] but task is already holding lock: [ 42.862413][ T615] ffffffffbb7d6630 ((netdev_chain).rwsem){++++}-{4:4}, at: blocking_notifier_call_chain+0x50/0x90 [ 42.862929][ T615] [ 42.862929][ T615] other info that might help us debug this: [ 42.863304][ T615] Possible unsafe locking scenario: [ 42.863304][ T615] [ 42.863674][ T615] CPU0 [ 42.863871][ T615] ---- [ 42.864061][ T615] lock((netdev_chain).rwsem); [ 42.864322][ T615] lock((netdev_chain).rwsem); [ 42.864575][ T615] [ 42.864575][ T615] *** DEADLOCK *** [ 42.864575][ T615] [ 42.864949][ T615] May be due to missing lock nesting notation [ 42.864949][ T615] [ 42.865312][ T615] 2 locks held by ip/615: [ 42.865518][ T615] #0: ffffffffbb7e7a48 (rtnl_mutex){+.+.}-{4:4}, at: rtnl_dellink+0x228/0xa40 [ 42.865972][ T615] #1: ffffffffbb7d6630 ((netdev_chain).rwsem){++++}-{4:4}, at: blocking_notifier_call_chain+0x50/0x90 [ 42.866396][ T615] [ 42.866396][ T615] stack backtrace: [ 42.866681][ T615] CPU: 3 UID: 0 PID: 615 Comm: ip Not tainted 6.13.0-rc5-virtme #1 [ 42.867003][ T615] Hardware name: Bochs Bochs, BIOS Bochs 01/01/2011 [ 42.867313][ T615] Call Trace: [ 42.867538][ T615] [ 42.867670][ T615] dump_stack_lvl+0x82/0xd0 [ 42.867930][ T615] print_deadlock_bug+0x40a/0x650 [ 42.868103][ T615] validate_chain+0x5bf/0xae0 [ 42.868301][ T615] ? __pfx_validate_chain+0x10/0x10 [ 42.868512][ T615] ? hlock_class+0x4e/0x130 [ 42.868683][ T615] ? mark_lock+0x38/0x3e0 [ 42.868838][ T615] __lock_acquire+0xb9a/0x1680 [ 42.869015][ T615] ? hlock_class+0x4e/0x130 [ 42.869184][ T615] lock_acquire.part.0+0xeb/0x330 [ 42.869358][ T615] ? blocking_notifier_call_chain+0x50/0x90 [ 42.869579][ T615] ? validate_chain+0x5a7/0xae0 [ 42.869754][ T615] ? __pfx_lock_acquire.part.0+0x10/0x10 [ 42.869953][ T615] ? trace_lock_acquire+0x14c/0x1f0 [ 42.870155][ T615] ? lock_acquire+0x32/0xc0 [ 42.870399][ T615] ? blocking_notifier_call_chain+0x50/0x90 [ 42.870733][ T615] down_read+0x9f/0x340 [ 42.870934][ T615] ? blocking_notifier_call_chain+0x50/0x90 [ 42.871254][ T615] ? __pfx_down_read+0x10/0x10 [ 42.871467][ T615] ? __lock_acquire+0xb9a/0x1680 [ 42.871639][ T615] blocking_notifier_call_chain+0x50/0x90 [ 42.871836][ T615] __netdev_upper_dev_unlink+0xd9/0x220 [ 42.872009][ T615] ? br_fdb_delete_by_port+0x36/0x260 [bridge] [ 42.872270][ T615] ? __pfx___netdev_upper_dev_unlink+0x10/0x10 [ 42.872479][ T615] ? find_held_lock+0x2c/0x110 [ 42.872648][ T615] ? del_nbp+0x377/0xbe0 [bridge] [ 42.872860][ T615] ? hlock_class+0x4e/0x130 [ 42.873029][ T615] netdev_upper_dev_unlink+0x71/0xa0 [ 42.873196][ T615] ? __pfx_netdev_upper_dev_unlink+0x10/0x10 [ 42.873435][ T615] ? mutex_spin_on_owner+0x1f2/0x2b0 [ 42.873606][ T615] ? rtnl_is_locked+0x15/0x20 [ 42.873850][ T615] del_nbp+0x602/0xbe0 [bridge] [ 42.874113][ T615] br_del_if+0xa3/0x1f0 [bridge] [ 42.874388][ T615] br_device_event+0x53d/0x800 [bridge] [ 42.874685][ T615] ? __pfx_br_device_event+0x10/0x10 [bridge] [ 42.875047][ T615] ? netconsole_netdev_event+0x2c1/0x620 [ 42.875309][ T615] ? packet_notifier+0x3b0/0x810 [ 42.875583][ T615] notifier_call_chain+0xcd/0x150 [ 42.875804][ T615] blocking_notifier_call_chain+0x66/0x90 [ 42.876032][ T615] unregister_netdevice_many_notify+0xa7b/0x1580 [ 42.876355][ T615] ? __lock_release+0x103/0x460 [ 42.876617][ T615] ? __pfx___lock_release+0x10/0x10 [ 42.876839][ T615] ? __pfx_unregister_netdevice_many_notify+0x10/0x10 [ 42.877171][ T615] ? hlock_class+0x4e/0x130 [ 42.877419][ T615] ? mark_held_locks+0x9e/0xe0 [ 42.877671][ T615] ? __pfx_vlan_info_rcu_free+0x10/0x10 [ 42.877933][ T615] ? lockdep_hardirqs_on_prepare+0x275/0x410 [ 42.878258][ T615] rtnl_dellink+0x348/0xa40 [ 42.878528][ T615] ? __pfx_rtnl_dellink+0x10/0x10 [ 42.878817][ T615] ? find_held_lock+0x2c/0x110 [ 42.879090][ T615] ? __lock_release+0x103/0x460 [ 42.879364][ T615] ? rtnetlink_rcv_msg+0x6ef/0xc10 [ 42.879632][ T615] ? __pfx___lock_release+0x10/0x10 [ 42.879894][ T615] ? __pfx_lock_acquire.part.0+0x10/0x10 [ 42.880164][ T615] ? rtnetlink_rcv_msg+0x6ef/0xc10 [ 42.880437][ T615] ? __pfx_rtnl_dellink+0x10/0x10 [ 42.880709][ T615] rtnetlink_rcv_msg+0x712/0xc10 [ 42.880977][ T615] ? __pfx_rtnetlink_rcv_msg+0x10/0x10 [ 42.881238][ T615] ? hlock_class+0x4e/0x130 [ 42.881471][ T615] ? mark_lock+0x38/0x3e0 [ 42.881640][ T615] ? __lock_acquire+0xb9a/0x1680 [ 42.881859][ T615] netlink_rcv_skb+0x130/0x360 [ 42.882105][ T615] ? __pfx_rtnetlink_rcv_msg+0x10/0x10 [ 42.882340][ T615] ? __pfx_netlink_rcv_skb+0x10/0x10 [ 42.882586][ T615] ? netlink_deliver_tap+0x13e/0x340 [ 42.882847][ T615] ? netlink_deliver_tap+0xc3/0x340 [ 42.883115][ T615] netlink_unicast+0x44b/0x710 [ 42.883380][ T615] ? __pfx_netlink_unicast+0x10/0x10 [ 42.883624][ T615] ? find_held_lock+0x2c/0x110 [ 42.883888][ T615] netlink_sendmsg+0x723/0xbe0 [ 42.884147][ T615] ? __pfx_netlink_sendmsg+0x10/0x10 [ 42.884422][ T615] ____sys_sendmsg+0x7ac/0xa10 [ 42.884699][ T615] ? __pfx_____sys_sendmsg+0x10/0x10 [ 42.884956][ T615] ? __pfx_copy_msghdr_from_user+0x10/0x10 [ 42.885275][ T615] ___sys_sendmsg+0xee/0x170 [ 42.885528][ T615] ? __pfx____sys_sendmsg+0x10/0x10 [ 42.885790][ T615] ? kasan_save_stack+0x34/0x50 [ 42.886061][ T615] ? kasan_save_stack+0x24/0x50 [ 42.886325][ T615] ? __kasan_record_aux_stack+0x8e/0xa0 [ 42.886602][ T615] ? __call_rcu_common.constprop.0+0xa1/0x4b0 [ 42.886923][ T615] ? __x64_sys_close+0x7c/0xd0 [ 42.887190][ T615] ? do_syscall_64+0xc1/0x1d0 [ 42.887458][ T615] ? entry_SYSCALL_64_after_hwframe+0x77/0x7f [ 42.887801][ T615] ? __lock_acquire+0xb9a/0x1680 [ 42.888074][ T615] ? find_held_lock+0x2c/0x110 [ 42.888349][ T615] ? __lock_release+0x103/0x460 [ 42.888610][ T615] ? __virt_addr_valid+0x22b/0x430 [ 42.888890][ T615] ? __pfx___lock_release+0x10/0x10 [ 42.889153][ T615] ? __pfx_lock_acquire.part.0+0x10/0x10 [ 42.889424][ T615] __sys_sendmsg+0x109/0x1a0 [ 42.889693][ T615] ? __pfx___sys_sendmsg+0x10/0x10 [ 42.889957][ T615] ? __pfx_slab_free_after_rcu_debug+0x10/0x10 [ 42.890300][ T615] do_syscall_64+0xc1/0x1d0 [ 42.890561][ T615] entry_SYSCALL_64_after_hwframe+0x77/0x7f [ 42.890893][ T615] RIP: 0033:0x7fdb28e759a7 [ 42.891165][ T615] Code: 0a 00 f7 d8 64 89 02 48 c7 c0 ff ff ff ff eb b9 0f 1f 00 f3 0f 1e fa 64 8b 04 25 18 00 00 00 85 c0 75 10 b8 2e 00 00 00 0f 05 <48> 3d 00 f0 ff ff 77 51 c3 48 83 ec 28 89 54 24 1c 48 89 74 24 10 [ 42.892094][ T615] RSP: 002b:00007fffa32d7d98 EFLAGS: 00000246 ORIG_RAX: 000000000000002e [ 42.892502][ T615] RAX: ffffffffffffffda RBX: 00007fffa32d84c0 RCX: 00007fdb28e759a7 [ 42.892887][ T615] RDX: 0000000000000000 RSI: 00007fffa32d7e00 RDI: 0000000000000005 [ 42.893283][ T615] RBP: 0000000000000002 R08: 0000000000000003 R09: 0000000000000078 [ 42.893688][ T615] R10: 00007fdb28d33ef8 R11: 0000000000000246 R12: 0000000000000002 [ 42.894097][ T615] R13: 00000000677900fd R14: 0000000000498600 R15: 0000000000000000 [ 42.894492][ T615] [ 43.142053][ T619] br1: port 2(veth1) entered disabled state [ 43.214299][ T620] veth1: left allmulticast mode [ 43.214549][ T620] veth1: left promiscuous mode [ 43.214875][ T620] br1: port 2(veth1) entered disabled state [ 43.266893][ T621] vx100: left allmulticast mode [ 43.267126][ T621] vx100: left promiscuous mode [ 43.267419][ T621] br1: port 1(vx100) entered disabled state