[ 10.952156][ T275] ip (275) used greatest stack depth: 23632 bytes left [ 11.115135][ T280] veth0: renamed from veth-hv-1 [ 11.880471][ T296] br0: port 1(vxlan0) entered blocking state [ 11.881449][ T296] br0: port 1(vxlan0) entered disabled state [ 11.881708][ T296] vxlan0: entered allmulticast mode [ 11.882969][ T296] vxlan0: entered promiscuous mode [ 11.966403][ T297] br0: port 1(vxlan0) entered blocking state [ 11.966912][ T297] br0: port 1(vxlan0) entered forwarding state [ 12.129440][ T300] veth0: renamed from veth-hv-2 [ 12.749476][ T308] br0: port 1(vxlan0) entered blocking state [ 12.749764][ T308] br0: port 1(vxlan0) entered disabled state [ 12.750011][ T308] vxlan0: entered allmulticast mode [ 12.751170][ T308] vxlan0: entered promiscuous mode [ 12.825062][ T309] br0: port 1(vxlan0) entered blocking state [ 12.825403][ T309] br0: port 1(vxlan0) entered forwarding state [ 13.256940][ T315] br0: port 2(veth-tap) entered blocking state [ 13.257258][ T315] br0: port 2(veth-tap) entered disabled state [ 13.257620][ T315] veth-tap: entered allmulticast mode [ 13.259888][ T315] veth-tap: entered promiscuous mode [ 13.659366][ T159] br0: port 2(veth-tap) entered blocking state [ 13.659788][ T159] br0: port 2(veth-tap) entered forwarding state [ 13.762356][ T321] ip (321) used greatest stack depth: 23432 bytes left [ 13.965230][ T325] br0: port 2(veth-tap) entered blocking state [ 13.965475][ T325] br0: port 2(veth-tap) entered disabled state [ 13.965766][ T325] veth-tap: entered allmulticast mode [ 13.967107][ T325] veth-tap: entered promiscuous mode [ 14.346177][ T159] br0: port 2(veth-tap) entered blocking state [ 14.346525][ T159] br0: port 2(veth-tap) entered forwarding state [ 14.792253][ T335] br0: port 1(vxlan0) entered disabled state [ 14.876958][ T336] br0: port 1(vxlan0) entered blocking state [ 14.877237][ T336] br0: port 1(vxlan0) entered forwarding state [ 15.042469][ T338] br0: port 1(vxlan0) entered disabled state [ 15.122325][ T339] br0: port 1(vxlan0) entered blocking state [ 15.122708][ T339] br0: port 1(vxlan0) entered forwarding state [ 15.822637][ T67] veth-tap: left allmulticast mode [ 15.822978][ T67] veth-tap: left promiscuous mode [ 15.823305][ T67] br0: port 2(veth-tap) entered disabled state [ 15.825432][ T67] vxlan0: left allmulticast mode [ 15.825631][ T67] vxlan0: left promiscuous mode [ 15.825935][ T67] br0: port 1(vxlan0) entered disabled state [ 16.163023][ T67] [ 16.163137][ T67] ============================================ [ 16.163312][ T67] WARNING: possible recursive locking detected [ 16.163482][ T67] 6.13.0-rc5-virtme #1 Not tainted [ 16.163627][ T67] -------------------------------------------- [ 16.163798][ T67] kworker/u16:1/67 is trying to acquire lock: [ 16.163969][ T67] ffffffffba7f6a70 ((netdev_chain).rwsem){++++}-{4:4}, at: blocking_notifier_call_chain+0x50/0x90 [ 16.164260][ T67] [ 16.164260][ T67] but task is already holding lock: [ 16.164467][ T67] ffffffffba7f6a70 ((netdev_chain).rwsem){++++}-{4:4}, at: blocking_notifier_call_chain+0x50/0x90 [ 16.164747][ T67] [ 16.164747][ T67] other info that might help us debug this: [ 16.164953][ T67] Possible unsafe locking scenario: [ 16.164953][ T67] [ 16.165154][ T67] CPU0 [ 16.165258][ T67] ---- [ 16.165365][ T67] lock((netdev_chain).rwsem); [ 16.165506][ T67] lock((netdev_chain).rwsem); [ 16.165642][ T67] [ 16.165642][ T67] *** DEADLOCK *** [ 16.165642][ T67] [ 16.165851][ T67] May be due to missing lock nesting notation [ 16.165851][ T67] [ 16.166060][ T67] 5 locks held by kworker/u16:1/67: [ 16.166200][ T67] #0: ffff8880010b5948 ((wq_completion)netns){+.+.}-{0:0}, at: process_one_work+0x7ec/0x16d0 [ 16.166491][ T67] #1: ffffc90000487da0 (net_cleanup_work){+.+.}-{0:0}, at: process_one_work+0xe0b/0x16d0 [ 16.166737][ T67] #2: ffffffffba7ec4d0 (pernet_ops_rwsem){++++}-{4:4}, at: cleanup_net+0xbc/0xba0 [ 16.166983][ T67] #3: ffffffffba807e88 (rtnl_mutex){+.+.}-{4:4}, at: default_device_exit_batch+0x81/0x2e0 [ 16.167264][ T67] #4: ffffffffba7f6a70 ((netdev_chain).rwsem){++++}-{4:4}, at: blocking_notifier_call_chain+0x50/0x90 [ 16.167549][ T67] [ 16.167549][ T67] stack backtrace: [ 16.167718][ T67] CPU: 2 UID: 0 PID: 67 Comm: kworker/u16:1 Not tainted 6.13.0-rc5-virtme #1 [ 16.167963][ T67] Hardware name: Bochs Bochs, BIOS Bochs 01/01/2011 [ 16.168140][ T67] Workqueue: netns cleanup_net [ 16.168283][ T67] Call Trace: [ 16.168387][ T67] [ 16.168458][ T67] dump_stack_lvl+0x82/0xd0 [ 16.168598][ T67] print_deadlock_bug+0x40a/0x650 [ 16.168752][ T67] validate_chain+0x5bf/0xae0 [ 16.168892][ T67] ? __pfx_validate_chain+0x10/0x10 [ 16.169038][ T67] ? hlock_class+0x4e/0x130 [ 16.169174][ T67] ? mark_lock+0x38/0x3e0 [ 16.169283][ T67] __lock_acquire+0xb9a/0x1680 [ 16.169426][ T67] lock_acquire.part.0+0xeb/0x330 [ 16.169565][ T67] ? blocking_notifier_call_chain+0x50/0x90 [ 16.169737][ T67] ? __lock_acquire+0xb9a/0x1680 [ 16.169876][ T67] ? __pfx_lock_acquire.part.0+0x10/0x10 [ 16.170016][ T67] ? trace_lock_acquire+0x14c/0x1f0 [ 16.170151][ T67] ? lock_acquire+0x32/0xc0 [ 16.170292][ T67] ? blocking_notifier_call_chain+0x50/0x90 [ 16.170468][ T67] down_read+0x9f/0x340 [ 16.170572][ T67] ? blocking_notifier_call_chain+0x50/0x90 [ 16.170746][ T67] ? __pfx_down_read+0x10/0x10 [ 16.170885][ T67] ? __lock_release+0x103/0x460 [ 16.171029][ T67] blocking_notifier_call_chain+0x50/0x90 [ 16.171171][ T67] __netdev_upper_dev_unlink+0xd9/0x220 [ 16.171310][ T67] ? mark_held_locks+0x9e/0xe0 [ 16.171448][ T67] ? __pfx___netdev_upper_dev_unlink+0x10/0x10 [ 16.171618][ T67] ? lockdep_hardirqs_on_prepare+0x275/0x410 [ 16.171795][ T67] ? __try_to_del_timer_sync+0xe7/0x140 [ 16.171935][ T67] ? __pfx___try_to_del_timer_sync+0x10/0x10 [ 16.172110][ T67] ? mark_held_locks+0x9e/0xe0 [ 16.172249][ T67] netdev_upper_dev_unlink+0x71/0xa0 [ 16.172394][ T67] ? __pfx_netdev_upper_dev_unlink+0x10/0x10 [ 16.172566][ T67] ? mutex_is_locked+0x1c/0x60 [ 16.172707][ T67] ? rtnl_is_locked+0x15/0x20 [ 16.172854][ T67] ? netdev_master_upper_dev_get+0x13/0x120 [ 16.173030][ T67] vrf_device_event+0x78/0xe0 [ 16.173175][ T67] notifier_call_chain+0xcd/0x150 [ 16.173316][ T67] blocking_notifier_call_chain+0x66/0x90 [ 16.173454][ T67] unregister_netdevice_many_notify+0xa83/0x1580 [ 16.173627][ T67] ? unregister_netdevice_queue+0x70/0x410 [ 16.173801][ T67] ? __pfx_unregister_netdevice_many_notify+0x10/0x10 [ 16.173970][ T67] ? __pfx_unregister_netdevice_queue+0x10/0x10 [ 16.174148][ T67] ? rtnl_is_locked+0x15/0x20 [ 16.174291][ T67] ? unregister_netdevice_queue+0x70/0x410 [ 16.174463][ T67] ? __pfx_unregister_netdevice_queue+0x10/0x10 [ 16.174634][ T67] ? vrf_dellink+0x100/0x150 [ 16.174773][ T67] ? __pfx_unregister_netdevice_queue+0x10/0x10 [ 16.174948][ T67] default_device_exit_batch+0x241/0x2e0 [ 16.175088][ T67] ? __pfx_default_device_exit_batch+0x10/0x10 [ 16.175264][ T67] ? ops_exit_list+0xb4/0x170 [ 16.175408][ T67] cleanup_net+0x4ef/0xba0 [ 16.175546][ T67] ? __pfx_lock_acquire.part.0+0x10/0x10 [ 16.175687][ T67] ? __pfx_cleanup_net+0x10/0x10 [ 16.175824][ T67] ? trace_lock_acquire+0x14c/0x1f0 [ 16.175963][ T67] ? lock_acquire+0x32/0xc0 [ 16.176103][ T67] ? process_one_work+0xe0b/0x16d0 [ 16.176245][ T67] process_one_work+0xe55/0x16d0 [ 16.176388][ T67] ? __pfx___lock_release+0x10/0x10 [ 16.176526][ T67] ? __pfx_process_one_work+0x10/0x10 [ 16.176662][ T67] ? assign_work+0x16c/0x240 [ 16.176805][ T67] worker_thread+0x58c/0xce0 [ 16.176944][ T67] ? lockdep_hardirqs_on_prepare+0x275/0x410 [ 16.177121][ T67] ? __pfx_worker_thread+0x10/0x10 [ 16.177262][ T67] ? __pfx_worker_thread+0x10/0x10 [ 16.177399][ T67] kthread+0x28a/0x350 [ 16.177505][ T67] ? __pfx_kthread+0x10/0x10 [ 16.177643][ T67] ret_from_fork+0x31/0x70 [ 16.177783][ T67] ? __pfx_kthread+0x10/0x10 [ 16.177923][ T67] ret_from_fork_asm+0x1a/0x30 [ 16.178067][ T67] [ 16.187528][ T67] br0: port 1(vxlan0) entered disabled state [ 16.195256][ T67] vxlan0 (unregistering): left allmulticast mode [ 16.195546][ T67] vxlan0 (unregistering): left promiscuous mode [ 16.195748][ T67] br0: port 1(vxlan0) entered disabled state [ 16.236120][ T67] veth-tap: left allmulticast mode [ 16.236296][ T67] veth-tap: left promiscuous mode [ 16.236556][ T67] br0: port 2(veth-tap) entered disabled state