[ 1648.446283][T19585] br0: port 1(gw_l) entered blocking state [ 1648.446795][T19585] br0: port 1(gw_l) entered disabled state [ 1648.447071][T19585] gw_l: entered allmulticast mode [ 1648.448523][T19585] gw_l: entered promiscuous mode [ 1648.450754][T19585] br0: port 1(gw_l) entered blocking state [ 1648.451101][T19585] br0: port 1(gw_l) entered forwarding state [ 1649.013459][T19587] br0: port 2(amtg) entered blocking state [ 1649.013946][T19587] br0: port 2(amtg) entered disabled state [ 1649.015754][T19587] amtg: entered allmulticast mode [ 1649.017318][T19587] amtg: entered promiscuous mode [ 1650.700048][T19598] br0: port 2(amtg) entered blocking state [ 1650.700653][T19598] br0: port 2(amtg) entered forwarding state [ 1651.483620][T19604] amtr: entered allmulticast mode [ 1651.484138][T19604] relay_gw: entered allmulticast mode [ 1651.484831][T19604] relay_src: entered allmulticast mode [ 2892.705332][ T66] br0: port 1(gw_l) entered disabled state [ 2892.797763][ T66] gw_l (unregistering): left allmulticast mode [ 2892.798134][ T66] gw_l (unregistering): left promiscuous mode [ 2892.798373][ T66] br0: port 1(gw_l) entered disabled state [ 2892.799764][ T66] [ 2892.799868][ T66] ============================================ [ 2892.800060][ T66] WARNING: possible recursive locking detected [ 2892.800261][ T66] 6.13.0-rc5-virtme #1 Not tainted [ 2892.800421][ T66] -------------------------------------------- [ 2892.800620][ T66] kworker/u16:1/66 is trying to acquire lock: [ 2892.800824][ T66] ffffffff93bf6a70 ((netdev_chain).rwsem){++++}-{4:4}, at: blocking_notifier_call_chain+0x50/0x90 [ 2892.801159][ T66] [ 2892.801159][ T66] but task is already holding lock: [ 2892.801395][ T66] ffffffff93bf6a70 ((netdev_chain).rwsem){++++}-{4:4}, at: blocking_notifier_call_chain+0x50/0x90 [ 2892.801710][ T66] [ 2892.801710][ T66] other info that might help us debug this: [ 2892.801918][ T66] Possible unsafe locking scenario: [ 2892.801918][ T66] [ 2892.802140][ T66] CPU0 [ 2892.802253][ T66] ---- [ 2892.802361][ T66] lock((netdev_chain).rwsem); [ 2892.802508][ T66] lock((netdev_chain).rwsem); [ 2892.802659][ T66] [ 2892.802659][ T66] *** DEADLOCK *** [ 2892.802659][ T66] [ 2892.802882][ T66] May be due to missing lock nesting notation [ 2892.802882][ T66] [ 2892.803110][ T66] 5 locks held by kworker/u16:1/66: [ 2892.803279][ T66] #0: ffff8880010b5948 ((wq_completion)netns){+.+.}-{0:0}, at: process_one_work+0x7ec/0x16d0 [ 2892.803611][ T66] #1: ffffc90000477da0 (net_cleanup_work){+.+.}-{0:0}, at: process_one_work+0xe0b/0x16d0 [ 2892.803910][ T66] #2: ffffffff93bec4d0 (pernet_ops_rwsem){++++}-{4:4}, at: cleanup_net+0xbc/0xba0 [ 2892.804208][ T66] #3: ffffffff93c07e88 (rtnl_mutex){+.+.}-{4:4}, at: default_device_exit_batch+0x81/0x2e0 [ 2892.804537][ T66] #4: ffffffff93bf6a70 ((netdev_chain).rwsem){++++}-{4:4}, at: blocking_notifier_call_chain+0x50/0x90 [ 2892.804869][ T66] [ 2892.804869][ T66] stack backtrace: [ 2892.805075][ T66] CPU: 3 UID: 0 PID: 66 Comm: kworker/u16:1 Not tainted 6.13.0-rc5-virtme #1 [ 2892.805361][ T66] Hardware name: Bochs Bochs, BIOS Bochs 01/01/2011 [ 2892.805567][ T66] Workqueue: netns cleanup_net [ 2892.805741][ T66] Call Trace: [ 2892.805867][ T66] [ 2892.805956][ T66] dump_stack_lvl+0x82/0xd0 [ 2892.806125][ T66] print_deadlock_bug+0x40a/0x650 [ 2892.806298][ T66] validate_chain+0x5bf/0xae0 [ 2892.806462][ T66] ? __pfx_validate_chain+0x10/0x10 [ 2892.806632][ T66] ? hlock_class+0x4e/0x130 [ 2892.806793][ T66] ? mark_lock+0x38/0x3e0 [ 2892.806922][ T66] __lock_acquire+0xb9a/0x1680 [ 2892.807082][ T66] lock_acquire.part.0+0xeb/0x330 [ 2892.807246][ T66] ? blocking_notifier_call_chain+0x50/0x90 [ 2892.807450][ T66] ? find_held_lock+0x2c/0x110 [ 2892.807605][ T66] ? __pfx_lock_acquire.part.0+0x10/0x10 [ 2892.807769][ T66] ? trace_lock_acquire+0x14c/0x1f0 [ 2892.807930][ T66] ? lock_acquire+0x32/0xc0 [ 2892.808099][ T66] ? blocking_notifier_call_chain+0x50/0x90 [ 2892.808302][ T66] down_read+0x9f/0x340 [ 2892.808425][ T66] ? blocking_notifier_call_chain+0x50/0x90 [ 2892.808628][ T66] ? __pfx_down_read+0x10/0x10 [ 2892.808790][ T66] ? lockdep_hardirqs_on_prepare+0x12b/0x410 [ 2892.808995][ T66] blocking_notifier_call_chain+0x50/0x90 [ 2892.809159][ T66] __netdev_upper_dev_unlink+0xd9/0x220 [ 2892.809338][ T66] ? __pfx___netdev_upper_dev_unlink+0x10/0x10 [ 2892.809541][ T66] ? find_held_lock+0x2c/0x110 [ 2892.809706][ T66] ? del_nbp+0x377/0xbe0 [ 2892.809835][ T66] ? hlock_class+0x4e/0x130 [ 2892.810000][ T66] netdev_upper_dev_unlink+0x71/0xa0 [ 2892.810160][ T66] ? __pfx_netdev_upper_dev_unlink+0x10/0x10 [ 2892.810364][ T66] ? mutex_is_locked+0x12/0x60 [ 2892.810525][ T66] ? rtnl_is_locked+0x15/0x20 [ 2892.810705][ T66] del_nbp+0x602/0xbe0 [ 2892.810835][ T66] br_del_if+0xa3/0x1f0 [ 2892.810962][ T66] br_device_event+0x53d/0x800 [ 2892.811127][ T66] ? __pfx_br_device_event+0x10/0x10 [ 2892.811279][ T66] ? packet_notifier+0x3b0/0x810 [ 2892.811431][ T66] notifier_call_chain+0xcd/0x150 [ 2892.811584][ T66] blocking_notifier_call_chain+0x66/0x90 [ 2892.811751][ T66] unregister_netdevice_many_notify+0xa83/0x1580 [ 2892.811949][ T66] ? __pfx_unregister_netdevice_many_notify+0x10/0x10 [ 2892.812153][ T66] ? default_device_exit_batch+0x81/0x2e0 [ 2892.812320][ T66] ? rtnl_is_locked+0x15/0x20 [ 2892.812479][ T66] ? unregister_netdevice_queue+0x70/0x410 [ 2892.812676][ T66] ? __pfx_unregister_netdevice_queue+0x10/0x10 [ 2892.812885][ T66] ? __pfx_unregister_netdevice_queue+0x10/0x10 [ 2892.813097][ T66] default_device_exit_batch+0x241/0x2e0 [ 2892.813261][ T66] ? __pfx_default_device_exit_batch+0x10/0x10 [ 2892.813462][ T66] ? ops_exit_list+0xb4/0x170 [ 2892.813632][ T66] cleanup_net+0x4ef/0xba0 [ 2892.813793][ T66] ? __pfx_lock_acquire.part.0+0x10/0x10 [ 2892.813958][ T66] ? __pfx_cleanup_net+0x10/0x10 [ 2892.814121][ T66] ? trace_lock_acquire+0x14c/0x1f0 [ 2892.814284][ T66] ? lock_acquire+0x32/0xc0 [ 2892.814447][ T66] ? process_one_work+0xe0b/0x16d0 [ 2892.814609][ T66] process_one_work+0xe55/0x16d0 [ 2892.814769][ T66] ? __pfx___lock_release+0x10/0x10 [ 2892.814931][ T66] ? __pfx_process_one_work+0x10/0x10 [ 2892.815096][ T66] ? assign_work+0x16c/0x240 [ 2892.815259][ T66] worker_thread+0x58c/0xce0 [ 2892.815424][ T66] ? lockdep_hardirqs_on_prepare+0x275/0x410 [ 2892.815625][ T66] ? __pfx_worker_thread+0x10/0x10 [ 2892.815789][ T66] ? __pfx_worker_thread+0x10/0x10 [ 2892.815949][ T66] kthread+0x28a/0x350 [ 2892.816079][ T66] ? __pfx_kthread+0x10/0x10 [ 2892.816241][ T66] ret_from_fork+0x31/0x70 [ 2892.816409][ T66] ? __pfx_kthread+0x10/0x10 [ 2892.816578][ T66] ret_from_fork_asm+0x1a/0x30 [ 2892.816747][ T66] [ 2892.869556][ T66] amtg: left allmulticast mode [ 2892.869749][ T66] amtg: left promiscuous mode [ 2892.870039][ T66] br0: port 2(amtg) entered disabled state [ 2892.917297][T19604] amtr: left allmulticast mode [ 2892.917570][T19604] relay_gw: left allmulticast mode [ 2892.917767][T19604] relay_src: left allmulticast mode