[ 11.592543][ T262] ip (262) used greatest stack depth: 24416 bytes left [ 16.246172][ T324] br1: port 1(vx10) entered blocking state [ 16.246715][ T324] br1: port 1(vx10) entered disabled state [ 16.247142][ T324] vx10: entered allmulticast mode [ 16.250126][ T324] vx10: entered promiscuous mode [ 16.251230][ T324] br1: port 1(vx10) entered blocking state [ 16.251622][ T324] br1: port 1(vx10) entered forwarding state [ 16.270412][ T324] ip (324) used greatest stack depth: 24368 bytes left [ 16.626181][ T329] br1: port 2(vx20) entered blocking state [ 16.626538][ T329] br1: port 2(vx20) entered disabled state [ 16.626858][ T329] vx20: entered allmulticast mode [ 16.628921][ T329] vx20: entered promiscuous mode [ 16.630093][ T329] br1: port 2(vx20) entered blocking state [ 16.630389][ T329] br1: port 2(vx20) entered forwarding state [ 16.800475][ T331] br1: port 3(veth1) entered blocking state [ 16.800838][ T331] br1: port 3(veth1) entered disabled state [ 16.801156][ T331] veth1: entered allmulticast mode [ 16.803594][ T331] veth1: entered promiscuous mode [ 16.900914][ T251] br1: port 3(veth1) entered blocking state [ 16.901403][ T251] br1: port 3(veth1) entered forwarding state [ 16.996507][ T333] br1: port 4(veth2) entered blocking state [ 16.996846][ T333] br1: port 4(veth2) entered disabled state [ 16.997154][ T333] veth2: entered allmulticast mode [ 16.999283][ T333] veth2: entered promiscuous mode [ 17.109631][ T37] br1: port 4(veth2) entered blocking state [ 17.110016][ T37] br1: port 4(veth2) entered forwarding state [ 17.380031][ T338] br1: port 5(vx4001) entered blocking state [ 17.380972][ T338] br1: port 5(vx4001) entered disabled state [ 17.381286][ T338] vx4001: entered allmulticast mode [ 17.383371][ T338] vx4001: entered promiscuous mode [ 17.384019][ T338] br1: port 5(vx4001) entered blocking state [ 17.384349][ T338] br1: port 5(vx4001) entered forwarding state [ 18.139921][ T347] 8021q: 802.1Q VLAN Support v1.8 [ 18.427448][ T350] br1: entered promiscuous mode [ 18.430234][ T350] br1: left promiscuous mode [ 18.442039][ T350] br1: entered promiscuous mode [ 18.450996][ T350] ip (350) used greatest stack depth: 24144 bytes left [ 24.221352][ T428] br1: port 1(vx10) entered blocking state [ 24.221643][ T428] br1: port 1(vx10) entered disabled state [ 24.223690][ T428] vx10: entered allmulticast mode [ 24.225881][ T428] vx10: entered promiscuous mode [ 24.235760][ T428] br1: port 1(vx10) entered blocking state [ 24.236175][ T428] br1: port 1(vx10) entered forwarding state [ 24.603642][ T432] br1: port 2(vx20) entered blocking state [ 24.603946][ T432] br1: port 2(vx20) entered disabled state [ 24.604231][ T432] vx20: entered allmulticast mode [ 24.607352][ T432] vx20: entered promiscuous mode [ 24.607991][ T432] br1: port 2(vx20) entered blocking state [ 24.608519][ T432] br1: port 2(vx20) entered forwarding state [ 24.966117][ T436] br1: port 3(vx4001) entered blocking state [ 24.966421][ T436] br1: port 3(vx4001) entered disabled state [ 24.966717][ T436] vx4001: entered allmulticast mode [ 24.968678][ T436] vx4001: entered promiscuous mode [ 24.969192][ T436] br1: port 3(vx4001) entered blocking state [ 24.969453][ T436] br1: port 3(vx4001) entered forwarding state [ 25.128479][ T438] br1: port 4(w1) entered blocking state [ 25.128731][ T438] br1: port 4(w1) entered disabled state [ 25.128975][ T438] w1: entered allmulticast mode [ 25.131024][ T438] w1: entered promiscuous mode [ 25.261069][ T58] br1: port 4(w1) entered blocking state [ 25.261410][ T58] br1: port 4(w1) entered forwarding state [ 25.428950][ T442] br1: port 5(w3) entered blocking state [ 25.429214][ T442] br1: port 5(w3) entered disabled state [ 25.429480][ T442] w3: entered allmulticast mode [ 25.431437][ T442] w3: entered promiscuous mode [ 25.539870][ T58] br1: port 5(w3) entered blocking state [ 25.540154][ T58] br1: port 5(w3) entered forwarding state [ 26.350641][ T452] br1: entered promiscuous mode [ 26.354878][ T452] br1: left promiscuous mode [ 26.368557][ T452] br1: entered promiscuous mode [ 27.127288][ C2] br1: received packet on vx20 with own address as source address (addr:00:00:5e:00:01:01, vlan:20) [ 27.132321][ C2] IPv6: vlan20-v: IPv6 duplicate address fe80::200:5eff:fe00:101 used by 00:00:5e:00:01:01 detected! [ 27.133634][ C2] IPv6: vlan20-v: IPv6 duplicate address 2001:db8:2::3 used by 00:00:5e:00:01:01 detected! [ 27.231949][ C3] br1: received packet on vx20 with own address as source address (addr:00:00:5e:00:01:01, vlan:20) [ 27.271123][ C1] br1: received packet on vx10 with own address as source address (addr:00:00:5e:00:01:01, vlan:10) [ 27.272859][ C1] IPv6: vlan10-v: IPv6 duplicate address fe80::200:5eff:fe00:101 used by 00:00:5e:00:01:01 detected! [ 28.140711][ C2] br1: received packet on vx10 with own address as source address (addr:00:00:5e:00:01:01, vlan:10) [ 28.141983][ C2] br1: received packet on vx20 with own address as source address (addr:00:00:5e:00:01:01, vlan:20) [ 28.407069][ C2] br1: received packet on vx20 with own address as source address (addr:00:00:5e:00:01:01, vlan:20) [ 28.870957][ C2] br1: received packet on vx10 with own address as source address (addr:00:00:5e:00:01:01, vlan:10) [ 45.499294][ T626] ip (626) used greatest stack depth: 23472 bytes left [ 45.586533][ T11] w3: left allmulticast mode [ 45.586945][ T11] w3: left promiscuous mode [ 45.587409][ T11] br1: port 5(w3) entered disabled state [ 45.599418][ T11] w1: left allmulticast mode [ 45.599733][ T11] w1: left promiscuous mode [ 45.600144][ T11] br1: port 4(w1) entered disabled state [ 45.601941][ T11] vx4001: left allmulticast mode [ 45.602500][ T11] vx4001: left promiscuous mode [ 45.602925][ T11] br1: port 3(vx4001) entered disabled state [ 45.606102][ T11] vx20: left allmulticast mode [ 45.606403][ T11] vx20: left promiscuous mode [ 45.606785][ T11] br1: port 2(vx20) entered disabled state [ 45.608525][ T11] vx10: left allmulticast mode [ 45.608798][ T11] vx10: left promiscuous mode [ 45.609174][ T11] br1: port 1(vx10) entered disabled state [ 45.632845][ T11] [ 45.632983][ T11] ============================================ [ 45.633191][ T11] WARNING: possible recursive locking detected [ 45.633396][ T11] 6.13.0-rc5-virtme #1 Not tainted [ 45.633593][ T11] -------------------------------------------- [ 45.633801][ T11] kworker/u16:0/11 is trying to acquire lock: [ 45.634014][ T11] ffffffff89dd6630 ((netdev_chain).rwsem){++++}-{4:4}, at: blocking_notifier_call_chain+0x50/0x90 [ 45.634384][ T11] [ 45.634384][ T11] but task is already holding lock: [ 45.634635][ T11] ffffffff89dd6630 ((netdev_chain).rwsem){++++}-{4:4}, at: blocking_notifier_call_chain+0x50/0x90 [ 45.635050][ T11] [ 45.635050][ T11] other info that might help us debug this: [ 45.635299][ T11] Possible unsafe locking scenario: [ 45.635299][ T11] [ 45.635541][ T11] CPU0 [ 45.635661][ T11] ---- [ 45.635778][ T11] lock((netdev_chain).rwsem); [ 45.635944][ T11] lock((netdev_chain).rwsem); [ 45.636106][ T11] [ 45.636106][ T11] *** DEADLOCK *** [ 45.636106][ T11] [ 45.636344][ T11] May be due to missing lock nesting notation [ 45.636344][ T11] [ 45.636583][ T11] 5 locks held by kworker/u16:0/11: [ 45.636755][ T11] #0: ffff8880010ad948 ((wq_completion)netns){+.+.}-{0:0}, at: process_one_work+0x7ec/0x16d0 [ 45.637083][ T11] #1: ffffc900000b7da0 (net_cleanup_work){+.+.}-{0:0}, at: process_one_work+0xe0b/0x16d0 [ 45.637375][ T11] #2: ffffffff89dcc090 (pernet_ops_rwsem){++++}-{4:4}, at: cleanup_net+0xbc/0xa50 [ 45.637673][ T11] #3: ffffffff89de7a48 (rtnl_mutex){+.+.}-{4:4}, at: cleanup_net+0x45e/0xa50 [ 45.637978][ T11] #4: ffffffff89dd6630 ((netdev_chain).rwsem){++++}-{4:4}, at: blocking_notifier_call_chain+0x50/0x90 [ 45.638308][ T11] [ 45.638308][ T11] stack backtrace: [ 45.638501][ T11] CPU: 2 UID: 0 PID: 11 Comm: kworker/u16:0 Not tainted 6.13.0-rc5-virtme #1 [ 45.638791][ T11] Hardware name: Bochs Bochs, BIOS Bochs 01/01/2011 [ 45.638999][ T11] Workqueue: netns cleanup_net [ 45.639170][ T11] Call Trace: [ 45.639296][ T11] [ 45.639384][ T11] dump_stack_lvl+0x82/0xd0 [ 45.639553][ T11] print_deadlock_bug+0x40a/0x650 [ 45.639718][ T11] validate_chain+0x5bf/0xae0 [ 45.639877][ T11] ? __pfx_validate_chain+0x10/0x10 [ 45.640046][ T11] ? hlock_class+0x4e/0x130 [ 45.640204][ T11] ? mark_lock+0x38/0x3e0 [ 45.640332][ T11] __lock_acquire+0xb9a/0x1680 [ 45.640497][ T11] ? spin_bug+0x1b1/0x1d0 [ 45.640642][ T11] ? debug_object_assert_init+0x2a9/0x370 [ 45.640813][ T11] lock_acquire.part.0+0xeb/0x330 [ 45.640974][ T11] ? blocking_notifier_call_chain+0x50/0x90 [ 45.641183][ T11] ? __pfx_lock_acquire.part.0+0x10/0x10 [ 45.641348][ T11] ? trace_lock_acquire+0x14c/0x1f0 [ 45.641508][ T11] ? lock_acquire+0x32/0xc0 [ 45.641673][ T11] ? blocking_notifier_call_chain+0x50/0x90 [ 45.641882][ T11] down_read+0x9f/0x340 [ 45.642008][ T11] ? blocking_notifier_call_chain+0x50/0x90 [ 45.642220][ T11] ? __pfx_down_read+0x10/0x10 [ 45.642395][ T11] blocking_notifier_call_chain+0x50/0x90 [ 45.642557][ T11] __dev_close_many+0xdf/0x2d0 [ 45.642720][ T11] ? __pfx___dev_close_many+0x10/0x10 [ 45.642880][ T11] dev_close_many+0x202/0x650 [ 45.643042][ T11] ? __lock_release+0x103/0x460 [ 45.643205][ T11] ? __pfx_dev_close_many+0x10/0x10 [ 45.643362][ T11] ? __pfx___lock_release+0x10/0x10 [ 45.643532][ T11] ? __pfx_lock_acquire.part.0+0x10/0x10 [ 45.643701][ T11] vlan_device_event+0x143b/0x2130 [8021q] [ 45.643909][ T11] ? net_generic+0xbb/0x1f0 [vxlan] [ 45.644091][ T11] ? __pfx_vlan_device_event+0x10/0x10 [8021q] [ 45.644294][ T11] ? __pfx_vxlan_netdevice_event+0x10/0x10 [vxlan] [ 45.644501][ T11] ? packet_notifier+0x3b0/0x810 [ 45.644671][ T11] notifier_call_chain+0xcd/0x150 [ 45.644826][ T11] blocking_notifier_call_chain+0x66/0x90 [ 45.644989][ T11] dev_close_many+0x2d8/0x650 [ 45.645152][ T11] ? __lock_release+0x103/0x460 [ 45.645314][ T11] ? __virt_addr_valid+0x22b/0x430 [ 45.645482][ T11] ? __pfx_dev_close_many+0x10/0x10 [ 45.645650][ T11] unregister_netdevice_many_notify+0x8e5/0x1580 [ 45.645951][ T11] ? __pfx_unregister_netdevice_many_notify+0x10/0x10 [ 45.646156][ T11] ? mark_held_locks+0x9e/0xe0 [ 45.646322][ T11] ? mutex_is_locked+0x1c/0x60 [ 45.646644][ T11] ? nexthop_net_exit_batch_rtnl+0x83/0x210 [ 45.646864][ T11] cleanup_net+0x4cf/0xa50 [ 45.647104][ T11] ? __pfx_lock_acquire.part.0+0x10/0x10 [ 45.647351][ T11] ? __pfx_cleanup_net+0x10/0x10 [ 45.647523][ T11] ? trace_lock_acquire+0x14c/0x1f0 [ 45.647684][ T11] ? lock_acquire+0x32/0xc0 [ 45.647846][ T11] ? process_one_work+0xe0b/0x16d0 [ 45.648008][ T11] process_one_work+0xe55/0x16d0 [ 45.648177][ T11] ? __pfx___lock_release+0x10/0x10 [ 45.648342][ T11] ? __pfx_process_one_work+0x10/0x10 [ 45.648600][ T11] ? assign_work+0x16c/0x240 [ 45.648772][ T11] worker_thread+0x58c/0xce0 [ 45.649016][ T11] ? __pfx_worker_thread+0x10/0x10 [ 45.649177][ T11] kthread+0x28a/0x350 [ 45.649393][ T11] ? __pfx_kthread+0x10/0x10 [ 45.649567][ T11] ret_from_fork+0x31/0x70 [ 45.649740][ T11] ? __pfx_kthread+0x10/0x10 [ 45.650078][ T11] ret_from_fork_asm+0x1a/0x30 [ 45.650259][ T11] [ 47.461780][ T650] br1: left promiscuous mode [ 47.962369][ T656] br1: port 4(veth2) entered disabled state [ 48.025062][ T657] veth2: left allmulticast mode [ 48.025302][ T657] veth2: left promiscuous mode [ 48.025600][ T657] br1: port 4(veth2) entered disabled state [ 48.146760][ T659] br1: port 3(veth1) entered disabled state [ 48.212563][ T660] veth1: left allmulticast mode [ 48.212764][ T660] veth1: left promiscuous mode [ 48.213026][ T660] br1: port 3(veth1) entered disabled state [ 48.326228][ T662] vx4001: left allmulticast mode [ 48.327016][ T662] vx4001: left promiscuous mode [ 48.327317][ T662] br1: port 5(vx4001) entered disabled state [ 48.626929][ T666] vx20: left allmulticast mode [ 48.627203][ T666] vx20: left promiscuous mode [ 48.627547][ T666] br1: port 2(vx20) entered disabled state [ 48.901467][ T670] vx10: left allmulticast mode [ 48.901724][ T670] vx10: left promiscuous mode [ 48.902054][ T670] br1: port 1(vx10) entered disabled state