[ 10.701182][ T248] ip (248) used greatest stack depth: 23248 bytes left [ 15.146393][ T324] br1: port 1(vx10) entered blocking state [ 15.146841][ T324] br1: port 1(vx10) entered disabled state [ 15.147217][ T324] vx10: entered allmulticast mode [ 15.149213][ T324] vx10: entered promiscuous mode [ 15.150133][ T324] br1: port 1(vx10) entered blocking state [ 15.150494][ T324] br1: port 1(vx10) entered forwarding state [ 15.508605][ T329] br1: port 2(vx20) entered blocking state [ 15.508884][ T329] br1: port 2(vx20) entered disabled state [ 15.509148][ T329] vx20: entered allmulticast mode [ 15.511103][ T329] vx20: entered promiscuous mode [ 15.512548][ T329] br1: port 2(vx20) entered blocking state [ 15.512819][ T329] br1: port 2(vx20) entered forwarding state [ 15.703673][ T331] br1: port 3(veth1) entered blocking state [ 15.704080][ T331] br1: port 3(veth1) entered disabled state [ 15.704371][ T331] veth1: entered allmulticast mode [ 15.706287][ T331] veth1: entered promiscuous mode [ 15.780990][ T37] br1: port 3(veth1) entered blocking state [ 15.781426][ T37] br1: port 3(veth1) entered forwarding state [ 15.861119][ T333] br1: port 4(veth2) entered blocking state [ 15.861407][ T333] br1: port 4(veth2) entered disabled state [ 15.861670][ T333] veth2: entered allmulticast mode [ 15.863918][ T333] veth2: entered promiscuous mode [ 15.942198][ T37] br1: port 4(veth2) entered blocking state [ 15.942531][ T37] br1: port 4(veth2) entered forwarding state [ 16.178838][ T338] br1: port 5(vx4001) entered blocking state [ 16.179107][ T338] br1: port 5(vx4001) entered disabled state [ 16.179851][ T338] vx4001: entered allmulticast mode [ 16.181900][ T338] vx4001: entered promiscuous mode [ 16.182519][ T338] br1: port 5(vx4001) entered blocking state [ 16.182766][ T338] br1: port 5(vx4001) entered forwarding state [ 16.825831][ T347] 8021q: 802.1Q VLAN Support v1.8 [ 17.095509][ T350] br1: entered promiscuous mode [ 17.097944][ T350] br1: left promiscuous mode [ 17.111043][ T350] br1: entered promiscuous mode [ 22.296537][ T429] br1: port 1(vx10) entered blocking state [ 22.296923][ T429] br1: port 1(vx10) entered disabled state [ 22.297231][ T429] vx10: entered allmulticast mode [ 22.300346][ T429] vx10: entered promiscuous mode [ 22.301173][ T429] br1: port 1(vx10) entered blocking state [ 22.301455][ T429] br1: port 1(vx10) entered forwarding state [ 22.611629][ T433] br1: port 2(vx20) entered blocking state [ 22.611912][ T433] br1: port 2(vx20) entered disabled state [ 22.612180][ T433] vx20: entered allmulticast mode [ 22.614091][ T433] vx20: entered promiscuous mode [ 22.614603][ T433] br1: port 2(vx20) entered blocking state [ 22.614844][ T433] br1: port 2(vx20) entered forwarding state [ 22.913270][ T437] br1: port 3(vx4001) entered blocking state [ 22.913577][ T437] br1: port 3(vx4001) entered disabled state [ 22.913851][ T437] vx4001: entered allmulticast mode [ 22.915709][ T437] vx4001: entered promiscuous mode [ 22.916211][ T437] br1: port 3(vx4001) entered blocking state [ 22.916477][ T437] br1: port 3(vx4001) entered forwarding state [ 23.063127][ T439] br1: port 4(w1) entered blocking state [ 23.063446][ T439] br1: port 4(w1) entered disabled state [ 23.063668][ T439] w1: entered allmulticast mode [ 23.065802][ T439] w1: entered promiscuous mode [ 23.150092][ T137] br1: port 4(w1) entered blocking state [ 23.150353][ T137] br1: port 4(w1) entered forwarding state [ 23.328203][ T442] br1: port 5(w3) entered blocking state [ 23.328578][ T442] br1: port 5(w3) entered disabled state [ 23.328932][ T442] w3: entered allmulticast mode [ 23.332013][ T442] w3: entered promiscuous mode [ 23.400076][ T40] br1: port 5(w3) entered blocking state [ 23.400381][ T40] br1: port 5(w3) entered forwarding state [ 24.207348][ T452] br1: entered promiscuous mode [ 24.209483][ T452] br1: left promiscuous mode [ 24.218945][ T452] br1: entered promiscuous mode [ 25.048878][ C3] br1: received packet on vx20 with own address as source address (addr:00:00:5e:00:01:01, vlan:20) [ 25.155423][ C0] br1: received packet on vx20 with own address as source address (addr:00:00:5e:00:01:01, vlan:20) [ 25.156042][ C0] IPv6: vlan20-v: IPv6 duplicate address 2001:db8:2::3 used by 00:00:5e:00:01:01 detected! [ 25.156743][ C1] br1: received packet on vx10 with own address as source address (addr:00:00:5e:00:01:01, vlan:10) [ 25.157759][ C1] br1: received packet on vx10 with own address as source address (addr:00:00:5e:00:01:01, vlan:10) [ 25.158295][ C1] IPv6: vlan10-v: IPv6 duplicate address fe80::200:5eff:fe00:101 used by 00:00:5e:00:01:01 detected! [ 25.217068][ C1] br1: received packet on vx20 with own address as source address (addr:00:00:5e:00:01:01, vlan:20) [ 25.218023][ C1] br1: received packet on vx20 with own address as source address (addr:00:00:5e:00:01:01, vlan:20) [ 25.218564][ C1] IPv6: vlan20-v: IPv6 duplicate address fe80::200:5eff:fe00:101 used by 00:00:5e:00:01:01 detected! [ 25.219232][ C1] br1: received packet on vx10 with own address as source address (addr:00:00:5e:00:01:01, vlan:10) [ 42.404797][ T68] w3: left allmulticast mode [ 42.405126][ T68] w3: left promiscuous mode [ 42.405508][ T68] br1: port 5(w3) entered disabled state [ 42.408631][ T68] w1: left allmulticast mode [ 42.408845][ T68] w1: left promiscuous mode [ 42.409159][ T68] br1: port 4(w1) entered disabled state [ 42.411985][ T68] vx4001: left allmulticast mode [ 42.412318][ T68] vx4001: left promiscuous mode [ 42.412638][ T68] br1: port 3(vx4001) entered disabled state [ 42.422765][ T68] vx20: left allmulticast mode [ 42.422996][ T68] vx20: left promiscuous mode [ 42.423445][ T68] br1: port 2(vx20) entered disabled state [ 42.426928][ T68] vx10: left allmulticast mode [ 42.427121][ T68] vx10: left promiscuous mode [ 42.427551][ T68] br1: port 1(vx10) entered disabled state [ 42.453240][ T68] [ 42.453371][ T68] ============================================ [ 42.453603][ T68] WARNING: possible recursive locking detected [ 42.453820][ T68] 6.13.0-rc5-virtme #1 Not tainted [ 42.454011][ T68] -------------------------------------------- [ 42.454279][ T68] kworker/u16:1/68 is trying to acquire lock: [ 42.454479][ T68] ffffffffb57d6630 ((netdev_chain).rwsem){++++}-{4:4}, at: blocking_notifier_call_chain+0x50/0x90 [ 42.454768][ T68] [ 42.454768][ T68] but task is already holding lock: [ 42.454971][ T68] ffffffffb57d6630 ((netdev_chain).rwsem){++++}-{4:4}, at: blocking_notifier_call_chain+0x50/0x90 [ 42.455258][ T68] [ 42.455258][ T68] other info that might help us debug this: [ 42.455488][ T68] Possible unsafe locking scenario: [ 42.455488][ T68] [ 42.455693][ T68] CPU0 [ 42.455799][ T68] ---- [ 42.455903][ T68] lock((netdev_chain).rwsem); [ 42.456047][ T68] lock((netdev_chain).rwsem); [ 42.456190][ T68] [ 42.456190][ T68] *** DEADLOCK *** [ 42.456190][ T68] [ 42.456410][ T68] May be due to missing lock nesting notation [ 42.456410][ T68] [ 42.456619][ T68] 5 locks held by kworker/u16:1/68: [ 42.456772][ T68] #0: ffff8880010ad948 ((wq_completion)netns){+.+.}-{0:0}, at: process_one_work+0x7ec/0x16d0 [ 42.457178][ T68] #1: ffffc90000497da0 (net_cleanup_work){+.+.}-{0:0}, at: process_one_work+0xe0b/0x16d0 [ 42.457566][ T68] #2: ffffffffb57cc090 (pernet_ops_rwsem){++++}-{4:4}, at: cleanup_net+0xbc/0xa50 [ 42.457942][ T68] #3: ffffffffb57e7a48 (rtnl_mutex){+.+.}-{4:4}, at: cleanup_net+0x45e/0xa50 [ 42.458317][ T68] #4: ffffffffb57d6630 ((netdev_chain).rwsem){++++}-{4:4}, at: blocking_notifier_call_chain+0x50/0x90 [ 42.458734][ T68] [ 42.458734][ T68] stack backtrace: [ 42.459011][ T68] CPU: 3 UID: 0 PID: 68 Comm: kworker/u16:1 Not tainted 6.13.0-rc5-virtme #1 [ 42.459394][ T68] Hardware name: Bochs Bochs, BIOS Bochs 01/01/2011 [ 42.459675][ T68] Workqueue: netns cleanup_net [ 42.459904][ T68] Call Trace: [ 42.460082][ T68] [ 42.460201][ T68] dump_stack_lvl+0x82/0xd0 [ 42.460432][ T68] print_deadlock_bug+0x40a/0x650 [ 42.460662][ T68] validate_chain+0x5bf/0xae0 [ 42.460893][ T68] ? __pfx_validate_chain+0x10/0x10 [ 42.461126][ T68] ? hlock_class+0x4e/0x130 [ 42.461357][ T68] ? mark_lock+0x38/0x3e0 [ 42.461535][ T68] __lock_acquire+0xb9a/0x1680 [ 42.461759][ T68] ? spin_bug+0x1b1/0x1d0 [ 42.461930][ T68] ? debug_object_assert_init+0x2a9/0x370 [ 42.462152][ T68] lock_acquire.part.0+0xeb/0x330 [ 42.462392][ T68] ? blocking_notifier_call_chain+0x50/0x90 [ 42.462671][ T68] ? __pfx_lock_acquire.part.0+0x10/0x10 [ 42.462893][ T68] ? trace_lock_acquire+0x14c/0x1f0 [ 42.463116][ T68] ? lock_acquire+0x32/0xc0 [ 42.463335][ T68] ? blocking_notifier_call_chain+0x50/0x90 [ 42.463597][ T68] down_read+0x9f/0x340 [ 42.463721][ T68] ? blocking_notifier_call_chain+0x50/0x90 [ 42.463896][ T68] ? __pfx_down_read+0x10/0x10 [ 42.464038][ T68] blocking_notifier_call_chain+0x50/0x90 [ 42.464179][ T68] __dev_close_many+0xdf/0x2d0 [ 42.464321][ T68] ? __pfx___dev_close_many+0x10/0x10 [ 42.464483][ T68] dev_close_many+0x202/0x650 [ 42.464633][ T68] ? __lock_release+0x103/0x460 [ 42.464776][ T68] ? __pfx_dev_close_many+0x10/0x10 [ 42.464959][ T68] ? __pfx___lock_release+0x10/0x10 [ 42.465100][ T68] ? __pfx_lock_acquire.part.0+0x10/0x10 [ 42.465269][ T68] vlan_device_event+0x143b/0x2130 [8021q] [ 42.465467][ T68] ? net_generic+0xbb/0x1f0 [vxlan] [ 42.465625][ T68] ? __pfx_vlan_device_event+0x10/0x10 [8021q] [ 42.465810][ T68] ? __pfx_vxlan_netdevice_event+0x10/0x10 [vxlan] [ 42.466003][ T68] ? packet_notifier+0x3b0/0x810 [ 42.466150][ T68] notifier_call_chain+0xcd/0x150 [ 42.466299][ T68] blocking_notifier_call_chain+0x66/0x90 [ 42.466450][ T68] dev_close_many+0x2d8/0x650 [ 42.466591][ T68] ? __lock_release+0x103/0x460 [ 42.466734][ T68] ? __virt_addr_valid+0x22b/0x430 [ 42.466960][ T68] ? __pfx_dev_close_many+0x10/0x10 [ 42.467190][ T68] unregister_netdevice_many_notify+0x8e5/0x1580 [ 42.467408][ T68] ? mark_held_locks+0x9e/0xe0 [ 42.467548][ T68] ? __pfx_unregister_netdevice_many_notify+0x10/0x10 [ 42.467752][ T68] ? mutex_is_locked+0x1c/0x60 [ 42.468067][ T68] ? rtnl_is_locked+0x15/0x20 [ 42.468282][ T68] ? __pfx_unregister_netdevice_queue+0x10/0x10 [ 42.468817][ T68] ? mutex_is_locked+0x1c/0x60 [ 42.469035][ T68] ? nexthop_net_exit_batch_rtnl+0x83/0x210 [ 42.469811][ T68] cleanup_net+0x4cf/0xa50 [ 42.470039][ T68] ? __pfx_lock_acquire.part.0+0x10/0x10 [ 42.470259][ T68] ? __pfx_cleanup_net+0x10/0x10 [ 42.470761][ T68] ? trace_lock_acquire+0x14c/0x1f0 [ 42.470994][ T68] ? lock_acquire+0x32/0xc0 [ 42.471219][ T68] ? process_one_work+0xe0b/0x16d0 [ 42.471441][ T68] process_one_work+0xe55/0x16d0 [ 42.471667][ T68] ? __pfx___lock_release+0x10/0x10 [ 42.471891][ T68] ? __pfx_process_one_work+0x10/0x10 [ 42.472257][ T68] ? assign_work+0x16c/0x240 [ 42.472489][ T68] worker_thread+0x58c/0xce0 [ 42.472711][ T68] ? lockdep_hardirqs_on_prepare+0x275/0x410 [ 42.472998][ T68] ? __pfx_worker_thread+0x10/0x10 [ 42.473223][ T68] ? __pfx_worker_thread+0x10/0x10 [ 42.473442][ T68] kthread+0x28a/0x350 [ 42.473614][ T68] ? __pfx_kthread+0x10/0x10 [ 42.473983][ T68] ret_from_fork+0x31/0x70 [ 42.474202][ T68] ? __pfx_kthread+0x10/0x10 [ 42.474411][ T68] ret_from_fork_asm+0x1a/0x30 [ 42.474749][ T68] [ 44.211719][ T650] br1: left promiscuous mode [ 44.709132][ T656] br1: port 4(veth2) entered disabled state [ 44.765714][ T657] veth2: left allmulticast mode [ 44.765986][ T657] veth2: left promiscuous mode [ 44.767017][ T657] br1: port 4(veth2) entered disabled state [ 44.876258][ T659] br1: port 3(veth1) entered disabled state [ 44.927946][ T660] veth1: left allmulticast mode [ 44.928136][ T660] veth1: left promiscuous mode [ 44.928411][ T660] br1: port 3(veth1) entered disabled state [ 45.038625][ T662] vx4001: left allmulticast mode [ 45.038816][ T662] vx4001: left promiscuous mode [ 45.039076][ T662] br1: port 5(vx4001) entered disabled state [ 45.319285][ T666] vx20: left allmulticast mode [ 45.321287][ T666] vx20: left promiscuous mode [ 45.321791][ T666] br1: port 2(vx20) entered disabled state [ 45.592758][ T670] vx10: left allmulticast mode [ 45.592975][ T670] vx10: left promiscuous mode [ 45.593244][ T670] br1: port 1(vx10) entered disabled state