[ 10.863589][ T248] ip (248) used greatest stack depth: 23440 bytes left [ 15.763013][ T326] br1: port 1(vx10) entered blocking state [ 15.763500][ T326] br1: port 1(vx10) entered disabled state [ 15.763918][ T326] vx10: entered allmulticast mode [ 15.765897][ T326] vx10: entered promiscuous mode [ 15.766901][ T326] br1: port 1(vx10) entered blocking state [ 15.767456][ T326] br1: port 1(vx10) entered forwarding state [ 16.103120][ T331] br1: port 2(vx20) entered blocking state [ 16.103449][ T331] br1: port 2(vx20) entered disabled state [ 16.103733][ T331] vx20: entered allmulticast mode [ 16.106115][ T331] vx20: entered promiscuous mode [ 16.106975][ T331] br1: port 2(vx20) entered blocking state [ 16.107269][ T331] br1: port 2(vx20) entered forwarding state [ 16.289435][ T333] br1: port 3(veth1) entered blocking state [ 16.289777][ T333] br1: port 3(veth1) entered disabled state [ 16.290076][ T333] veth1: entered allmulticast mode [ 16.292471][ T333] veth1: entered promiscuous mode [ 16.381035][ T39] br1: port 3(veth1) entered blocking state [ 16.381552][ T39] br1: port 3(veth1) entered forwarding state [ 16.467784][ T335] br1: port 4(veth2) entered blocking state [ 16.468122][ T335] br1: port 4(veth2) entered disabled state [ 16.468431][ T335] veth2: entered allmulticast mode [ 16.470330][ T335] veth2: entered promiscuous mode [ 16.550014][ T40] br1: port 4(veth2) entered blocking state [ 16.550345][ T40] br1: port 4(veth2) entered forwarding state [ 16.847575][ T340] br1: port 5(vx4001) entered blocking state [ 16.847877][ T340] br1: port 5(vx4001) entered disabled state [ 16.848152][ T340] vx4001: entered allmulticast mode [ 16.850502][ T340] vx4001: entered promiscuous mode [ 16.851111][ T340] br1: port 5(vx4001) entered blocking state [ 16.851392][ T340] br1: port 5(vx4001) entered forwarding state [ 17.564050][ T349] 8021q: 802.1Q VLAN Support v1.8 [ 17.845464][ T352] br1: entered promiscuous mode [ 17.848167][ T352] br1: left promiscuous mode [ 17.855454][ T352] br1: entered promiscuous mode [ 23.699658][ T429] br1: port 1(vx10) entered blocking state [ 23.699949][ T429] br1: port 1(vx10) entered disabled state [ 23.700222][ T429] vx10: entered allmulticast mode [ 23.702193][ T429] vx10: entered promiscuous mode [ 23.703033][ T429] br1: port 1(vx10) entered blocking state [ 23.703281][ T429] br1: port 1(vx10) entered forwarding state [ 24.072724][ T433] br1: port 2(vx20) entered blocking state [ 24.073070][ T433] br1: port 2(vx20) entered disabled state [ 24.073393][ T433] vx20: entered allmulticast mode [ 24.075596][ T433] vx20: entered promiscuous mode [ 24.076132][ T433] br1: port 2(vx20) entered blocking state [ 24.076402][ T433] br1: port 2(vx20) entered forwarding state [ 24.446234][ T437] br1: port 3(vx4001) entered blocking state [ 24.446680][ T437] br1: port 3(vx4001) entered disabled state [ 24.447088][ T437] vx4001: entered allmulticast mode [ 24.449934][ T437] vx4001: entered promiscuous mode [ 24.450660][ T437] br1: port 3(vx4001) entered blocking state [ 24.450982][ T437] br1: port 3(vx4001) entered forwarding state [ 24.614784][ T439] br1: port 4(w1) entered blocking state [ 24.615282][ T439] br1: port 4(w1) entered disabled state [ 24.616078][ T439] w1: entered allmulticast mode [ 24.617978][ T439] w1: entered promiscuous mode [ 24.700337][ T45] br1: port 4(w1) entered blocking state [ 24.700745][ T45] br1: port 4(w1) entered forwarding state [ 24.867075][ T442] br1: port 5(w3) entered blocking state [ 24.868189][ T442] br1: port 5(w3) entered disabled state [ 24.868437][ T442] w3: entered allmulticast mode [ 24.870335][ T442] w3: entered promiscuous mode [ 24.951621][ T39] br1: port 5(w3) entered blocking state [ 24.952005][ T39] br1: port 5(w3) entered forwarding state [ 25.762296][ T452] br1: entered promiscuous mode [ 25.764696][ T452] br1: left promiscuous mode [ 25.793072][ T452] br1: entered promiscuous mode [ 26.692911][ C1] br1: received packet on vx10 with own address as source address (addr:00:00:5e:00:01:01, vlan:10) [ 26.696432][ C2] br1: received packet on vx10 with own address as source address (addr:00:00:5e:00:01:01, vlan:10) [ 26.696931][ C2] IPv6: vlan10-v: IPv6 duplicate address 2001:db8:1::3 used by 00:00:5e:00:01:01 detected! [ 26.820156][ C1] br1: received packet on vx20 with own address as source address (addr:00:00:5e:00:01:01, vlan:20) [ 26.884103][ C1] br1: received packet on vx20 with own address as source address (addr:00:00:5e:00:01:01, vlan:20) [ 26.885757][ C1] br1: received packet on vx20 with own address as source address (addr:00:00:5e:00:01:01, vlan:20) [ 26.886469][ C1] IPv6: vlan20-v: IPv6 duplicate address 2001:db8:2::3 used by 00:00:5e:00:01:01 detected! [ 27.140914][ C2] br1: received packet on vx20 with own address as source address (addr:00:00:5e:00:01:01, vlan:20) [ 27.141866][ C2] br1: received packet on vx20 with own address as source address (addr:00:00:5e:00:01:01, vlan:20) [ 27.142378][ C2] IPv6: vlan20-v: IPv6 duplicate address fe80::200:5eff:fe00:101 used by 00:00:5e:00:01:01 detected! [ 44.696963][ T67] w3: left allmulticast mode [ 44.697322][ T67] w3: left promiscuous mode [ 44.697712][ T67] br1: port 5(w3) entered disabled state [ 44.709240][ T67] w1: left allmulticast mode [ 44.709515][ T67] w1: left promiscuous mode [ 44.709956][ T67] br1: port 4(w1) entered disabled state [ 44.711473][ T67] vx4001: left allmulticast mode [ 44.711781][ T67] vx4001: left promiscuous mode [ 44.712115][ T67] br1: port 3(vx4001) entered disabled state [ 44.713812][ T67] vx20: left allmulticast mode [ 44.714020][ T67] vx20: left promiscuous mode [ 44.714327][ T67] br1: port 2(vx20) entered disabled state [ 44.715861][ T67] vx10: left allmulticast mode [ 44.716087][ T67] vx10: left promiscuous mode [ 44.716423][ T67] br1: port 1(vx10) entered disabled state [ 44.747570][ T67] [ 44.747714][ T67] ============================================ [ 44.747915][ T67] WARNING: possible recursive locking detected [ 44.748130][ T67] 6.13.0-rc5-virtme #1 Not tainted [ 44.748268][ T67] -------------------------------------------- [ 44.748445][ T67] kworker/u16:1/67 is trying to acquire lock: [ 44.748633][ T67] ffffffffae1d6630 ((netdev_chain).rwsem){++++}-{4:4}, at: blocking_notifier_call_chain+0x50/0x90 [ 44.748934][ T67] [ 44.748934][ T67] but task is already holding lock: [ 44.749178][ T67] ffffffffae1d6630 ((netdev_chain).rwsem){++++}-{4:4}, at: blocking_notifier_call_chain+0x50/0x90 [ 44.749468][ T67] [ 44.749468][ T67] other info that might help us debug this: [ 44.749681][ T67] Possible unsafe locking scenario: [ 44.749681][ T67] [ 44.749887][ T67] CPU0 [ 44.750015][ T67] ---- [ 44.750122][ T67] lock((netdev_chain).rwsem); [ 44.750267][ T67] lock((netdev_chain).rwsem); [ 44.750404][ T67] [ 44.750404][ T67] *** DEADLOCK *** [ 44.750404][ T67] [ 44.750681][ T67] May be due to missing lock nesting notation [ 44.750681][ T67] [ 44.750884][ T67] 5 locks held by kworker/u16:1/67: [ 44.751048][ T67] #0: ffff8880010ad948 ((wq_completion)netns){+.+.}-{0:0}, at: process_one_work+0x7ec/0x16d0 [ 44.751345][ T67] #1: ffffc90000487da0 (net_cleanup_work){+.+.}-{0:0}, at: process_one_work+0xe0b/0x16d0 [ 44.751596][ T67] #2: ffffffffae1cc090 (pernet_ops_rwsem){++++}-{4:4}, at: cleanup_net+0xbc/0xa50 [ 44.751927][ T67] #3: ffffffffae1e7a48 (rtnl_mutex){+.+.}-{4:4}, at: cleanup_net+0x45e/0xa50 [ 44.752177][ T67] #4: ffffffffae1d6630 ((netdev_chain).rwsem){++++}-{4:4}, at: blocking_notifier_call_chain+0x50/0x90 [ 44.752542][ T67] [ 44.752542][ T67] stack backtrace: [ 44.752716][ T67] CPU: 2 UID: 0 PID: 67 Comm: kworker/u16:1 Not tainted 6.13.0-rc5-virtme #1 [ 44.752966][ T67] Hardware name: Bochs Bochs, BIOS Bochs 01/01/2011 [ 44.753220][ T67] Workqueue: netns cleanup_net [ 44.753368][ T67] Call Trace: [ 44.753554][ T67] [ 44.753628][ T67] dump_stack_lvl+0x82/0xd0 [ 44.753776][ T67] print_deadlock_bug+0x40a/0x650 [ 44.753996][ T67] validate_chain+0x5bf/0xae0 [ 44.754139][ T67] ? __pfx_validate_chain+0x10/0x10 [ 44.754281][ T67] ? hlock_class+0x4e/0x130 [ 44.754420][ T67] ? mark_lock+0x38/0x3e0 [ 44.754528][ T67] __lock_acquire+0xb9a/0x1680 [ 44.754752][ T67] ? spin_bug+0x1b1/0x1d0 [ 44.754861][ T67] ? debug_object_assert_init+0x2a9/0x370 [ 44.755004][ T67] lock_acquire.part.0+0xeb/0x330 [ 44.755222][ T67] ? blocking_notifier_call_chain+0x50/0x90 [ 44.755404][ T67] ? __pfx_lock_acquire.part.0+0x10/0x10 [ 44.755627][ T67] ? trace_lock_acquire+0x14c/0x1f0 [ 44.755777][ T67] ? lock_acquire+0x32/0xc0 [ 44.755933][ T67] ? blocking_notifier_call_chain+0x50/0x90 [ 44.756127][ T67] down_read+0x9f/0x340 [ 44.756239][ T67] ? blocking_notifier_call_chain+0x50/0x90 [ 44.756413][ T67] ? __pfx_down_read+0x10/0x10 [ 44.756632][ T67] blocking_notifier_call_chain+0x50/0x90 [ 44.756773][ T67] __dev_close_many+0xdf/0x2d0 [ 44.756918][ T67] ? __pfx___dev_close_many+0x10/0x10 [ 44.757064][ T67] dev_close_many+0x202/0x650 [ 44.757282][ T67] ? __lock_release+0x103/0x460 [ 44.757423][ T67] ? __pfx_dev_close_many+0x10/0x10 [ 44.757562][ T67] ? __pfx___lock_release+0x10/0x10 [ 44.757701][ T67] ? __pfx_lock_acquire.part.0+0x10/0x10 [ 44.757847][ T67] vlan_device_event+0x143b/0x2130 [8021q] [ 44.758028][ T67] ? net_generic+0xbb/0x1f0 [vxlan] [ 44.758184][ T67] ? __pfx_vlan_device_event+0x10/0x10 [8021q] [ 44.758363][ T67] ? __pfx_vxlan_netdevice_event+0x10/0x10 [vxlan] [ 44.758546][ T67] ? packet_notifier+0x3b0/0x810 [ 44.758695][ T67] notifier_call_chain+0xcd/0x150 [ 44.758842][ T67] blocking_notifier_call_chain+0x66/0x90 [ 44.758984][ T67] dev_close_many+0x2d8/0x650 [ 44.759126][ T67] ? __lock_release+0x103/0x460 [ 44.759272][ T67] ? __virt_addr_valid+0x22b/0x430 [ 44.759417][ T67] ? __pfx_dev_close_many+0x10/0x10 [ 44.759640][ T67] unregister_netdevice_many_notify+0x8e5/0x1580 [ 44.759817][ T67] ? mark_held_locks+0x9e/0xe0 [ 44.759957][ T67] ? __pfx_unregister_netdevice_many_notify+0x10/0x10 [ 44.760131][ T67] ? mutex_is_locked+0x1c/0x60 [ 44.760357][ T67] ? rtnl_is_locked+0x15/0x20 [ 44.760499][ T67] ? __pfx_unregister_netdevice_queue+0x10/0x10 [ 44.760681][ T67] ? mutex_is_locked+0x1c/0x60 [ 44.760909][ T67] ? nexthop_net_exit_batch_rtnl+0x83/0x210 [ 44.761095][ T67] cleanup_net+0x4cf/0xa50 [ 44.761241][ T67] ? __pfx_lock_acquire.part.0+0x10/0x10 [ 44.761382][ T67] ? __pfx_cleanup_net+0x10/0x10 [ 44.761600][ T67] ? trace_lock_acquire+0x14c/0x1f0 [ 44.761748][ T67] ? lock_acquire+0x32/0xc0 [ 44.761889][ T67] ? process_one_work+0xe0b/0x16d0 [ 44.762161][ T67] process_one_work+0xe55/0x16d0 [ 44.762386][ T67] ? __pfx___lock_release+0x10/0x10 [ 44.762530][ T67] ? __pfx_process_one_work+0x10/0x10 [ 44.762671][ T67] ? assign_work+0x16c/0x240 [ 44.762813][ T67] worker_thread+0x58c/0xce0 [ 44.763041][ T67] ? lockdep_hardirqs_on_prepare+0x275/0x410 [ 44.763221][ T67] ? __pfx_worker_thread+0x10/0x10 [ 44.763363][ T67] ? __pfx_worker_thread+0x10/0x10 [ 44.763504][ T67] kthread+0x28a/0x350 [ 44.763611][ T67] ? __pfx_kthread+0x10/0x10 [ 44.763762][ T67] ret_from_fork+0x31/0x70 [ 44.763922][ T67] ? __pfx_kthread+0x10/0x10 [ 44.764081][ T67] ret_from_fork_asm+0x1a/0x30 [ 44.764319][ T67] [ 46.634731][ T650] br1: left promiscuous mode [ 47.129512][ T656] br1: port 4(veth2) entered disabled state [ 47.196747][ T657] veth2: left allmulticast mode [ 47.197038][ T657] veth2: left promiscuous mode [ 47.197435][ T657] br1: port 4(veth2) entered disabled state [ 47.343268][ T659] br1: port 3(veth1) entered disabled state [ 47.400867][ T660] veth1: left allmulticast mode [ 47.401106][ T660] veth1: left promiscuous mode [ 47.401389][ T660] br1: port 3(veth1) entered disabled state [ 47.520893][ T662] vx4001: left allmulticast mode [ 47.521118][ T662] vx4001: left promiscuous mode [ 47.521397][ T662] br1: port 5(vx4001) entered disabled state [ 47.798753][ T666] vx20: left allmulticast mode [ 47.798992][ T666] vx20: left promiscuous mode [ 47.799283][ T666] br1: port 2(vx20) entered disabled state [ 48.074268][ T670] vx10: left allmulticast mode [ 48.074976][ T670] vx10: left promiscuous mode [ 48.075276][ T670] br1: port 1(vx10) entered disabled state