[ 10.328571][ T249] ip (249) used greatest stack depth: 24328 bytes left [ 10.654280][ T258] ip (258) used greatest stack depth: 23632 bytes left [ 14.859521][ T326] br1: port 1(vx10) entered blocking state [ 14.860030][ T326] br1: port 1(vx10) entered disabled state [ 14.860460][ T326] vx10: entered allmulticast mode [ 14.862442][ T326] vx10: entered promiscuous mode [ 14.863485][ T326] br1: port 1(vx10) entered blocking state [ 14.863867][ T326] br1: port 1(vx10) entered forwarding state [ 15.213783][ T331] br1: port 2(vx20) entered blocking state [ 15.214078][ T331] br1: port 2(vx20) entered disabled state [ 15.214382][ T331] vx20: entered allmulticast mode [ 15.216902][ T331] vx20: entered promiscuous mode [ 15.217746][ T331] br1: port 2(vx20) entered blocking state [ 15.217996][ T331] br1: port 2(vx20) entered forwarding state [ 15.394225][ T333] br1: port 3(veth1) entered blocking state [ 15.394524][ T333] br1: port 3(veth1) entered disabled state [ 15.394799][ T333] veth1: entered allmulticast mode [ 15.397044][ T333] veth1: entered promiscuous mode [ 15.483044][ T37] br1: port 3(veth1) entered blocking state [ 15.483479][ T37] br1: port 3(veth1) entered forwarding state [ 15.556514][ T335] br1: port 4(veth2) entered blocking state [ 15.556823][ T335] br1: port 4(veth2) entered disabled state [ 15.557115][ T335] veth2: entered allmulticast mode [ 15.559486][ T335] veth2: entered promiscuous mode [ 15.645216][ T40] br1: port 4(veth2) entered blocking state [ 15.645532][ T40] br1: port 4(veth2) entered forwarding state [ 15.892167][ T340] br1: port 5(vx4001) entered blocking state [ 15.892479][ T340] br1: port 5(vx4001) entered disabled state [ 15.892752][ T340] vx4001: entered allmulticast mode [ 15.894675][ T340] vx4001: entered promiscuous mode [ 15.895190][ T340] br1: port 5(vx4001) entered blocking state [ 15.895440][ T340] br1: port 5(vx4001) entered forwarding state [ 16.592818][ T349] 8021q: 802.1Q VLAN Support v1.8 [ 16.869999][ T352] br1: entered promiscuous mode [ 16.872659][ T352] br1: left promiscuous mode [ 16.884009][ T352] br1: entered promiscuous mode [ 22.519992][ T431] br1: port 1(vx10) entered blocking state [ 22.520417][ T431] br1: port 1(vx10) entered disabled state [ 22.520708][ T431] vx10: entered allmulticast mode [ 22.522708][ T431] vx10: entered promiscuous mode [ 22.523613][ T431] br1: port 1(vx10) entered blocking state [ 22.523881][ T431] br1: port 1(vx10) entered forwarding state [ 22.879492][ T435] br1: port 2(vx20) entered blocking state [ 22.879888][ T435] br1: port 2(vx20) entered disabled state [ 22.880184][ T435] vx20: entered allmulticast mode [ 22.882567][ T435] vx20: entered promiscuous mode [ 22.883122][ T435] br1: port 2(vx20) entered blocking state [ 22.883398][ T435] br1: port 2(vx20) entered forwarding state [ 23.230306][ T439] br1: port 3(vx4001) entered blocking state [ 23.230748][ T439] br1: port 3(vx4001) entered disabled state [ 23.231199][ T439] vx4001: entered allmulticast mode [ 23.245144][ T439] vx4001: entered promiscuous mode [ 23.246132][ T439] br1: port 3(vx4001) entered blocking state [ 23.246578][ T439] br1: port 3(vx4001) entered forwarding state [ 23.436154][ T441] br1: port 4(w1) entered blocking state [ 23.436687][ T441] br1: port 4(w1) entered disabled state [ 23.436944][ T441] w1: entered allmulticast mode [ 23.438975][ T441] w1: entered promiscuous mode [ 23.554505][ T39] br1: port 4(w1) entered blocking state [ 23.554790][ T39] br1: port 4(w1) entered forwarding state [ 23.746505][ T444] br1: port 5(w3) entered blocking state [ 23.746813][ T444] br1: port 5(w3) entered disabled state [ 23.748922][ T444] w3: entered allmulticast mode [ 23.750919][ T444] w3: entered promiscuous mode [ 23.835669][ T37] br1: port 5(w3) entered blocking state [ 23.835997][ T37] br1: port 5(w3) entered forwarding state [ 24.631840][ T454] br1: entered promiscuous mode [ 24.634633][ T454] br1: left promiscuous mode [ 24.657104][ T454] br1: entered promiscuous mode [ 25.357912][ C1] br1: received packet on vx10 with own address as source address (addr:00:00:5e:00:01:01, vlan:10) [ 25.361562][ C1] IPv6: vlan10-v: IPv6 duplicate address fe80::200:5eff:fe00:101 used by 00:00:5e:00:01:01 detected! [ 25.405782][ C1] br1: received packet on vx20 with own address as source address (addr:00:00:5e:00:01:01, vlan:20) [ 25.549835][ C1] br1: received packet on vx10 with own address as source address (addr:00:00:5e:00:01:01, vlan:10) [ 25.589751][ C1] br1: received packet on vx20 with own address as source address (addr:00:00:5e:00:01:01, vlan:20) [ 25.591142][ C1] IPv6: vlan20-v: IPv6 duplicate address 2001:db8:2::3 used by 00:00:5e:00:01:01 detected! [ 26.356448][ C0] br1: received packet on vx10 with own address as source address (addr:00:00:5e:00:01:01, vlan:10) [ 26.357507][ C0] br1: received packet on vx20 with own address as source address (addr:00:00:5e:00:01:01, vlan:20) [ 26.382586][ C1] br1: received packet on vx20 with own address as source address (addr:00:00:5e:00:01:01, vlan:20) [ 26.384650][ C1] br1: received packet on vx20 with own address as source address (addr:00:00:5e:00:01:01, vlan:20) [ 43.235101][ T11] w3: left allmulticast mode [ 43.235775][ T11] w3: left promiscuous mode [ 43.236166][ T11] br1: port 5(w3) entered disabled state [ 43.239075][ T11] w1: left allmulticast mode [ 43.239355][ T11] w1: left promiscuous mode [ 43.239656][ T11] br1: port 4(w1) entered disabled state [ 43.241451][ T11] vx4001: left allmulticast mode [ 43.241667][ T11] vx4001: left promiscuous mode [ 43.242045][ T11] br1: port 3(vx4001) entered disabled state [ 43.244279][ T11] vx20: left allmulticast mode [ 43.244467][ T11] vx20: left promiscuous mode [ 43.244765][ T11] br1: port 2(vx20) entered disabled state [ 43.246251][ T11] vx10: left allmulticast mode [ 43.246485][ T11] vx10: left promiscuous mode [ 43.246781][ T11] br1: port 1(vx10) entered disabled state [ 43.278240][ T11] [ 43.278363][ T11] ============================================ [ 43.278544][ T11] WARNING: possible recursive locking detected [ 43.278724][ T11] 6.13.0-rc5-virtme #1 Not tainted [ 43.278868][ T11] -------------------------------------------- [ 43.279044][ T11] kworker/u16:0/11 is trying to acquire lock: [ 43.279241][ T11] ffffffff901d6630 ((netdev_chain).rwsem){++++}-{4:4}, at: blocking_notifier_call_chain+0x50/0x90 [ 43.279533][ T11] [ 43.279533][ T11] but task is already holding lock: [ 43.279737][ T11] ffffffff901d6630 ((netdev_chain).rwsem){++++}-{4:4}, at: blocking_notifier_call_chain+0x50/0x90 [ 43.280014][ T11] [ 43.280014][ T11] other info that might help us debug this: [ 43.280222][ T11] Possible unsafe locking scenario: [ 43.280222][ T11] [ 43.280436][ T11] CPU0 [ 43.280541][ T11] ---- [ 43.280647][ T11] lock((netdev_chain).rwsem); [ 43.280792][ T11] lock((netdev_chain).rwsem); [ 43.280930][ T11] [ 43.280930][ T11] *** DEADLOCK *** [ 43.280930][ T11] [ 43.281133][ T11] May be due to missing lock nesting notation [ 43.281133][ T11] [ 43.281341][ T11] 5 locks held by kworker/u16:0/11: [ 43.281483][ T11] #0: ffff8880010ad948 ((wq_completion)netns){+.+.}-{0:0}, at: process_one_work+0x7ec/0x16d0 [ 43.281770][ T11] #1: ffffc900000b7da0 (net_cleanup_work){+.+.}-{0:0}, at: process_one_work+0xe0b/0x16d0 [ 43.282020][ T11] #2: ffffffff901cc090 (pernet_ops_rwsem){++++}-{4:4}, at: cleanup_net+0xbc/0xa50 [ 43.282272][ T11] #3: ffffffff901e7a48 (rtnl_mutex){+.+.}-{4:4}, at: cleanup_net+0x45e/0xa50 [ 43.282521][ T11] #4: ffffffff901d6630 ((netdev_chain).rwsem){++++}-{4:4}, at: blocking_notifier_call_chain+0x50/0x90 [ 43.282815][ T11] [ 43.282815][ T11] stack backtrace: [ 43.282987][ T11] CPU: 1 UID: 0 PID: 11 Comm: kworker/u16:0 Not tainted 6.13.0-rc5-virtme #1 [ 43.283235][ T11] Hardware name: Bochs Bochs, BIOS Bochs 01/01/2011 [ 43.283415][ T11] Workqueue: netns cleanup_net [ 43.283563][ T11] Call Trace: [ 43.283674][ T11] [ 43.283751][ T11] dump_stack_lvl+0x82/0xd0 [ 43.283900][ T11] print_deadlock_bug+0x40a/0x650 [ 43.284043][ T11] validate_chain+0x5bf/0xae0 [ 43.284185][ T11] ? __pfx_validate_chain+0x10/0x10 [ 43.284336][ T11] ? hlock_class+0x4e/0x130 [ 43.284473][ T11] ? mark_lock+0x38/0x3e0 [ 43.284580][ T11] __lock_acquire+0xb9a/0x1680 [ 43.284720][ T11] ? spin_bug+0x1b1/0x1d0 [ 43.284829][ T11] ? debug_object_assert_init+0x2a9/0x370 [ 43.284971][ T11] lock_acquire.part.0+0xeb/0x330 [ 43.285109][ T11] ? blocking_notifier_call_chain+0x50/0x90 [ 43.285288][ T11] ? __pfx_lock_acquire.part.0+0x10/0x10 [ 43.285434][ T11] ? trace_lock_acquire+0x14c/0x1f0 [ 43.285573][ T11] ? lock_acquire+0x32/0xc0 [ 43.285713][ T11] ? blocking_notifier_call_chain+0x50/0x90 [ 43.285934][ T11] down_read+0x9f/0x340 [ 43.286056][ T11] ? blocking_notifier_call_chain+0x50/0x90 [ 43.286228][ T11] ? __pfx_down_read+0x10/0x10 [ 43.286379][ T11] blocking_notifier_call_chain+0x50/0x90 [ 43.286520][ T11] __dev_close_many+0xdf/0x2d0 [ 43.286660][ T11] ? __pfx___dev_close_many+0x10/0x10 [ 43.286802][ T11] dev_close_many+0x202/0x650 [ 43.286941][ T11] ? __lock_release+0x103/0x460 [ 43.287082][ T11] ? __pfx_dev_close_many+0x10/0x10 [ 43.287221][ T11] ? __pfx___lock_release+0x10/0x10 [ 43.287365][ T11] ? __pfx_lock_acquire.part.0+0x10/0x10 [ 43.287506][ T11] vlan_device_event+0x143b/0x2130 [8021q] [ 43.287685][ T11] ? net_generic+0xbb/0x1f0 [vxlan] [ 43.287839][ T11] ? __pfx_vlan_device_event+0x10/0x10 [8021q] [ 43.288014][ T11] ? __pfx_vxlan_netdevice_event+0x10/0x10 [vxlan] [ 43.288201][ T11] ? packet_notifier+0x3b0/0x810 [ 43.288354][ T11] notifier_call_chain+0xcd/0x150 [ 43.288496][ T11] blocking_notifier_call_chain+0x66/0x90 [ 43.288716][ T11] dev_close_many+0x2d8/0x650 [ 43.288857][ T11] ? __lock_release+0x103/0x460 [ 43.288997][ T11] ? __virt_addr_valid+0x22b/0x430 [ 43.289140][ T11] ? __pfx_dev_close_many+0x10/0x10 [ 43.289360][ T11] unregister_netdevice_many_notify+0x8e5/0x1580 [ 43.289548][ T11] ? mark_held_locks+0x9e/0xe0 [ 43.289690][ T11] ? __pfx_unregister_netdevice_many_notify+0x10/0x10 [ 43.289939][ T11] ? mutex_is_locked+0x1c/0x60 [ 43.290081][ T11] ? rtnl_is_locked+0x15/0x20 [ 43.290224][ T11] ? __pfx_unregister_netdevice_queue+0x10/0x10 [ 43.290399][ T11] ? mutex_is_locked+0x1c/0x60 [ 43.290618][ T11] ? nexthop_net_exit_batch_rtnl+0x83/0x210 [ 43.290797][ T11] cleanup_net+0x4cf/0xa50 [ 43.290937][ T11] ? __pfx_lock_acquire.part.0+0x10/0x10 [ 43.291076][ T11] ? __pfx_cleanup_net+0x10/0x10 [ 43.291305][ T11] ? trace_lock_acquire+0x14c/0x1f0 [ 43.291444][ T11] ? lock_acquire+0x32/0xc0 [ 43.291582][ T11] ? process_one_work+0xe0b/0x16d0 [ 43.291724][ T11] process_one_work+0xe55/0x16d0 [ 43.291942][ T11] ? __pfx___lock_release+0x10/0x10 [ 43.292086][ T11] ? __pfx_process_one_work+0x10/0x10 [ 43.292230][ T11] ? assign_work+0x16c/0x240 [ 43.292372][ T11] worker_thread+0x58c/0xce0 [ 43.292593][ T11] ? __pfx_worker_thread+0x10/0x10 [ 43.292731][ T11] kthread+0x28a/0x350 [ 43.292857][ T11] ? __pfx_kthread+0x10/0x10 [ 43.292998][ T11] ret_from_fork+0x31/0x70 [ 43.293223][ T11] ? __pfx_kthread+0x10/0x10 [ 43.293362][ T11] ret_from_fork_asm+0x1a/0x30 [ 43.293510][ T11] [ 45.091920][ T652] br1: left promiscuous mode [ 45.571607][ T658] br1: port 4(veth2) entered disabled state [ 45.631034][ T659] veth2: left allmulticast mode [ 45.631237][ T659] veth2: left promiscuous mode [ 45.631542][ T659] br1: port 4(veth2) entered disabled state [ 45.737374][ T661] br1: port 3(veth1) entered disabled state [ 45.800039][ T662] veth1: left allmulticast mode [ 45.800238][ T662] veth1: left promiscuous mode [ 45.800711][ T662] br1: port 3(veth1) entered disabled state [ 45.923748][ T664] vx4001: left allmulticast mode [ 45.923951][ T664] vx4001: left promiscuous mode [ 45.924221][ T664] br1: port 5(vx4001) entered disabled state [ 46.183473][ T668] vx20: left allmulticast mode [ 46.183724][ T668] vx20: left promiscuous mode [ 46.184027][ T668] br1: port 2(vx20) entered disabled state [ 46.446669][ T672] vx10: left allmulticast mode [ 46.446866][ T672] vx10: left promiscuous mode [ 46.447129][ T672] br1: port 1(vx10) entered disabled state