[ 13.435638][ T291] ip (291) used greatest stack depth: 24256 bytes left [ 16.211462][ T328] br1: port 1(vx10) entered blocking state [ 16.211973][ T328] br1: port 1(vx10) entered disabled state [ 16.213181][ T328] vx10: entered allmulticast mode [ 16.215266][ T328] vx10: entered promiscuous mode [ 16.216325][ T328] br1: port 1(vx10) entered blocking state [ 16.216696][ T328] br1: port 1(vx10) entered forwarding state [ 16.597676][ T333] br1: port 2(vx20) entered blocking state [ 16.598004][ T333] br1: port 2(vx20) entered disabled state [ 16.598312][ T333] vx20: entered allmulticast mode [ 16.600329][ T333] vx20: entered promiscuous mode [ 16.601746][ T333] br1: port 2(vx20) entered blocking state [ 16.602028][ T333] br1: port 2(vx20) entered forwarding state [ 16.770907][ T335] br1: port 3(veth1) entered blocking state [ 16.771621][ T335] br1: port 3(veth1) entered disabled state [ 16.771935][ T335] veth1: entered allmulticast mode [ 16.774164][ T335] veth1: entered promiscuous mode [ 16.850354][ T39] br1: port 3(veth1) entered blocking state [ 16.850818][ T39] br1: port 3(veth1) entered forwarding state [ 17.005079][ T338] br1: port 4(veth2) entered blocking state [ 17.005688][ T338] br1: port 4(veth2) entered disabled state [ 17.005969][ T338] veth2: entered allmulticast mode [ 17.008217][ T338] veth2: entered promiscuous mode [ 17.102158][ T39] br1: port 4(veth2) entered blocking state [ 17.102493][ T39] br1: port 4(veth2) entered forwarding state [ 17.793243][ T348] 8021q: 802.1Q VLAN Support v1.8 [ 18.078923][ T351] br1: entered promiscuous mode [ 18.082953][ T351] br1: left promiscuous mode [ 18.094308][ T351] br1: entered promiscuous mode [ 18.104122][ T351] ip (351) used greatest stack depth: 24016 bytes left [ 23.800113][ T430] br1: port 1(vx10) entered blocking state [ 23.801520][ T430] br1: port 1(vx10) entered disabled state [ 23.801799][ T430] vx10: entered allmulticast mode [ 23.803746][ T430] vx10: entered promiscuous mode [ 23.804574][ T430] br1: port 1(vx10) entered blocking state [ 23.804847][ T430] br1: port 1(vx10) entered forwarding state [ 24.130374][ T434] br1: port 2(vx20) entered blocking state [ 24.130806][ T434] br1: port 2(vx20) entered disabled state [ 24.131208][ T434] vx20: entered allmulticast mode [ 24.134729][ T434] vx20: entered promiscuous mode [ 24.135686][ T434] br1: port 2(vx20) entered blocking state [ 24.135980][ T434] br1: port 2(vx20) entered forwarding state [ 24.314260][ T436] br1: port 3(w1) entered blocking state [ 24.314672][ T436] br1: port 3(w1) entered disabled state [ 24.314949][ T436] w1: entered allmulticast mode [ 24.316978][ T436] w1: entered promiscuous mode [ 24.384729][ T40] br1: port 3(w1) entered blocking state [ 24.385006][ T40] br1: port 3(w1) entered forwarding state [ 24.534689][ T439] br1: port 4(w3) entered blocking state [ 24.534969][ T439] br1: port 4(w3) entered disabled state [ 24.535215][ T439] w3: entered allmulticast mode [ 24.538144][ T439] w3: entered promiscuous mode [ 24.620525][ T45] br1: port 4(w3) entered blocking state [ 24.620753][ T45] br1: port 4(w3) entered forwarding state [ 25.523369][ T449] br1: entered promiscuous mode [ 25.525873][ T449] br1: left promiscuous mode [ 25.542834][ T449] br1: entered promiscuous mode [ 26.285168][ C0] br1: received packet on vx10 with own address as source address (addr:00:00:5e:00:01:01, vlan:10) [ 26.797309][ C0] br1: received packet on vx20 with own address as source address (addr:00:00:5e:00:01:01, vlan:20) [ 26.861165][ C1] br1: received packet on vx10 with own address as source address (addr:00:00:5e:00:01:01, vlan:10) [ 26.862398][ C1] br1: received packet on vx10 with own address as source address (addr:00:00:5e:00:01:01, vlan:10) [ 27.118613][ C1] br1: received packet on vx20 with own address as source address (addr:00:00:5e:00:01:01, vlan:20) [ 27.120129][ C1] br1: received packet on vx20 with own address as source address (addr:00:00:5e:00:01:01, vlan:20) [ 27.356842][ C1] br1: received packet on vx10 with own address as source address (addr:00:00:5e:00:01:01, vlan:10) [ 28.141798][ C1] br1: received packet on vx20 with own address as source address (addr:00:00:5e:00:01:01, vlan:20) [ 31.212947][ C1] br1: received packet on vx10 with own address as source address (addr:00:00:5e:00:01:01, vlan:10) [ 31.724981][ C1] br1: received packet on vx20 with own address as source address (addr:00:00:5e:00:01:01, vlan:20) [ 40.429097][ C0] br1: received packet on vx20 with own address as source address (addr:00:00:5e:00:01:01, vlan:20) [ 40.429622][ C0] br1: received packet on vx10 with own address as source address (addr:00:00:5e:00:01:01, vlan:10) [ 50.650488][ T673] GACT probability NOT on [ 55.739442][ T781] ip (781) used greatest stack depth: 23064 bytes left [ 55.827656][ T67] w3: left allmulticast mode [ 55.828003][ T67] w3: left promiscuous mode [ 55.828539][ T67] br1: port 4(w3) entered disabled state [ 55.831156][ T67] w1: left allmulticast mode [ 55.832172][ T67] w1: left promiscuous mode [ 55.832521][ T67] br1: port 3(w1) entered disabled state [ 55.833909][ T67] vx20: left allmulticast mode [ 55.834118][ T67] vx20: left promiscuous mode [ 55.834456][ T67] br1: port 2(vx20) entered disabled state [ 55.839220][ T67] vx10: left allmulticast mode [ 55.839468][ T67] vx10: left promiscuous mode [ 55.839787][ T67] br1: port 1(vx10) entered disabled state [ 55.871306][ T67] [ 55.871427][ T67] ============================================ [ 55.871609][ T67] WARNING: possible recursive locking detected [ 55.871790][ T67] 6.13.0-rc5-virtme #1 Not tainted [ 55.871953][ T67] -------------------------------------------- [ 55.872136][ T67] kworker/u16:1/67 is trying to acquire lock: [ 55.872311][ T67] ffffffffb03d6630 ((netdev_chain).rwsem){++++}-{4:4}, at: blocking_notifier_call_chain+0x50/0x90 [ 55.872615][ T67] [ 55.872615][ T67] but task is already holding lock: [ 55.872816][ T67] ffffffffb03d6630 ((netdev_chain).rwsem){++++}-{4:4}, at: blocking_notifier_call_chain+0x50/0x90 [ 55.873096][ T67] [ 55.873096][ T67] other info that might help us debug this: [ 55.873300][ T67] Possible unsafe locking scenario: [ 55.873300][ T67] [ 55.873505][ T67] CPU0 [ 55.873613][ T67] ---- [ 55.873719][ T67] lock((netdev_chain).rwsem); [ 55.873863][ T67] lock((netdev_chain).rwsem); [ 55.874001][ T67] [ 55.874001][ T67] *** DEADLOCK *** [ 55.874001][ T67] [ 55.874207][ T67] May be due to missing lock nesting notation [ 55.874207][ T67] [ 55.874414][ T67] 5 locks held by kworker/u16:1/67: [ 55.874552][ T67] #0: ffff8880010ad948 ((wq_completion)netns){+.+.}-{0:0}, at: process_one_work+0x7ec/0x16d0 [ 55.874852][ T67] #1: ffffc90000487da0 (net_cleanup_work){+.+.}-{0:0}, at: process_one_work+0xe0b/0x16d0 [ 55.875097][ T67] #2: ffffffffb03cc090 (pernet_ops_rwsem){++++}-{4:4}, at: cleanup_net+0xbc/0xa50 [ 55.875349][ T67] #3: ffffffffb03e7a48 (rtnl_mutex){+.+.}-{4:4}, at: cleanup_net+0x45e/0xa50 [ 55.875596][ T67] #4: ffffffffb03d6630 ((netdev_chain).rwsem){++++}-{4:4}, at: blocking_notifier_call_chain+0x50/0x90 [ 55.875879][ T67] [ 55.875879][ T67] stack backtrace: [ 55.876072][ T67] CPU: 3 UID: 0 PID: 67 Comm: kworker/u16:1 Not tainted 6.13.0-rc5-virtme #1 [ 55.876318][ T67] Hardware name: Bochs Bochs, BIOS Bochs 01/01/2011 [ 55.876495][ T67] Workqueue: netns cleanup_net [ 55.876641][ T67] Call Trace: [ 55.876745][ T67] [ 55.876819][ T67] dump_stack_lvl+0x82/0xd0 [ 55.876963][ T67] print_deadlock_bug+0x40a/0x650 [ 55.877106][ T67] validate_chain+0x5bf/0xae0 [ 55.877246][ T67] ? __pfx_validate_chain+0x10/0x10 [ 55.877389][ T67] ? hlock_class+0x4e/0x130 [ 55.877528][ T67] ? mark_lock+0x38/0x3e0 [ 55.877638][ T67] __lock_acquire+0xb9a/0x1680 [ 55.877778][ T67] ? spin_bug+0x1b1/0x1d0 [ 55.877888][ T67] ? debug_object_assert_init+0x2a9/0x370 [ 55.878031][ T67] lock_acquire.part.0+0xeb/0x330 [ 55.878175][ T67] ? blocking_notifier_call_chain+0x50/0x90 [ 55.878351][ T67] ? __pfx_lock_acquire.part.0+0x10/0x10 [ 55.878498][ T67] ? trace_lock_acquire+0x14c/0x1f0 [ 55.878637][ T67] ? lock_acquire+0x32/0xc0 [ 55.878777][ T67] ? blocking_notifier_call_chain+0x50/0x90 [ 55.878950][ T67] down_read+0x9f/0x340 [ 55.879058][ T67] ? blocking_notifier_call_chain+0x50/0x90 [ 55.879231][ T67] ? __pfx_down_read+0x10/0x10 [ 55.879372][ T67] blocking_notifier_call_chain+0x50/0x90 [ 55.879517][ T67] __dev_close_many+0xdf/0x2d0 [ 55.879655][ T67] ? __pfx___dev_close_many+0x10/0x10 [ 55.879795][ T67] ? mark_held_locks+0x9e/0xe0 [ 55.879941][ T67] dev_close_many+0x202/0x650 [ 55.880084][ T67] ? __pfx_dev_close_many+0x10/0x10 [ 55.880236][ T67] vlan_device_event+0x143b/0x2130 [8021q] [ 55.880422][ T67] ? net_generic+0xbb/0x1f0 [vxlan] [ 55.880580][ T67] ? __pfx_vlan_device_event+0x10/0x10 [8021q] [ 55.880752][ T67] ? __pfx_vxlan_netdevice_event+0x10/0x10 [vxlan] [ 55.880939][ T67] ? packet_notifier+0x3b0/0x810 [ 55.881086][ T67] notifier_call_chain+0xcd/0x150 [ 55.881230][ T67] blocking_notifier_call_chain+0x66/0x90 [ 55.881372][ T67] dev_close_many+0x2d8/0x650 [ 55.881516][ T67] ? __lock_release+0x103/0x460 [ 55.881654][ T67] ? __virt_addr_valid+0x22b/0x430 [ 55.881799][ T67] ? __pfx_dev_close_many+0x10/0x10 [ 55.881941][ T67] unregister_netdevice_many_notify+0x8e5/0x1580 [ 55.882118][ T67] ? mark_held_locks+0x9e/0xe0 [ 55.882260][ T67] ? __pfx_unregister_netdevice_many_notify+0x10/0x10 [ 55.882436][ T67] ? mutex_is_locked+0x1c/0x60 [ 55.882574][ T67] ? rtnl_is_locked+0x15/0x20 [ 55.882719][ T67] ? __pfx_unregister_netdevice_queue+0x10/0x10 [ 55.882893][ T67] ? mutex_is_locked+0x1c/0x60 [ 55.883033][ T67] ? nexthop_net_exit_batch_rtnl+0x83/0x210 [ 55.883209][ T67] cleanup_net+0x4cf/0xa50 [ 55.883348][ T67] ? __pfx_lock_acquire.part.0+0x10/0x10 [ 55.883488][ T67] ? __pfx_cleanup_net+0x10/0x10 [ 55.883630][ T67] ? trace_lock_acquire+0x14c/0x1f0 [ 55.883768][ T67] ? lock_acquire+0x32/0xc0 [ 55.883908][ T67] ? process_one_work+0xe0b/0x16d0 [ 55.884050][ T67] process_one_work+0xe55/0x16d0 [ 55.884191][ T67] ? __pfx___lock_release+0x10/0x10 [ 55.884329][ T67] ? __pfx_process_one_work+0x10/0x10 [ 55.884471][ T67] ? assign_work+0x16c/0x240 [ 55.884619][ T67] worker_thread+0x58c/0xce0 [ 55.884782][ T67] ? lockdep_hardirqs_on_prepare+0x275/0x410 [ 55.884960][ T67] ? __pfx_worker_thread+0x10/0x10 [ 55.885101][ T67] ? __pfx_worker_thread+0x10/0x10 [ 55.885241][ T67] kthread+0x28a/0x350 [ 55.885347][ T67] ? __pfx_kthread+0x10/0x10 [ 55.885489][ T67] ret_from_fork+0x31/0x70 [ 55.885632][ T67] ? __pfx_kthread+0x10/0x10 [ 55.885771][ T67] ret_from_fork_asm+0x1a/0x30 [ 55.885921][ T67] [ 57.499666][ T804] br1: left promiscuous mode [ 57.954115][ T810] br1: port 4(veth2) entered disabled state [ 58.012826][ T811] veth2: left allmulticast mode [ 58.013014][ T811] veth2: left promiscuous mode [ 58.013267][ T811] br1: port 4(veth2) entered disabled state [ 58.131803][ T813] br1: port 3(veth1) entered disabled state [ 58.181651][ T814] veth1: left allmulticast mode [ 58.181837][ T814] veth1: left promiscuous mode [ 58.182096][ T814] br1: port 3(veth1) entered disabled state [ 58.299219][ T816] vx20: left allmulticast mode [ 58.299536][ T816] vx20: left promiscuous mode [ 58.299973][ T816] br1: port 2(vx20) entered disabled state [ 58.571811][ T820] vx10: left allmulticast mode [ 58.571993][ T820] vx10: left promiscuous mode [ 58.572256][ T820] br1: port 1(vx10) entered disabled state