[ 11.892523][ T272] ip (272) used greatest stack depth: 23728 bytes left [ 13.603135][ T294] 8021q: 802.1Q VLAN Support v1.8 [ 17.418223][ T341] gre: GRE over IPv4 demultiplexor driver [ 17.432400][ T341] ip_gre: GRE over IPv4 tunneling driver [ 18.511588][ T359] lag: Mode changed to "loadbalance" [ 18.769133][ T362] lag: Port device veth2 added [ 18.769884][ T255] [ 18.769992][ T255] ====================================================== [ 18.770187][ T255] WARNING: possible circular locking dependency detected [ 18.770474][ T255] 6.13.0-rc5-virtme #1 Not tainted [ 18.770705][ T255] ------------------------------------------------------ [ 18.770917][ T255] kworker/u20:1/255 is trying to acquire lock: [ 18.771114][ T255] ffff88800e912d80 (team->team_lock_key){+.+.}-{4:4}, at: team_port_change_check+0x43/0x130 [ 18.771518][ T255] [ 18.771518][ T255] but task is already holding lock: [ 18.771752][ T255] ffffffff911d6630 ((netdev_chain).rwsem){++++}-{4:4}, at: blocking_notifier_call_chain+0x50/0x90 [ 18.772130][ T255] [ 18.772130][ T255] which lock already depends on the new lock. [ 18.772130][ T255] [ 18.772454][ T255] [ 18.772454][ T255] the existing dependency chain (in reverse order) is: [ 18.772773][ T255] [ 18.772773][ T255] -> #1 ((netdev_chain).rwsem){++++}-{4:4}: [ 18.773018][ T255] __lock_acquire+0xb9a/0x1680 [ 18.773184][ T255] lock_acquire.part.0+0xeb/0x330 [ 18.773373][ T255] down_read+0x9f/0x340 [ 18.773571][ T255] blocking_notifier_call_chain+0x50/0x90 [ 18.773771][ T255] __dev_open+0x15c/0x430 [ 18.773934][ T255] dev_open+0xc9/0x130 [ 18.774148][ T255] team_port_add+0x90a/0x1ce0 [ 18.774341][ T255] team_add_slave+0x41/0x80 [ 18.774512][ T255] do_setlink.constprop.0+0x80e/0x2300 [ 18.774734][ T255] rtnl_newlink+0x69c/0xa70 [ 18.774909][ T255] rtnetlink_rcv_msg+0x712/0xc10 [ 18.775103][ T255] netlink_rcv_skb+0x130/0x360 [ 18.775299][ T255] netlink_unicast+0x44b/0x710 [ 18.775477][ T255] netlink_sendmsg+0x723/0xbe0 [ 18.775681][ T255] ____sys_sendmsg+0x7ac/0xa10 [ 18.775869][ T255] ___sys_sendmsg+0xee/0x170 [ 18.776067][ T255] __sys_sendmsg+0x109/0x1a0 [ 18.776242][ T255] do_syscall_64+0xc1/0x1d0 [ 18.776412][ T255] entry_SYSCALL_64_after_hwframe+0x77/0x7f [ 18.776614][ T255] [ 18.776614][ T255] -> #0 (team->team_lock_key){+.+.}-{4:4}: [ 18.776902][ T255] check_prev_add+0x182/0xce0 [ 18.777066][ T255] validate_chain+0x7bd/0xae0 [ 18.777276][ T255] __lock_acquire+0xb9a/0x1680 [ 18.777507][ T255] lock_acquire.part.0+0xeb/0x330 [ 18.777728][ T255] __mutex_lock+0x169/0xbc0 [ 18.777891][ T255] team_port_change_check+0x43/0x130 [ 18.778080][ T255] team_device_event+0x3fe/0x520 [ 18.778248][ T255] notifier_call_chain+0xcd/0x150 [ 18.778412][ T255] blocking_notifier_call_chain+0x66/0x90 [ 18.778614][ T255] netdev_state_change+0xf5/0x120 [ 18.778771][ T255] linkwatch_do_dev+0xd2/0x100 [ 18.778922][ T255] __linkwatch_run_queue+0x1df/0x650 [ 18.779113][ T255] linkwatch_event+0x40/0x60 [ 18.779282][ T255] process_one_work+0xe55/0x16d0 [ 18.779453][ T255] worker_thread+0x58c/0xce0 [ 18.779609][ T255] kthread+0x28a/0x350 [ 18.779765][ T255] ret_from_fork+0x31/0x70 [ 18.779927][ T255] ret_from_fork_asm+0x1a/0x30 [ 18.780120][ T255] [ 18.780120][ T255] other info that might help us debug this: [ 18.780120][ T255] [ 18.780457][ T255] Possible unsafe locking scenario: [ 18.780457][ T255] [ 18.780744][ T255] CPU0 CPU1 [ 18.780901][ T255] ---- ---- [ 18.781053][ T255] rlock((netdev_chain).rwsem); [ 18.781219][ T255] lock(team->team_lock_key); [ 18.781455][ T255] lock((netdev_chain).rwsem); [ 18.781684][ T255] lock(team->team_lock_key); [ 18.781841][ T255] [ 18.781841][ T255] *** DEADLOCK *** [ 18.781841][ T255] [ 18.782065][ T255] 4 locks held by kworker/u20:1/255: [ 18.782226][ T255] #0: ffff8880010a9d48 ((wq_completion)events_unbound){+.+.}-{0:0}, at: process_one_work+0x7ec/0x16d0 [ 18.782547][ T255] #1: ffffc90000657da0 ((linkwatch_work).work){+.+.}-{0:0}, at: process_one_work+0xe0b/0x16d0 [ 18.782871][ T255] #2: ffffffff911e7a48 (rtnl_mutex){+.+.}-{4:4}, at: linkwatch_event+0xe/0x60 [ 18.783150][ T255] #3: ffffffff911d6630 ((netdev_chain).rwsem){++++}-{4:4}, at: blocking_notifier_call_chain+0x50/0x90 [ 18.783479][ T255] [ 18.783479][ T255] stack backtrace: [ 18.783670][ T255] CPU: 0 UID: 0 PID: 255 Comm: kworker/u20:1 Not tainted 6.13.0-rc5-virtme #1 [ 18.783943][ T255] Hardware name: Bochs Bochs, BIOS Bochs 01/01/2011 [ 18.784145][ T255] Workqueue: events_unbound linkwatch_event [ 18.784422][ T255] Call Trace: [ 18.784547][ T255] [ 18.784629][ T255] dump_stack_lvl+0x82/0xd0 [ 18.784789][ T255] print_circular_bug+0x2e2/0x4b0 [ 18.784948][ T255] check_noncircular+0x2f9/0x3e0 [ 18.785134][ T255] ? __pfx___lock_release+0x10/0x10 [ 18.785376][ T255] ? __pfx_check_noncircular+0x10/0x10 [ 18.785609][ T255] ? trace_ipi_send_cpu.constprop.0+0x117/0x190 [ 18.785813][ T255] ? alloc_chain_hlocks+0x4e0/0x720 [ 18.785969][ T255] check_prev_add+0x182/0xce0 [ 18.786129][ T255] validate_chain+0x7bd/0xae0 [ 18.786336][ T255] ? rcu_read_lock_any_held+0x43/0xb0 [ 18.786510][ T255] ? __pfx_validate_chain+0x10/0x10 [ 18.786668][ T255] ? hlock_class+0x4e/0x130 [ 18.786820][ T255] ? mark_lock+0x38/0x3e0 [ 18.786946][ T255] ? rcu_read_lock_any_held+0x43/0xb0 [ 18.787103][ T255] __lock_acquire+0xb9a/0x1680 [ 18.787276][ T255] lock_acquire.part.0+0xeb/0x330 [ 18.787432][ T255] ? team_port_change_check+0x43/0x130 [ 18.787587][ T255] ? validate_chain+0x1fe/0xae0 [ 18.787748][ T255] ? __pfx_lock_acquire.part.0+0x10/0x10 [ 18.787906][ T255] ? trace_lock_acquire+0x14c/0x1f0 [ 18.788068][ T255] ? lock_acquire+0x32/0xc0 [ 18.788239][ T255] ? team_port_change_check+0x43/0x130 [ 18.788398][ T255] __mutex_lock+0x169/0xbc0 [ 18.788556][ T255] ? team_port_change_check+0x43/0x130 [ 18.788713][ T255] ? team_port_change_check+0x43/0x130 [ 18.788869][ T255] ? __pfx___mutex_lock+0x10/0x10 [ 18.789026][ T255] ? lock_acquire.part.0+0xeb/0x330 [ 18.789190][ T255] ? blocking_notifier_call_chain+0x50/0x90 [ 18.789412][ T255] ? find_held_lock+0x2c/0x110 [ 18.789578][ T255] ? __pfx_fib_sync_up+0x10/0x10 [ 18.789741][ T255] ? __pfx_rwsem_read_trylock+0x10/0x10 [ 18.789904][ T255] ? team_port_change_check+0x43/0x130 [ 18.790059][ T255] team_port_change_check+0x43/0x130 [ 18.790223][ T255] team_device_event+0x3fe/0x520 [ 18.790384][ T255] ? trace_notifier_run+0xe7/0x150 [ 18.790542][ T255] notifier_call_chain+0xcd/0x150 [ 18.790703][ T255] blocking_notifier_call_chain+0x66/0x90 [ 18.790853][ T255] netdev_state_change+0xf5/0x120 [ 18.791011][ T255] ? __pfx_netdev_state_change+0x10/0x10 [ 18.791176][ T255] linkwatch_do_dev+0xd2/0x100 [ 18.791341][ T255] __linkwatch_run_queue+0x1df/0x650 [ 18.791505][ T255] ? __pfx___linkwatch_run_queue+0x10/0x10 [ 18.791696][ T255] ? lock_acquire+0x32/0xc0 [ 18.791854][ T255] linkwatch_event+0x40/0x60 [ 18.792015][ T255] process_one_work+0xe55/0x16d0 [ 18.792181][ T255] ? __pfx___lock_release+0x10/0x10 [ 18.792394][ T255] ? __pfx_process_one_work+0x10/0x10 [ 18.792584][ T255] ? assign_work+0x16c/0x240 [ 18.792773][ T255] worker_thread+0x58c/0xce0 [ 18.792968][ T255] ? lockdep_hardirqs_on_prepare+0x275/0x410 [ 18.793242][ T255] ? __pfx_worker_thread+0x10/0x10 [ 18.793478][ T255] ? __pfx_worker_thread+0x10/0x10 [ 18.793714][ T255] kthread+0x28a/0x350 [ 18.793834][ T255] ? __pfx_kthread+0x10/0x10 [ 18.793993][ T255] ret_from_fork+0x31/0x70 [ 18.794163][ T255] ? __pfx_kthread+0x10/0x10 [ 18.794376][ T255] ret_from_fork_asm+0x1a/0x30 [ 18.794552][ T255] [ 18.968999][ T365] lag: Port device veth4 added [ 19.089624][ T367] 8021q: adding VLAN 0 to HW filter on device lag [ 19.168999][ T368] br1: port 1(lag) entered blocking state [ 19.169367][ T368] br1: port 1(lag) entered disabled state [ 19.169665][ T368] lag: entered allmulticast mode [ 19.169840][ T368] veth2: entered allmulticast mode [ 19.170017][ T368] veth4: entered allmulticast mode [ 19.171810][ T368] veth2: entered promiscuous mode [ 19.172041][ T368] veth4: entered promiscuous mode [ 19.232304][ T369] br1: port 1(lag) entered blocking state [ 19.232621][ T369] br1: port 1(lag) entered forwarding state [ 19.750523][ T378] GACT probability NOT on [ 25.918879][ T405] Mirror/redirect action on [ 40.360725][ T37] br1: port 1(lag) entered disabled state [ 46.192547][ T37] br1: port 1(lag) entered blocking state [ 46.192796][ T37] br1: port 1(lag) entered forwarding state [ 61.029915][ T255] br1: port 1(lag) entered disabled state [ 66.846241][ T255] br1: port 1(lag) entered blocking state [ 66.846577][ T255] br1: port 1(lag) entered forwarding state [ 67.415307][ T510] lag: left allmulticast mode [ 67.416250][ T510] veth2: left allmulticast mode [ 67.416460][ T510] veth4: left allmulticast mode [ 67.416624][ T510] veth2: left promiscuous mode [ 67.416783][ T510] veth4: left promiscuous mode [ 67.416935][ T510] br1: port 1(lag) entered disabled state [ 67.492790][ T359] lag: Port device veth4 removed [ 67.497361][ T359] lag: Port device veth2 removed