[ 10.563952][ T249] ip (249) used greatest stack depth: 23792 bytes left [ 12.898834][ T295] 8021q: 802.1Q VLAN Support v1.8 [ 16.375666][ T343] gre: GRE over IPv4 demultiplexor driver [ 16.389473][ T343] ip_gre: GRE over IPv4 tunneling driver [ 17.342747][ T360] lag: Mode changed to "loadbalance" [ 17.525549][ T363] lag: Port device veth2 added [ 17.526919][ T37] [ 17.527012][ T37] ====================================================== [ 17.527242][ T37] WARNING: possible circular locking dependency detected [ 17.527434][ T37] 6.13.0-rc5-virtme #1 Not tainted [ 17.527597][ T37] ------------------------------------------------------ [ 17.527789][ T37] kworker/u17:0/37 is trying to acquire lock: [ 17.527968][ T37] ffff888004efad80 (team->team_lock_key){+.+.}-{4:4}, at: team_port_change_check+0x43/0x130 [ 17.528272][ T37] [ 17.528272][ T37] but task is already holding lock: [ 17.528495][ T37] ffffffffa43d6630 ((netdev_chain).rwsem){++++}-{4:4}, at: blocking_notifier_call_chain+0x50/0x90 [ 17.528783][ T37] [ 17.528783][ T37] which lock already depends on the new lock. [ 17.528783][ T37] [ 17.529059][ T37] [ 17.529059][ T37] the existing dependency chain (in reverse order) is: [ 17.529314][ T37] [ 17.529314][ T37] -> #1 ((netdev_chain).rwsem){++++}-{4:4}: [ 17.529546][ T37] __lock_acquire+0xb9a/0x1680 [ 17.529691][ T37] lock_acquire.part.0+0xeb/0x330 [ 17.529830][ T37] down_read+0x9f/0x340 [ 17.529975][ T37] blocking_notifier_call_chain+0x50/0x90 [ 17.530157][ T37] __dev_open+0x15c/0x430 [ 17.530301][ T37] dev_open+0xc9/0x130 [ 17.530445][ T37] team_port_add+0x90a/0x1ce0 [ 17.530594][ T37] team_add_slave+0x41/0x80 [ 17.530735][ T37] do_setlink.constprop.0+0x80e/0x2300 [ 17.530913][ T37] rtnl_newlink+0x69c/0xa70 [ 17.531054][ T37] rtnetlink_rcv_msg+0x712/0xc10 [ 17.531190][ T37] netlink_rcv_skb+0x130/0x360 [ 17.531329][ T37] netlink_unicast+0x44b/0x710 [ 17.531468][ T37] netlink_sendmsg+0x723/0xbe0 [ 17.531629][ T37] ____sys_sendmsg+0x7ac/0xa10 [ 17.531781][ T37] ___sys_sendmsg+0xee/0x170 [ 17.531928][ T37] __sys_sendmsg+0x109/0x1a0 [ 17.532069][ T37] do_syscall_64+0xc1/0x1d0 [ 17.532208][ T37] entry_SYSCALL_64_after_hwframe+0x77/0x7f [ 17.532383][ T37] [ 17.532383][ T37] -> #0 (team->team_lock_key){+.+.}-{4:4}: [ 17.532598][ T37] check_prev_add+0x182/0xce0 [ 17.532752][ T37] validate_chain+0x7bd/0xae0 [ 17.532892][ T37] __lock_acquire+0xb9a/0x1680 [ 17.533032][ T37] lock_acquire.part.0+0xeb/0x330 [ 17.533169][ T37] __mutex_lock+0x169/0xbc0 [ 17.533325][ T37] team_port_change_check+0x43/0x130 [ 17.533501][ T37] team_device_event+0x3fe/0x520 [ 17.533644][ T37] notifier_call_chain+0xcd/0x150 [ 17.533788][ T37] blocking_notifier_call_chain+0x66/0x90 [ 17.533960][ T37] netdev_state_change+0xf5/0x120 [ 17.534099][ T37] linkwatch_do_dev+0xd2/0x100 [ 17.534236][ T37] __linkwatch_run_queue+0x1df/0x650 [ 17.534405][ T37] linkwatch_event+0x40/0x60 [ 17.534541][ T37] process_one_work+0xe55/0x16d0 [ 17.534684][ T37] worker_thread+0x58c/0xce0 [ 17.534820][ T37] kthread+0x28a/0x350 [ 17.534957][ T37] ret_from_fork+0x31/0x70 [ 17.535095][ T37] ret_from_fork_asm+0x1a/0x30 [ 17.535234][ T37] [ 17.535234][ T37] other info that might help us debug this: [ 17.535234][ T37] [ 17.535501][ T37] Possible unsafe locking scenario: [ 17.535501][ T37] [ 17.535710][ T37] CPU0 CPU1 [ 17.535857][ T37] ---- ---- [ 17.535995][ T37] rlock((netdev_chain).rwsem); [ 17.536149][ T37] lock(team->team_lock_key); [ 17.536397][ T37] lock((netdev_chain).rwsem); [ 17.536682][ T37] lock(team->team_lock_key); [ 17.536886][ T37] [ 17.536886][ T37] *** DEADLOCK *** [ 17.536886][ T37] [ 17.537167][ T37] 4 locks held by kworker/u17:0/37: [ 17.537358][ T37] #0: ffff8880010a9d48 ((wq_completion)events_unbound){+.+.}-{0:0}, at: process_one_work+0x7ec/0x16d0 [ 17.537753][ T37] #1: ffffc9000029fda0 ((linkwatch_work).work){+.+.}-{0:0}, at: process_one_work+0xe0b/0x16d0 [ 17.538155][ T37] #2: ffffffffa43e7a48 (rtnl_mutex){+.+.}-{4:4}, at: linkwatch_event+0xe/0x60 [ 17.538518][ T37] #3: ffffffffa43d6630 ((netdev_chain).rwsem){++++}-{4:4}, at: blocking_notifier_call_chain+0x50/0x90 [ 17.538914][ T37] [ 17.538914][ T37] stack backtrace: [ 17.539153][ T37] CPU: 0 UID: 0 PID: 37 Comm: kworker/u17:0 Not tainted 6.13.0-rc5-virtme #1 [ 17.539444][ T37] Hardware name: Bochs Bochs, BIOS Bochs 01/01/2011 [ 17.539621][ T37] Workqueue: events_unbound linkwatch_event [ 17.539803][ T37] Call Trace: [ 17.539913][ T37] [ 17.539993][ T37] dump_stack_lvl+0x82/0xd0 [ 17.540139][ T37] print_circular_bug+0x2e2/0x4b0 [ 17.540290][ T37] check_noncircular+0x2f9/0x3e0 [ 17.540436][ T37] ? __pfx_check_noncircular+0x10/0x10 [ 17.540587][ T37] ? __bfs+0x3f9/0x6c0 [ 17.540693][ T37] ? __pfx_usage_accumulate+0x10/0x10 [ 17.540835][ T37] ? alloc_chain_hlocks+0x4e0/0x720 [ 17.540976][ T37] ? unwind_next_frame+0x1478/0x1d00 [ 17.541124][ T37] check_prev_add+0x182/0xce0 [ 17.541276][ T37] ? ret_from_fork_asm+0x1a/0x30 [ 17.541423][ T37] validate_chain+0x7bd/0xae0 [ 17.541574][ T37] ? ret_from_fork_asm+0x1a/0x30 [ 17.541725][ T37] ? __pfx_validate_chain+0x10/0x10 [ 17.541865][ T37] ? hlock_class+0x4e/0x130 [ 17.542007][ T37] ? mark_lock+0x38/0x3e0 [ 17.542116][ T37] __lock_acquire+0xb9a/0x1680 [ 17.542259][ T37] lock_acquire.part.0+0xeb/0x330 [ 17.542407][ T37] ? team_port_change_check+0x43/0x130 [ 17.542545][ T37] ? validate_chain+0x1fe/0xae0 [ 17.542690][ T37] ? __pfx_lock_acquire.part.0+0x10/0x10 [ 17.542847][ T37] ? trace_lock_acquire+0x14c/0x1f0 [ 17.542989][ T37] ? lock_acquire+0x32/0xc0 [ 17.543138][ T37] ? team_port_change_check+0x43/0x130 [ 17.543289][ T37] __mutex_lock+0x169/0xbc0 [ 17.543436][ T37] ? team_port_change_check+0x43/0x130 [ 17.543585][ T37] ? team_port_change_check+0x43/0x130 [ 17.543736][ T37] ? __pfx___mutex_lock+0x10/0x10 [ 17.543883][ T37] ? lock_acquire.part.0+0xeb/0x330 [ 17.544039][ T37] ? blocking_notifier_call_chain+0x50/0x90 [ 17.544232][ T37] ? find_held_lock+0x2c/0x110 [ 17.544377][ T37] ? __pfx_fib_sync_up+0x10/0x10 [ 17.544529][ T37] ? __pfx_rwsem_read_trylock+0x10/0x10 [ 17.544676][ T37] ? team_port_change_check+0x43/0x130 [ 17.544828][ T37] team_port_change_check+0x43/0x130 [ 17.544977][ T37] team_device_event+0x3fe/0x520 [ 17.545131][ T37] ? trace_notifier_run+0xe7/0x150 [ 17.545288][ T37] notifier_call_chain+0xcd/0x150 [ 17.545448][ T37] blocking_notifier_call_chain+0x66/0x90 [ 17.545603][ T37] netdev_state_change+0xf5/0x120 [ 17.545748][ T37] ? __pfx_netdev_state_change+0x10/0x10 [ 17.545893][ T37] linkwatch_do_dev+0xd2/0x100 [ 17.546035][ T37] __linkwatch_run_queue+0x1df/0x650 [ 17.546176][ T37] ? __pfx___linkwatch_run_queue+0x10/0x10 [ 17.546351][ T37] ? lock_acquire+0x32/0xc0 [ 17.546491][ T37] linkwatch_event+0x40/0x60 [ 17.546630][ T37] process_one_work+0xe55/0x16d0 [ 17.546792][ T37] ? __pfx___lock_release+0x10/0x10 [ 17.546942][ T37] ? __pfx_process_one_work+0x10/0x10 [ 17.547086][ T37] ? assign_work+0x16c/0x240 [ 17.547229][ T37] worker_thread+0x58c/0xce0 [ 17.547386][ T37] ? __pfx_worker_thread+0x10/0x10 [ 17.547527][ T37] kthread+0x28a/0x350 [ 17.547632][ T37] ? __pfx_kthread+0x10/0x10 [ 17.547772][ T37] ret_from_fork+0x31/0x70 [ 17.547911][ T37] ? __pfx_kthread+0x10/0x10 [ 17.548064][ T37] ret_from_fork_asm+0x1a/0x30 [ 17.548261][ T37] [ 17.755160][ T367] lag: Port device veth4 added [ 17.872387][ T369] 8021q: adding VLAN 0 to HW filter on device lag [ 17.930776][ T370] br1: port 1(lag) entered blocking state [ 17.931037][ T370] br1: port 1(lag) entered disabled state [ 17.931231][ T370] lag: entered allmulticast mode [ 17.931389][ T370] veth2: entered allmulticast mode [ 17.931535][ T370] veth4: entered allmulticast mode [ 17.933265][ T370] veth2: entered promiscuous mode [ 17.933458][ T370] veth4: entered promiscuous mode [ 17.984714][ T371] br1: port 1(lag) entered blocking state [ 17.984973][ T371] br1: port 1(lag) entered forwarding state [ 18.432022][ T380] GACT probability NOT on [ 24.533103][ T407] Mirror/redirect action on [ 38.902308][ T138] br1: port 1(lag) entered disabled state [ 44.690014][ T37] br1: port 1(lag) entered blocking state [ 44.690216][ T37] br1: port 1(lag) entered forwarding state [ 59.491112][ T37] br1: port 1(lag) entered disabled state [ 65.385710][ T39] br1: port 1(lag) entered blocking state [ 65.385915][ T39] br1: port 1(lag) entered forwarding state [ 65.946365][ T511] lag: left allmulticast mode [ 65.946629][ T511] veth2: left allmulticast mode [ 65.946812][ T511] veth4: left allmulticast mode [ 65.946994][ T511] veth2: left promiscuous mode [ 65.947184][ T511] veth4: left promiscuous mode [ 65.947805][ T511] br1: port 1(lag) entered disabled state [ 66.022115][ T360] lag: Port device veth4 removed [ 66.030667][ T360] lag: Port device veth2 removed