[ 13.377456][ T293] 8021q: 802.1Q VLAN Support v1.8 [ 17.164351][ T340] gre: GRE over IPv4 demultiplexor driver [ 17.179082][ T340] ip_gre: GRE over IPv4 tunneling driver [ 18.226625][ T358] lag: Mode changed to "loadbalance" [ 18.420405][ T361] lag: Port device veth2 added [ 18.421165][ T151] [ 18.421263][ T151] ====================================================== [ 18.421444][ T151] WARNING: possible circular locking dependency detected [ 18.421685][ T151] 6.13.0-rc5-virtme #1 Not tainted [ 18.421835][ T151] ------------------------------------------------------ [ 18.422006][ T151] kworker/u17:2/151 is trying to acquire lock: [ 18.422191][ T151] ffff88800e942d80 (team->team_lock_key){+.+.}-{4:4}, at: team_port_change_check+0x43/0x130 [ 18.422489][ T151] [ 18.422489][ T151] but task is already holding lock: [ 18.422701][ T151] ffffffffafdd6630 ((netdev_chain).rwsem){++++}-{4:4}, at: blocking_notifier_call_chain+0x50/0x90 [ 18.422998][ T151] [ 18.422998][ T151] which lock already depends on the new lock. [ 18.422998][ T151] [ 18.423273][ T151] [ 18.423273][ T151] the existing dependency chain (in reverse order) is: [ 18.423521][ T151] [ 18.423521][ T151] -> #1 ((netdev_chain).rwsem){++++}-{4:4}: [ 18.423757][ T151] __lock_acquire+0xb9a/0x1680 [ 18.423985][ T151] lock_acquire.part.0+0xeb/0x330 [ 18.424203][ T151] down_read+0x9f/0x340 [ 18.424406][ T151] blocking_notifier_call_chain+0x50/0x90 [ 18.424669][ T151] __dev_open+0x15c/0x430 [ 18.424883][ T151] dev_open+0xc9/0x130 [ 18.425103][ T151] team_port_add+0x90a/0x1ce0 [ 18.425327][ T151] team_add_slave+0x41/0x80 [ 18.425554][ T151] do_setlink.constprop.0+0x80e/0x2300 [ 18.425833][ T151] rtnl_newlink+0x69c/0xa70 [ 18.426051][ T151] rtnetlink_rcv_msg+0x712/0xc10 [ 18.426274][ T151] netlink_rcv_skb+0x130/0x360 [ 18.426490][ T151] netlink_unicast+0x44b/0x710 [ 18.426710][ T151] netlink_sendmsg+0x723/0xbe0 [ 18.426921][ T151] ____sys_sendmsg+0x7ac/0xa10 [ 18.427147][ T151] ___sys_sendmsg+0xee/0x170 [ 18.427372][ T151] __sys_sendmsg+0x109/0x1a0 [ 18.427600][ T151] do_syscall_64+0xc1/0x1d0 [ 18.427829][ T151] entry_SYSCALL_64_after_hwframe+0x77/0x7f [ 18.428126][ T151] [ 18.428126][ T151] -> #0 (team->team_lock_key){+.+.}-{4:4}: [ 18.428492][ T151] check_prev_add+0x182/0xce0 [ 18.428723][ T151] validate_chain+0x7bd/0xae0 [ 18.428953][ T151] __lock_acquire+0xb9a/0x1680 [ 18.429186][ T151] lock_acquire.part.0+0xeb/0x330 [ 18.429412][ T151] __mutex_lock+0x169/0xbc0 [ 18.429636][ T151] team_port_change_check+0x43/0x130 [ 18.429955][ T151] team_device_event+0x3fe/0x520 [ 18.430208][ T151] notifier_call_chain+0xcd/0x150 [ 18.430444][ T151] blocking_notifier_call_chain+0x66/0x90 [ 18.430736][ T151] netdev_state_change+0xf5/0x120 [ 18.430984][ T151] linkwatch_do_dev+0xd2/0x100 [ 18.431237][ T151] __linkwatch_run_queue+0x1df/0x650 [ 18.431516][ T151] linkwatch_event+0x40/0x60 [ 18.431753][ T151] process_one_work+0xe55/0x16d0 [ 18.431978][ T151] worker_thread+0x58c/0xce0 [ 18.432201][ T151] kthread+0x28a/0x350 [ 18.432428][ T151] ret_from_fork+0x31/0x70 [ 18.432656][ T151] ret_from_fork_asm+0x1a/0x30 [ 18.432887][ T151] [ 18.432887][ T151] other info that might help us debug this: [ 18.432887][ T151] [ 18.433336][ T151] Possible unsafe locking scenario: [ 18.433336][ T151] [ 18.433666][ T151] CPU0 CPU1 [ 18.433890][ T151] ---- ---- [ 18.434111][ T151] rlock((netdev_chain).rwsem); [ 18.434344][ T151] lock(team->team_lock_key); [ 18.434678][ T151] lock((netdev_chain).rwsem); [ 18.435021][ T151] lock(team->team_lock_key); [ 18.435244][ T151] [ 18.435244][ T151] *** DEADLOCK *** [ 18.435244][ T151] [ 18.435565][ T151] 4 locks held by kworker/u17:2/151: [ 18.435791][ T151] #0: ffff8880010a9d48 ((wq_completion)events_unbound){+.+.}-{0:0}, at: process_one_work+0x7ec/0x16d0 [ 18.436246][ T151] #1: ffffc900006c7da0 ((linkwatch_work).work){+.+.}-{0:0}, at: process_one_work+0xe0b/0x16d0 [ 18.436712][ T151] #2: ffffffffafde7a48 (rtnl_mutex){+.+.}-{4:4}, at: linkwatch_event+0xe/0x60 [ 18.437114][ T151] #3: ffffffffafdd6630 ((netdev_chain).rwsem){++++}-{4:4}, at: blocking_notifier_call_chain+0x50/0x90 [ 18.437601][ T151] [ 18.437601][ T151] stack backtrace: [ 18.437900][ T151] CPU: 1 UID: 0 PID: 151 Comm: kworker/u17:2 Not tainted 6.13.0-rc5-virtme #1 [ 18.438315][ T151] Hardware name: Bochs Bochs, BIOS Bochs 01/01/2011 [ 18.438611][ T151] Workqueue: events_unbound linkwatch_event [ 18.438901][ T151] Call Trace: [ 18.439072][ T151] [ 18.439205][ T151] dump_stack_lvl+0x82/0xd0 [ 18.439436][ T151] print_circular_bug+0x2e2/0x4b0 [ 18.439642][ T151] check_noncircular+0x2f9/0x3e0 [ 18.439869][ T151] ? mark_lock+0x38/0x3e0 [ 18.440046][ T151] ? __pfx_check_noncircular+0x10/0x10 [ 18.440260][ T151] ? __lock_acquire+0xb9a/0x1680 [ 18.440487][ T151] ? alloc_chain_hlocks+0x4e0/0x720 [ 18.440714][ T151] check_prev_add+0x182/0xce0 [ 18.440935][ T151] validate_chain+0x7bd/0xae0 [ 18.441167][ T151] ? __pfx_validate_chain+0x10/0x10 [ 18.441399][ T151] ? hlock_class+0x4e/0x130 [ 18.441619][ T151] ? mark_lock+0x38/0x3e0 [ 18.441780][ T151] __lock_acquire+0xb9a/0x1680 [ 18.442002][ T151] lock_acquire.part.0+0xeb/0x330 [ 18.442219][ T151] ? team_port_change_check+0x43/0x130 [ 18.442438][ T151] ? validate_chain+0x1fe/0xae0 [ 18.442655][ T151] ? __pfx_lock_acquire.part.0+0x10/0x10 [ 18.442867][ T151] ? trace_lock_acquire+0x14c/0x1f0 [ 18.443079][ T151] ? lock_acquire+0x32/0xc0 [ 18.443301][ T151] ? team_port_change_check+0x43/0x130 [ 18.443644][ T151] __mutex_lock+0x169/0xbc0 [ 18.443870][ T151] ? team_port_change_check+0x43/0x130 [ 18.444082][ T151] ? team_port_change_check+0x43/0x130 [ 18.444301][ T151] ? __pfx___mutex_lock+0x10/0x10 [ 18.444523][ T151] ? lock_acquire.part.0+0xeb/0x330 [ 18.444761][ T151] ? blocking_notifier_call_chain+0x50/0x90 [ 18.445028][ T151] ? find_held_lock+0x2c/0x110 [ 18.445255][ T151] ? __pfx_fib_sync_up+0x10/0x10 [ 18.445488][ T151] ? __pfx_rwsem_read_trylock+0x10/0x10 [ 18.445709][ T151] ? team_port_change_check+0x43/0x130 [ 18.445932][ T151] team_port_change_check+0x43/0x130 [ 18.446157][ T151] team_device_event+0x3fe/0x520 [ 18.446390][ T151] ? trace_notifier_run+0xe7/0x150 [ 18.446612][ T151] notifier_call_chain+0xcd/0x150 [ 18.446834][ T151] blocking_notifier_call_chain+0x66/0x90 [ 18.447071][ T151] netdev_state_change+0xf5/0x120 [ 18.447312][ T151] ? __pfx_netdev_state_change+0x10/0x10 [ 18.447549][ T151] linkwatch_do_dev+0xd2/0x100 [ 18.447773][ T151] __linkwatch_run_queue+0x1df/0x650 [ 18.448001][ T151] ? __pfx___linkwatch_run_queue+0x10/0x10 [ 18.448274][ T151] ? lock_acquire+0x32/0xc0 [ 18.448499][ T151] linkwatch_event+0x40/0x60 [ 18.448711][ T151] process_one_work+0xe55/0x16d0 [ 18.448940][ T151] ? __pfx___lock_release+0x10/0x10 [ 18.449145][ T151] ? __pfx_process_one_work+0x10/0x10 [ 18.449373][ T151] ? assign_work+0x16c/0x240 [ 18.449600][ T151] worker_thread+0x58c/0xce0 [ 18.449819][ T151] ? lockdep_hardirqs_on_prepare+0x275/0x410 [ 18.450079][ T151] ? __pfx_worker_thread+0x10/0x10 [ 18.450297][ T151] ? __pfx_worker_thread+0x10/0x10 [ 18.450509][ T151] kthread+0x28a/0x350 [ 18.450673][ T151] ? __pfx_kthread+0x10/0x10 [ 18.450897][ T151] ret_from_fork+0x31/0x70 [ 18.451134][ T151] ? __pfx_kthread+0x10/0x10 [ 18.451354][ T151] ret_from_fork_asm+0x1a/0x30 [ 18.451588][ T151] [ 18.692458][ T365] lag: Port device veth4 added [ 18.823552][ T367] 8021q: adding VLAN 0 to HW filter on device lag [ 18.884087][ T368] br1: port 1(lag) entered blocking state [ 18.884320][ T368] br1: port 1(lag) entered disabled state [ 18.884519][ T368] lag: entered allmulticast mode [ 18.884672][ T368] veth2: entered allmulticast mode [ 18.884814][ T368] veth4: entered allmulticast mode [ 18.886635][ T368] veth2: entered promiscuous mode [ 18.886823][ T368] veth4: entered promiscuous mode [ 18.948105][ T369] br1: port 1(lag) entered blocking state [ 18.948340][ T369] br1: port 1(lag) entered forwarding state [ 19.469446][ T378] GACT probability NOT on [ 25.632971][ T405] Mirror/redirect action on [ 40.054991][ T412] br1: port 1(lag) entered disabled state [ 45.897191][ T70] br1: port 1(lag) entered blocking state [ 45.897445][ T70] br1: port 1(lag) entered forwarding state [ 60.797737][ T412] br1: port 1(lag) entered disabled state [ 66.649304][ T151] br1: port 1(lag) entered blocking state [ 66.649558][ T151] br1: port 1(lag) entered forwarding state [ 67.259608][ T510] lag: left allmulticast mode [ 67.259845][ T510] veth2: left allmulticast mode [ 67.260016][ T510] veth4: left allmulticast mode [ 67.260220][ T510] veth2: left promiscuous mode [ 67.260405][ T510] veth4: left promiscuous mode [ 67.260564][ T510] br1: port 1(lag) entered disabled state [ 67.330513][ T358] lag: Port device veth4 removed [ 67.333724][ T358] lag: Port device veth2 removed