[ 851.168727][ T4565] lag1: Mode changed to "loadbalance" [ 851.272702][ T4567] 8021q: adding VLAN 0 to HW filter on device lag1 [ 851.532668][ T4574] [ 851.532832][ T4574] ====================================================== [ 851.533053][ T4574] WARNING: possible circular locking dependency detected [ 851.533287][ T4574] 6.14.0-rc2-virtme #1 Not tainted [ 851.533442][ T4574] ------------------------------------------------------ [ 851.533641][ T4574] ip/4574 is trying to acquire lock: [ 851.533829][ T4574] ffff8880077d9cb0 (&dev->lock){+.+.}-{4:4}, at: __dev_open+0x57e/0x710 [ 851.534071][ T4574] [ 851.534071][ T4574] but task is already holding lock: [ 851.534294][ T4574] ffff88800a6b2d80 (team->team_lock_key){+.+.}-{4:4}, at: team_add_slave+0x33/0x80 [ 851.534575][ T4574] [ 851.534575][ T4574] which lock already depends on the new lock. [ 851.534575][ T4574] [ 851.534931][ T4574] [ 851.534931][ T4574] the existing dependency chain (in reverse order) is: [ 851.535192][ T4574] [ 851.535192][ T4574] -> #1 (team->team_lock_key){+.+.}-{4:4}: [ 851.535468][ T4574] __lock_acquire+0xb9a/0x1680 [ 851.535661][ T4574] lock_acquire.part.0+0xeb/0x330 [ 851.535832][ T4574] __mutex_lock+0x169/0xc40 [ 851.535994][ T4574] team_set_mac_address+0x120/0x250 [ 851.536187][ T4574] netif_set_mac_address+0x28a/0x410 [ 851.536411][ T4574] do_setlink.constprop.0+0x1980/0x25f0 [ 851.536627][ T4574] rtnl_setlink+0x2b5/0x5a0 [ 851.536809][ T4574] rtnetlink_rcv_msg+0x712/0xc10 [ 851.536971][ T4574] netlink_rcv_skb+0x130/0x360 [ 851.537152][ T4574] netlink_unicast+0x44b/0x710 [ 851.537321][ T4574] netlink_sendmsg+0x723/0xbe0 [ 851.537482][ T4574] __sys_sendto+0x3c3/0x450 [ 851.537658][ T4574] __x64_sys_sendto+0xe0/0x1c0 [ 851.537837][ T4574] do_syscall_64+0xc1/0x1d0 [ 851.538007][ T4574] entry_SYSCALL_64_after_hwframe+0x77/0x7f [ 851.538214][ T4574] [ 851.538214][ T4574] -> #0 (&dev->lock){+.+.}-{4:4}: [ 851.538459][ T4574] check_prev_add+0x182/0xce0 [ 851.538633][ T4574] validate_chain+0x7bd/0xae0 [ 851.538817][ T4574] __lock_acquire+0xb9a/0x1680 [ 851.538980][ T4574] lock_acquire.part.0+0xeb/0x330 [ 851.539141][ T4574] __mutex_lock+0x169/0xc40 [ 851.539292][ T4574] __dev_open+0x57e/0x710 [ 851.539441][ T4574] netif_open+0xc9/0x130 [ 851.539589][ T4574] dev_open+0x9f/0x1a0 [ 851.539759][ T4574] team_port_add+0x90a/0x1ce0 [ 851.539919][ T4574] team_add_slave+0x41/0x80 [ 851.540073][ T4574] do_set_master+0x302/0x610 [ 851.540222][ T4574] do_setlink.constprop.0+0x87b/0x25f0 [ 851.540410][ T4574] rtnl_newlink+0x69c/0xa70 [ 851.540560][ T4574] rtnetlink_rcv_msg+0x712/0xc10 [ 851.540722][ T4574] netlink_rcv_skb+0x130/0x360 [ 851.540878][ T4574] netlink_unicast+0x44b/0x710 [ 851.541029][ T4574] netlink_sendmsg+0x723/0xbe0 [ 851.541178][ T4574] ____sys_sendmsg+0x7ac/0xa10 [ 851.541329][ T4574] ___sys_sendmsg+0xee/0x170 [ 851.541478][ T4574] __sys_sendmsg+0x109/0x1a0 [ 851.541625][ T4574] do_syscall_64+0xc1/0x1d0 [ 851.541787][ T4574] entry_SYSCALL_64_after_hwframe+0x77/0x7f [ 851.541984][ T4574] [ 851.541984][ T4574] other info that might help us debug this: [ 851.541984][ T4574] [ 851.542279][ T4574] Possible unsafe locking scenario: [ 851.542279][ T4574] [ 851.542500][ T4574] CPU0 CPU1 [ 851.542648][ T4574] ---- ---- [ 851.542822][ T4574] lock(team->team_lock_key); [ 851.542986][ T4574] lock(&dev->lock); [ 851.543181][ T4574] lock(team->team_lock_key); [ 851.543402][ T4574] lock(&dev->lock); [ 851.543515][ T4574] [ 851.543515][ T4574] *** DEADLOCK *** [ 851.543515][ T4574] [ 851.543756][ T4574] 2 locks held by ip/4574: [ 851.543920][ T4574] #0: ffffffffa8feeb48 (rtnl_mutex){+.+.}-{4:4}, at: rtnl_newlink+0x653/0xa70 [ 851.544189][ T4574] #1: ffff88800a6b2d80 (team->team_lock_key){+.+.}-{4:4}, at: team_add_slave+0x33/0x80 [ 851.544454][ T4574] [ 851.544454][ T4574] stack backtrace: [ 851.544635][ T4574] CPU: 2 UID: 0 PID: 4574 Comm: ip Not tainted 6.14.0-rc2-virtme #1 [ 851.544640][ T4574] Hardware name: Bochs Bochs, BIOS Bochs 01/01/2011 [ 851.544642][ T4574] Call Trace: [ 851.544644][ T4574] [ 851.544646][ T4574] dump_stack_lvl+0x82/0xd0 [ 851.544653][ T4574] print_circular_bug+0x1b7/0x240 [ 851.544659][ T4574] check_noncircular+0x2f9/0x3e0 [ 851.544663][ T4574] ? __pfx_check_noncircular+0x10/0x10 [ 851.544667][ T4574] ? find_held_lock+0x2c/0x110 [ 851.544673][ T4574] ? alloc_chain_hlocks+0x4e0/0x720 [ 851.544677][ T4574] check_prev_add+0x182/0xce0 [ 851.544682][ T4574] validate_chain+0x7bd/0xae0 [ 851.544686][ T4574] ? stack_depot_save_flags+0x27e/0x440 [ 851.544692][ T4574] ? __pfx_validate_chain+0x10/0x10 [ 851.544695][ T4574] ? hlock_class+0x4e/0x130 [ 851.544697][ T4574] ? mark_lock+0x38/0x3e0 [ 851.544701][ T4574] __lock_acquire+0xb9a/0x1680 [ 851.544706][ T4574] ? mark_lock+0x38/0x3e0 [ 851.544710][ T4574] lock_acquire.part.0+0xeb/0x330 [ 851.544713][ T4574] ? __dev_open+0x57e/0x710 [ 851.544716][ T4574] ? validate_chain+0x162/0xae0 [ 851.544719][ T4574] ? __pfx_lock_acquire.part.0+0x10/0x10 [ 851.544722][ T4574] ? trace_lock_acquire+0x14c/0x1f0 [ 851.544725][ T4574] ? lock_acquire+0x32/0xc0 [ 851.544728][ T4574] ? __dev_open+0x57e/0x710 [ 851.544731][ T4574] __mutex_lock+0x169/0xc40 [ 851.544734][ T4574] ? __dev_open+0x57e/0x710 [ 851.544737][ T4574] ? __dev_open+0x57e/0x710 [ 851.544739][ T4574] ? __pfx___mutex_lock+0x10/0x10 [ 851.544743][ T4574] ? packet_notifier+0x3b0/0x810 [ 851.544752][ T4574] ? __dev_open+0x57e/0x710 [ 851.544754][ T4574] __dev_open+0x57e/0x710 [ 851.544757][ T4574] ? mark_held_locks+0x9e/0xe0 [ 851.544760][ T4574] ? __pfx___dev_open+0x10/0x10 [ 851.544762][ T4574] ? trace_percpu_alloc_percpu+0x149/0x1d0 [ 851.544769][ T4574] netif_open+0xc9/0x130 [ 851.544771][ T4574] ? __pfx_netif_open+0x10/0x10 [ 851.544775][ T4574] dev_open+0x9f/0x1a0 [ 851.544780][ T4574] team_port_add+0x90a/0x1ce0 [ 851.544785][ T4574] ? __pfx_team_port_add+0x10/0x10 [ 851.544788][ T4574] ? rcu_read_lock_any_held+0x43/0xb0 [ 851.544793][ T4574] ? validate_chain+0x1fe/0xae0 [ 851.544798][ T4574] team_add_slave+0x41/0x80 [ 851.544802][ T4574] do_set_master+0x302/0x610 [ 851.544807][ T4574] do_setlink.constprop.0+0x87b/0x25f0 [ 851.544811][ T4574] ? __pfx_do_setlink.constprop.0+0x10/0x10 [ 851.544815][ T4574] ? lock_acquire.part.0+0xeb/0x330 [ 851.544818][ T4574] ? rtnl_newlink+0x653/0xa70 [ 851.544821][ T4574] ? rtnl_newlink+0xb9/0xa70 [ 851.544823][ T4574] ? rtnetlink_rcv_msg+0x712/0xc10 [ 851.544826][ T4574] ? __mutex_trylock_common+0xfa/0x260 [ 851.544830][ T4574] ? __pfx___mutex_trylock_common+0x10/0x10 [ 851.544834][ T4574] ? lock_acquire+0x32/0xc0 [ 851.544837][ T4574] ? trace_contention_end+0xef/0x150 [ 851.544841][ T4574] ? __mutex_lock+0x190/0xc40 [ 851.544843][ T4574] ? __pfx_do_raw_spin_lock+0x10/0x10 [ 851.544847][ T4574] ? rtnl_newlink+0x653/0xa70 [ 851.544850][ T4574] ? __pfx___mutex_lock+0x10/0x10 [ 851.544852][ T4574] ? __pfx_validate_nla+0x10/0x10 [ 851.544857][ T4574] ? trace_cap_capable+0x11e/0x1a0 [ 851.544862][ T4574] ? __rtnl_newlink+0x3f2/0xa80 [ 851.544867][ T4574] rtnl_newlink+0x69c/0xa70 [ 851.544871][ T4574] ? __pfx_rtnl_newlink+0x10/0x10 [ 851.544874][ T4574] ? find_held_lock+0x2c/0x110 [ 851.544879][ T4574] ? __pfx___lock_release+0x10/0x10 [ 851.544882][ T4574] ? __pfx_lock_acquire.part.0+0x10/0x10 [ 851.544887][ T4574] ? rtnetlink_rcv_msg+0x6ef/0xc10 [ 851.544890][ T4574] ? __pfx_rtnl_newlink+0x10/0x10 [ 851.544893][ T4574] rtnetlink_rcv_msg+0x712/0xc10 [ 851.544896][ T4574] ? __pfx_rtnetlink_rcv_msg+0x10/0x10 [ 851.544899][ T4574] ? hlock_class+0x4e/0x130 [ 851.544901][ T4574] ? mark_lock+0x38/0x3e0 [ 851.544906][ T4574] ? __lock_acquire+0xb9a/0x1680 [ 851.544910][ T4574] netlink_rcv_skb+0x130/0x360 [ 851.544914][ T4574] ? __pfx_rtnetlink_rcv_msg+0x10/0x10 [ 851.544917][ T4574] ? __pfx_netlink_rcv_skb+0x10/0x10 [ 851.544925][ T4574] ? netlink_deliver_tap+0x13e/0x340 [ 851.544929][ T4574] ? netlink_deliver_tap+0xc3/0x340 [ 851.544932][ T4574] netlink_unicast+0x44b/0x710 [ 851.544937][ T4574] ? __pfx_netlink_unicast+0x10/0x10 [ 851.544940][ T4574] ? find_held_lock+0x2c/0x110 [ 851.544945][ T4574] netlink_sendmsg+0x723/0xbe0 [ 851.544950][ T4574] ? __pfx_netlink_sendmsg+0x10/0x10 [ 851.544956][ T4574] ____sys_sendmsg+0x7ac/0xa10 [ 851.544960][ T4574] ? __pfx_____sys_sendmsg+0x10/0x10 [ 851.544963][ T4574] ? __pfx_copy_msghdr_from_user+0x10/0x10 [ 851.544970][ T4574] ___sys_sendmsg+0xee/0x170 [ 851.544974][ T4574] ? __pfx____sys_sendmsg+0x10/0x10 [ 851.544978][ T4574] ? kasan_save_stack+0x34/0x50 [ 851.544982][ T4574] ? kasan_save_stack+0x24/0x50 [ 851.544984][ T4574] ? kasan_record_aux_stack+0x8c/0xa0 [ 851.544987][ T4574] ? __call_rcu_common.constprop.0+0xa1/0x5b0 [ 851.544991][ T4574] ? __x64_sys_close+0x7c/0xd0 [ 851.544994][ T4574] ? do_syscall_64+0xc1/0x1d0 [ 851.544997][ T4574] ? entry_SYSCALL_64_after_hwframe+0x77/0x7f [ 851.545001][ T4574] ? __lock_acquire+0xb9a/0x1680 [ 851.545004][ T4574] ? lock_downgrade+0x52/0x110 [ 851.545008][ T4574] ? __pfx_do_raw_spin_lock+0x10/0x10 [ 851.545011][ T4574] ? find_held_lock+0x2c/0x110 [ 851.545015][ T4574] ? __lock_release+0x103/0x460 [ 851.545018][ T4574] ? __virt_addr_valid+0x22b/0x430 [ 851.545022][ T4574] ? __pfx___lock_release+0x10/0x10 [ 851.545025][ T4574] ? __pfx_lock_acquire.part.0+0x10/0x10 [ 851.545029][ T4574] __sys_sendmsg+0x109/0x1a0 [ 851.545033][ T4574] ? __pfx___sys_sendmsg+0x10/0x10 [ 851.545041][ T4574] do_syscall_64+0xc1/0x1d0 [ 851.545044][ T4574] entry_SYSCALL_64_after_hwframe+0x77/0x7f [ 851.545047][ T4574] RIP: 0033:0x7f2233cc69a7 [ 851.545051][ T4574] Code: 0a 00 f7 d8 64 89 02 48 c7 c0 ff ff ff ff eb b9 0f 1f 00 f3 0f 1e fa 64 8b 04 25 18 00 00 00 85 c0 75 10 b8 2e 00 00 00 0f 05 <48> 3d 00 f0 ff ff 77 51 c3 48 83 ec 28 89 54 24 1c 48 89 74 24 10 [ 851.545054][ T4574] RSP: 002b:00007ffe77f1aca8 EFLAGS: 00000246 ORIG_RAX: 000000000000002e [ 851.545057][ T4574] RAX: ffffffffffffffda RBX: 00007ffe77f1b3d0 RCX: 00007f2233cc69a7 [ 851.545059][ T4574] RDX: 0000000000000000 RSI: 00007ffe77f1ad10 RDI: 0000000000000005 [ 851.545061][ T4574] RBP: 0000000000000004 R08: 0000000000000003 R09: 0000000000000000 [ 851.545063][ T4574] R10: 000000003a3a7910 R11: 0000000000000246 R12: 0000000000000004 [ 851.545065][ T4574] R13: 0000000067b2d6a8 R14: 0000000000498600 R15: 0000000000000000 [ 851.545070][ T4574] [ 851.585657][ T4574] lag1: Port device veth0 added [ 851.591124][ T4574] ip (4574) used greatest stack depth: 23504 bytes left [ 851.647205][ T4583] lag1: Port device veth6 added [ 851.940825][ T4589] 8021q: adding VLAN 0 to HW filter on device lag1 [ 852.459080][ T4600] lag4: Mode changed to "loadbalance" [ 852.530009][ T4602] 8021q: adding VLAN 0 to HW filter on device lag4 [ 852.781761][ T4607] lag4: Port device veth3 added [ 852.854210][ T4608] lag4: Port device veth5 added [ 853.182646][ T4614] 8021q: adding VLAN 0 to HW filter on device lag4 [ 853.745598][ T4625] lag2: Mode changed to "loadbalance" [ 853.817214][ T4627] 8021q: adding VLAN 0 to HW filter on device lag2 [ 854.074381][ T4632] lag2: Port device veth1 added [ 854.152745][ T4633] lag2: Port device veth7 added [ 854.365253][ T4638] br1: port 1(lag2) entered blocking state [ 854.365507][ T4638] br1: port 1(lag2) entered disabled state [ 854.365745][ T4638] lag2: entered allmulticast mode [ 854.365902][ T4638] veth1: entered allmulticast mode [ 854.366065][ T4638] veth7: entered allmulticast mode [ 854.367990][ T4638] veth1: entered promiscuous mode [ 854.368215][ T4638] veth7: entered promiscuous mode [ 854.549275][ T4642] br1: port 1(lag2) entered blocking state [ 854.549524][ T4642] br1: port 1(lag2) entered forwarding state [ 854.807746][ T4648] lag3: Mode changed to "loadbalance" [ 854.886513][ T4650] 8021q: adding VLAN 0 to HW filter on device lag3 [ 855.158394][ T4655] lag3: Port device veth2 added [ 855.244223][ T4656] lag3: Port device veth4 added [ 864.147098][ T4720] veth7: left allmulticast mode [ 864.148548][ T4720] lag2: Port device veth7 removed [ 864.213405][ T4721] veth7: left promiscuous mode [ 877.087874][ T4771] veth7: entered allmulticast mode [ 877.088137][ T4771] veth7: entered promiscuous mode [ 877.089074][ T4771] lag2: Port device veth7 added [ 877.349000][ T4782] veth1: left allmulticast mode [ 877.350377][ T4782] lag2: Port device veth1 removed [ 877.410159][ T4783] veth1: left promiscuous mode [ 890.240713][ T2706] br1: port 1(lag2) entered disabled state [ 890.301148][ T4833] veth7: left allmulticast mode [ 890.302671][ T4833] lag2: Port device veth7 removed [ 890.356808][ T4834] veth7: left promiscuous mode [ 890.526656][ T4844] veth1: entered allmulticast mode [ 890.526879][ T4844] veth1: entered promiscuous mode [ 890.527677][ T4844] lag2: Port device veth1 added [ 890.539137][ T258] br1: port 1(lag2) entered blocking state [ 890.539443][ T258] br1: port 1(lag2) entered forwarding state [ 890.785922][ T4855] veth7: entered allmulticast mode [ 890.786155][ T4855] veth7: entered promiscuous mode [ 890.786977][ T4855] lag2: Port device veth7 added [ 903.619915][ T4905] lag2: left allmulticast mode [ 903.620225][ T4905] veth1: left allmulticast mode [ 903.620466][ T4905] veth7: left allmulticast mode [ 903.620725][ T4905] veth1: left promiscuous mode [ 903.620994][ T4905] veth7: left promiscuous mode [ 903.622105][ T4905] br1: port 1(lag2) entered disabled state [ 905.749729][ T4907] br1: port 1(lag2) entered blocking state [ 905.750112][ T4907] br1: port 1(lag2) entered disabled state [ 905.751868][ T4907] lag2: entered allmulticast mode [ 905.752125][ T4907] veth1: entered allmulticast mode [ 905.752389][ T4907] veth7: entered allmulticast mode [ 905.755101][ T4907] veth1: entered promiscuous mode [ 905.755459][ T4907] veth7: entered promiscuous mode [ 905.756493][ T4907] br1: port 1(lag2) entered blocking state [ 905.756835][ T4907] br1: port 1(lag2) entered forwarding state [ 918.668906][ T4957] lag3: Port device veth2 removed [ 931.637700][ T5008] lag3: Port device veth2 added [ 931.916681][ T5019] lag3: Port device veth4 removed [ 944.911080][ T5070] lag3: Port device veth2 removed [ 945.182598][ T5081] lag3: Port device veth4 added [ 945.480464][ T5092] lag3: Port device veth2 added [ 958.363163][ T5142] br1: port 2(lag3) entered blocking state [ 958.363774][ T5142] br1: port 2(lag3) entered disabled state [ 958.364034][ T5142] lag3: entered allmulticast mode [ 958.364195][ T5142] veth4: entered allmulticast mode [ 958.364345][ T5142] veth2: entered allmulticast mode [ 958.365980][ T5142] lag3: entered promiscuous mode [ 958.366149][ T5142] veth4: entered promiscuous mode [ 958.366521][ T5142] veth2: entered promiscuous mode [ 958.367005][ T5142] lag2: entered promiscuous mode [ 958.368146][ T5142] br1: port 2(lag3) entered blocking state [ 958.368367][ T5142] br1: port 2(lag3) entered forwarding state [ 960.469426][ T5144] lag3: left allmulticast mode [ 960.469640][ T5144] veth4: left allmulticast mode [ 960.469801][ T5144] veth2: left allmulticast mode [ 960.469957][ T5144] lag3: left promiscuous mode [ 960.470101][ T5144] veth4: left promiscuous mode [ 960.471138][ T5144] veth2: left promiscuous mode [ 960.471664][ T5144] br1: port 2(lag3) entered disabled state [ 960.472466][ T5144] lag2: left promiscuous mode [ 973.680443][ T5197] lag3: Port device veth4 removed [ 973.750551][ T5198] lag3: Port device veth2 removed [ 974.431367][ T40] br1: port 1(lag2) entered disabled state [ 974.562220][ T5213] lag2: left allmulticast mode [ 974.562531][ T5213] veth1: left allmulticast mode [ 974.562767][ T5213] veth7: left allmulticast mode [ 974.563038][ T5213] br1: port 1(lag2) entered disabled state [ 974.763352][ T5215] lag2: Port device veth7 removed [ 974.830586][ T5216] lag2: Port device veth1 removed [ 976.053012][ T5229] lag4: Port device veth5 removed [ 976.133775][ T5230] lag4: Port device veth3 removed [ 977.289644][ T5245] lag1: Port device veth6 removed [ 977.352622][ T5246] lag1: Port device veth0 removed