[ 831.276927][T11000] lag1: Mode changed to "loadbalance" [ 831.369153][T11002] 8021q: adding VLAN 0 to HW filter on device lag1 [ 831.794692][T11009] [ 831.794813][T11009] ====================================================== [ 831.795001][T11009] WARNING: possible circular locking dependency detected [ 831.795191][T11009] 6.14.0-rc2-virtme #1 Not tainted [ 831.795342][T11009] ------------------------------------------------------ [ 831.795554][T11009] ip/11009 is trying to acquire lock: [ 831.795713][T11009] ffff888008769cb0 (&dev->lock){+.+.}-{4:4}, at: __dev_open+0x57e/0x710 [ 831.795956][T11009] [ 831.795956][T11009] but task is already holding lock: [ 831.796176][T11009] ffff88801052ad80 (team->team_lock_key){+.+.}-{4:4}, at: team_add_slave+0x33/0x80 [ 831.796483][T11009] [ 831.796483][T11009] which lock already depends on the new lock. [ 831.796483][T11009] [ 831.796813][T11009] [ 831.796813][T11009] the existing dependency chain (in reverse order) is: [ 831.797072][T11009] [ 831.797072][T11009] -> #1 (team->team_lock_key){+.+.}-{4:4}: [ 831.797302][T11009] __lock_acquire+0xb9a/0x1680 [ 831.797468][T11009] lock_acquire.part.0+0xeb/0x330 [ 831.797633][T11009] __mutex_lock+0x169/0xc40 [ 831.797788][T11009] team_set_mac_address+0x120/0x250 [ 831.797954][T11009] netif_set_mac_address+0x28a/0x410 [ 831.798149][T11009] do_setlink.constprop.0+0x1980/0x25f0 [ 831.798340][T11009] rtnl_setlink+0x2b5/0x5a0 [ 831.798508][T11009] rtnetlink_rcv_msg+0x712/0xc10 [ 831.798659][T11009] netlink_rcv_skb+0x130/0x360 [ 831.798816][T11009] netlink_unicast+0x44b/0x710 [ 831.798965][T11009] netlink_sendmsg+0x723/0xbe0 [ 831.799122][T11009] __sys_sendto+0x3c3/0x450 [ 831.799282][T11009] __x64_sys_sendto+0xe0/0x1c0 [ 831.799442][T11009] do_syscall_64+0xc1/0x1d0 [ 831.799606][T11009] entry_SYSCALL_64_after_hwframe+0x77/0x7f [ 831.799795][T11009] [ 831.799795][T11009] -> #0 (&dev->lock){+.+.}-{4:4}: [ 831.800021][T11009] check_prev_add+0x182/0xce0 [ 831.800185][T11009] validate_chain+0x7bd/0xae0 [ 831.800350][T11009] __lock_acquire+0xb9a/0x1680 [ 831.800511][T11009] lock_acquire.part.0+0xeb/0x330 [ 831.800664][T11009] __mutex_lock+0x169/0xc40 [ 831.800817][T11009] __dev_open+0x57e/0x710 [ 831.800968][T11009] netif_open+0xc9/0x130 [ 831.801125][T11009] dev_open+0x9f/0x1a0 [ 831.801278][T11009] team_port_add+0x90a/0x1ce0 [ 831.801446][T11009] team_add_slave+0x41/0x80 [ 831.801597][T11009] do_set_master+0x302/0x610 [ 831.801744][T11009] do_setlink.constprop.0+0x87b/0x25f0 [ 831.801931][T11009] rtnl_newlink+0x69c/0xa70 [ 831.802083][T11009] rtnetlink_rcv_msg+0x712/0xc10 [ 831.802238][T11009] netlink_rcv_skb+0x130/0x360 [ 831.802398][T11009] netlink_unicast+0x44b/0x710 [ 831.802559][T11009] netlink_sendmsg+0x723/0xbe0 [ 831.802720][T11009] ____sys_sendmsg+0x7ac/0xa10 [ 831.802887][T11009] ___sys_sendmsg+0xee/0x170 [ 831.803051][T11009] __sys_sendmsg+0x109/0x1a0 [ 831.803212][T11009] do_syscall_64+0xc1/0x1d0 [ 831.803370][T11009] entry_SYSCALL_64_after_hwframe+0x77/0x7f [ 831.803574][T11009] [ 831.803574][T11009] other info that might help us debug this: [ 831.803574][T11009] [ 831.803867][T11009] Possible unsafe locking scenario: [ 831.803867][T11009] [ 831.804111][T11009] CPU0 CPU1 [ 831.804267][T11009] ---- ---- [ 831.804418][T11009] lock(team->team_lock_key); [ 831.804577][T11009] lock(&dev->lock); [ 831.804769][T11009] lock(team->team_lock_key); [ 831.804989][T11009] lock(&dev->lock); [ 831.805107][T11009] [ 831.805107][T11009] *** DEADLOCK *** [ 831.805107][T11009] [ 831.805337][T11009] 2 locks held by ip/11009: [ 831.805497][T11009] #0: ffffffffb35eeb48 (rtnl_mutex){+.+.}-{4:4}, at: rtnl_newlink+0x653/0xa70 [ 831.805785][T11009] #1: ffff88801052ad80 (team->team_lock_key){+.+.}-{4:4}, at: team_add_slave+0x33/0x80 [ 831.806067][T11009] [ 831.806067][T11009] stack backtrace: [ 831.806257][T11009] CPU: 3 UID: 0 PID: 11009 Comm: ip Not tainted 6.14.0-rc2-virtme #1 [ 831.806261][T11009] Hardware name: Bochs Bochs, BIOS Bochs 01/01/2011 [ 831.806264][T11009] Call Trace: [ 831.806265][T11009] [ 831.806267][T11009] dump_stack_lvl+0x82/0xd0 [ 831.806275][T11009] print_circular_bug+0x1b7/0x240 [ 831.806280][T11009] check_noncircular+0x2f9/0x3e0 [ 831.806283][T11009] ? __pfx_check_noncircular+0x10/0x10 [ 831.806287][T11009] ? find_held_lock+0x2c/0x110 [ 831.806293][T11009] ? alloc_chain_hlocks+0x4e0/0x720 [ 831.806296][T11009] check_prev_add+0x182/0xce0 [ 831.806301][T11009] validate_chain+0x7bd/0xae0 [ 831.806305][T11009] ? stack_depot_save_flags+0x27e/0x440 [ 831.806311][T11009] ? __pfx_validate_chain+0x10/0x10 [ 831.806314][T11009] ? hlock_class+0x4e/0x130 [ 831.806317][T11009] ? mark_lock+0x38/0x3e0 [ 831.806321][T11009] __lock_acquire+0xb9a/0x1680 [ 831.806327][T11009] lock_acquire.part.0+0xeb/0x330 [ 831.806330][T11009] ? __dev_open+0x57e/0x710 [ 831.806333][T11009] ? __pfx_lock_acquire.part.0+0x10/0x10 [ 831.806337][T11009] ? trace_lock_acquire+0x14c/0x1f0 [ 831.806340][T11009] ? lock_acquire+0x32/0xc0 [ 831.806343][T11009] ? __dev_open+0x57e/0x710 [ 831.806346][T11009] __mutex_lock+0x169/0xc40 [ 831.806348][T11009] ? __dev_open+0x57e/0x710 [ 831.806354][T11009] ? __dev_open+0x57e/0x710 [ 831.806356][T11009] ? __pfx___mutex_lock+0x10/0x10 [ 831.806361][T11009] ? packet_notifier+0x3b0/0x810 [ 831.806368][T11009] ? __dev_open+0x57e/0x710 [ 831.806370][T11009] __dev_open+0x57e/0x710 [ 831.806372][T11009] ? mark_held_locks+0x9e/0xe0 [ 831.806375][T11009] ? __pfx___dev_open+0x10/0x10 [ 831.806378][T11009] ? trace_percpu_alloc_percpu+0x149/0x1d0 [ 831.806384][T11009] netif_open+0xc9/0x130 [ 831.806387][T11009] ? __pfx_netif_open+0x10/0x10 [ 831.806391][T11009] dev_open+0x9f/0x1a0 [ 831.806395][T11009] team_port_add+0x90a/0x1ce0 [ 831.806400][T11009] ? __pfx_team_port_add+0x10/0x10 [ 831.806403][T11009] ? rcu_read_lock_any_held+0x43/0xb0 [ 831.806408][T11009] ? validate_chain+0x1fe/0xae0 [ 831.806413][T11009] team_add_slave+0x41/0x80 [ 831.806417][T11009] do_set_master+0x302/0x610 [ 831.806421][T11009] do_setlink.constprop.0+0x87b/0x25f0 [ 831.806429][T11009] ? __pfx_do_setlink.constprop.0+0x10/0x10 [ 831.806434][T11009] ? lock_acquire.part.0+0xeb/0x330 [ 831.806439][T11009] ? rtnl_newlink+0x653/0xa70 [ 831.806444][T11009] ? rtnl_newlink+0xb9/0xa70 [ 831.806446][T11009] ? rtnetlink_rcv_msg+0x712/0xc10 [ 831.806449][T11009] ? __mutex_trylock_common+0xfa/0x260 [ 831.806453][T11009] ? __pfx___mutex_trylock_common+0x10/0x10 [ 831.806457][T11009] ? lock_acquire+0x32/0xc0 [ 831.806460][T11009] ? trace_contention_end+0xef/0x150 [ 831.806464][T11009] ? __mutex_lock+0x190/0xc40 [ 831.806466][T11009] ? __pfx_do_raw_spin_lock+0x10/0x10 [ 831.806470][T11009] ? rtnl_newlink+0x653/0xa70 [ 831.806473][T11009] ? __pfx___mutex_lock+0x10/0x10 [ 831.806475][T11009] ? __pfx_validate_nla+0x10/0x10 [ 831.806480][T11009] ? trace_cap_capable+0x11e/0x1a0 [ 831.806486][T11009] ? __rtnl_newlink+0x3f2/0xa80 [ 831.806490][T11009] rtnl_newlink+0x69c/0xa70 [ 831.806494][T11009] ? __pfx_rtnl_newlink+0x10/0x10 [ 831.806497][T11009] ? find_held_lock+0x2c/0x110 [ 831.806502][T11009] ? __pfx___lock_release+0x10/0x10 [ 831.806504][T11009] ? __pfx_lock_acquire.part.0+0x10/0x10 [ 831.806509][T11009] ? rtnetlink_rcv_msg+0x6ef/0xc10 [ 831.806512][T11009] ? __pfx_rtnl_newlink+0x10/0x10 [ 831.806515][T11009] rtnetlink_rcv_msg+0x712/0xc10 [ 831.806519][T11009] ? __pfx_rtnetlink_rcv_msg+0x10/0x10 [ 831.806522][T11009] ? hlock_class+0x4e/0x130 [ 831.806524][T11009] ? mark_lock+0x38/0x3e0 [ 831.806528][T11009] ? __lock_acquire+0xb9a/0x1680 [ 831.806532][T11009] netlink_rcv_skb+0x130/0x360 [ 831.806535][T11009] ? __pfx_rtnetlink_rcv_msg+0x10/0x10 [ 831.806539][T11009] ? __pfx_netlink_rcv_skb+0x10/0x10 [ 831.806546][T11009] ? netlink_deliver_tap+0x13e/0x340 [ 831.806549][T11009] ? netlink_deliver_tap+0xc3/0x340 [ 831.806553][T11009] netlink_unicast+0x44b/0x710 [ 831.806557][T11009] ? __pfx_netlink_unicast+0x10/0x10 [ 831.806560][T11009] ? find_held_lock+0x2c/0x110 [ 831.806565][T11009] netlink_sendmsg+0x723/0xbe0 [ 831.806570][T11009] ? __pfx_netlink_sendmsg+0x10/0x10 [ 831.806575][T11009] ____sys_sendmsg+0x7ac/0xa10 [ 831.806579][T11009] ? __pfx_____sys_sendmsg+0x10/0x10 [ 831.806582][T11009] ? __pfx_copy_msghdr_from_user+0x10/0x10 [ 831.806589][T11009] ___sys_sendmsg+0xee/0x170 [ 831.806593][T11009] ? __pfx____sys_sendmsg+0x10/0x10 [ 831.806597][T11009] ? kasan_save_stack+0x34/0x50 [ 831.806601][T11009] ? kasan_save_stack+0x24/0x50 [ 831.806603][T11009] ? kasan_record_aux_stack+0x8c/0xa0 [ 831.806606][T11009] ? __call_rcu_common.constprop.0+0xa1/0x5b0 [ 831.806610][T11009] ? __x64_sys_close+0x7c/0xd0 [ 831.806614][T11009] ? do_syscall_64+0xc1/0x1d0 [ 831.806616][T11009] ? entry_SYSCALL_64_after_hwframe+0x77/0x7f [ 831.806619][T11009] ? __lock_acquire+0xb9a/0x1680 [ 831.806622][T11009] ? lock_downgrade+0x52/0x110 [ 831.806626][T11009] ? __pfx_do_raw_spin_lock+0x10/0x10 [ 831.806630][T11009] ? find_held_lock+0x2c/0x110 [ 831.806634][T11009] ? __lock_release+0x103/0x460 [ 831.806636][T11009] ? __virt_addr_valid+0x22b/0x430 [ 831.806641][T11009] ? __pfx___lock_release+0x10/0x10 [ 831.806644][T11009] ? __pfx_lock_acquire.part.0+0x10/0x10 [ 831.806648][T11009] __sys_sendmsg+0x109/0x1a0 [ 831.806651][T11009] ? __pfx___sys_sendmsg+0x10/0x10 [ 831.806659][T11009] do_syscall_64+0xc1/0x1d0 [ 831.806662][T11009] entry_SYSCALL_64_after_hwframe+0x77/0x7f [ 831.806665][T11009] RIP: 0033:0x7feb764999a7 [ 831.806669][T11009] Code: 0a 00 f7 d8 64 89 02 48 c7 c0 ff ff ff ff eb b9 0f 1f 00 f3 0f 1e fa 64 8b 04 25 18 00 00 00 85 c0 75 10 b8 2e 00 00 00 0f 05 <48> 3d 00 f0 ff ff 77 51 c3 48 83 ec 28 89 54 24 1c 48 89 74 24 10 [ 831.806672][T11009] RSP: 002b:00007fffab41cb68 EFLAGS: 00000246 ORIG_RAX: 000000000000002e [ 831.806676][T11009] RAX: ffffffffffffffda RBX: 00007fffab41d290 RCX: 00007feb764999a7 [ 831.806678][T11009] RDX: 0000000000000000 RSI: 00007fffab41cbd0 RDI: 0000000000000005 [ 831.806679][T11009] RBP: 0000000000000004 R08: 0000000000000003 R09: 0000000000000000 [ 831.806681][T11009] R10: 0000000003067910 R11: 0000000000000246 R12: 0000000000000004 [ 831.806682][T11009] R13: 0000000067b35566 R14: 0000000000498600 R15: 0000000000000000 [ 831.806688][T11009] [ 831.826327][T11009] lag1: Port device veth0 added [ 831.846106][T11009] ip (11009) used greatest stack depth: 23536 bytes left [ 831.886043][T11010] lag1: Port device veth6 added [ 832.205435][T11016] 8021q: adding VLAN 0 to HW filter on device lag1 [ 833.449257][T11039] lag4: Mode changed to "loadbalance" [ 833.531330][T11041] 8021q: adding VLAN 0 to HW filter on device lag4 [ 833.834043][T11047] lag4: Port device veth3 added [ 833.927532][T11048] lag4: Port device veth5 added [ 834.239007][T11054] 8021q: adding VLAN 0 to HW filter on device lag4 [ 835.543727][T11077] lag2: Mode changed to "loadbalance" [ 835.624359][T11079] 8021q: adding VLAN 0 to HW filter on device lag2 [ 835.973750][T11085] lag2: Port device veth1 added [ 836.062133][T11086] lag2: Port device veth7 added [ 836.656094][T11099] br1: port 1(lag2.100) entered blocking state [ 836.656360][T11099] br1: port 1(lag2.100) entered disabled state [ 836.656689][T11099] lag2.100: entered allmulticast mode [ 836.656850][T11099] lag2: entered allmulticast mode [ 836.657013][T11099] veth1: entered allmulticast mode [ 836.657180][T11099] veth7: entered allmulticast mode [ 836.659261][T11099] lag2.100: entered promiscuous mode [ 836.659473][T11099] lag2: entered promiscuous mode [ 836.659646][T11099] veth1: entered promiscuous mode [ 836.659996][T11099] veth7: entered promiscuous mode [ 836.982934][T11106] br2: port 1(lag2.200) entered blocking state [ 836.983329][T11106] br2: port 1(lag2.200) entered disabled state [ 836.984079][T11106] lag2.200: entered allmulticast mode [ 836.986707][T11106] lag2.200: entered promiscuous mode [ 837.167342][T11109] br1: port 1(lag2.100) entered blocking state [ 837.167651][T11109] br1: port 1(lag2.100) entered forwarding state [ 837.236379][T11110] br2: port 1(lag2.200) entered blocking state [ 837.236710][T11110] br2: port 1(lag2.200) entered forwarding state [ 837.631881][T11118] lag3: Mode changed to "loadbalance" [ 837.699120][T11120] 8021q: adding VLAN 0 to HW filter on device lag3 [ 838.002906][T11126] lag3: Port device veth2 added [ 838.075439][T11127] lag3: Port device veth4 added [ 849.630822][T11208] veth7: left promiscuous mode [ 849.631248][T11208] veth7: left allmulticast mode [ 849.632757][T11208] lag2: Port device veth7 removed [ 864.957810][T11269] veth7: entered promiscuous mode [ 864.958202][T11269] veth7: entered allmulticast mode [ 864.959184][T11269] lag2: Port device veth7 added [ 865.221620][T11280] veth1: left promiscuous mode [ 865.222090][T11280] veth1: left allmulticast mode [ 865.223540][T11280] lag2: Port device veth1 removed [ 880.464290][ T119] br1: port 1(lag2.100) entered disabled state [ 880.465369][ T119] br2: port 1(lag2.200) entered disabled state [ 880.527413][T11341] veth7: left promiscuous mode [ 880.527888][T11341] veth7: left allmulticast mode [ 880.537774][T11341] lag2: Port device veth7 removed [ 880.769806][T11352] veth1: entered promiscuous mode [ 880.770249][T11352] veth1: entered allmulticast mode [ 880.771059][T11352] lag2: Port device veth1 added [ 880.801145][ T9769] br1: port 1(lag2.100) entered blocking state [ 880.801468][ T9769] br1: port 1(lag2.100) entered forwarding state [ 880.802201][ T9769] br2: port 1(lag2.200) entered blocking state [ 880.802461][ T9769] br2: port 1(lag2.200) entered forwarding state [ 881.007560][T11363] veth7: entered promiscuous mode [ 881.007952][T11363] veth7: entered allmulticast mode [ 881.008729][T11363] lag2: Port device veth7 added [ 896.134585][T11423] lag2.200: left allmulticast mode [ 896.134947][T11423] lag2.200: left promiscuous mode [ 896.136230][T11423] br2: port 1(lag2.200) entered disabled state [ 896.211337][T11424] lag2.100: left allmulticast mode [ 896.211588][T11424] lag2: left allmulticast mode [ 896.211779][T11424] veth1: left allmulticast mode [ 896.211985][T11424] veth7: left allmulticast mode [ 896.212175][T11424] lag2.100: left promiscuous mode [ 896.212346][T11424] lag2: left promiscuous mode [ 896.212534][T11424] veth1: left promiscuous mode [ 896.213035][T11424] veth7: left promiscuous mode [ 896.213781][T11424] br1: port 1(lag2.100) entered disabled state [ 898.330360][T11426] br1: port 1(lag2.100) entered blocking state [ 898.330620][T11426] br1: port 1(lag2.100) entered disabled state [ 898.330855][T11426] lag2.100: entered allmulticast mode [ 898.331004][T11426] lag2: entered allmulticast mode [ 898.331154][T11426] veth1: entered allmulticast mode [ 898.331307][T11426] veth7: entered allmulticast mode [ 898.340567][T11426] lag2.100: entered promiscuous mode [ 898.340785][T11426] lag2: entered promiscuous mode [ 898.341009][T11426] veth1: entered promiscuous mode [ 898.341370][T11426] veth7: entered promiscuous mode [ 898.342537][T11426] br1: port 1(lag2.100) entered blocking state [ 898.342739][T11426] br1: port 1(lag2.100) entered forwarding state [ 898.429389][T11427] br2: port 1(lag2.200) entered blocking state [ 898.429791][T11427] br2: port 1(lag2.200) entered disabled state [ 898.430143][T11427] lag2.200: entered allmulticast mode [ 898.431825][T11427] lag2.200: entered promiscuous mode [ 898.433185][T11427] br2: port 1(lag2.200) entered blocking state [ 898.433420][T11427] br2: port 1(lag2.200) entered forwarding state [ 913.638920][T11487] lag3: Port device veth2 removed [ 928.953468][T11548] lag3: Port device veth2 added [ 929.201679][T11559] lag3: Port device veth4 removed [ 944.381850][T11620] lag3: Port device veth2 removed [ 944.619454][T11631] lag3: Port device veth4 added [ 944.897857][T11642] lag3: Port device veth2 added [ 960.661016][T11712] lag3: Port device veth4 removed [ 960.725019][T11713] lag3: Port device veth2 removed [ 961.366140][T11725] br2: port 1(lag2.200) entered disabled state [ 961.441722][T11726] br1: port 1(lag2.100) entered disabled state [ 961.646223][T11729] lag2.200: left allmulticast mode [ 961.646450][T11729] lag2.200: left promiscuous mode [ 961.646947][T11729] br2: port 1(lag2.200) entered disabled state [ 961.847850][T11731] lag2.100: left allmulticast mode [ 961.848127][T11731] lag2: left allmulticast mode [ 961.848366][T11731] veth1: left allmulticast mode [ 961.849781][T11731] veth7: left allmulticast mode [ 961.850050][T11731] lag2.100: left promiscuous mode [ 961.850287][T11731] lag2: left promiscuous mode [ 961.850542][T11731] veth1: left promiscuous mode [ 961.851104][T11731] veth7: left promiscuous mode [ 961.852038][T11731] br1: port 1(lag2.100) entered disabled state [ 962.232504][T11735] lag2: Port device veth7 removed [ 962.289036][T11736] lag2: Port device veth1 removed [ 963.587282][T11753] lag4: Port device veth5 removed [ 963.646002][T11754] lag4: Port device veth3 removed [ 964.894105][T11769] lag1: Port device veth6 removed [ 964.957902][T11770] lag1: Port device veth0 removed