[ 104.197471][ T1145] lag1: Mode changed to "loadbalance" [ 104.291904][ T1146] 8021q: adding VLAN 0 to HW filter on device lag1 [ 104.709072][ T1155] [ 104.709217][ T1155] ====================================================== [ 104.709422][ T1155] WARNING: possible circular locking dependency detected [ 104.709622][ T1155] 6.14.0-rc2-virtme #1 Not tainted [ 104.709782][ T1155] ------------------------------------------------------ [ 104.709982][ T1155] ip/1155 is trying to acquire lock: [ 104.710146][ T1155] ffff88800716ccb0 (&dev->lock){+.+.}-{4:4}, at: __dev_open+0x57e/0x710 [ 104.710411][ T1155] [ 104.710411][ T1155] but task is already holding lock: [ 104.710663][ T1155] ffff88800faaad80 (team->team_lock_key){+.+.}-{4:4}, at: team_add_slave+0x33/0x80 [ 104.710969][ T1155] [ 104.710969][ T1155] which lock already depends on the new lock. [ 104.710969][ T1155] [ 104.711294][ T1155] [ 104.711294][ T1155] the existing dependency chain (in reverse order) is: [ 104.711583][ T1155] [ 104.711583][ T1155] -> #1 (team->team_lock_key){+.+.}-{4:4}: [ 104.711838][ T1155] __lock_acquire+0xb9a/0x1680 [ 104.712014][ T1155] lock_acquire.part.0+0xeb/0x330 [ 104.712176][ T1155] __mutex_lock+0x169/0xc40 [ 104.712341][ T1155] team_set_mac_address+0x120/0x250 [ 104.712502][ T1155] netif_set_mac_address+0x28a/0x410 [ 104.712711][ T1155] do_setlink.constprop.0+0x1980/0x25f0 [ 104.712933][ T1155] rtnl_setlink+0x2b5/0x5a0 [ 104.713100][ T1155] rtnetlink_rcv_msg+0x712/0xc10 [ 104.713261][ T1155] netlink_rcv_skb+0x130/0x360 [ 104.713440][ T1155] netlink_unicast+0x44b/0x710 [ 104.713601][ T1155] netlink_sendmsg+0x723/0xbe0 [ 104.713758][ T1155] __sys_sendto+0x3c3/0x450 [ 104.713923][ T1155] __x64_sys_sendto+0xe0/0x1c0 [ 104.714081][ T1155] do_syscall_64+0xc1/0x1d0 [ 104.714250][ T1155] entry_SYSCALL_64_after_hwframe+0x77/0x7f [ 104.714453][ T1155] [ 104.714453][ T1155] -> #0 (&dev->lock){+.+.}-{4:4}: [ 104.714698][ T1155] check_prev_add+0x182/0xce0 [ 104.714864][ T1155] validate_chain+0x7bd/0xae0 [ 104.715024][ T1155] __lock_acquire+0xb9a/0x1680 [ 104.715185][ T1155] lock_acquire.part.0+0xeb/0x330 [ 104.715353][ T1155] __mutex_lock+0x169/0xc40 [ 104.715513][ T1155] __dev_open+0x57e/0x710 [ 104.715671][ T1155] netif_open+0xc9/0x130 [ 104.715833][ T1155] dev_open+0x9f/0x1a0 [ 104.715997][ T1155] team_port_add+0x90a/0x1ce0 [ 104.716158][ T1155] team_add_slave+0x41/0x80 [ 104.716321][ T1155] do_set_master+0x302/0x610 [ 104.716481][ T1155] do_setlink.constprop.0+0x87b/0x25f0 [ 104.716685][ T1155] rtnl_newlink+0x69c/0xa70 [ 104.716844][ T1155] rtnetlink_rcv_msg+0x712/0xc10 [ 104.717003][ T1155] netlink_rcv_skb+0x130/0x360 [ 104.717166][ T1155] netlink_unicast+0x44b/0x710 [ 104.717329][ T1155] netlink_sendmsg+0x723/0xbe0 [ 104.717488][ T1155] ____sys_sendmsg+0x7ac/0xa10 [ 104.717655][ T1155] ___sys_sendmsg+0xee/0x170 [ 104.717821][ T1155] __sys_sendmsg+0x109/0x1a0 [ 104.717985][ T1155] do_syscall_64+0xc1/0x1d0 [ 104.718148][ T1155] entry_SYSCALL_64_after_hwframe+0x77/0x7f [ 104.718353][ T1155] [ 104.718353][ T1155] other info that might help us debug this: [ 104.718353][ T1155] [ 104.718672][ T1155] Possible unsafe locking scenario: [ 104.718672][ T1155] [ 104.718917][ T1155] CPU0 CPU1 [ 104.719079][ T1155] ---- ---- [ 104.719245][ T1155] lock(team->team_lock_key); [ 104.719410][ T1155] lock(&dev->lock); [ 104.719615][ T1155] lock(team->team_lock_key); [ 104.719861][ T1155] lock(&dev->lock); [ 104.719995][ T1155] [ 104.719995][ T1155] *** DEADLOCK *** [ 104.719995][ T1155] [ 104.720233][ T1155] 2 locks held by ip/1155: [ 104.720399][ T1155] #0: ffffffffa7beeb48 (rtnl_mutex){+.+.}-{4:4}, at: rtnl_newlink+0x653/0xa70 [ 104.720689][ T1155] #1: ffff88800faaad80 (team->team_lock_key){+.+.}-{4:4}, at: team_add_slave+0x33/0x80 [ 104.720981][ T1155] [ 104.720981][ T1155] stack backtrace: [ 104.721191][ T1155] CPU: 3 UID: 0 PID: 1155 Comm: ip Not tainted 6.14.0-rc2-virtme #1 [ 104.721195][ T1155] Hardware name: Bochs Bochs, BIOS Bochs 01/01/2011 [ 104.721198][ T1155] Call Trace: [ 104.721200][ T1155] [ 104.721202][ T1155] dump_stack_lvl+0x82/0xd0 [ 104.721211][ T1155] print_circular_bug+0x1b7/0x240 [ 104.721216][ T1155] check_noncircular+0x2f9/0x3e0 [ 104.721220][ T1155] ? __pfx_check_noncircular+0x10/0x10 [ 104.721224][ T1155] ? find_held_lock+0x2c/0x110 [ 104.721231][ T1155] ? alloc_chain_hlocks+0x4e0/0x720 [ 104.721235][ T1155] check_prev_add+0x182/0xce0 [ 104.721239][ T1155] validate_chain+0x7bd/0xae0 [ 104.721244][ T1155] ? stack_depot_save_flags+0x27e/0x440 [ 104.721251][ T1155] ? __pfx_validate_chain+0x10/0x10 [ 104.721254][ T1155] ? hlock_class+0x4e/0x130 [ 104.721256][ T1155] ? mark_lock+0x38/0x3e0 [ 104.721261][ T1155] __lock_acquire+0xb9a/0x1680 [ 104.721266][ T1155] ? mark_lock+0x38/0x3e0 [ 104.721269][ T1155] lock_acquire.part.0+0xeb/0x330 [ 104.721273][ T1155] ? __dev_open+0x57e/0x710 [ 104.721276][ T1155] ? validate_chain+0x162/0xae0 [ 104.721279][ T1155] ? __pfx_lock_acquire.part.0+0x10/0x10 [ 104.721282][ T1155] ? trace_lock_acquire+0x14c/0x1f0 [ 104.721286][ T1155] ? lock_acquire+0x32/0xc0 [ 104.721289][ T1155] ? __dev_open+0x57e/0x710 [ 104.721292][ T1155] __mutex_lock+0x169/0xc40 [ 104.721294][ T1155] ? __dev_open+0x57e/0x710 [ 104.721298][ T1155] ? __dev_open+0x57e/0x710 [ 104.721300][ T1155] ? __pfx___mutex_lock+0x10/0x10 [ 104.721304][ T1155] ? mutex_is_locked+0x1c/0x60 [ 104.721307][ T1155] ? rtnl_is_locked+0x15/0x20 [ 104.721315][ T1155] ? mirred_device_event+0x3d/0x1e0 [act_mirred] [ 104.721324][ T1155] ? __dev_open+0x57e/0x710 [ 104.721326][ T1155] __dev_open+0x57e/0x710 [ 104.721328][ T1155] ? mark_held_locks+0x9e/0xe0 [ 104.721331][ T1155] ? __pfx___dev_open+0x10/0x10 [ 104.721334][ T1155] ? trace_percpu_alloc_percpu+0x149/0x1d0 [ 104.721342][ T1155] netif_open+0xc9/0x130 [ 104.721344][ T1155] ? __pfx_netif_open+0x10/0x10 [ 104.721348][ T1155] dev_open+0x9f/0x1a0 [ 104.721353][ T1155] team_port_add+0x90a/0x1ce0 [ 104.721358][ T1155] ? __pfx_team_port_add+0x10/0x10 [ 104.721361][ T1155] ? rcu_read_lock_any_held+0x43/0xb0 [ 104.721368][ T1155] ? validate_chain+0x1fe/0xae0 [ 104.721373][ T1155] team_add_slave+0x41/0x80 [ 104.721377][ T1155] do_set_master+0x302/0x610 [ 104.721382][ T1155] do_setlink.constprop.0+0x87b/0x25f0 [ 104.721386][ T1155] ? __pfx_do_setlink.constprop.0+0x10/0x10 [ 104.721390][ T1155] ? lock_acquire.part.0+0xeb/0x330 [ 104.721394][ T1155] ? rtnl_newlink+0x653/0xa70 [ 104.721396][ T1155] ? rtnl_newlink+0xb9/0xa70 [ 104.721399][ T1155] ? rtnetlink_rcv_msg+0x712/0xc10 [ 104.721402][ T1155] ? __mutex_trylock_common+0xfa/0x260 [ 104.721406][ T1155] ? __pfx___mutex_trylock_common+0x10/0x10 [ 104.721409][ T1155] ? lock_acquire+0x32/0xc0 [ 104.721413][ T1155] ? trace_contention_end+0xef/0x150 [ 104.721416][ T1155] ? __mutex_lock+0x190/0xc40 [ 104.721419][ T1155] ? __pfx_do_raw_spin_lock+0x10/0x10 [ 104.721423][ T1155] ? rtnl_newlink+0x653/0xa70 [ 104.721426][ T1155] ? __pfx___mutex_lock+0x10/0x10 [ 104.721428][ T1155] ? __pfx_validate_nla+0x10/0x10 [ 104.721434][ T1155] ? trace_cap_capable+0x11e/0x1a0 [ 104.721440][ T1155] ? __rtnl_newlink+0x3f2/0xa80 [ 104.721445][ T1155] rtnl_newlink+0x69c/0xa70 [ 104.721449][ T1155] ? __pfx_rtnl_newlink+0x10/0x10 [ 104.721452][ T1155] ? find_held_lock+0x2c/0x110 [ 104.721457][ T1155] ? __pfx___lock_release+0x10/0x10 [ 104.721460][ T1155] ? __pfx_lock_acquire.part.0+0x10/0x10 [ 104.721465][ T1155] ? rtnetlink_rcv_msg+0x6ef/0xc10 [ 104.721468][ T1155] ? __pfx_rtnl_newlink+0x10/0x10 [ 104.721471][ T1155] rtnetlink_rcv_msg+0x712/0xc10 [ 104.721475][ T1155] ? __pfx_rtnetlink_rcv_msg+0x10/0x10 [ 104.721478][ T1155] ? hlock_class+0x4e/0x130 [ 104.721480][ T1155] ? mark_lock+0x38/0x3e0 [ 104.721484][ T1155] ? __lock_acquire+0xb9a/0x1680 [ 104.721488][ T1155] netlink_rcv_skb+0x130/0x360 [ 104.721492][ T1155] ? __pfx_rtnetlink_rcv_msg+0x10/0x10 [ 104.721495][ T1155] ? __pfx_netlink_rcv_skb+0x10/0x10 [ 104.721503][ T1155] ? netlink_deliver_tap+0x13e/0x340 [ 104.721506][ T1155] ? netlink_deliver_tap+0xc3/0x340 [ 104.721510][ T1155] netlink_unicast+0x44b/0x710 [ 104.721514][ T1155] ? __pfx_netlink_unicast+0x10/0x10 [ 104.721517][ T1155] ? find_held_lock+0x2c/0x110 [ 104.721522][ T1155] netlink_sendmsg+0x723/0xbe0 [ 104.721527][ T1155] ? __pfx_netlink_sendmsg+0x10/0x10 [ 104.721533][ T1155] ____sys_sendmsg+0x7ac/0xa10 [ 104.721536][ T1155] ? __pfx_____sys_sendmsg+0x10/0x10 [ 104.721539][ T1155] ? __pfx_copy_msghdr_from_user+0x10/0x10 [ 104.721546][ T1155] ___sys_sendmsg+0xee/0x170 [ 104.721550][ T1155] ? __pfx____sys_sendmsg+0x10/0x10 [ 104.721554][ T1155] ? kasan_save_stack+0x34/0x50 [ 104.721560][ T1155] ? kasan_save_stack+0x24/0x50 [ 104.721563][ T1155] ? kasan_record_aux_stack+0x8c/0xa0 [ 104.721567][ T1155] ? __call_rcu_common.constprop.0+0xa1/0x5b0 [ 104.721570][ T1155] ? __x64_sys_close+0x7c/0xd0 [ 104.721575][ T1155] ? do_syscall_64+0xc1/0x1d0 [ 104.721577][ T1155] ? entry_SYSCALL_64_after_hwframe+0x77/0x7f [ 104.721579][ T1155] ? __lock_acquire+0xb9a/0x1680 [ 104.721582][ T1155] ? lock_downgrade+0x52/0x110 [ 104.721586][ T1155] ? __pfx_do_raw_spin_lock+0x10/0x10 [ 104.721590][ T1155] ? find_held_lock+0x2c/0x110 [ 104.721594][ T1155] ? __lock_release+0x103/0x460 [ 104.721597][ T1155] ? __virt_addr_valid+0x22b/0x430 [ 104.721601][ T1155] ? __pfx___lock_release+0x10/0x10 [ 104.721604][ T1155] ? __pfx_lock_acquire.part.0+0x10/0x10 [ 104.721609][ T1155] __sys_sendmsg+0x109/0x1a0 [ 104.721612][ T1155] ? __pfx___sys_sendmsg+0x10/0x10 [ 104.721620][ T1155] do_syscall_64+0xc1/0x1d0 [ 104.721623][ T1155] entry_SYSCALL_64_after_hwframe+0x77/0x7f [ 104.721626][ T1155] RIP: 0033:0x7f7510c669a7 [ 104.721630][ T1155] Code: 0a 00 f7 d8 64 89 02 48 c7 c0 ff ff ff ff eb b9 0f 1f 00 f3 0f 1e fa 64 8b 04 25 18 00 00 00 85 c0 75 10 b8 2e 00 00 00 0f 05 <48> 3d 00 f0 ff ff 77 51 c3 48 83 ec 28 89 54 24 1c 48 89 74 24 10 [ 104.721633][ T1155] RSP: 002b:00007ffc643183c8 EFLAGS: 00000246 ORIG_RAX: 000000000000002e [ 104.721637][ T1155] RAX: ffffffffffffffda RBX: 00007ffc64318af0 RCX: 00007f7510c669a7 [ 104.721639][ T1155] RDX: 0000000000000000 RSI: 00007ffc64318430 RDI: 0000000000000005 [ 104.721641][ T1155] RBP: 0000000000000004 R08: 0000000000000003 R09: 0000000000000000 [ 104.721642][ T1155] R10: 000000002c58d910 R11: 0000000000000246 R12: 0000000000000004 [ 104.721644][ T1155] R13: 0000000067b302ab R14: 0000000000498600 R15: 0000000000000000 [ 104.721649][ T1155] [ 104.741928][ T1155] lag1: Port device veth0 added [ 104.760087][ T1155] ip (1155) used greatest stack depth: 23504 bytes left [ 104.813042][ T1156] lag1: Port device veth6 added [ 105.145721][ T1162] 8021q: adding VLAN 0 to HW filter on device lag1 [ 106.391090][ T1185] lag4: Mode changed to "loadbalance" [ 106.458917][ T1187] 8021q: adding VLAN 0 to HW filter on device lag4 [ 106.761904][ T1193] lag4: Port device veth3 added [ 106.819902][ T1194] lag4: Port device veth5 added [ 107.139441][ T1200] 8021q: adding VLAN 0 to HW filter on device lag4 [ 108.307219][ T1223] lag2: Mode changed to "loadbalance" [ 108.395983][ T1225] 8021q: adding VLAN 0 to HW filter on device lag2 [ 108.676021][ T1231] lag2: Port device veth1 added [ 108.750870][ T1232] lag2: Port device veth7 added [ 109.328822][ T1245] br1: port 1(lag2.100) entered blocking state [ 109.329116][ T1245] br1: port 1(lag2.100) entered disabled state [ 109.329602][ T1245] lag2.100: entered allmulticast mode [ 109.329834][ T1245] lag2: entered allmulticast mode [ 109.330019][ T1245] veth1: entered allmulticast mode [ 109.330215][ T1245] veth7: entered allmulticast mode [ 109.332540][ T1245] lag2.100: entered promiscuous mode [ 109.332780][ T1245] lag2: entered promiscuous mode [ 109.332964][ T1245] veth1: entered promiscuous mode [ 109.333360][ T1245] veth7: entered promiscuous mode [ 109.612852][ T1252] br2: port 1(lag2.200) entered blocking state [ 109.613108][ T1252] br2: port 1(lag2.200) entered disabled state [ 109.613363][ T1252] lag2.200: entered allmulticast mode [ 109.614960][ T1252] lag2.200: entered promiscuous mode [ 109.789526][ T1255] br1: port 1(lag2.100) entered blocking state [ 109.789782][ T1255] br1: port 1(lag2.100) entered forwarding state [ 109.846181][ T1256] br2: port 1(lag2.200) entered blocking state [ 109.846464][ T1256] br2: port 1(lag2.200) entered forwarding state [ 110.201634][ T1264] lag3: Mode changed to "loadbalance" [ 110.263402][ T1266] 8021q: adding VLAN 0 to HW filter on device lag3 [ 110.543366][ T1272] lag3: Port device veth2 added [ 110.617246][ T1273] lag3: Port device veth4 added [ 122.006947][ T1354] veth7: left promiscuous mode [ 122.007390][ T1354] veth7: left allmulticast mode [ 122.008697][ T1354] lag2: Port device veth7 removed [ 137.241021][ T1415] veth7: entered promiscuous mode [ 137.241400][ T1415] veth7: entered allmulticast mode [ 137.242152][ T1415] lag2: Port device veth7 added [ 137.447795][ T1426] veth1: left promiscuous mode [ 137.448172][ T1426] veth1: left allmulticast mode [ 137.456789][ T1426] lag2: Port device veth1 removed [ 152.659731][ T217] br1: port 1(lag2.100) entered disabled state [ 152.660957][ T217] br2: port 1(lag2.200) entered disabled state [ 152.710915][ T1487] veth7: left promiscuous mode [ 152.711308][ T1487] veth7: left allmulticast mode [ 152.712606][ T1487] lag2: Port device veth7 removed [ 152.929237][ T1498] veth1: entered promiscuous mode [ 152.929700][ T1498] veth1: entered allmulticast mode [ 152.930466][ T1498] lag2: Port device veth1 added [ 152.957445][ T66] br1: port 1(lag2.100) entered blocking state [ 152.957756][ T66] br1: port 1(lag2.100) entered forwarding state [ 152.958343][ T66] br2: port 1(lag2.200) entered blocking state [ 152.958529][ T66] br2: port 1(lag2.200) entered forwarding state [ 153.181454][ T1510] veth7: entered promiscuous mode [ 153.181842][ T1510] veth7: entered allmulticast mode [ 153.182600][ T1510] lag2: Port device veth7 added [ 168.200236][ T1570] lag2.200: left allmulticast mode [ 168.200447][ T1570] lag2.200: left promiscuous mode [ 168.200963][ T1570] br2: port 1(lag2.200) entered disabled state [ 168.261109][ T1571] lag2.100: left allmulticast mode [ 168.261487][ T1571] lag2: left allmulticast mode [ 168.261672][ T1571] veth1: left allmulticast mode [ 168.261855][ T1571] veth7: left allmulticast mode [ 168.262044][ T1571] lag2.100: left promiscuous mode [ 168.262216][ T1571] lag2: left promiscuous mode [ 168.262389][ T1571] veth1: left promiscuous mode [ 168.262808][ T1571] veth7: left promiscuous mode [ 168.263545][ T1571] br1: port 1(lag2.100) entered disabled state [ 170.357625][ T1573] br1: port 1(lag2.100) entered blocking state [ 170.357851][ T1573] br1: port 1(lag2.100) entered disabled state [ 170.358066][ T1573] lag2.100: entered allmulticast mode [ 170.358210][ T1573] lag2: entered allmulticast mode [ 170.358372][ T1573] veth1: entered allmulticast mode [ 170.358523][ T1573] veth7: entered allmulticast mode [ 170.360086][ T1573] lag2.100: entered promiscuous mode [ 170.360267][ T1573] lag2: entered promiscuous mode [ 170.360526][ T1573] veth1: entered promiscuous mode [ 170.361062][ T1573] veth7: entered promiscuous mode [ 170.362358][ T1573] br1: port 1(lag2.100) entered blocking state [ 170.362556][ T1573] br1: port 1(lag2.100) entered forwarding state [ 170.417592][ T1574] br2: port 1(lag2.200) entered blocking state [ 170.417820][ T1574] br2: port 1(lag2.200) entered disabled state [ 170.418037][ T1574] lag2.200: entered allmulticast mode [ 170.420642][ T1574] lag2.200: entered promiscuous mode [ 170.421554][ T1574] br2: port 1(lag2.200) entered blocking state [ 170.421747][ T1574] br2: port 1(lag2.200) entered forwarding state [ 185.480265][ T1634] lag3: Port device veth2 removed [ 200.583600][ T1695] lag3: Port device veth2 added [ 200.806552][ T1706] lag3: Port device veth4 removed [ 215.889277][ T1767] lag3: Port device veth2 removed [ 216.107836][ T1778] lag3: Port device veth4 added [ 216.374704][ T1789] lag3: Port device veth2 added [ 232.117053][ T1859] lag3: Port device veth4 removed [ 232.188303][ T1860] lag3: Port device veth2 removed [ 232.855154][ T1872] br2: port 1(lag2.200) entered disabled state [ 232.902379][ T1873] br1: port 1(lag2.100) entered disabled state [ 233.077969][ T1876] lag2.200: left allmulticast mode [ 233.078168][ T1876] lag2.200: left promiscuous mode [ 233.078859][ T1876] br2: port 1(lag2.200) entered disabled state [ 233.230830][ T1878] lag2.100: left allmulticast mode [ 233.231049][ T1878] lag2: left allmulticast mode [ 233.231224][ T1878] veth1: left allmulticast mode [ 233.231408][ T1878] veth7: left allmulticast mode [ 233.231584][ T1878] lag2.100: left promiscuous mode [ 233.231739][ T1878] lag2: left promiscuous mode [ 233.231898][ T1878] veth1: left promiscuous mode [ 233.232263][ T1878] veth7: left promiscuous mode [ 233.232909][ T1878] br1: port 1(lag2.100) entered disabled state [ 233.563551][ T1882] lag2: Port device veth7 removed [ 233.617240][ T1883] lag2: Port device veth1 removed [ 234.803944][ T1900] lag4: Port device veth5 removed [ 234.890008][ T1901] lag4: Port device veth3 removed [ 236.121727][ T1916] lag1: Port device veth6 removed [ 236.167063][ T1917] lag1: Port device veth0 removed