[ 1387.139716][T18128] lag1: Mode changed to "loadbalance" [ 1387.235120][T18130] 8021q: adding VLAN 0 to HW filter on device lag1 [ 1387.616307][T18146] [ 1387.616436][T18146] ====================================================== [ 1387.616643][T18146] WARNING: possible circular locking dependency detected [ 1387.616847][T18146] 6.14.0-rc2-virtme #1 Not tainted [ 1387.617023][T18146] ------------------------------------------------------ [ 1387.617223][T18146] ip/18146 is trying to acquire lock: [ 1387.617382][T18146] ffff888007c59cb0 (&dev->lock){+.+.}-{4:4}, at: __dev_open+0x57e/0x710 [ 1387.617633][T18146] [ 1387.617633][T18146] but task is already holding lock: [ 1387.617881][T18146] ffff88800bb5ad80 (team->team_lock_key){+.+.}-{4:4}, at: team_add_slave+0x33/0x80 [ 1387.618177][T18146] [ 1387.618177][T18146] which lock already depends on the new lock. [ 1387.618177][T18146] [ 1387.618499][T18146] [ 1387.618499][T18146] the existing dependency chain (in reverse order) is: [ 1387.618777][T18146] [ 1387.618777][T18146] -> #1 (team->team_lock_key){+.+.}-{4:4}: [ 1387.619024][T18146] __lock_acquire+0xb9a/0x1680 [ 1387.619195][T18146] lock_acquire.part.0+0xeb/0x330 [ 1387.619356][T18146] __mutex_lock+0x169/0xc40 [ 1387.619522][T18146] team_set_mac_address+0x120/0x250 [ 1387.619686][T18146] netif_set_mac_address+0x28a/0x410 [ 1387.619886][T18146] do_setlink.constprop.0+0x1980/0x25f0 [ 1387.620098][T18146] rtnl_setlink+0x2b5/0x5a0 [ 1387.620264][T18146] rtnetlink_rcv_msg+0x712/0xc10 [ 1387.620429][T18146] netlink_rcv_skb+0x130/0x360 [ 1387.620596][T18146] netlink_unicast+0x44b/0x710 [ 1387.620764][T18146] netlink_sendmsg+0x723/0xbe0 [ 1387.620925][T18146] __sys_sendto+0x3c3/0x450 [ 1387.621087][T18146] __x64_sys_sendto+0xe0/0x1c0 [ 1387.621251][T18146] do_syscall_64+0xc1/0x1d0 [ 1387.621411][T18146] entry_SYSCALL_64_after_hwframe+0x77/0x7f [ 1387.621612][T18146] [ 1387.621612][T18146] -> #0 (&dev->lock){+.+.}-{4:4}: [ 1387.621860][T18146] check_prev_add+0x182/0xce0 [ 1387.622023][T18146] validate_chain+0x7bd/0xae0 [ 1387.622186][T18146] __lock_acquire+0xb9a/0x1680 [ 1387.622349][T18146] lock_acquire.part.0+0xeb/0x330 [ 1387.622509][T18146] __mutex_lock+0x169/0xc40 [ 1387.622671][T18146] __dev_open+0x57e/0x710 [ 1387.622834][T18146] netif_open+0xc9/0x130 [ 1387.623004][T18146] dev_open+0x9f/0x1a0 [ 1387.623165][T18146] team_port_add+0x90a/0x1ce0 [ 1387.623328][T18146] team_add_slave+0x41/0x80 [ 1387.623491][T18146] do_set_master+0x302/0x610 [ 1387.623653][T18146] do_setlink.constprop.0+0x87b/0x25f0 [ 1387.623854][T18146] rtnl_newlink+0x69c/0xa70 [ 1387.624016][T18146] rtnetlink_rcv_msg+0x712/0xc10 [ 1387.624184][T18146] netlink_rcv_skb+0x130/0x360 [ 1387.624348][T18146] netlink_unicast+0x44b/0x710 [ 1387.624510][T18146] netlink_sendmsg+0x723/0xbe0 [ 1387.624670][T18146] ____sys_sendmsg+0x7ac/0xa10 [ 1387.624832][T18146] ___sys_sendmsg+0xee/0x170 [ 1387.624996][T18146] __sys_sendmsg+0x109/0x1a0 [ 1387.625164][T18146] do_syscall_64+0xc1/0x1d0 [ 1387.625325][T18146] entry_SYSCALL_64_after_hwframe+0x77/0x7f [ 1387.625525][T18146] [ 1387.625525][T18146] other info that might help us debug this: [ 1387.625525][T18146] [ 1387.625839][T18146] Possible unsafe locking scenario: [ 1387.625839][T18146] [ 1387.626078][T18146] CPU0 CPU1 [ 1387.626238][T18146] ---- ---- [ 1387.626402][T18146] lock(team->team_lock_key); [ 1387.626563][T18146] lock(&dev->lock); [ 1387.626766][T18146] lock(team->team_lock_key); [ 1387.627007][T18146] lock(&dev->lock); [ 1387.627132][T18146] [ 1387.627132][T18146] *** DEADLOCK *** [ 1387.627132][T18146] [ 1387.627367][T18146] 2 locks held by ip/18146: [ 1387.627538][T18146] #0: ffffffff85feeb48 (rtnl_mutex){+.+.}-{4:4}, at: rtnl_newlink+0x653/0xa70 [ 1387.627827][T18146] #1: ffff88800bb5ad80 (team->team_lock_key){+.+.}-{4:4}, at: team_add_slave+0x33/0x80 [ 1387.628116][T18146] [ 1387.628116][T18146] stack backtrace: [ 1387.628313][T18146] CPU: 3 UID: 0 PID: 18146 Comm: ip Not tainted 6.14.0-rc2-virtme #1 [ 1387.628317][T18146] Hardware name: Bochs Bochs, BIOS Bochs 01/01/2011 [ 1387.628319][T18146] Call Trace: [ 1387.628321][T18146] [ 1387.628323][T18146] dump_stack_lvl+0x82/0xd0 [ 1387.628329][T18146] print_circular_bug+0x1b7/0x240 [ 1387.628333][T18146] check_noncircular+0x2f9/0x3e0 [ 1387.628337][T18146] ? __pfx_check_noncircular+0x10/0x10 [ 1387.628341][T18146] ? find_held_lock+0x2c/0x110 [ 1387.628346][T18146] ? alloc_chain_hlocks+0x4e0/0x720 [ 1387.628350][T18146] check_prev_add+0x182/0xce0 [ 1387.628355][T18146] validate_chain+0x7bd/0xae0 [ 1387.628359][T18146] ? stack_depot_save_flags+0x27e/0x440 [ 1387.628365][T18146] ? __pfx_validate_chain+0x10/0x10 [ 1387.628368][T18146] ? hlock_class+0x4e/0x130 [ 1387.628370][T18146] ? mark_lock+0x38/0x3e0 [ 1387.628374][T18146] __lock_acquire+0xb9a/0x1680 [ 1387.628379][T18146] ? mark_lock+0x38/0x3e0 [ 1387.628382][T18146] lock_acquire.part.0+0xeb/0x330 [ 1387.628385][T18146] ? __dev_open+0x57e/0x710 [ 1387.628388][T18146] ? validate_chain+0x162/0xae0 [ 1387.628391][T18146] ? __pfx_lock_acquire.part.0+0x10/0x10 [ 1387.628394][T18146] ? trace_lock_acquire+0x14c/0x1f0 [ 1387.628398][T18146] ? lock_acquire+0x32/0xc0 [ 1387.628400][T18146] ? __dev_open+0x57e/0x710 [ 1387.628403][T18146] __mutex_lock+0x169/0xc40 [ 1387.628405][T18146] ? __dev_open+0x57e/0x710 [ 1387.628409][T18146] ? __dev_open+0x57e/0x710 [ 1387.628411][T18146] ? __pfx___mutex_lock+0x10/0x10 [ 1387.628415][T18146] ? packet_notifier+0x3b0/0x810 [ 1387.628423][T18146] ? __dev_open+0x57e/0x710 [ 1387.628425][T18146] __dev_open+0x57e/0x710 [ 1387.628427][T18146] ? mark_held_locks+0x9e/0xe0 [ 1387.628430][T18146] ? __pfx___dev_open+0x10/0x10 [ 1387.628433][T18146] ? trace_percpu_alloc_percpu+0x149/0x1d0 [ 1387.628439][T18146] netif_open+0xc9/0x130 [ 1387.628441][T18146] ? __pfx_netif_open+0x10/0x10 [ 1387.628445][T18146] dev_open+0x9f/0x1a0 [ 1387.628449][T18146] team_port_add+0x90a/0x1ce0 [ 1387.628454][T18146] ? __pfx_team_port_add+0x10/0x10 [ 1387.628457][T18146] ? rcu_read_lock_any_held+0x43/0xb0 [ 1387.628462][T18146] ? validate_chain+0x1fe/0xae0 [ 1387.628467][T18146] team_add_slave+0x41/0x80 [ 1387.628471][T18146] do_set_master+0x302/0x610 [ 1387.628475][T18146] do_setlink.constprop.0+0x87b/0x25f0 [ 1387.628480][T18146] ? __pfx_do_setlink.constprop.0+0x10/0x10 [ 1387.628484][T18146] ? lock_acquire.part.0+0xeb/0x330 [ 1387.628487][T18146] ? rtnl_newlink+0x653/0xa70 [ 1387.628490][T18146] ? rtnl_newlink+0xb9/0xa70 [ 1387.628492][T18146] ? rtnetlink_rcv_msg+0x712/0xc10 [ 1387.628495][T18146] ? __mutex_trylock_common+0xfa/0x260 [ 1387.628498][T18146] ? __pfx___mutex_trylock_common+0x10/0x10 [ 1387.628502][T18146] ? lock_acquire+0x32/0xc0 [ 1387.628505][T18146] ? trace_contention_end+0xef/0x150 [ 1387.628509][T18146] ? __mutex_lock+0x190/0xc40 [ 1387.628511][T18146] ? __pfx_do_raw_spin_lock+0x10/0x10 [ 1387.628515][T18146] ? rtnl_newlink+0x653/0xa70 [ 1387.628518][T18146] ? __pfx___mutex_lock+0x10/0x10 [ 1387.628520][T18146] ? __pfx_validate_nla+0x10/0x10 [ 1387.628525][T18146] ? trace_cap_capable+0x11e/0x1a0 [ 1387.628530][T18146] ? __rtnl_newlink+0x3f2/0xa80 [ 1387.628535][T18146] rtnl_newlink+0x69c/0xa70 [ 1387.628539][T18146] ? __pfx_rtnl_newlink+0x10/0x10 [ 1387.628542][T18146] ? find_held_lock+0x2c/0x110 [ 1387.628546][T18146] ? __pfx___lock_release+0x10/0x10 [ 1387.628549][T18146] ? __pfx_lock_acquire.part.0+0x10/0x10 [ 1387.628554][T18146] ? rtnetlink_rcv_msg+0x6ef/0xc10 [ 1387.628557][T18146] ? __pfx_rtnl_newlink+0x10/0x10 [ 1387.628560][T18146] rtnetlink_rcv_msg+0x712/0xc10 [ 1387.628564][T18146] ? __pfx_rtnetlink_rcv_msg+0x10/0x10 [ 1387.628566][T18146] ? hlock_class+0x4e/0x130 [ 1387.628569][T18146] ? mark_lock+0x38/0x3e0 [ 1387.628573][T18146] ? __lock_acquire+0xb9a/0x1680 [ 1387.628576][T18146] netlink_rcv_skb+0x130/0x360 [ 1387.628579][T18146] ? __pfx_rtnetlink_rcv_msg+0x10/0x10 [ 1387.628583][T18146] ? __pfx_netlink_rcv_skb+0x10/0x10 [ 1387.628590][T18146] ? netlink_deliver_tap+0x13e/0x340 [ 1387.628594][T18146] ? netlink_deliver_tap+0xc3/0x340 [ 1387.628597][T18146] netlink_unicast+0x44b/0x710 [ 1387.628601][T18146] ? __pfx_netlink_unicast+0x10/0x10 [ 1387.628605][T18146] ? find_held_lock+0x2c/0x110 [ 1387.628610][T18146] netlink_sendmsg+0x723/0xbe0 [ 1387.628614][T18146] ? __pfx_netlink_sendmsg+0x10/0x10 [ 1387.628620][T18146] ____sys_sendmsg+0x7ac/0xa10 [ 1387.628623][T18146] ? __pfx_____sys_sendmsg+0x10/0x10 [ 1387.628626][T18146] ? __pfx_copy_msghdr_from_user+0x10/0x10 [ 1387.628633][T18146] ___sys_sendmsg+0xee/0x170 [ 1387.628637][T18146] ? __pfx____sys_sendmsg+0x10/0x10 [ 1387.628641][T18146] ? kasan_save_stack+0x34/0x50 [ 1387.628645][T18146] ? kasan_save_stack+0x24/0x50 [ 1387.628648][T18146] ? kasan_record_aux_stack+0x8c/0xa0 [ 1387.628652][T18146] ? __call_rcu_common.constprop.0+0xa1/0x5b0 [ 1387.628656][T18146] ? __x64_sys_close+0x7c/0xd0 [ 1387.628659][T18146] ? do_syscall_64+0xc1/0x1d0 [ 1387.628661][T18146] ? entry_SYSCALL_64_after_hwframe+0x77/0x7f [ 1387.628664][T18146] ? __lock_acquire+0xb9a/0x1680 [ 1387.628667][T18146] ? lock_downgrade+0x52/0x110 [ 1387.628671][T18146] ? __pfx_do_raw_spin_lock+0x10/0x10 [ 1387.628674][T18146] ? find_held_lock+0x2c/0x110 [ 1387.628679][T18146] ? __lock_release+0x103/0x460 [ 1387.628681][T18146] ? __virt_addr_valid+0x22b/0x430 [ 1387.628685][T18146] ? __pfx___lock_release+0x10/0x10 [ 1387.628688][T18146] ? __pfx_lock_acquire.part.0+0x10/0x10 [ 1387.628692][T18146] __sys_sendmsg+0x109/0x1a0 [ 1387.628695][T18146] ? __pfx___sys_sendmsg+0x10/0x10 [ 1387.628703][T18146] do_syscall_64+0xc1/0x1d0 [ 1387.628706][T18146] entry_SYSCALL_64_after_hwframe+0x77/0x7f [ 1387.628709][T18146] RIP: 0033:0x7f5a80dec9a7 [ 1387.628713][T18146] Code: 0a 00 f7 d8 64 89 02 48 c7 c0 ff ff ff ff eb b9 0f 1f 00 f3 0f 1e fa 64 8b 04 25 18 00 00 00 85 c0 75 10 b8 2e 00 00 00 0f 05 <48> 3d 00 f0 ff ff 77 51 c3 48 83 ec 28 89 54 24 1c 48 89 74 24 10 [ 1387.628716][T18146] RSP: 002b:00007ffcc6c5ec38 EFLAGS: 00000246 ORIG_RAX: 000000000000002e [ 1387.628719][T18146] RAX: ffffffffffffffda RBX: 00007ffcc6c5f360 RCX: 00007f5a80dec9a7 [ 1387.628721][T18146] RDX: 0000000000000000 RSI: 00007ffcc6c5eca0 RDI: 0000000000000005 [ 1387.628723][T18146] RBP: 0000000000000004 R08: 0000000000000003 R09: 0000000000000000 [ 1387.628725][T18146] R10: 0000000013544910 R11: 0000000000000246 R12: 0000000000000004 [ 1387.628726][T18146] R13: 0000000067b3578a R14: 0000000000498600 R15: 0000000000000000 [ 1387.628731][T18146] [ 1387.649880][T18146] lag1: Port device veth0 added [ 1387.711705][T18147] lag1: Port device veth6 added [ 1388.032409][T18153] 8021q: adding VLAN 0 to HW filter on device lag1 [ 1389.193149][T18176] lag4: Mode changed to "loadbalance" [ 1389.263639][T18177] 8021q: adding VLAN 0 to HW filter on device lag4 [ 1389.566258][T18184] lag4: Port device veth3 added [ 1389.627297][T18185] lag4: Port device veth5 added [ 1389.955389][T18191] 8021q: adding VLAN 0 to HW filter on device lag4 [ 1391.141019][T18214] lag2: Mode changed to "loadbalance" [ 1391.223587][T18216] 8021q: adding VLAN 0 to HW filter on device lag2 [ 1391.506491][T18222] lag2: Port device veth1 added [ 1391.581943][T18223] lag2: Port device veth7 added [ 1392.429466][T18239] br1: port 1(lag2.100) entered blocking state [ 1392.429701][T18239] br1: port 1(lag2.100) entered disabled state [ 1392.429931][T18239] lag2.100: entered allmulticast mode [ 1392.430091][T18239] lag2: entered allmulticast mode [ 1392.430248][T18239] veth1: entered allmulticast mode [ 1392.430407][T18239] veth7: entered allmulticast mode [ 1392.432284][T18239] lag2.100: entered promiscuous mode [ 1392.432450][T18239] lag2: entered promiscuous mode [ 1392.432605][T18239] veth1: entered promiscuous mode [ 1392.432946][T18239] veth7: entered promiscuous mode [ 1392.713758][T18246] br2: port 1(lag2.200) entered blocking state [ 1392.713989][T18246] br2: port 1(lag2.200) entered disabled state [ 1392.714233][T18246] lag2.200: entered allmulticast mode [ 1392.715828][T18246] lag2.200: entered promiscuous mode [ 1392.897037][T18249] br1: port 1(lag2.100) entered blocking state [ 1392.897295][T18249] br1: port 1(lag2.100) entered forwarding state [ 1392.959030][T18250] br2: port 1(lag2.200) entered blocking state [ 1392.959344][T18250] br2: port 1(lag2.200) entered forwarding state [ 1393.310662][T18258] lag3: Mode changed to "loadbalance" [ 1393.389773][T18259] 8021q: adding VLAN 0 to HW filter on device lag3 [ 1393.695106][T18266] lag3: Port device veth2 added [ 1393.759246][T18267] lag3: Port device veth4 added [ 1405.196958][T18348] veth7: left promiscuous mode [ 1405.197362][T18348] veth7: left allmulticast mode [ 1405.200170][T18348] lag2: Port device veth7 removed [ 1420.352058][T18409] veth7: entered promiscuous mode [ 1420.352547][T18409] veth7: entered allmulticast mode [ 1420.353334][T18409] lag2: Port device veth7 added [ 1420.591873][T18420] veth1: left promiscuous mode [ 1420.592269][T18420] veth1: left allmulticast mode [ 1420.593565][T18420] lag2: Port device veth1 removed [ 1435.645244][ T151] br1: port 1(lag2.100) entered disabled state [ 1435.646380][ T151] br2: port 1(lag2.200) entered disabled state [ 1435.707195][T18481] veth7: left promiscuous mode [ 1435.708137][T18481] veth7: left allmulticast mode [ 1435.709299][T18481] lag2: Port device veth7 removed [ 1435.961166][T18492] veth1: entered promiscuous mode [ 1435.961672][T18492] veth1: entered allmulticast mode [ 1435.962586][T18492] lag2: Port device veth1 added [ 1435.989284][ T68] br1: port 1(lag2.100) entered blocking state [ 1435.989570][ T68] br1: port 1(lag2.100) entered forwarding state [ 1435.990216][ T68] br2: port 1(lag2.200) entered blocking state [ 1435.990453][ T68] br2: port 1(lag2.200) entered forwarding state [ 1436.206103][T18503] veth7: entered promiscuous mode [ 1436.206524][T18503] veth7: entered allmulticast mode [ 1436.207526][T18503] lag2: Port device veth7 added [ 1451.308598][T18563] lag2.200: left allmulticast mode [ 1451.308803][T18563] lag2.200: left promiscuous mode [ 1451.309347][T18563] br2: port 1(lag2.200) entered disabled state [ 1451.383591][T18564] lag2.100: left allmulticast mode [ 1451.383809][T18564] lag2: left allmulticast mode [ 1451.383981][T18564] veth1: left allmulticast mode [ 1451.384136][T18564] veth7: left allmulticast mode [ 1451.384307][T18564] lag2.100: left promiscuous mode [ 1451.384465][T18564] lag2: left promiscuous mode [ 1451.384616][T18564] veth1: left promiscuous mode [ 1451.385022][T18564] veth7: left promiscuous mode [ 1451.385723][T18564] br1: port 1(lag2.100) entered disabled state [ 1453.495241][T18566] br1: port 1(lag2.100) entered blocking state [ 1453.495532][T18566] br1: port 1(lag2.100) entered disabled state [ 1453.495776][T18566] lag2.100: entered allmulticast mode [ 1453.495933][T18566] lag2: entered allmulticast mode [ 1453.496090][T18566] veth1: entered allmulticast mode [ 1453.496253][T18566] veth7: entered allmulticast mode [ 1453.497877][T18566] lag2.100: entered promiscuous mode [ 1453.498058][T18566] lag2: entered promiscuous mode [ 1453.498221][T18566] veth1: entered promiscuous mode [ 1453.498874][T18566] veth7: entered promiscuous mode [ 1453.500880][T18566] br1: port 1(lag2.100) entered blocking state [ 1453.501128][T18566] br1: port 1(lag2.100) entered forwarding state [ 1453.567479][T18567] br2: port 1(lag2.200) entered blocking state [ 1453.567748][T18567] br2: port 1(lag2.200) entered disabled state [ 1453.567986][T18567] lag2.200: entered allmulticast mode [ 1453.569595][T18567] lag2.200: entered promiscuous mode [ 1453.570605][T18567] br2: port 1(lag2.200) entered blocking state [ 1453.570818][T18567] br2: port 1(lag2.200) entered forwarding state [ 1468.650187][T18627] lag3: Port device veth2 removed [ 1483.774261][T18688] lag3: Port device veth2 added [ 1483.987722][T18699] lag3: Port device veth4 removed [ 1498.976151][T18760] lag3: Port device veth2 removed [ 1499.203398][T18771] lag3: Port device veth4 added [ 1499.423778][T18782] lag3: Port device veth2 added [ 1514.951236][T18852] lag3: Port device veth4 removed [ 1515.011842][T18853] lag3: Port device veth2 removed [ 1515.528291][T18865] br2: port 1(lag2.200) entered disabled state [ 1515.582817][T18866] br1: port 1(lag2.100) entered disabled state [ 1515.760005][T18869] lag2.200: left allmulticast mode [ 1515.760211][T18869] lag2.200: left promiscuous mode [ 1515.760687][T18869] br2: port 1(lag2.200) entered disabled state [ 1515.927349][T18871] lag2.100: left allmulticast mode [ 1515.927565][T18871] lag2: left allmulticast mode [ 1515.927722][T18871] veth1: left allmulticast mode [ 1515.927880][T18871] veth7: left allmulticast mode [ 1515.928042][T18871] lag2.100: left promiscuous mode [ 1515.928189][T18871] lag2: left promiscuous mode [ 1515.928336][T18871] veth1: left promiscuous mode [ 1515.928700][T18871] veth7: left promiscuous mode [ 1515.929339][T18871] br1: port 1(lag2.100) entered disabled state [ 1516.273007][T18875] lag2: Port device veth7 removed [ 1516.319249][T18876] lag2: Port device veth1 removed [ 1517.609196][T18893] lag4: Port device veth5 removed [ 1517.666130][T18894] lag4: Port device veth3 removed [ 1518.907182][T18909] lag1: Port device veth6 removed [ 1518.975107][T18910] lag1: Port device veth0 removed