[ 1406.453609][T18209] lag1: Mode changed to "loadbalance" [ 1406.546943][T18211] 8021q: adding VLAN 0 to HW filter on device lag1 [ 1406.901390][T18221] [ 1406.901581][T18221] ====================================================== [ 1406.901771][T18221] WARNING: possible circular locking dependency detected [ 1406.901967][T18221] 6.14.0-rc2-virtme #1 Not tainted [ 1406.902115][T18221] ------------------------------------------------------ [ 1406.902313][T18221] ip/18221 is trying to acquire lock: [ 1406.902464][T18221] ffff888008241cb0 (&dev->lock){+.+.}-{4:4}, at: __dev_open+0x57e/0x710 [ 1406.902702][T18221] [ 1406.902702][T18221] but task is already holding lock: [ 1406.902934][T18221] ffff8880110b2d80 (team->team_lock_key){+.+.}-{4:4}, at: team_add_slave+0x33/0x80 [ 1406.903229][T18221] [ 1406.903229][T18221] which lock already depends on the new lock. [ 1406.903229][T18221] [ 1406.903537][T18221] [ 1406.903537][T18221] the existing dependency chain (in reverse order) is: [ 1406.903804][T18221] [ 1406.903804][T18221] -> #1 (team->team_lock_key){+.+.}-{4:4}: [ 1406.904040][T18221] __lock_acquire+0xb9a/0x1680 [ 1406.904199][T18221] lock_acquire.part.0+0xeb/0x330 [ 1406.904344][T18221] __mutex_lock+0x169/0xc40 [ 1406.904489][T18221] team_set_mac_address+0x120/0x250 [ 1406.904635][T18221] netif_set_mac_address+0x28a/0x410 [ 1406.904821][T18221] do_setlink.constprop.0+0x1980/0x25f0 [ 1406.905003][T18221] rtnl_setlink+0x2b5/0x5a0 [ 1406.905151][T18221] rtnetlink_rcv_msg+0x712/0xc10 [ 1406.905300][T18221] netlink_rcv_skb+0x130/0x360 [ 1406.905446][T18221] netlink_unicast+0x44b/0x710 [ 1406.905590][T18221] netlink_sendmsg+0x723/0xbe0 [ 1406.905736][T18221] __sys_sendto+0x3c3/0x450 [ 1406.905881][T18221] __x64_sys_sendto+0xe0/0x1c0 [ 1406.906025][T18221] do_syscall_64+0xc1/0x1d0 [ 1406.906169][T18221] entry_SYSCALL_64_after_hwframe+0x77/0x7f [ 1406.906354][T18221] [ 1406.906354][T18221] -> #0 (&dev->lock){+.+.}-{4:4}: [ 1406.906571][T18221] check_prev_add+0x182/0xce0 [ 1406.906742][T18221] validate_chain+0x7bd/0xae0 [ 1406.906890][T18221] __lock_acquire+0xb9a/0x1680 [ 1406.907038][T18221] lock_acquire.part.0+0xeb/0x330 [ 1406.907182][T18221] __mutex_lock+0x169/0xc40 [ 1406.907330][T18221] __dev_open+0x57e/0x710 [ 1406.907478][T18221] netif_open+0xc9/0x130 [ 1406.907622][T18221] dev_open+0x9f/0x1a0 [ 1406.907768][T18221] team_port_add+0x90a/0x1ce0 [ 1406.907914][T18221] team_add_slave+0x41/0x80 [ 1406.908064][T18221] do_set_master+0x302/0x610 [ 1406.908211][T18221] do_setlink.constprop.0+0x87b/0x25f0 [ 1406.908393][T18221] rtnl_newlink+0x69c/0xa70 [ 1406.908543][T18221] rtnetlink_rcv_msg+0x712/0xc10 [ 1406.908690][T18221] netlink_rcv_skb+0x130/0x360 [ 1406.908842][T18221] netlink_unicast+0x44b/0x710 [ 1406.908987][T18221] netlink_sendmsg+0x723/0xbe0 [ 1406.909137][T18221] ____sys_sendmsg+0x7ac/0xa10 [ 1406.909284][T18221] ___sys_sendmsg+0xee/0x170 [ 1406.909435][T18221] __sys_sendmsg+0x109/0x1a0 [ 1406.909584][T18221] do_syscall_64+0xc1/0x1d0 [ 1406.909764][T18221] entry_SYSCALL_64_after_hwframe+0x77/0x7f [ 1406.909943][T18221] [ 1406.909943][T18221] other info that might help us debug this: [ 1406.909943][T18221] [ 1406.910239][T18221] Possible unsafe locking scenario: [ 1406.910239][T18221] [ 1406.910459][T18221] CPU0 CPU1 [ 1406.910605][T18221] ---- ---- [ 1406.910748][T18221] lock(team->team_lock_key); [ 1406.910894][T18221] lock(&dev->lock); [ 1406.911075][T18221] lock(team->team_lock_key); [ 1406.911292][T18221] lock(&dev->lock); [ 1406.911411][T18221] [ 1406.911411][T18221] *** DEADLOCK *** [ 1406.911411][T18221] [ 1406.911625][T18221] 2 locks held by ip/18221: [ 1406.911771][T18221] #0: ffffffff961eeb48 (rtnl_mutex){+.+.}-{4:4}, at: rtnl_newlink+0x653/0xa70 [ 1406.912042][T18221] #1: ffff8880110b2d80 (team->team_lock_key){+.+.}-{4:4}, at: team_add_slave+0x33/0x80 [ 1406.912304][T18221] [ 1406.912304][T18221] stack backtrace: [ 1406.912489][T18221] CPU: 3 UID: 0 PID: 18221 Comm: ip Not tainted 6.14.0-rc2-virtme #1 [ 1406.912494][T18221] Hardware name: Bochs Bochs, BIOS Bochs 01/01/2011 [ 1406.912496][T18221] Call Trace: [ 1406.912498][T18221] [ 1406.912500][T18221] dump_stack_lvl+0x82/0xd0 [ 1406.912507][T18221] print_circular_bug+0x1b7/0x240 [ 1406.912511][T18221] check_noncircular+0x2f9/0x3e0 [ 1406.912515][T18221] ? __pfx_check_noncircular+0x10/0x10 [ 1406.912519][T18221] ? find_held_lock+0x2c/0x110 [ 1406.912526][T18221] ? alloc_chain_hlocks+0x4e0/0x720 [ 1406.912529][T18221] check_prev_add+0x182/0xce0 [ 1406.912534][T18221] validate_chain+0x7bd/0xae0 [ 1406.912539][T18221] ? stack_depot_save_flags+0x27e/0x440 [ 1406.912544][T18221] ? __pfx_validate_chain+0x10/0x10 [ 1406.912547][T18221] ? hlock_class+0x4e/0x130 [ 1406.912549][T18221] ? mark_lock+0x38/0x3e0 [ 1406.912554][T18221] __lock_acquire+0xb9a/0x1680 [ 1406.912559][T18221] ? mark_lock+0x38/0x3e0 [ 1406.912562][T18221] lock_acquire.part.0+0xeb/0x330 [ 1406.912565][T18221] ? __dev_open+0x57e/0x710 [ 1406.912568][T18221] ? validate_chain+0x162/0xae0 [ 1406.912571][T18221] ? __pfx_lock_acquire.part.0+0x10/0x10 [ 1406.912574][T18221] ? trace_lock_acquire+0x14c/0x1f0 [ 1406.912578][T18221] ? lock_acquire+0x32/0xc0 [ 1406.912580][T18221] ? __dev_open+0x57e/0x710 [ 1406.912583][T18221] __mutex_lock+0x169/0xc40 [ 1406.912586][T18221] ? __dev_open+0x57e/0x710 [ 1406.912589][T18221] ? __dev_open+0x57e/0x710 [ 1406.912591][T18221] ? __pfx___mutex_lock+0x10/0x10 [ 1406.912598][T18221] ? packet_notifier+0x3b0/0x810 [ 1406.912606][T18221] ? __dev_open+0x57e/0x710 [ 1406.912608][T18221] __dev_open+0x57e/0x710 [ 1406.912610][T18221] ? mark_held_locks+0x9e/0xe0 [ 1406.912613][T18221] ? __pfx___dev_open+0x10/0x10 [ 1406.912615][T18221] ? trace_percpu_alloc_percpu+0x149/0x1d0 [ 1406.912621][T18221] netif_open+0xc9/0x130 [ 1406.912624][T18221] ? __pfx_netif_open+0x10/0x10 [ 1406.912628][T18221] dev_open+0x9f/0x1a0 [ 1406.912632][T18221] team_port_add+0x90a/0x1ce0 [ 1406.912637][T18221] ? __pfx_team_port_add+0x10/0x10 [ 1406.912640][T18221] ? rcu_read_lock_any_held+0x43/0xb0 [ 1406.912645][T18221] ? validate_chain+0x1fe/0xae0 [ 1406.912649][T18221] team_add_slave+0x41/0x80 [ 1406.912653][T18221] do_set_master+0x302/0x610 [ 1406.912658][T18221] do_setlink.constprop.0+0x87b/0x25f0 [ 1406.912662][T18221] ? __pfx_do_setlink.constprop.0+0x10/0x10 [ 1406.912666][T18221] ? lock_acquire.part.0+0xeb/0x330 [ 1406.912669][T18221] ? rtnl_newlink+0x653/0xa70 [ 1406.912672][T18221] ? rtnl_newlink+0xb9/0xa70 [ 1406.912674][T18221] ? rtnetlink_rcv_msg+0x712/0xc10 [ 1406.912678][T18221] ? __mutex_trylock_common+0xfa/0x260 [ 1406.912681][T18221] ? __pfx___mutex_trylock_common+0x10/0x10 [ 1406.912685][T18221] ? lock_acquire+0x32/0xc0 [ 1406.912688][T18221] ? trace_contention_end+0xef/0x150 [ 1406.912692][T18221] ? __mutex_lock+0x190/0xc40 [ 1406.912695][T18221] ? __pfx_do_raw_spin_lock+0x10/0x10 [ 1406.912698][T18221] ? rtnl_newlink+0x653/0xa70 [ 1406.912702][T18221] ? __pfx___mutex_lock+0x10/0x10 [ 1406.912704][T18221] ? __pfx_validate_nla+0x10/0x10 [ 1406.912709][T18221] ? trace_cap_capable+0x11e/0x1a0 [ 1406.912714][T18221] ? __rtnl_newlink+0x3f2/0xa80 [ 1406.912719][T18221] rtnl_newlink+0x69c/0xa70 [ 1406.912724][T18221] ? __pfx_rtnl_newlink+0x10/0x10 [ 1406.912726][T18221] ? find_held_lock+0x2c/0x110 [ 1406.912731][T18221] ? __pfx___lock_release+0x10/0x10 [ 1406.912734][T18221] ? __pfx_lock_acquire.part.0+0x10/0x10 [ 1406.912739][T18221] ? rtnetlink_rcv_msg+0x6ef/0xc10 [ 1406.912742][T18221] ? __pfx_rtnl_newlink+0x10/0x10 [ 1406.912745][T18221] rtnetlink_rcv_msg+0x712/0xc10 [ 1406.912748][T18221] ? __pfx_rtnetlink_rcv_msg+0x10/0x10 [ 1406.912751][T18221] ? hlock_class+0x4e/0x130 [ 1406.912753][T18221] ? mark_lock+0x38/0x3e0 [ 1406.912758][T18221] ? __lock_acquire+0xb9a/0x1680 [ 1406.912761][T18221] netlink_rcv_skb+0x130/0x360 [ 1406.912764][T18221] ? __pfx_rtnetlink_rcv_msg+0x10/0x10 [ 1406.912768][T18221] ? __pfx_netlink_rcv_skb+0x10/0x10 [ 1406.912776][T18221] ? netlink_deliver_tap+0x13e/0x340 [ 1406.912779][T18221] ? netlink_deliver_tap+0xc3/0x340 [ 1406.912783][T18221] netlink_unicast+0x44b/0x710 [ 1406.912787][T18221] ? __pfx_netlink_unicast+0x10/0x10 [ 1406.912791][T18221] ? find_held_lock+0x2c/0x110 [ 1406.912796][T18221] netlink_sendmsg+0x723/0xbe0 [ 1406.912800][T18221] ? __pfx_netlink_sendmsg+0x10/0x10 [ 1406.912806][T18221] ____sys_sendmsg+0x7ac/0xa10 [ 1406.912809][T18221] ? __pfx_____sys_sendmsg+0x10/0x10 [ 1406.912812][T18221] ? __pfx_copy_msghdr_from_user+0x10/0x10 [ 1406.912819][T18221] ___sys_sendmsg+0xee/0x170 [ 1406.912823][T18221] ? __pfx____sys_sendmsg+0x10/0x10 [ 1406.912827][T18221] ? kasan_save_stack+0x34/0x50 [ 1406.912831][T18221] ? kasan_save_stack+0x24/0x50 [ 1406.912834][T18221] ? kasan_record_aux_stack+0x8c/0xa0 [ 1406.912836][T18221] ? __call_rcu_common.constprop.0+0xa1/0x5b0 [ 1406.912840][T18221] ? __x64_sys_close+0x7c/0xd0 [ 1406.912844][T18221] ? do_syscall_64+0xc1/0x1d0 [ 1406.912846][T18221] ? entry_SYSCALL_64_after_hwframe+0x77/0x7f [ 1406.912849][T18221] ? __lock_acquire+0xb9a/0x1680 [ 1406.912852][T18221] ? lock_downgrade+0x52/0x110 [ 1406.912856][T18221] ? __pfx_do_raw_spin_lock+0x10/0x10 [ 1406.912859][T18221] ? find_held_lock+0x2c/0x110 [ 1406.912864][T18221] ? __lock_release+0x103/0x460 [ 1406.912866][T18221] ? __virt_addr_valid+0x22b/0x430 [ 1406.912870][T18221] ? __pfx___lock_release+0x10/0x10 [ 1406.912873][T18221] ? __pfx_lock_acquire.part.0+0x10/0x10 [ 1406.912877][T18221] __sys_sendmsg+0x109/0x1a0 [ 1406.912881][T18221] ? __pfx___sys_sendmsg+0x10/0x10 [ 1406.912889][T18221] do_syscall_64+0xc1/0x1d0 [ 1406.912892][T18221] entry_SYSCALL_64_after_hwframe+0x77/0x7f [ 1406.912895][T18221] RIP: 0033:0x7fd31bc689a7 [ 1406.912899][T18221] Code: 0a 00 f7 d8 64 89 02 48 c7 c0 ff ff ff ff eb b9 0f 1f 00 f3 0f 1e fa 64 8b 04 25 18 00 00 00 85 c0 75 10 b8 2e 00 00 00 0f 05 <48> 3d 00 f0 ff ff 77 51 c3 48 83 ec 28 89 54 24 1c 48 89 74 24 10 [ 1406.912902][T18221] RSP: 002b:00007ffd54643118 EFLAGS: 00000246 ORIG_RAX: 000000000000002e [ 1406.912905][T18221] RAX: ffffffffffffffda RBX: 00007ffd54643840 RCX: 00007fd31bc689a7 [ 1406.912907][T18221] RDX: 0000000000000000 RSI: 00007ffd54643180 RDI: 0000000000000005 [ 1406.912909][T18221] RBP: 0000000000000004 R08: 0000000000000003 R09: 0000000000000000 [ 1406.912910][T18221] R10: 0000000002676910 R11: 0000000000000246 R12: 0000000000000004 [ 1406.912912][T18221] R13: 0000000067b302aa R14: 0000000000498600 R15: 0000000000000000 [ 1406.912917][T18221] [ 1406.932081][T18221] lag1: Port device veth0 added [ 1407.000593][T18227] lag1: Port device veth6 added [ 1407.341082][T18233] 8021q: adding VLAN 0 to HW filter on device lag1 [ 1407.884486][T18244] lag4: Mode changed to "loadbalance" [ 1407.971953][T18246] 8021q: adding VLAN 0 to HW filter on device lag4 [ 1408.219776][T18251] lag4: Port device veth3 added [ 1408.307937][T18252] lag4: Port device veth5 added [ 1408.648604][T18258] 8021q: adding VLAN 0 to HW filter on device lag4 [ 1409.158735][T18269] lag2: Mode changed to "loadbalance" [ 1409.227349][T18271] 8021q: adding VLAN 0 to HW filter on device lag2 [ 1409.469005][T18276] lag2: Port device veth1 added [ 1409.547208][T18277] lag2: Port device veth7 added [ 1409.779976][T18282] br1: port 1(lag2) entered blocking state [ 1409.780341][T18282] br1: port 1(lag2) entered disabled state [ 1409.780669][T18282] lag2: entered allmulticast mode [ 1409.780885][T18282] veth1: entered allmulticast mode [ 1409.781118][T18282] veth7: entered allmulticast mode [ 1409.783264][T18282] veth1: entered promiscuous mode [ 1409.783559][T18282] veth7: entered promiscuous mode [ 1409.968346][T18286] br1: port 1(lag2) entered blocking state [ 1409.968655][T18286] br1: port 1(lag2) entered forwarding state [ 1410.241267][T18292] lag3: Mode changed to "loadbalance" [ 1410.305567][T18294] 8021q: adding VLAN 0 to HW filter on device lag3 [ 1410.547216][T18299] lag3: Port device veth2 added [ 1410.613042][T18300] lag3: Port device veth4 added [ 1419.485160][T18363] veth7: left allmulticast mode [ 1419.487133][T18363] lag2: Port device veth7 removed [ 1419.535109][T18364] veth7: left promiscuous mode [ 1432.370169][T18414] veth7: entered allmulticast mode [ 1432.370392][T18414] veth7: entered promiscuous mode [ 1432.371275][T18414] lag2: Port device veth7 added [ 1432.605782][T18425] veth1: left allmulticast mode [ 1432.607481][T18425] lag2: Port device veth1 removed [ 1432.669755][T18426] veth1: left promiscuous mode [ 1445.416567][T17273] br1: port 1(lag2) entered disabled state [ 1445.471991][T18476] veth7: left allmulticast mode [ 1445.473334][T18476] lag2: Port device veth7 removed [ 1445.529029][T18477] veth7: left promiscuous mode [ 1445.698213][T18487] veth1: entered allmulticast mode [ 1445.698433][T18487] veth1: entered promiscuous mode [ 1445.699181][T18487] lag2: Port device veth1 added [ 1445.725464][ T150] br1: port 1(lag2) entered blocking state [ 1445.725744][ T150] br1: port 1(lag2) entered forwarding state [ 1445.928286][T18498] veth7: entered allmulticast mode [ 1445.928504][T18498] veth7: entered promiscuous mode [ 1445.929250][T18498] lag2: Port device veth7 added [ 1458.712615][T18548] lag2: left allmulticast mode [ 1458.712799][T18548] veth1: left allmulticast mode [ 1458.712948][T18548] veth7: left allmulticast mode [ 1458.713109][T18548] veth1: left promiscuous mode [ 1458.713271][T18548] veth7: left promiscuous mode [ 1458.713433][T18548] br1: port 1(lag2) entered disabled state [ 1460.815788][T18550] br1: port 1(lag2) entered blocking state [ 1460.816144][T18550] br1: port 1(lag2) entered disabled state [ 1460.816376][T18550] lag2: entered allmulticast mode [ 1460.817253][T18550] veth1: entered allmulticast mode [ 1460.817531][T18550] veth7: entered allmulticast mode [ 1460.819466][T18550] veth1: entered promiscuous mode [ 1460.819670][T18550] veth7: entered promiscuous mode [ 1460.820299][T18550] br1: port 1(lag2) entered blocking state [ 1460.820528][T18550] br1: port 1(lag2) entered forwarding state [ 1473.633059][T18600] lag3: Port device veth2 removed [ 1486.485925][T18651] lag3: Port device veth2 added [ 1486.727369][T18662] lag3: Port device veth4 removed [ 1499.585226][T18713] lag3: Port device veth2 removed [ 1499.813636][T18724] lag3: Port device veth4 added [ 1500.047937][T18735] lag3: Port device veth2 added [ 1512.978886][T18785] br1: port 2(lag3) entered blocking state [ 1512.979271][T18785] br1: port 2(lag3) entered disabled state [ 1512.979648][T18785] lag3: entered allmulticast mode [ 1512.979895][T18785] veth4: entered allmulticast mode [ 1512.980148][T18785] veth2: entered allmulticast mode [ 1512.983107][T18785] lag3: entered promiscuous mode [ 1512.983336][T18785] veth4: entered promiscuous mode [ 1512.983795][T18785] veth2: entered promiscuous mode [ 1512.984339][T18785] lag2: entered promiscuous mode [ 1512.985695][T18785] br1: port 2(lag3) entered blocking state [ 1512.985930][T18785] br1: port 2(lag3) entered forwarding state [ 1515.095079][T18787] lag3: left allmulticast mode [ 1515.095302][T18787] veth4: left allmulticast mode [ 1515.095495][T18787] veth2: left allmulticast mode [ 1515.095668][T18787] lag3: left promiscuous mode [ 1515.095844][T18787] veth4: left promiscuous mode [ 1515.096249][T18787] veth2: left promiscuous mode [ 1515.096826][T18787] br1: port 2(lag3) entered disabled state [ 1515.098763][T18787] lag2: left promiscuous mode [ 1528.298961][T18840] lag3: Port device veth4 removed [ 1528.359540][T18841] lag3: Port device veth2 removed [ 1528.952711][ T5152] br1: port 1(lag2) entered disabled state [ 1529.066193][T18856] lag2: left allmulticast mode [ 1529.067231][T18856] veth1: left allmulticast mode [ 1529.067596][T18856] veth7: left allmulticast mode [ 1529.067805][T18856] br1: port 1(lag2) entered disabled state [ 1529.224385][T18858] lag2: Port device veth7 removed [ 1529.277650][T18859] lag2: Port device veth1 removed [ 1530.354563][T18874] lag4: Port device veth5 removed [ 1530.415957][T18875] lag4: Port device veth3 removed [ 1531.495135][T18888] lag1: Port device veth6 removed [ 1531.558154][T18889] lag1: Port device veth0 removed