[ 12.201603][ T253] ip (253) used greatest stack depth: 23136 bytes left [ 14.880862][ T301] lag1: Mode changed to "loadbalance" [ 15.414862][ T310] lag1: Port device veth0 added [ 15.500411][ T311] lag1: Port device veth6 added [ 16.643476][ T323] 8021q: 802.1Q VLAN Support v1.8 [ 16.644053][ T323] 8021q: adding VLAN 0 to HW filter on device lag1 [ 18.077300][ T342] lag4: Mode changed to "loadbalance" [ 18.167589][ T344] 8021q: adding VLAN 0 to HW filter on device lag4 [ 18.565296][ T350] [ 18.565474][ T350] ====================================================== [ 18.565766][ T350] WARNING: possible circular locking dependency detected [ 18.566111][ T350] 6.13.0-rc5-virtme #1 Not tainted [ 18.566349][ T350] ------------------------------------------------------ [ 18.566616][ T350] ip/350 is trying to acquire lock: [ 18.566842][ T350] ffffffff8c1d6630 ((netdev_chain).rwsem){++++}-{4:4}, at: blocking_notifier_call_chain+0x50/0x90 [ 18.567242][ T350] [ 18.567242][ T350] but task is already holding lock: [ 18.567557][ T350] ffff88800e46ad80 (team->team_lock_key#2){+.+.}-{4:4}, at: team_add_slave+0x33/0x80 [ 18.567958][ T350] [ 18.567958][ T350] which lock already depends on the new lock. [ 18.567958][ T350] [ 18.568421][ T350] [ 18.568421][ T350] the existing dependency chain (in reverse order) is: [ 18.568816][ T350] [ 18.568816][ T350] -> #1 (team->team_lock_key#2){+.+.}-{4:4}: [ 18.569180][ T350] __lock_acquire+0xb9a/0x1680 [ 18.569421][ T350] lock_acquire.part.0+0xeb/0x330 [ 18.569653][ T350] __mutex_lock+0x169/0xbc0 [ 18.569884][ T350] team_vlan_rx_add_vid+0x3b/0x1b0 [ 18.570113][ T350] vlan_vid_add+0x2bc/0x860 [ 18.570354][ T350] vlan_device_event+0x2f4/0x2130 [8021q] [ 18.570649][ T350] notifier_call_chain+0xcd/0x150 [ 18.570887][ T350] blocking_notifier_call_chain+0x66/0x90 [ 18.571179][ T350] __dev_notify_flags+0xe6/0x250 [ 18.571400][ T350] dev_change_flags+0xec/0x160 [ 18.571634][ T350] do_setlink.constprop.0+0x79d/0x2300 [ 18.571946][ T350] rtnl_newlink+0x69c/0xa70 [ 18.572174][ T350] rtnetlink_rcv_msg+0x712/0xc10 [ 18.572414][ T350] netlink_rcv_skb+0x130/0x360 [ 18.572645][ T350] netlink_unicast+0x44b/0x710 [ 18.572857][ T350] netlink_sendmsg+0x723/0xbe0 [ 18.573089][ T350] ____sys_sendmsg+0x7ac/0xa10 [ 18.573330][ T350] ___sys_sendmsg+0xee/0x170 [ 18.573575][ T350] __sys_sendmsg+0x109/0x1a0 [ 18.573815][ T350] do_syscall_64+0xc1/0x1d0 [ 18.574053][ T350] entry_SYSCALL_64_after_hwframe+0x77/0x7f [ 18.574351][ T350] [ 18.574351][ T350] -> #0 ((netdev_chain).rwsem){++++}-{4:4}: [ 18.574740][ T350] check_prev_add+0x182/0xce0 [ 18.574995][ T350] validate_chain+0x7bd/0xae0 [ 18.575230][ T350] __lock_acquire+0xb9a/0x1680 [ 18.575463][ T350] lock_acquire.part.0+0xeb/0x330 [ 18.575710][ T350] down_read+0x9f/0x340 [ 18.575950][ T350] blocking_notifier_call_chain+0x50/0x90 [ 18.576260][ T350] __dev_open+0x15c/0x430 [ 18.576547][ T350] dev_open+0xc9/0x130 [ 18.576787][ T350] team_port_add+0x90a/0x1ce0 [ 18.577012][ T350] team_add_slave+0x41/0x80 [ 18.577229][ T350] do_setlink.constprop.0+0x80e/0x2300 [ 18.577515][ T350] rtnl_newlink+0x69c/0xa70 [ 18.577742][ T350] rtnetlink_rcv_msg+0x712/0xc10 [ 18.577984][ T350] netlink_rcv_skb+0x130/0x360 [ 18.578220][ T350] netlink_unicast+0x44b/0x710 [ 18.578454][ T350] netlink_sendmsg+0x723/0xbe0 [ 18.578686][ T350] ____sys_sendmsg+0x7ac/0xa10 [ 18.578918][ T350] ___sys_sendmsg+0xee/0x170 [ 18.579153][ T350] __sys_sendmsg+0x109/0x1a0 [ 18.579402][ T350] do_syscall_64+0xc1/0x1d0 [ 18.579642][ T350] entry_SYSCALL_64_after_hwframe+0x77/0x7f [ 18.579940][ T350] [ 18.579940][ T350] other info that might help us debug this: [ 18.579940][ T350] [ 18.580395][ T350] Possible unsafe locking scenario: [ 18.580395][ T350] [ 18.580759][ T350] CPU0 CPU1 [ 18.581010][ T350] ---- ---- [ 18.581247][ T350] lock(team->team_lock_key#2); [ 18.581491][ T350] lock((netdev_chain).rwsem); [ 18.581896][ T350] lock(team->team_lock_key#2); [ 18.582127][ T350] rlock((netdev_chain).rwsem); [ 18.582279][ T350] [ 18.582279][ T350] *** DEADLOCK *** [ 18.582279][ T350] [ 18.582517][ T350] 2 locks held by ip/350: [ 18.582651][ T350] #0: ffffffff8c1e7a48 (rtnl_mutex){+.+.}-{4:4}, at: rtnl_newlink+0x653/0xa70 [ 18.582948][ T350] #1: ffff88800e46ad80 (team->team_lock_key#2){+.+.}-{4:4}, at: team_add_slave+0x33/0x80 [ 18.583239][ T350] [ 18.583239][ T350] stack backtrace: [ 18.583446][ T350] CPU: 0 UID: 0 PID: 350 Comm: ip Not tainted 6.13.0-rc5-virtme #1 [ 18.583701][ T350] Hardware name: Bochs Bochs, BIOS Bochs 01/01/2011 [ 18.583908][ T350] Call Trace: [ 18.584037][ T350] [ 18.584127][ T350] dump_stack_lvl+0x82/0xd0 [ 18.584303][ T350] print_circular_bug+0x2e2/0x4b0 [ 18.584457][ T350] check_noncircular+0x2f9/0x3e0 [ 18.584630][ T350] ? __pfx_check_noncircular+0x10/0x10 [ 18.584801][ T350] ? is_bpf_text_address+0x67/0x120 [ 18.584985][ T350] ? alloc_chain_hlocks+0x4e0/0x720 [ 18.585169][ T350] check_prev_add+0x182/0xce0 [ 18.585334][ T350] validate_chain+0x7bd/0xae0 [ 18.585509][ T350] ? __pfx_validate_chain+0x10/0x10 [ 18.585666][ T350] ? hlock_class+0x4e/0x130 [ 18.585816][ T350] ? mark_lock+0x38/0x3e0 [ 18.585943][ T350] __lock_acquire+0xb9a/0x1680 [ 18.586096][ T350] lock_acquire.part.0+0xeb/0x330 [ 18.586256][ T350] ? blocking_notifier_call_chain+0x50/0x90 [ 18.586447][ T350] ? __pfx_lock_acquire.part.0+0x10/0x10 [ 18.586598][ T350] ? trace_lock_acquire+0x14c/0x1f0 [ 18.586754][ T350] ? lock_acquire+0x32/0xc0 [ 18.586907][ T350] ? blocking_notifier_call_chain+0x50/0x90 [ 18.587097][ T350] down_read+0x9f/0x340 [ 18.587209][ T350] ? blocking_notifier_call_chain+0x50/0x90 [ 18.587403][ T350] ? __pfx_down_read+0x10/0x10 [ 18.587554][ T350] blocking_notifier_call_chain+0x50/0x90 [ 18.587708][ T350] __dev_open+0x15c/0x430 [ 18.587822][ T350] ? trace_percpu_alloc_percpu+0x149/0x1d0 [ 18.588016][ T350] ? __pfx___dev_open+0x10/0x10 [ 18.588165][ T350] ? pcpu_alloc_noprof+0x4fe/0xab0 [ 18.588317][ T350] dev_open+0xc9/0x130 [ 18.588438][ T350] ? __pfx_dev_open+0x10/0x10 [ 18.588590][ T350] team_port_add+0x90a/0x1ce0 [ 18.588742][ T350] ? __pfx_team_port_add+0x10/0x10 [ 18.588892][ T350] ? __pfx_validate_chain+0x10/0x10 [ 18.589042][ T350] ? __pfx_validate_linkmsg+0x10/0x10 [ 18.589198][ T350] team_add_slave+0x41/0x80 [ 18.589354][ T350] do_setlink.constprop.0+0x80e/0x2300 [ 18.589514][ T350] ? __pfx_do_setlink.constprop.0+0x10/0x10 [ 18.589700][ T350] ? lock_acquire.part.0+0xeb/0x330 [ 18.589851][ T350] ? rtnl_newlink+0x653/0xa70 [ 18.590002][ T350] ? rtnl_newlink+0xb9/0xa70 [ 18.590155][ T350] ? rtnetlink_rcv_msg+0x712/0xc10 [ 18.590306][ T350] ? __mutex_trylock_common+0xfa/0x260 [ 18.590459][ T350] ? __pfx___mutex_trylock_common+0x10/0x10 [ 18.590650][ T350] ? lock_acquire+0x32/0xc0 [ 18.590801][ T350] ? trace_contention_end+0xef/0x150 [ 18.590953][ T350] ? __mutex_lock+0x190/0xbc0 [ 18.591103][ T350] ? __pfx_do_raw_spin_lock+0x10/0x10 [ 18.591255][ T350] ? rtnl_newlink+0x653/0xa70 [ 18.591407][ T350] ? __pfx___mutex_lock+0x10/0x10 [ 18.591587][ T350] ? __rtnl_newlink+0x3f2/0xa80 [ 18.591748][ T350] rtnl_newlink+0x69c/0xa70 [ 18.591914][ T350] ? __pfx_rtnl_newlink+0x10/0x10 [ 18.592061][ T350] ? find_held_lock+0x2c/0x110 [ 18.592213][ T350] ? __pfx___lock_release+0x10/0x10 [ 18.592368][ T350] ? __pfx_lock_acquire.part.0+0x10/0x10 [ 18.592521][ T350] ? rtnetlink_rcv_msg+0x6ef/0xc10 [ 18.592669][ T350] ? __pfx_rtnl_newlink+0x10/0x10 [ 18.592825][ T350] rtnetlink_rcv_msg+0x712/0xc10 [ 18.592977][ T350] ? __pfx_rtnetlink_rcv_msg+0x10/0x10 [ 18.593125][ T350] ? hlock_class+0x4e/0x130 [ 18.593273][ T350] ? mark_lock+0x38/0x3e0 [ 18.593388][ T350] ? __lock_acquire+0xb9a/0x1680 [ 18.593540][ T350] netlink_rcv_skb+0x130/0x360 [ 18.593690][ T350] ? __pfx_rtnetlink_rcv_msg+0x10/0x10 [ 18.593838][ T350] ? __pfx_netlink_rcv_skb+0x10/0x10 [ 18.593997][ T350] ? netlink_deliver_tap+0x13e/0x340 [ 18.594155][ T350] ? netlink_deliver_tap+0xc3/0x340 [ 18.594344][ T350] netlink_unicast+0x44b/0x710 [ 18.594494][ T350] ? __pfx_netlink_unicast+0x10/0x10 [ 18.594643][ T350] ? find_held_lock+0x2c/0x110 [ 18.594804][ T350] netlink_sendmsg+0x723/0xbe0 [ 18.594955][ T350] ? __pfx_netlink_sendmsg+0x10/0x10 [ 18.595116][ T350] ____sys_sendmsg+0x7ac/0xa10 [ 18.595267][ T350] ? __pfx_____sys_sendmsg+0x10/0x10 [ 18.595438][ T350] ? __pfx_copy_msghdr_from_user+0x10/0x10 [ 18.595631][ T350] ___sys_sendmsg+0xee/0x170 [ 18.595780][ T350] ? __pfx____sys_sendmsg+0x10/0x10 [ 18.595932][ T350] ? kasan_save_stack+0x34/0x50 [ 18.596083][ T350] ? kasan_save_stack+0x24/0x50 [ 18.596245][ T350] ? __kasan_record_aux_stack+0x8e/0xa0 [ 18.596397][ T350] ? __call_rcu_common.constprop.0+0xa1/0x4b0 [ 18.596589][ T350] ? __x64_sys_close+0x7c/0xd0 [ 18.596745][ T350] ? do_syscall_64+0xc1/0x1d0 [ 18.596894][ T350] ? entry_SYSCALL_64_after_hwframe+0x77/0x7f [ 18.597076][ T350] ? __lock_acquire+0xb9a/0x1680 [ 18.597227][ T350] ? find_held_lock+0x2c/0x110 [ 18.597388][ T350] ? __lock_release+0x103/0x460 [ 18.597540][ T350] ? __virt_addr_valid+0x22b/0x430 [ 18.597693][ T350] ? __pfx___lock_release+0x10/0x10 [ 18.597933][ T350] ? __pfx_lock_acquire.part.0+0x10/0x10 [ 18.598082][ T350] __sys_sendmsg+0x109/0x1a0 [ 18.598231][ T350] ? __pfx___sys_sendmsg+0x10/0x10 [ 18.598384][ T350] ? __pfx_slab_free_after_rcu_debug+0x10/0x10 [ 18.598686][ T350] do_syscall_64+0xc1/0x1d0 [ 18.598836][ T350] entry_SYSCALL_64_after_hwframe+0x77/0x7f [ 18.599019][ T350] RIP: 0033:0x7ff5c3fc79a7 [ 18.599267][ T350] Code: 0a 00 f7 d8 64 89 02 48 c7 c0 ff ff ff ff eb b9 0f 1f 00 f3 0f 1e fa 64 8b 04 25 18 00 00 00 85 c0 75 10 b8 2e 00 00 00 0f 05 <48> 3d 00 f0 ff ff 77 51 c3 48 83 ec 28 89 54 24 1c 48 89 74 24 10 [ 18.599831][ T350] RSP: 002b:00007ffe29686bd8 EFLAGS: 00000246 ORIG_RAX: 000000000000002e [ 18.600147][ T350] RAX: ffffffffffffffda RBX: 00007ffe29687300 RCX: 00007ff5c3fc79a7 [ 18.600372][ T350] RDX: 0000000000000000 RSI: 00007ffe29686c40 RDI: 0000000000000005 [ 18.600735][ T350] RBP: 0000000000000004 R08: 0000000000000003 R09: 0000000000000000 [ 18.600960][ T350] R10: 00000000207c7910 R11: 0000000000000246 R12: 0000000000000004 [ 18.601187][ T350] R13: 00000000677951cf R14: 0000000000498600 R15: 0000000000000000 [ 18.601502][ T350] [ 18.610752][ T350] lag4: Port device veth3 added [ 18.691637][ T351] lag4: Port device veth5 added [ 19.054274][ T357] 8021q: adding VLAN 0 to HW filter on device lag4 [ 20.399501][ T380] lag2: Mode changed to "loadbalance" [ 20.484017][ T382] 8021q: adding VLAN 0 to HW filter on device lag2 [ 20.791986][ T388] lag2: Port device veth1 added [ 20.872501][ T389] lag2: Port device veth7 added [ 21.623938][ T403] br1: port 1(lag2.100) entered blocking state [ 21.624811][ T403] br1: port 1(lag2.100) entered disabled state [ 21.625112][ T403] lag2.100: entered allmulticast mode [ 21.625298][ T403] lag2: entered allmulticast mode [ 21.625480][ T403] veth1: entered allmulticast mode [ 21.625671][ T403] veth7: entered allmulticast mode [ 21.628008][ T403] lag2.100: entered promiscuous mode [ 21.628219][ T403] lag2: entered promiscuous mode [ 21.628408][ T403] veth1: entered promiscuous mode [ 21.628777][ T403] veth7: entered promiscuous mode [ 21.929046][ T410] br2: port 1(lag2.200) entered blocking state [ 21.929336][ T410] br2: port 1(lag2.200) entered disabled state [ 21.929561][ T410] lag2.200: entered allmulticast mode [ 21.931549][ T410] lag2.200: entered promiscuous mode [ 22.139111][ T413] br1: port 1(lag2.100) entered blocking state [ 22.139530][ T413] br1: port 1(lag2.100) entered forwarding state [ 22.213108][ T414] br2: port 1(lag2.200) entered blocking state [ 22.213381][ T414] br2: port 1(lag2.200) entered forwarding state [ 22.605329][ T422] lag3: Mode changed to "loadbalance" [ 22.679183][ T424] 8021q: adding VLAN 0 to HW filter on device lag3 [ 22.960046][ T430] lag3: Port device veth2 added [ 23.035584][ T431] lag3: Port device veth4 added [ 34.619673][ T512] veth7: left promiscuous mode [ 34.620096][ T512] veth7: left allmulticast mode [ 34.621250][ T512] lag2: Port device veth7 removed [ 49.915437][ T573] veth7: entered promiscuous mode [ 49.915829][ T573] veth7: entered allmulticast mode [ 49.916615][ T573] lag2: Port device veth7 added [ 50.223340][ T584] veth1: left promiscuous mode [ 50.223770][ T584] veth1: left allmulticast mode [ 50.225324][ T584] lag2: Port device veth1 removed [ 65.528094][ T37] br1: port 1(lag2.100) entered disabled state [ 65.529404][ T37] br2: port 1(lag2.200) entered disabled state [ 65.591833][ T645] veth7: left promiscuous mode [ 65.592402][ T645] veth7: left allmulticast mode [ 65.594362][ T645] lag2: Port device veth7 removed [ 65.839100][ T656] veth1: entered promiscuous mode [ 65.839608][ T656] veth1: entered allmulticast mode [ 65.840618][ T656] lag2: Port device veth1 added [ 65.863010][ T380] br1: port 1(lag2.100) entered blocking state [ 65.863382][ T380] br1: port 1(lag2.100) entered forwarding state [ 65.866162][ T380] br2: port 1(lag2.200) entered blocking state [ 65.866425][ T380] br2: port 1(lag2.200) entered forwarding state [ 66.118240][ T667] veth7: entered promiscuous mode [ 66.118679][ T667] veth7: entered allmulticast mode [ 66.119476][ T667] lag2: Port device veth7 added [ 81.316589][ T727] lag2.200: left allmulticast mode [ 81.316819][ T727] lag2.200: left promiscuous mode [ 81.317367][ T727] br2: port 1(lag2.200) entered disabled state [ 81.382862][ T728] lag2.100: left allmulticast mode [ 81.383087][ T728] lag2: left allmulticast mode [ 81.383268][ T728] veth1: left allmulticast mode [ 81.383437][ T728] veth7: left allmulticast mode [ 81.383608][ T728] lag2.100: left promiscuous mode [ 81.383786][ T728] lag2: left promiscuous mode [ 81.383963][ T728] veth1: left promiscuous mode [ 81.384389][ T728] veth7: left promiscuous mode [ 81.385106][ T728] br1: port 1(lag2.100) entered disabled state [ 83.482275][ T730] br1: port 1(lag2.100) entered blocking state [ 83.482587][ T730] br1: port 1(lag2.100) entered disabled state [ 83.482820][ T730] lag2.100: entered allmulticast mode [ 83.482994][ T730] lag2: entered allmulticast mode [ 83.483152][ T730] veth1: entered allmulticast mode [ 83.483318][ T730] veth7: entered allmulticast mode [ 83.485116][ T730] lag2.100: entered promiscuous mode [ 83.485305][ T730] lag2: entered promiscuous mode [ 83.485466][ T730] veth1: entered promiscuous mode [ 83.485826][ T730] veth7: entered promiscuous mode [ 83.488100][ T730] br1: port 1(lag2.100) entered blocking state [ 83.488335][ T730] br1: port 1(lag2.100) entered forwarding state [ 83.547338][ T731] br2: port 1(lag2.200) entered blocking state [ 83.547588][ T731] br2: port 1(lag2.200) entered disabled state [ 83.547820][ T731] lag2.200: entered allmulticast mode [ 83.549917][ T731] lag2.200: entered promiscuous mode [ 83.551080][ T731] br2: port 1(lag2.200) entered blocking state [ 83.551308][ T731] br2: port 1(lag2.200) entered forwarding state [ 98.710221][ T791] lag3: Port device veth2 removed [ 113.963152][ T852] lag3: Port device veth2 added [ 114.244261][ T863] lag3: Port device veth4 removed [ 129.501373][ T924] lag3: Port device veth2 removed [ 129.784750][ T935] lag3: Port device veth4 added [ 130.079220][ T946] lag3: Port device veth2 added [ 146.024218][ T1016] lag3: Port device veth4 removed [ 146.090211][ T1017] lag3: Port device veth2 removed [ 146.794866][ T1029] br2: port 1(lag2.200) entered disabled state [ 146.860400][ T1030] br1: port 1(lag2.100) entered disabled state [ 147.063878][ T1033] lag2.200: left allmulticast mode [ 147.064235][ T1033] lag2.200: left promiscuous mode [ 147.064991][ T1033] br2: port 1(lag2.200) entered disabled state [ 147.284608][ T1035] lag2.100: left allmulticast mode [ 147.284809][ T1035] lag2: left allmulticast mode [ 147.284968][ T1035] veth1: left allmulticast mode [ 147.285140][ T1035] veth7: left allmulticast mode [ 147.285298][ T1035] lag2.100: left promiscuous mode [ 147.285443][ T1035] lag2: left promiscuous mode [ 147.285591][ T1035] veth1: left promiscuous mode [ 147.285959][ T1035] veth7: left promiscuous mode [ 147.286977][ T1035] br1: port 1(lag2.100) entered disabled state [ 147.641377][ T1039] lag2: Port device veth7 removed [ 147.722129][ T1040] lag2: Port device veth1 removed [ 149.123532][ T1057] lag4: Port device veth5 removed [ 149.188957][ T1058] lag4: Port device veth3 removed [ 150.511486][ T1073] lag1: Port device veth6 removed [ 150.567985][ T1074] lag1: Port device veth0 removed