[ 11.849383][ T261] ip (261) used greatest stack depth: 23248 bytes left [ 14.009718][ T298] lag1: Mode changed to "loadbalance" [ 14.555655][ T307] lag1: Port device veth0 added [ 14.645741][ T308] lag1: Port device veth6 added [ 15.686920][ T320] 8021q: 802.1Q VLAN Support v1.8 [ 15.687441][ T320] 8021q: adding VLAN 0 to HW filter on device lag1 [ 16.978001][ T339] lag4: Mode changed to "loadbalance" [ 17.093265][ T341] 8021q: adding VLAN 0 to HW filter on device lag4 [ 17.514827][ T347] [ 17.514967][ T347] ====================================================== [ 17.515263][ T347] WARNING: possible circular locking dependency detected [ 17.515501][ T347] 6.13.0-rc5-virtme #1 Not tainted [ 17.515674][ T347] ------------------------------------------------------ [ 17.515898][ T347] ip/347 is trying to acquire lock: [ 17.516148][ T347] ffffffff9edd6630 ((netdev_chain).rwsem){++++}-{4:4}, at: blocking_notifier_call_chain+0x50/0x90 [ 17.516527][ T347] [ 17.516527][ T347] but task is already holding lock: [ 17.516782][ T347] ffff88800d59ad80 (team->team_lock_key#2){+.+.}-{4:4}, at: team_add_slave+0x33/0x80 [ 17.517173][ T347] [ 17.517173][ T347] which lock already depends on the new lock. [ 17.517173][ T347] [ 17.517581][ T347] [ 17.517581][ T347] the existing dependency chain (in reverse order) is: [ 17.517887][ T347] [ 17.517887][ T347] -> #1 (team->team_lock_key#2){+.+.}-{4:4}: [ 17.518227][ T347] __lock_acquire+0xb9a/0x1680 [ 17.518415][ T347] lock_acquire.part.0+0xeb/0x330 [ 17.518594][ T347] __mutex_lock+0x169/0xbc0 [ 17.518821][ T347] team_vlan_rx_add_vid+0x3b/0x1b0 [ 17.519006][ T347] vlan_vid_add+0x2bc/0x860 [ 17.519255][ T347] vlan_device_event+0x2f4/0x2130 [8021q] [ 17.519496][ T347] notifier_call_chain+0xcd/0x150 [ 17.519673][ T347] blocking_notifier_call_chain+0x66/0x90 [ 17.519917][ T347] __dev_notify_flags+0xe6/0x250 [ 17.520138][ T347] dev_change_flags+0xec/0x160 [ 17.520315][ T347] do_setlink.constprop.0+0x79d/0x2300 [ 17.520533][ T347] rtnl_newlink+0x69c/0xa70 [ 17.520707][ T347] rtnetlink_rcv_msg+0x712/0xc10 [ 17.520900][ T347] netlink_rcv_skb+0x130/0x360 [ 17.521079][ T347] netlink_unicast+0x44b/0x710 [ 17.521269][ T347] netlink_sendmsg+0x723/0xbe0 [ 17.521447][ T347] ____sys_sendmsg+0x7ac/0xa10 [ 17.521623][ T347] ___sys_sendmsg+0xee/0x170 [ 17.521806][ T347] __sys_sendmsg+0x109/0x1a0 [ 17.521981][ T347] do_syscall_64+0xc1/0x1d0 [ 17.522188][ T347] entry_SYSCALL_64_after_hwframe+0x77/0x7f [ 17.522415][ T347] [ 17.522415][ T347] -> #0 ((netdev_chain).rwsem){++++}-{4:4}: [ 17.522688][ T347] check_prev_add+0x182/0xce0 [ 17.522870][ T347] validate_chain+0x7bd/0xae0 [ 17.523044][ T347] __lock_acquire+0xb9a/0x1680 [ 17.523292][ T347] lock_acquire.part.0+0xeb/0x330 [ 17.523489][ T347] down_read+0x9f/0x340 [ 17.523672][ T347] blocking_notifier_call_chain+0x50/0x90 [ 17.523899][ T347] __dev_open+0x15c/0x430 [ 17.524073][ T347] dev_open+0xc9/0x130 [ 17.524256][ T347] team_port_add+0x90a/0x1ce0 [ 17.524434][ T347] team_add_slave+0x41/0x80 [ 17.524607][ T347] do_setlink.constprop.0+0x80e/0x2300 [ 17.524827][ T347] rtnl_newlink+0x69c/0xa70 [ 17.525000][ T347] rtnetlink_rcv_msg+0x712/0xc10 [ 17.525193][ T347] netlink_rcv_skb+0x130/0x360 [ 17.525369][ T347] netlink_unicast+0x44b/0x710 [ 17.525540][ T347] netlink_sendmsg+0x723/0xbe0 [ 17.525713][ T347] ____sys_sendmsg+0x7ac/0xa10 [ 17.525888][ T347] ___sys_sendmsg+0xee/0x170 [ 17.526113][ T347] __sys_sendmsg+0x109/0x1a0 [ 17.526317][ T347] do_syscall_64+0xc1/0x1d0 [ 17.526493][ T347] entry_SYSCALL_64_after_hwframe+0x77/0x7f [ 17.526767][ T347] [ 17.526767][ T347] other info that might help us debug this: [ 17.526767][ T347] [ 17.527145][ T347] Possible unsafe locking scenario: [ 17.527145][ T347] [ 17.527462][ T347] CPU0 CPU1 [ 17.527667][ T347] ---- ---- [ 17.527867][ T347] lock(team->team_lock_key#2); [ 17.528068][ T347] lock((netdev_chain).rwsem); [ 17.528338][ T347] lock(team->team_lock_key#2); [ 17.528606][ T347] rlock((netdev_chain).rwsem); [ 17.528778][ T347] [ 17.528778][ T347] *** DEADLOCK *** [ 17.528778][ T347] [ 17.529016][ T347] 2 locks held by ip/347: [ 17.529165][ T347] #0: ffffffff9ede7a48 (rtnl_mutex){+.+.}-{4:4}, at: rtnl_newlink+0x653/0xa70 [ 17.529460][ T347] #1: ffff88800d59ad80 (team->team_lock_key#2){+.+.}-{4:4}, at: team_add_slave+0x33/0x80 [ 17.529753][ T347] [ 17.529753][ T347] stack backtrace: [ 17.529968][ T347] CPU: 0 UID: 0 PID: 347 Comm: ip Not tainted 6.13.0-rc5-virtme #1 [ 17.530261][ T347] Hardware name: Bochs Bochs, BIOS Bochs 01/01/2011 [ 17.530482][ T347] Call Trace: [ 17.530629][ T347] [ 17.530720][ T347] dump_stack_lvl+0x82/0xd0 [ 17.530920][ T347] print_circular_bug+0x2e2/0x4b0 [ 17.531104][ T347] check_noncircular+0x2f9/0x3e0 [ 17.531295][ T347] ? __pfx_check_noncircular+0x10/0x10 [ 17.531473][ T347] ? is_bpf_text_address+0x67/0x120 [ 17.531659][ T347] ? alloc_chain_hlocks+0x4e0/0x720 [ 17.531833][ T347] check_prev_add+0x182/0xce0 [ 17.532009][ T347] validate_chain+0x7bd/0xae0 [ 17.532196][ T347] ? __pfx_validate_chain+0x10/0x10 [ 17.532371][ T347] ? hlock_class+0x4e/0x130 [ 17.532542][ T347] ? mark_lock+0x38/0x3e0 [ 17.532679][ T347] __lock_acquire+0xb9a/0x1680 [ 17.532856][ T347] lock_acquire.part.0+0xeb/0x330 [ 17.533039][ T347] ? blocking_notifier_call_chain+0x50/0x90 [ 17.533272][ T347] ? __pfx_lock_acquire.part.0+0x10/0x10 [ 17.533449][ T347] ? trace_lock_acquire+0x14c/0x1f0 [ 17.533625][ T347] ? lock_acquire+0x32/0xc0 [ 17.533798][ T347] ? blocking_notifier_call_chain+0x50/0x90 [ 17.534013][ T347] down_read+0x9f/0x340 [ 17.534169][ T347] ? blocking_notifier_call_chain+0x50/0x90 [ 17.534396][ T347] ? __pfx_down_read+0x10/0x10 [ 17.534572][ T347] blocking_notifier_call_chain+0x50/0x90 [ 17.534760][ T347] __dev_open+0x15c/0x430 [ 17.534938][ T347] ? trace_percpu_alloc_percpu+0x149/0x1d0 [ 17.535198][ T347] ? __pfx___dev_open+0x10/0x10 [ 17.535435][ T347] ? pcpu_alloc_noprof+0x4fe/0xab0 [ 17.535615][ T347] dev_open+0xc9/0x130 [ 17.535755][ T347] ? __pfx_dev_open+0x10/0x10 [ 17.535934][ T347] team_port_add+0x90a/0x1ce0 [ 17.536121][ T347] ? __pfx_team_port_add+0x10/0x10 [ 17.536298][ T347] ? __pfx_validate_chain+0x10/0x10 [ 17.536473][ T347] ? __pfx_validate_linkmsg+0x10/0x10 [ 17.536655][ T347] team_add_slave+0x41/0x80 [ 17.536830][ T347] do_setlink.constprop.0+0x80e/0x2300 [ 17.537052][ T347] ? __pfx_do_setlink.constprop.0+0x10/0x10 [ 17.537314][ T347] ? lock_acquire.part.0+0xeb/0x330 [ 17.537533][ T347] ? rtnl_newlink+0x653/0xa70 [ 17.537758][ T347] ? rtnl_newlink+0xb9/0xa70 [ 17.537938][ T347] ? rtnetlink_rcv_msg+0x712/0xc10 [ 17.538119][ T347] ? __mutex_trylock_common+0xfa/0x260 [ 17.538297][ T347] ? __pfx___mutex_trylock_common+0x10/0x10 [ 17.538522][ T347] ? lock_acquire+0x32/0xc0 [ 17.538697][ T347] ? trace_contention_end+0xef/0x150 [ 17.538873][ T347] ? __mutex_lock+0x190/0xbc0 [ 17.539053][ T347] ? __pfx_do_raw_spin_lock+0x10/0x10 [ 17.539251][ T347] ? rtnl_newlink+0x653/0xa70 [ 17.539425][ T347] ? __pfx___mutex_lock+0x10/0x10 [ 17.539603][ T347] ? __rtnl_newlink+0x3f2/0xa80 [ 17.539785][ T347] rtnl_newlink+0x69c/0xa70 [ 17.539962][ T347] ? __pfx_rtnl_newlink+0x10/0x10 [ 17.540157][ T347] ? find_held_lock+0x2c/0x110 [ 17.540336][ T347] ? __pfx___lock_release+0x10/0x10 [ 17.540508][ T347] ? __pfx_lock_acquire.part.0+0x10/0x10 [ 17.540684][ T347] ? rtnetlink_rcv_msg+0x6ef/0xc10 [ 17.540856][ T347] ? __pfx_rtnl_newlink+0x10/0x10 [ 17.541026][ T347] rtnetlink_rcv_msg+0x712/0xc10 [ 17.541209][ T347] ? __pfx_rtnetlink_rcv_msg+0x10/0x10 [ 17.541379][ T347] ? hlock_class+0x4e/0x130 [ 17.541555][ T347] ? mark_lock+0x38/0x3e0 [ 17.541682][ T347] ? __lock_acquire+0xb9a/0x1680 [ 17.541859][ T347] netlink_rcv_skb+0x130/0x360 [ 17.542026][ T347] ? __pfx_rtnetlink_rcv_msg+0x10/0x10 [ 17.542196][ T347] ? __pfx_netlink_rcv_skb+0x10/0x10 [ 17.542381][ T347] ? netlink_deliver_tap+0x13e/0x340 [ 17.542556][ T347] ? netlink_deliver_tap+0xc3/0x340 [ 17.542734][ T347] netlink_unicast+0x44b/0x710 [ 17.542911][ T347] ? __pfx_netlink_unicast+0x10/0x10 [ 17.543095][ T347] ? find_held_lock+0x2c/0x110 [ 17.543301][ T347] netlink_sendmsg+0x723/0xbe0 [ 17.543478][ T347] ? __pfx_netlink_sendmsg+0x10/0x10 [ 17.543664][ T347] ____sys_sendmsg+0x7ac/0xa10 [ 17.543843][ T347] ? __pfx_____sys_sendmsg+0x10/0x10 [ 17.544020][ T347] ? __pfx_copy_msghdr_from_user+0x10/0x10 [ 17.544289][ T347] ___sys_sendmsg+0xee/0x170 [ 17.544475][ T347] ? __pfx____sys_sendmsg+0x10/0x10 [ 17.544655][ T347] ? kasan_save_stack+0x34/0x50 [ 17.544834][ T347] ? kasan_save_stack+0x24/0x50 [ 17.545012][ T347] ? __kasan_record_aux_stack+0x8e/0xa0 [ 17.545187][ T347] ? __call_rcu_common.constprop.0+0xa1/0x4b0 [ 17.545407][ T347] ? __x64_sys_close+0x7c/0xd0 [ 17.545580][ T347] ? do_syscall_64+0xc1/0x1d0 [ 17.545751][ T347] ? entry_SYSCALL_64_after_hwframe+0x77/0x7f [ 17.545970][ T347] ? __lock_acquire+0xb9a/0x1680 [ 17.546150][ T347] ? find_held_lock+0x2c/0x110 [ 17.546336][ T347] ? __lock_release+0x103/0x460 [ 17.546514][ T347] ? __virt_addr_valid+0x22b/0x430 [ 17.546695][ T347] ? __pfx___lock_release+0x10/0x10 [ 17.546870][ T347] ? __pfx_lock_acquire.part.0+0x10/0x10 [ 17.547046][ T347] __sys_sendmsg+0x109/0x1a0 [ 17.547218][ T347] ? __pfx___sys_sendmsg+0x10/0x10 [ 17.547393][ T347] ? __pfx_slab_free_after_rcu_debug+0x10/0x10 [ 17.547634][ T347] do_syscall_64+0xc1/0x1d0 [ 17.547847][ T347] entry_SYSCALL_64_after_hwframe+0x77/0x7f [ 17.548067][ T347] RIP: 0033:0x7f2d823a99a7 [ 17.548283][ T347] Code: 0a 00 f7 d8 64 89 02 48 c7 c0 ff ff ff ff eb b9 0f 1f 00 f3 0f 1e fa 64 8b 04 25 18 00 00 00 85 c0 75 10 b8 2e 00 00 00 0f 05 <48> 3d 00 f0 ff ff 77 51 c3 48 83 ec 28 89 54 24 1c 48 89 74 24 10 [ 17.548978][ T347] RSP: 002b:00007fff08ac5d08 EFLAGS: 00000246 ORIG_RAX: 000000000000002e [ 17.549287][ T347] RAX: ffffffffffffffda RBX: 00007fff08ac6430 RCX: 00007f2d823a99a7 [ 17.549605][ T347] RDX: 0000000000000000 RSI: 00007fff08ac5d70 RDI: 0000000000000005 [ 17.549866][ T347] RBP: 0000000000000004 R08: 0000000000000003 R09: 0000000000000000 [ 17.550175][ T347] R10: 000000001cfc8910 R11: 0000000000000246 R12: 0000000000000004 [ 17.550457][ T347] R13: 000000006779293f R14: 0000000000498600 R15: 0000000000000000 [ 17.550722][ T347] [ 17.553243][ T347] lag4: Port device veth3 added [ 17.563055][ T347] ip (347) used greatest stack depth: 23160 bytes left [ 17.632260][ T348] lag4: Port device veth5 added [ 18.033260][ T354] 8021q: adding VLAN 0 to HW filter on device lag4 [ 19.284676][ T377] lag2: Mode changed to "loadbalance" [ 19.356822][ T379] 8021q: adding VLAN 0 to HW filter on device lag2 [ 19.634073][ T385] lag2: Port device veth1 added [ 19.699100][ T386] lag2: Port device veth7 added [ 20.396917][ T401] br1: port 1(lag2.100) entered blocking state [ 20.397153][ T401] br1: port 1(lag2.100) entered disabled state [ 20.397380][ T401] lag2.100: entered allmulticast mode [ 20.397520][ T401] lag2: entered allmulticast mode [ 20.398619][ T401] veth1: entered allmulticast mode [ 20.398822][ T401] veth7: entered allmulticast mode [ 20.400799][ T401] lag2.100: entered promiscuous mode [ 20.401010][ T401] lag2: entered promiscuous mode [ 20.401167][ T401] veth1: entered promiscuous mode [ 20.401513][ T401] veth7: entered promiscuous mode [ 20.670201][ T408] br2: port 1(lag2.200) entered blocking state [ 20.670729][ T408] br2: port 1(lag2.200) entered disabled state [ 20.671681][ T408] lag2.200: entered allmulticast mode [ 20.674250][ T408] lag2.200: entered promiscuous mode [ 20.845614][ T411] br1: port 1(lag2.100) entered blocking state [ 20.845865][ T411] br1: port 1(lag2.100) entered forwarding state [ 20.917139][ T412] br2: port 1(lag2.200) entered blocking state [ 20.917487][ T412] br2: port 1(lag2.200) entered forwarding state [ 21.310284][ T420] lag3: Mode changed to "loadbalance" [ 21.403103][ T421] 8021q: adding VLAN 0 to HW filter on device lag3 [ 21.720190][ T428] lag3: Port device veth2 added [ 21.798789][ T429] lag3: Port device veth4 added [ 33.331553][ T511] veth7: left promiscuous mode [ 33.331989][ T511] veth7: left allmulticast mode [ 33.333147][ T511] lag2: Port device veth7 removed [ 48.565209][ T572] veth7: entered promiscuous mode [ 48.565600][ T572] veth7: entered allmulticast mode [ 48.566361][ T572] lag2: Port device veth7 added [ 48.818091][ T583] veth1: left promiscuous mode [ 48.818484][ T583] veth1: left allmulticast mode [ 48.819647][ T583] lag2: Port device veth1 removed [ 64.050296][ T37] br1: port 1(lag2.100) entered disabled state [ 64.051520][ T37] br2: port 1(lag2.200) entered disabled state [ 64.110675][ T644] veth7: left promiscuous mode [ 64.111092][ T644] veth7: left allmulticast mode [ 64.112976][ T644] lag2: Port device veth7 removed [ 64.372177][ T655] veth1: entered promiscuous mode [ 64.372555][ T655] veth1: entered allmulticast mode [ 64.373299][ T655] lag2: Port device veth1 added [ 64.390957][ T262] br1: port 1(lag2.100) entered blocking state [ 64.391225][ T262] br1: port 1(lag2.100) entered forwarding state [ 64.392200][ T262] br2: port 1(lag2.200) entered blocking state [ 64.392434][ T262] br2: port 1(lag2.200) entered forwarding state [ 64.617348][ T667] veth7: entered promiscuous mode [ 64.617718][ T667] veth7: entered allmulticast mode [ 64.618462][ T667] lag2: Port device veth7 added [ 79.894446][ T727] lag2.200: left allmulticast mode [ 79.894703][ T727] lag2.200: left promiscuous mode [ 79.895311][ T727] br2: port 1(lag2.200) entered disabled state [ 79.958553][ T728] lag2.100: left allmulticast mode [ 79.958815][ T728] lag2: left allmulticast mode [ 79.958991][ T728] veth1: left allmulticast mode [ 79.959179][ T728] veth7: left allmulticast mode [ 79.959353][ T728] lag2.100: left promiscuous mode [ 79.959508][ T728] lag2: left promiscuous mode [ 79.959685][ T728] veth1: left promiscuous mode [ 79.960108][ T728] veth7: left promiscuous mode [ 79.960853][ T728] br1: port 1(lag2.100) entered disabled state [ 82.085179][ T730] br1: port 1(lag2.100) entered blocking state [ 82.085466][ T730] br1: port 1(lag2.100) entered disabled state [ 82.085817][ T730] lag2.100: entered allmulticast mode [ 82.085976][ T730] lag2: entered allmulticast mode [ 82.086134][ T730] veth1: entered allmulticast mode [ 82.086293][ T730] veth7: entered allmulticast mode [ 82.087983][ T730] lag2.100: entered promiscuous mode [ 82.088157][ T730] lag2: entered promiscuous mode [ 82.088312][ T730] veth1: entered promiscuous mode [ 82.088797][ T730] veth7: entered promiscuous mode [ 82.090240][ T730] br1: port 1(lag2.100) entered blocking state [ 82.090592][ T730] br1: port 1(lag2.100) entered forwarding state [ 82.175578][ T731] br2: port 1(lag2.200) entered blocking state [ 82.175970][ T731] br2: port 1(lag2.200) entered disabled state [ 82.176289][ T731] lag2.200: entered allmulticast mode [ 82.178892][ T731] lag2.200: entered promiscuous mode [ 82.180759][ T731] br2: port 1(lag2.200) entered blocking state [ 82.181083][ T731] br2: port 1(lag2.200) entered forwarding state [ 97.427332][ T791] lag3: Port device veth2 removed [ 112.692461][ T852] lag3: Port device veth2 added [ 112.982281][ T863] lag3: Port device veth4 removed [ 128.234777][ T924] lag3: Port device veth2 removed [ 128.524909][ T935] lag3: Port device veth4 added [ 128.784993][ T946] lag3: Port device veth2 added [ 144.669065][ T1016] lag3: Port device veth4 removed [ 144.735236][ T1017] lag3: Port device veth2 removed [ 145.400223][ T1028] br2: port 1(lag2.200) entered disabled state [ 145.455343][ T1029] br1: port 1(lag2.100) entered disabled state [ 145.667640][ T1032] lag2.200: left allmulticast mode [ 145.667853][ T1032] lag2.200: left promiscuous mode [ 145.668349][ T1032] br2: port 1(lag2.200) entered disabled state [ 145.852983][ T1034] lag2.100: left allmulticast mode [ 145.853200][ T1034] lag2: left allmulticast mode [ 145.853371][ T1034] veth1: left allmulticast mode [ 145.853560][ T1034] veth7: left allmulticast mode [ 145.853763][ T1034] lag2.100: left promiscuous mode [ 145.853951][ T1034] lag2: left promiscuous mode [ 145.854147][ T1034] veth1: left promiscuous mode [ 145.854589][ T1034] veth7: left promiscuous mode [ 145.855272][ T1034] br1: port 1(lag2.100) entered disabled state [ 146.253518][ T1038] lag2: Port device veth7 removed [ 146.318024][ T1039] lag2: Port device veth1 removed [ 147.705518][ T1054] lag4: Port device veth5 removed [ 147.789773][ T1055] lag4: Port device veth3 removed [ 149.036499][ T1071] lag1: Port device veth6 removed [ 149.099673][ T1072] lag1: Port device veth0 removed