[ 13.318272][ T299] lag1: Mode changed to "loadbalance" [ 13.825452][ T308] lag1: Port device veth0 added [ 13.920102][ T309] lag1: Port device veth6 added [ 14.951557][ T321] 8021q: 802.1Q VLAN Support v1.8 [ 14.951887][ T321] 8021q: adding VLAN 0 to HW filter on device lag1 [ 16.281167][ T340] lag4: Mode changed to "loadbalance" [ 16.389891][ T342] 8021q: adding VLAN 0 to HW filter on device lag4 [ 16.783312][ T348] [ 16.783450][ T348] ====================================================== [ 16.783657][ T348] WARNING: possible circular locking dependency detected [ 16.783857][ T348] 6.13.0-rc5-virtme #1 Not tainted [ 16.784019][ T348] ------------------------------------------------------ [ 16.784219][ T348] ip/348 is trying to acquire lock: [ 16.784379][ T348] ffffffff935d6630 ((netdev_chain).rwsem){++++}-{4:4}, at: blocking_notifier_call_chain+0x50/0x90 [ 16.784720][ T348] [ 16.784720][ T348] but task is already holding lock: [ 16.784955][ T348] ffff888009fdad80 (team->team_lock_key#2){+.+.}-{4:4}, at: team_add_slave+0x33/0x80 [ 16.785257][ T348] [ 16.785257][ T348] which lock already depends on the new lock. [ 16.785257][ T348] [ 16.785571][ T348] [ 16.785571][ T348] the existing dependency chain (in reverse order) is: [ 16.785841][ T348] [ 16.785841][ T348] -> #1 (team->team_lock_key#2){+.+.}-{4:4}: [ 16.786088][ T348] __lock_acquire+0xb9a/0x1680 [ 16.786254][ T348] lock_acquire.part.0+0xeb/0x330 [ 16.786416][ T348] __mutex_lock+0x169/0xbc0 [ 16.786579][ T348] team_vlan_rx_add_vid+0x3b/0x1b0 [ 16.786742][ T348] vlan_vid_add+0x2bc/0x860 [ 16.786908][ T348] vlan_device_event+0x2f4/0x2130 [8021q] [ 16.787117][ T348] notifier_call_chain+0xcd/0x150 [ 16.787280][ T348] blocking_notifier_call_chain+0x66/0x90 [ 16.787477][ T348] __dev_notify_flags+0xe6/0x250 [ 16.787639][ T348] dev_change_flags+0xec/0x160 [ 16.787798][ T348] do_setlink.constprop.0+0x79d/0x2300 [ 16.787998][ T348] rtnl_newlink+0x69c/0xa70 [ 16.788158][ T348] rtnetlink_rcv_msg+0x712/0xc10 [ 16.788317][ T348] netlink_rcv_skb+0x130/0x360 [ 16.788481][ T348] netlink_unicast+0x44b/0x710 [ 16.788661][ T348] netlink_sendmsg+0x723/0xbe0 [ 16.788825][ T348] ____sys_sendmsg+0x7ac/0xa10 [ 16.789003][ T348] ___sys_sendmsg+0xee/0x170 [ 16.789167][ T348] __sys_sendmsg+0x109/0x1a0 [ 16.789325][ T348] do_syscall_64+0xc1/0x1d0 [ 16.789484][ T348] entry_SYSCALL_64_after_hwframe+0x77/0x7f [ 16.789685][ T348] [ 16.789685][ T348] -> #0 ((netdev_chain).rwsem){++++}-{4:4}: [ 16.789934][ T348] check_prev_add+0x182/0xce0 [ 16.790102][ T348] validate_chain+0x7bd/0xae0 [ 16.790258][ T348] __lock_acquire+0xb9a/0x1680 [ 16.790424][ T348] lock_acquire.part.0+0xeb/0x330 [ 16.790583][ T348] down_read+0x9f/0x340 [ 16.790741][ T348] blocking_notifier_call_chain+0x50/0x90 [ 16.790936][ T348] __dev_open+0x15c/0x430 [ 16.791100][ T348] dev_open+0xc9/0x130 [ 16.791268][ T348] team_port_add+0x90a/0x1ce0 [ 16.791432][ T348] team_add_slave+0x41/0x80 [ 16.791591][ T348] do_setlink.constprop.0+0x80e/0x2300 [ 16.791788][ T348] rtnl_newlink+0x69c/0xa70 [ 16.791946][ T348] rtnetlink_rcv_msg+0x712/0xc10 [ 16.792106][ T348] netlink_rcv_skb+0x130/0x360 [ 16.792264][ T348] netlink_unicast+0x44b/0x710 [ 16.792424][ T348] netlink_sendmsg+0x723/0xbe0 [ 16.792581][ T348] ____sys_sendmsg+0x7ac/0xa10 [ 16.792738][ T348] ___sys_sendmsg+0xee/0x170 [ 16.792898][ T348] __sys_sendmsg+0x109/0x1a0 [ 16.793060][ T348] do_syscall_64+0xc1/0x1d0 [ 16.793220][ T348] entry_SYSCALL_64_after_hwframe+0x77/0x7f [ 16.793416][ T348] [ 16.793416][ T348] other info that might help us debug this: [ 16.793416][ T348] [ 16.793722][ T348] Possible unsafe locking scenario: [ 16.793722][ T348] [ 16.793959][ T348] CPU0 CPU1 [ 16.794115][ T348] ---- ---- [ 16.794276][ T348] lock(team->team_lock_key#2); [ 16.794447][ T348] lock((netdev_chain).rwsem); [ 16.794695][ T348] lock(team->team_lock_key#2); [ 16.794940][ T348] rlock((netdev_chain).rwsem); [ 16.795119][ T348] [ 16.795119][ T348] *** DEADLOCK *** [ 16.795119][ T348] [ 16.795353][ T348] 2 locks held by ip/348: [ 16.795477][ T348] #0: ffffffff935e7a48 (rtnl_mutex){+.+.}-{4:4}, at: rtnl_newlink+0x653/0xa70 [ 16.795761][ T348] #1: ffff888009fdad80 (team->team_lock_key#2){+.+.}-{4:4}, at: team_add_slave+0x33/0x80 [ 16.796053][ T348] [ 16.796053][ T348] stack backtrace: [ 16.796254][ T348] CPU: 2 UID: 0 PID: 348 Comm: ip Not tainted 6.13.0-rc5-virtme #1 [ 16.796495][ T348] Hardware name: Bochs Bochs, BIOS Bochs 01/01/2011 [ 16.796695][ T348] Call Trace: [ 16.796821][ T348] [ 16.796906][ T348] dump_stack_lvl+0x82/0xd0 [ 16.797069][ T348] print_circular_bug+0x2e2/0x4b0 [ 16.797233][ T348] check_noncircular+0x2f9/0x3e0 [ 16.797393][ T348] ? __pfx_check_noncircular+0x10/0x10 [ 16.797603][ T348] ? is_bpf_text_address+0x67/0x120 [ 16.797801][ T348] ? alloc_chain_hlocks+0x4e0/0x720 [ 16.797963][ T348] check_prev_add+0x182/0xce0 [ 16.798132][ T348] validate_chain+0x7bd/0xae0 [ 16.798294][ T348] ? __pfx_validate_chain+0x10/0x10 [ 16.798461][ T348] ? hlock_class+0x4e/0x130 [ 16.798624][ T348] ? mark_lock+0x38/0x3e0 [ 16.798750][ T348] __lock_acquire+0xb9a/0x1680 [ 16.798913][ T348] lock_acquire.part.0+0xeb/0x330 [ 16.799072][ T348] ? blocking_notifier_call_chain+0x50/0x90 [ 16.799267][ T348] ? __pfx_lock_acquire.part.0+0x10/0x10 [ 16.799426][ T348] ? trace_lock_acquire+0x14c/0x1f0 [ 16.799585][ T348] ? lock_acquire+0x32/0xc0 [ 16.799744][ T348] ? blocking_notifier_call_chain+0x50/0x90 [ 16.799941][ T348] down_read+0x9f/0x340 [ 16.800063][ T348] ? blocking_notifier_call_chain+0x50/0x90 [ 16.800260][ T348] ? __pfx_down_read+0x10/0x10 [ 16.800421][ T348] blocking_notifier_call_chain+0x50/0x90 [ 16.800582][ T348] __dev_open+0x15c/0x430 [ 16.800705][ T348] ? trace_percpu_alloc_percpu+0x149/0x1d0 [ 16.800909][ T348] ? __pfx___dev_open+0x10/0x10 [ 16.801068][ T348] ? pcpu_alloc_noprof+0x4fe/0xab0 [ 16.801231][ T348] dev_open+0xc9/0x130 [ 16.801352][ T348] ? __pfx_dev_open+0x10/0x10 [ 16.801519][ T348] team_port_add+0x90a/0x1ce0 [ 16.801690][ T348] ? __pfx_team_port_add+0x10/0x10 [ 16.801848][ T348] ? __pfx_validate_chain+0x10/0x10 [ 16.802011][ T348] ? __pfx_validate_linkmsg+0x10/0x10 [ 16.802178][ T348] team_add_slave+0x41/0x80 [ 16.802336][ T348] do_setlink.constprop.0+0x80e/0x2300 [ 16.802497][ T348] ? __pfx_do_setlink.constprop.0+0x10/0x10 [ 16.802698][ T348] ? lock_acquire.part.0+0xeb/0x330 [ 16.802858][ T348] ? rtnl_newlink+0x653/0xa70 [ 16.803016][ T348] ? rtnl_newlink+0xb9/0xa70 [ 16.803181][ T348] ? rtnetlink_rcv_msg+0x712/0xc10 [ 16.803344][ T348] ? __mutex_trylock_common+0xfa/0x260 [ 16.803506][ T348] ? __pfx___mutex_trylock_common+0x10/0x10 [ 16.803703][ T348] ? lock_acquire+0x32/0xc0 [ 16.803861][ T348] ? trace_contention_end+0xef/0x150 [ 16.804020][ T348] ? __mutex_lock+0x190/0xbc0 [ 16.804180][ T348] ? __pfx_do_raw_spin_lock+0x10/0x10 [ 16.804343][ T348] ? rtnl_newlink+0x653/0xa70 [ 16.804500][ T348] ? __pfx___mutex_lock+0x10/0x10 [ 16.804670][ T348] ? __rtnl_newlink+0x3f2/0xa80 [ 16.804827][ T348] rtnl_newlink+0x69c/0xa70 [ 16.804986][ T348] ? __pfx_rtnl_newlink+0x10/0x10 [ 16.805151][ T348] ? find_held_lock+0x2c/0x110 [ 16.805314][ T348] ? __pfx___lock_release+0x10/0x10 [ 16.805474][ T348] ? __pfx_lock_acquire.part.0+0x10/0x10 [ 16.805641][ T348] ? rtnetlink_rcv_msg+0x6ef/0xc10 [ 16.805800][ T348] ? __pfx_rtnl_newlink+0x10/0x10 [ 16.805958][ T348] rtnetlink_rcv_msg+0x712/0xc10 [ 16.806121][ T348] ? __pfx_rtnetlink_rcv_msg+0x10/0x10 [ 16.806284][ T348] ? hlock_class+0x4e/0x130 [ 16.806445][ T348] ? mark_lock+0x38/0x3e0 [ 16.806569][ T348] ? __lock_acquire+0xb9a/0x1680 [ 16.806734][ T348] netlink_rcv_skb+0x130/0x360 [ 16.806897][ T348] ? __pfx_rtnetlink_rcv_msg+0x10/0x10 [ 16.807058][ T348] ? __pfx_netlink_rcv_skb+0x10/0x10 [ 16.807225][ T348] ? netlink_deliver_tap+0x13e/0x340 [ 16.807388][ T348] ? netlink_deliver_tap+0xc3/0x340 [ 16.807549][ T348] netlink_unicast+0x44b/0x710 [ 16.807709][ T348] ? __pfx_netlink_unicast+0x10/0x10 [ 16.807878][ T348] ? find_held_lock+0x2c/0x110 [ 16.808041][ T348] netlink_sendmsg+0x723/0xbe0 [ 16.808202][ T348] ? __pfx_netlink_sendmsg+0x10/0x10 [ 16.808364][ T348] ____sys_sendmsg+0x7ac/0xa10 [ 16.808528][ T348] ? __pfx_____sys_sendmsg+0x10/0x10 [ 16.808687][ T348] ? __pfx_copy_msghdr_from_user+0x10/0x10 [ 16.808891][ T348] ___sys_sendmsg+0xee/0x170 [ 16.809055][ T348] ? __pfx____sys_sendmsg+0x10/0x10 [ 16.809216][ T348] ? kasan_save_stack+0x34/0x50 [ 16.809377][ T348] ? kasan_save_stack+0x24/0x50 [ 16.809535][ T348] ? __kasan_record_aux_stack+0x8e/0xa0 [ 16.809695][ T348] ? __call_rcu_common.constprop.0+0xa1/0x4b0 [ 16.809893][ T348] ? __x64_sys_close+0x7c/0xd0 [ 16.810054][ T348] ? do_syscall_64+0xc1/0x1d0 [ 16.810219][ T348] ? entry_SYSCALL_64_after_hwframe+0x77/0x7f [ 16.810424][ T348] ? __lock_acquire+0xb9a/0x1680 [ 16.810585][ T348] ? find_held_lock+0x2c/0x110 [ 16.810742][ T348] ? __lock_release+0x103/0x460 [ 16.810901][ T348] ? __virt_addr_valid+0x22b/0x430 [ 16.811062][ T348] ? __pfx___lock_release+0x10/0x10 [ 16.811221][ T348] ? __pfx_lock_acquire.part.0+0x10/0x10 [ 16.811384][ T348] __sys_sendmsg+0x109/0x1a0 [ 16.811543][ T348] ? __pfx___sys_sendmsg+0x10/0x10 [ 16.811703][ T348] ? __pfx_slab_free_after_rcu_debug+0x10/0x10 [ 16.811904][ T348] do_syscall_64+0xc1/0x1d0 [ 16.812060][ T348] entry_SYSCALL_64_after_hwframe+0x77/0x7f [ 16.812258][ T348] RIP: 0033:0x7fa3b4fbe9a7 [ 16.812425][ T348] Code: 0a 00 f7 d8 64 89 02 48 c7 c0 ff ff ff ff eb b9 0f 1f 00 f3 0f 1e fa 64 8b 04 25 18 00 00 00 85 c0 75 10 b8 2e 00 00 00 0f 05 <48> 3d 00 f0 ff ff 77 51 c3 48 83 ec 28 89 54 24 1c 48 89 74 24 10 [ 16.812987][ T348] RSP: 002b:00007ffc4a6f9158 EFLAGS: 00000246 ORIG_RAX: 000000000000002e [ 16.813227][ T348] RAX: ffffffffffffffda RBX: 00007ffc4a6f9880 RCX: 00007fa3b4fbe9a7 [ 16.813464][ T348] RDX: 0000000000000000 RSI: 00007ffc4a6f91c0 RDI: 0000000000000005 [ 16.813711][ T348] RBP: 0000000000000004 R08: 0000000000000003 R09: 0000000000000000 [ 16.813949][ T348] R10: 0000000000ba8910 R11: 0000000000000246 R12: 0000000000000004 [ 16.814191][ T348] R13: 000000006779559a R14: 0000000000498600 R15: 0000000000000000 [ 16.814435][ T348] [ 16.824624][ T348] lag4: Port device veth3 added [ 16.830332][ T348] ip (348) used greatest stack depth: 23472 bytes left [ 16.878572][ T349] lag4: Port device veth5 added [ 17.211060][ T355] 8021q: adding VLAN 0 to HW filter on device lag4 [ 18.436310][ T378] lag2: Mode changed to "loadbalance" [ 18.516618][ T380] 8021q: adding VLAN 0 to HW filter on device lag2 [ 18.815116][ T386] lag2: Port device veth1 added [ 18.893444][ T387] lag2: Port device veth7 added [ 19.666710][ T402] br1: port 1(lag2.100) entered blocking state [ 19.666993][ T402] br1: port 1(lag2.100) entered disabled state [ 19.667266][ T402] lag2.100: entered allmulticast mode [ 19.667428][ T402] lag2: entered allmulticast mode [ 19.667606][ T402] veth1: entered allmulticast mode [ 19.667771][ T402] veth7: entered allmulticast mode [ 19.669974][ T402] lag2.100: entered promiscuous mode [ 19.670183][ T402] lag2: entered promiscuous mode [ 19.670351][ T402] veth1: entered promiscuous mode [ 19.671038][ T402] veth7: entered promiscuous mode [ 19.979400][ T409] br2: port 1(lag2.200) entered blocking state [ 19.979929][ T409] br2: port 1(lag2.200) entered disabled state [ 19.980155][ T409] lag2.200: entered allmulticast mode [ 19.981962][ T409] lag2.200: entered promiscuous mode [ 20.188244][ T412] br1: port 1(lag2.100) entered blocking state [ 20.188511][ T412] br1: port 1(lag2.100) entered forwarding state [ 20.262683][ T413] br2: port 1(lag2.200) entered blocking state [ 20.263042][ T413] br2: port 1(lag2.200) entered forwarding state [ 20.681898][ T421] lag3: Mode changed to "loadbalance" [ 20.758876][ T423] 8021q: adding VLAN 0 to HW filter on device lag3 [ 21.066066][ T429] lag3: Port device veth2 added [ 21.143443][ T430] lag3: Port device veth4 added [ 32.730776][ T511] veth7: left promiscuous mode [ 32.731334][ T511] veth7: left allmulticast mode [ 32.732861][ T511] lag2: Port device veth7 removed [ 47.956890][ T572] veth7: entered promiscuous mode [ 47.957406][ T572] veth7: entered allmulticast mode [ 47.958813][ T572] lag2: Port device veth7 added [ 48.182474][ T583] veth1: left promiscuous mode [ 48.182958][ T583] veth1: left allmulticast mode [ 48.184399][ T583] lag2: Port device veth1 removed [ 63.383134][ T137] br1: port 1(lag2.100) entered disabled state [ 63.385342][ T137] br2: port 1(lag2.200) entered disabled state [ 63.472879][ T645] veth7: left promiscuous mode [ 63.473438][ T645] veth7: left allmulticast mode [ 63.475430][ T645] lag2: Port device veth7 removed [ 63.766439][ T656] veth1: entered promiscuous mode [ 63.767217][ T656] veth1: entered allmulticast mode [ 63.768638][ T656] lag2: Port device veth1 added [ 63.802309][ T137] br1: port 1(lag2.100) entered blocking state [ 63.802549][ T137] br1: port 1(lag2.100) entered forwarding state [ 63.804005][ T137] br2: port 1(lag2.200) entered blocking state [ 63.804215][ T137] br2: port 1(lag2.200) entered forwarding state [ 64.038156][ T667] veth7: entered promiscuous mode [ 64.038591][ T667] veth7: entered allmulticast mode [ 64.039403][ T667] lag2: Port device veth7 added [ 79.088029][ T728] lag2.200: left allmulticast mode [ 79.088254][ T728] lag2.200: left promiscuous mode [ 79.088806][ T728] br2: port 1(lag2.200) entered disabled state [ 79.175020][ T729] lag2.100: left allmulticast mode [ 79.175242][ T729] lag2: left allmulticast mode [ 79.175419][ T729] veth1: left allmulticast mode [ 79.175598][ T729] veth7: left allmulticast mode [ 79.175781][ T729] lag2.100: left promiscuous mode [ 79.175948][ T729] lag2: left promiscuous mode [ 79.176111][ T729] veth1: left promiscuous mode [ 79.176520][ T729] veth7: left promiscuous mode [ 79.177236][ T729] br1: port 1(lag2.100) entered disabled state [ 81.276771][ T731] br1: port 1(lag2.100) entered blocking state [ 81.277128][ T731] br1: port 1(lag2.100) entered disabled state [ 81.277370][ T731] lag2.100: entered allmulticast mode [ 81.277520][ T731] lag2: entered allmulticast mode [ 81.278560][ T731] veth1: entered allmulticast mode [ 81.278924][ T731] veth7: entered allmulticast mode [ 81.280626][ T731] lag2.100: entered promiscuous mode [ 81.280795][ T731] lag2: entered promiscuous mode [ 81.280946][ T731] veth1: entered promiscuous mode [ 81.281304][ T731] veth7: entered promiscuous mode [ 81.283077][ T731] br1: port 1(lag2.100) entered blocking state [ 81.283287][ T731] br1: port 1(lag2.100) entered forwarding state [ 81.341953][ T732] br2: port 1(lag2.200) entered blocking state [ 81.342190][ T732] br2: port 1(lag2.200) entered disabled state [ 81.342413][ T732] lag2.200: entered allmulticast mode [ 81.345086][ T732] lag2.200: entered promiscuous mode [ 81.346025][ T732] br2: port 1(lag2.200) entered blocking state [ 81.346217][ T732] br2: port 1(lag2.200) entered forwarding state [ 96.493016][ T792] lag3: Port device veth2 removed [ 111.695927][ T853] lag3: Port device veth2 added [ 111.986647][ T864] lag3: Port device veth4 removed [ 127.248476][ T925] lag3: Port device veth2 removed [ 127.488454][ T936] lag3: Port device veth4 added [ 127.729132][ T947] lag3: Port device veth2 added [ 143.533731][ T1017] lag3: Port device veth4 removed [ 143.592894][ T1018] lag3: Port device veth2 removed [ 144.193369][ T1030] br2: port 1(lag2.200) entered disabled state [ 144.263920][ T1031] br1: port 1(lag2.100) entered disabled state [ 144.451924][ T1034] lag2.200: left allmulticast mode [ 144.452220][ T1034] lag2.200: left promiscuous mode [ 144.452906][ T1034] br2: port 1(lag2.200) entered disabled state [ 144.632622][ T1036] lag2.100: left allmulticast mode [ 144.632816][ T1036] lag2: left allmulticast mode [ 144.632968][ T1036] veth1: left allmulticast mode [ 144.633117][ T1036] veth7: left allmulticast mode [ 144.633273][ T1036] lag2.100: left promiscuous mode [ 144.633413][ T1036] lag2: left promiscuous mode [ 144.633557][ T1036] veth1: left promiscuous mode [ 144.633930][ T1036] veth7: left promiscuous mode [ 144.634600][ T1036] br1: port 1(lag2.100) entered disabled state [ 145.014012][ T1040] lag2: Port device veth7 removed [ 145.082493][ T1041] lag2: Port device veth1 removed [ 146.368504][ T1057] lag4: Port device veth5 removed [ 146.433406][ T1058] lag4: Port device veth3 removed [ 147.801958][ T1073] lag1: Port device veth6 removed [ 147.888289][ T1074] lag1: Port device veth0 removed