[ 11.888732][ T262] ip (262) used greatest stack depth: 22904 bytes left [ 14.111511][ T299] lag1: Mode changed to "loadbalance" [ 14.650287][ T308] lag1: Port device veth0 added [ 14.746978][ T309] lag1: Port device veth6 added [ 15.775028][ T321] 8021q: 802.1Q VLAN Support v1.8 [ 15.775465][ T321] 8021q: adding VLAN 0 to HW filter on device lag1 [ 17.212236][ T340] lag4: Mode changed to "loadbalance" [ 17.306535][ T342] 8021q: adding VLAN 0 to HW filter on device lag4 [ 17.768625][ T348] [ 17.768792][ T348] ====================================================== [ 17.769078][ T348] WARNING: possible circular locking dependency detected [ 17.769374][ T348] 6.13.0-rc5-virtme #1 Not tainted [ 17.769610][ T348] ------------------------------------------------------ [ 17.769892][ T348] ip/348 is trying to acquire lock: [ 17.770125][ T348] ffffffff863d6630 ((netdev_chain).rwsem){++++}-{4:4}, at: blocking_notifier_call_chain+0x50/0x90 [ 17.770635][ T348] [ 17.770635][ T348] but task is already holding lock: [ 17.771013][ T348] ffff88800dedad80 (team->team_lock_key#2){+.+.}-{4:4}, at: team_add_slave+0x33/0x80 [ 17.771289][ T348] [ 17.771289][ T348] which lock already depends on the new lock. [ 17.771289][ T348] [ 17.771583][ T348] [ 17.771583][ T348] the existing dependency chain (in reverse order) is: [ 17.771833][ T348] [ 17.771833][ T348] -> #1 (team->team_lock_key#2){+.+.}-{4:4}: [ 17.772061][ T348] __lock_acquire+0xb9a/0x1680 [ 17.772214][ T348] lock_acquire.part.0+0xeb/0x330 [ 17.772376][ T348] __mutex_lock+0x169/0xbc0 [ 17.772528][ T348] team_vlan_rx_add_vid+0x3b/0x1b0 [ 17.772681][ T348] vlan_vid_add+0x2bc/0x860 [ 17.772837][ T348] vlan_device_event+0x2f4/0x2130 [8021q] [ 17.773039][ T348] notifier_call_chain+0xcd/0x150 [ 17.773196][ T348] blocking_notifier_call_chain+0x66/0x90 [ 17.773382][ T348] __dev_notify_flags+0xe6/0x250 [ 17.773537][ T348] dev_change_flags+0xec/0x160 [ 17.773695][ T348] do_setlink.constprop.0+0x79d/0x2300 [ 17.773889][ T348] rtnl_newlink+0x69c/0xa70 [ 17.774038][ T348] rtnetlink_rcv_msg+0x712/0xc10 [ 17.774188][ T348] netlink_rcv_skb+0x130/0x360 [ 17.774349][ T348] netlink_unicast+0x44b/0x710 [ 17.774501][ T348] netlink_sendmsg+0x723/0xbe0 [ 17.774662][ T348] ____sys_sendmsg+0x7ac/0xa10 [ 17.774815][ T348] ___sys_sendmsg+0xee/0x170 [ 17.774966][ T348] __sys_sendmsg+0x109/0x1a0 [ 17.775111][ T348] do_syscall_64+0xc1/0x1d0 [ 17.775263][ T348] entry_SYSCALL_64_after_hwframe+0x77/0x7f [ 17.775466][ T348] [ 17.775466][ T348] -> #0 ((netdev_chain).rwsem){++++}-{4:4}: [ 17.775697][ T348] check_prev_add+0x182/0xce0 [ 17.775852][ T348] validate_chain+0x7bd/0xae0 [ 17.776008][ T348] __lock_acquire+0xb9a/0x1680 [ 17.776163][ T348] lock_acquire.part.0+0xeb/0x330 [ 17.776317][ T348] down_read+0x9f/0x340 [ 17.776476][ T348] blocking_notifier_call_chain+0x50/0x90 [ 17.776682][ T348] __dev_open+0x15c/0x430 [ 17.776850][ T348] dev_open+0xc9/0x130 [ 17.777017][ T348] team_port_add+0x90a/0x1ce0 [ 17.777179][ T348] team_add_slave+0x41/0x80 [ 17.777345][ T348] do_setlink.constprop.0+0x80e/0x2300 [ 17.777548][ T348] rtnl_newlink+0x69c/0xa70 [ 17.777716][ T348] rtnetlink_rcv_msg+0x712/0xc10 [ 17.777882][ T348] netlink_rcv_skb+0x130/0x360 [ 17.778043][ T348] netlink_unicast+0x44b/0x710 [ 17.778208][ T348] netlink_sendmsg+0x723/0xbe0 [ 17.778371][ T348] ____sys_sendmsg+0x7ac/0xa10 [ 17.778533][ T348] ___sys_sendmsg+0xee/0x170 [ 17.778695][ T348] __sys_sendmsg+0x109/0x1a0 [ 17.778864][ T348] do_syscall_64+0xc1/0x1d0 [ 17.779008][ T348] entry_SYSCALL_64_after_hwframe+0x77/0x7f [ 17.779188][ T348] [ 17.779188][ T348] other info that might help us debug this: [ 17.779188][ T348] [ 17.779473][ T348] Possible unsafe locking scenario: [ 17.779473][ T348] [ 17.779691][ T348] CPU0 CPU1 [ 17.779836][ T348] ---- ---- [ 17.779986][ T348] lock(team->team_lock_key#2); [ 17.780136][ T348] lock((netdev_chain).rwsem); [ 17.780352][ T348] lock(team->team_lock_key#2); [ 17.780573][ T348] rlock((netdev_chain).rwsem); [ 17.780720][ T348] [ 17.780720][ T348] *** DEADLOCK *** [ 17.780720][ T348] [ 17.780947][ T348] 2 locks held by ip/348: [ 17.781058][ T348] #0: ffffffff863e7a48 (rtnl_mutex){+.+.}-{4:4}, at: rtnl_newlink+0x653/0xa70 [ 17.781327][ T348] #1: ffff88800dedad80 (team->team_lock_key#2){+.+.}-{4:4}, at: team_add_slave+0x33/0x80 [ 17.781590][ T348] [ 17.781590][ T348] stack backtrace: [ 17.781771][ T348] CPU: 2 UID: 0 PID: 348 Comm: ip Not tainted 6.13.0-rc5-virtme #1 [ 17.782001][ T348] Hardware name: Bochs Bochs, BIOS Bochs 01/01/2011 [ 17.782185][ T348] Call Trace: [ 17.782299][ T348] [ 17.782377][ T348] dump_stack_lvl+0x82/0xd0 [ 17.782532][ T348] print_circular_bug+0x2e2/0x4b0 [ 17.782691][ T348] check_noncircular+0x2f9/0x3e0 [ 17.782840][ T348] ? __pfx_check_noncircular+0x10/0x10 [ 17.782993][ T348] ? is_bpf_text_address+0x67/0x120 [ 17.783149][ T348] ? alloc_chain_hlocks+0x4e0/0x720 [ 17.783296][ T348] check_prev_add+0x182/0xce0 [ 17.783443][ T348] validate_chain+0x7bd/0xae0 [ 17.783595][ T348] ? __pfx_validate_chain+0x10/0x10 [ 17.783743][ T348] ? hlock_class+0x4e/0x130 [ 17.783891][ T348] ? mark_lock+0x38/0x3e0 [ 17.784002][ T348] __lock_acquire+0xb9a/0x1680 [ 17.784158][ T348] lock_acquire.part.0+0xeb/0x330 [ 17.784305][ T348] ? blocking_notifier_call_chain+0x50/0x90 [ 17.784492][ T348] ? __pfx_lock_acquire.part.0+0x10/0x10 [ 17.784679][ T348] ? trace_lock_acquire+0x14c/0x1f0 [ 17.784826][ T348] ? lock_acquire+0x32/0xc0 [ 17.785016][ T348] ? blocking_notifier_call_chain+0x50/0x90 [ 17.785203][ T348] down_read+0x9f/0x340 [ 17.785323][ T348] ? blocking_notifier_call_chain+0x50/0x90 [ 17.785503][ T348] ? __pfx_down_read+0x10/0x10 [ 17.785653][ T348] blocking_notifier_call_chain+0x50/0x90 [ 17.785799][ T348] __dev_open+0x15c/0x430 [ 17.785910][ T348] ? trace_percpu_alloc_percpu+0x149/0x1d0 [ 17.786094][ T348] ? __pfx___dev_open+0x10/0x10 [ 17.786239][ T348] ? pcpu_alloc_noprof+0x4fe/0xab0 [ 17.786395][ T348] dev_open+0xc9/0x130 [ 17.786508][ T348] ? __pfx_dev_open+0x10/0x10 [ 17.786659][ T348] team_port_add+0x90a/0x1ce0 [ 17.786807][ T348] ? __pfx_team_port_add+0x10/0x10 [ 17.786955][ T348] ? __pfx_validate_chain+0x10/0x10 [ 17.787100][ T348] ? __pfx_validate_linkmsg+0x10/0x10 [ 17.787250][ T348] team_add_slave+0x41/0x80 [ 17.787394][ T348] do_setlink.constprop.0+0x80e/0x2300 [ 17.787545][ T348] ? __pfx_do_setlink.constprop.0+0x10/0x10 [ 17.787729][ T348] ? lock_acquire.part.0+0xeb/0x330 [ 17.787900][ T348] ? rtnl_newlink+0x653/0xa70 [ 17.788047][ T348] ? rtnl_newlink+0xb9/0xa70 [ 17.788192][ T348] ? rtnetlink_rcv_msg+0x712/0xc10 [ 17.788347][ T348] ? __mutex_trylock_common+0xfa/0x260 [ 17.788500][ T348] ? __pfx___mutex_trylock_common+0x10/0x10 [ 17.788682][ T348] ? lock_acquire+0x32/0xc0 [ 17.788830][ T348] ? trace_contention_end+0xef/0x150 [ 17.788980][ T348] ? __mutex_lock+0x190/0xbc0 [ 17.789132][ T348] ? __pfx_do_raw_spin_lock+0x10/0x10 [ 17.789310][ T348] ? rtnl_newlink+0x653/0xa70 [ 17.789456][ T348] ? __pfx___mutex_lock+0x10/0x10 [ 17.789616][ T348] ? __rtnl_newlink+0x3f2/0xa80 [ 17.789780][ T348] rtnl_newlink+0x69c/0xa70 [ 17.789935][ T348] ? __pfx_rtnl_newlink+0x10/0x10 [ 17.790090][ T348] ? find_held_lock+0x2c/0x110 [ 17.790257][ T348] ? __pfx___lock_release+0x10/0x10 [ 17.790405][ T348] ? __pfx_lock_acquire.part.0+0x10/0x10 [ 17.790558][ T348] ? rtnetlink_rcv_msg+0x6ef/0xc10 [ 17.790707][ T348] ? __pfx_rtnl_newlink+0x10/0x10 [ 17.790869][ T348] rtnetlink_rcv_msg+0x712/0xc10 [ 17.791013][ T348] ? __pfx_rtnetlink_rcv_msg+0x10/0x10 [ 17.791157][ T348] ? hlock_class+0x4e/0x130 [ 17.791317][ T348] ? mark_lock+0x38/0x3e0 [ 17.791440][ T348] ? __lock_acquire+0xb9a/0x1680 [ 17.791595][ T348] netlink_rcv_skb+0x130/0x360 [ 17.791744][ T348] ? __pfx_rtnetlink_rcv_msg+0x10/0x10 [ 17.791903][ T348] ? __pfx_netlink_rcv_skb+0x10/0x10 [ 17.792078][ T348] ? netlink_deliver_tap+0x13e/0x340 [ 17.792241][ T348] ? netlink_deliver_tap+0xc3/0x340 [ 17.792405][ T348] netlink_unicast+0x44b/0x710 [ 17.792568][ T348] ? __pfx_netlink_unicast+0x10/0x10 [ 17.792728][ T348] ? find_held_lock+0x2c/0x110 [ 17.792903][ T348] netlink_sendmsg+0x723/0xbe0 [ 17.793056][ T348] ? __pfx_netlink_sendmsg+0x10/0x10 [ 17.793211][ T348] ____sys_sendmsg+0x7ac/0xa10 [ 17.793358][ T348] ? __pfx_____sys_sendmsg+0x10/0x10 [ 17.793512][ T348] ? __pfx_copy_msghdr_from_user+0x10/0x10 [ 17.793708][ T348] ___sys_sendmsg+0xee/0x170 [ 17.793859][ T348] ? __pfx____sys_sendmsg+0x10/0x10 [ 17.794020][ T348] ? kasan_save_stack+0x34/0x50 [ 17.794183][ T348] ? kasan_save_stack+0x24/0x50 [ 17.794352][ T348] ? __kasan_record_aux_stack+0x8e/0xa0 [ 17.794516][ T348] ? __call_rcu_common.constprop.0+0xa1/0x4b0 [ 17.794721][ T348] ? __x64_sys_close+0x7c/0xd0 [ 17.794884][ T348] ? do_syscall_64+0xc1/0x1d0 [ 17.795040][ T348] ? entry_SYSCALL_64_after_hwframe+0x77/0x7f [ 17.795223][ T348] ? __lock_acquire+0xb9a/0x1680 [ 17.795381][ T348] ? find_held_lock+0x2c/0x110 [ 17.795545][ T348] ? __lock_release+0x103/0x460 [ 17.795700][ T348] ? __virt_addr_valid+0x22b/0x430 [ 17.795862][ T348] ? __pfx___lock_release+0x10/0x10 [ 17.796023][ T348] ? __pfx_lock_acquire.part.0+0x10/0x10 [ 17.796186][ T348] __sys_sendmsg+0x109/0x1a0 [ 17.796349][ T348] ? __pfx___sys_sendmsg+0x10/0x10 [ 17.796503][ T348] ? __pfx_slab_free_after_rcu_debug+0x10/0x10 [ 17.796716][ T348] do_syscall_64+0xc1/0x1d0 [ 17.796868][ T348] entry_SYSCALL_64_after_hwframe+0x77/0x7f [ 17.797047][ T348] RIP: 0033:0x7fd4b45899a7 [ 17.797205][ T348] Code: 0a 00 f7 d8 64 89 02 48 c7 c0 ff ff ff ff eb b9 0f 1f 00 f3 0f 1e fa 64 8b 04 25 18 00 00 00 85 c0 75 10 b8 2e 00 00 00 0f 05 <48> 3d 00 f0 ff ff 77 51 c3 48 83 ec 28 89 54 24 1c 48 89 74 24 10 [ 17.797746][ T348] RSP: 002b:00007ffc76d5dc78 EFLAGS: 00000246 ORIG_RAX: 000000000000002e [ 17.797989][ T348] RAX: ffffffffffffffda RBX: 00007ffc76d5e3a0 RCX: 00007fd4b45899a7 [ 17.798230][ T348] RDX: 0000000000000000 RSI: 00007ffc76d5dce0 RDI: 0000000000000005 [ 17.798457][ T348] RBP: 0000000000000004 R08: 0000000000000003 R09: 0000000000000000 [ 17.798678][ T348] R10: 000000001a7d4910 R11: 0000000000000246 R12: 0000000000000004 [ 17.798898][ T348] R13: 000000006778fea1 R14: 0000000000498600 R15: 0000000000000000 [ 17.799119][ T348] [ 17.807691][ T348] lag4: Port device veth3 added [ 17.879007][ T349] lag4: Port device veth5 added [ 18.219918][ T355] 8021q: adding VLAN 0 to HW filter on device lag4 [ 19.477465][ T378] lag2: Mode changed to "loadbalance" [ 19.548831][ T380] 8021q: adding VLAN 0 to HW filter on device lag2 [ 19.861195][ T386] lag2: Port device veth1 added [ 19.936346][ T387] lag2: Port device veth7 added [ 20.712898][ T402] br1: port 1(lag2.100) entered blocking state [ 20.713127][ T402] br1: port 1(lag2.100) entered disabled state [ 20.713351][ T402] lag2.100: entered allmulticast mode [ 20.713540][ T402] lag2: entered allmulticast mode [ 20.713686][ T402] veth1: entered allmulticast mode [ 20.713834][ T402] veth7: entered allmulticast mode [ 20.715724][ T402] lag2.100: entered promiscuous mode [ 20.715911][ T402] lag2: entered promiscuous mode [ 20.716060][ T402] veth1: entered promiscuous mode [ 20.716404][ T402] veth7: entered promiscuous mode [ 21.000578][ T409] br2: port 1(lag2.200) entered blocking state [ 21.000883][ T409] br2: port 1(lag2.200) entered disabled state [ 21.001125][ T409] lag2.200: entered allmulticast mode [ 21.002791][ T409] lag2.200: entered promiscuous mode [ 21.191598][ T412] br1: port 1(lag2.100) entered blocking state [ 21.191891][ T412] br1: port 1(lag2.100) entered forwarding state [ 21.259608][ T413] br2: port 1(lag2.200) entered blocking state [ 21.259859][ T413] br2: port 1(lag2.200) entered forwarding state [ 21.637282][ T421] lag3: Mode changed to "loadbalance" [ 21.717536][ T422] 8021q: adding VLAN 0 to HW filter on device lag3 [ 22.019200][ T429] lag3: Port device veth2 added [ 22.102134][ T430] lag3: Port device veth4 added [ 33.760723][ T512] veth7: left promiscuous mode [ 33.761169][ T512] veth7: left allmulticast mode [ 33.762425][ T512] lag2: Port device veth7 removed [ 49.079325][ T573] veth7: entered promiscuous mode [ 49.079730][ T573] veth7: entered allmulticast mode [ 49.080522][ T573] lag2: Port device veth7 added [ 49.338521][ T584] veth1: left promiscuous mode [ 49.338986][ T584] veth1: left allmulticast mode [ 49.341574][ T584] lag2: Port device veth1 removed [ 64.657523][ T152] br1: port 1(lag2.100) entered disabled state [ 64.658958][ T152] br2: port 1(lag2.200) entered disabled state [ 64.716709][ T645] veth7: left promiscuous mode [ 64.717179][ T645] veth7: left allmulticast mode [ 64.718406][ T645] lag2: Port device veth7 removed [ 64.958890][ T656] veth1: entered promiscuous mode [ 64.959231][ T656] veth1: entered allmulticast mode [ 64.960808][ T656] lag2: Port device veth1 added [ 64.974932][ T421] br2: port 1(lag2.200) entered blocking state [ 64.975204][ T421] br2: port 1(lag2.200) entered forwarding state [ 64.975930][ T37] br1: port 1(lag2.100) entered blocking state [ 64.976141][ T37] br1: port 1(lag2.100) entered forwarding state [ 65.205352][ T667] veth7: entered promiscuous mode [ 65.205713][ T667] veth7: entered allmulticast mode [ 65.206460][ T667] lag2: Port device veth7 added [ 80.260826][ T727] lag2.200: left allmulticast mode [ 80.261038][ T727] lag2.200: left promiscuous mode [ 80.261567][ T727] br2: port 1(lag2.200) entered disabled state [ 80.327608][ T728] lag2.100: left allmulticast mode [ 80.327805][ T728] lag2: left allmulticast mode [ 80.327963][ T728] veth1: left allmulticast mode [ 80.328123][ T728] veth7: left allmulticast mode [ 80.328276][ T728] lag2.100: left promiscuous mode [ 80.328428][ T728] lag2: left promiscuous mode [ 80.328573][ T728] veth1: left promiscuous mode [ 80.328947][ T728] veth7: left promiscuous mode [ 80.329630][ T728] br1: port 1(lag2.100) entered disabled state [ 82.431740][ T730] br1: port 1(lag2.100) entered blocking state [ 82.432029][ T730] br1: port 1(lag2.100) entered disabled state [ 82.432303][ T730] lag2.100: entered allmulticast mode [ 82.433200][ T730] lag2: entered allmulticast mode [ 82.433526][ T730] veth1: entered allmulticast mode [ 82.433717][ T730] veth7: entered allmulticast mode [ 82.435450][ T730] lag2.100: entered promiscuous mode [ 82.435645][ T730] lag2: entered promiscuous mode [ 82.435807][ T730] veth1: entered promiscuous mode [ 82.436162][ T730] veth7: entered promiscuous mode [ 82.437320][ T730] br1: port 1(lag2.100) entered blocking state [ 82.437623][ T730] br1: port 1(lag2.100) entered forwarding state [ 82.497736][ T731] br2: port 1(lag2.200) entered blocking state [ 82.498065][ T731] br2: port 1(lag2.200) entered disabled state [ 82.498383][ T731] lag2.200: entered allmulticast mode [ 82.500833][ T731] lag2.200: entered promiscuous mode [ 82.502195][ T731] br2: port 1(lag2.200) entered blocking state [ 82.502489][ T731] br2: port 1(lag2.200) entered forwarding state [ 97.695388][ T791] lag3: Port device veth2 removed [ 112.830848][ T852] lag3: Port device veth2 added [ 113.111592][ T863] lag3: Port device veth4 removed [ 128.327985][ T924] lag3: Port device veth2 removed [ 128.579558][ T935] lag3: Port device veth4 added [ 128.825176][ T946] lag3: Port device veth2 added [ 144.619697][ T1016] lag3: Port device veth4 removed [ 144.676563][ T1017] lag3: Port device veth2 removed [ 145.329190][ T1029] br2: port 1(lag2.200) entered disabled state [ 145.415161][ T1030] br1: port 1(lag2.100) entered disabled state [ 145.617006][ T1033] lag2.200: left allmulticast mode [ 145.617215][ T1033] lag2.200: left promiscuous mode [ 145.617694][ T1033] br2: port 1(lag2.200) entered disabled state [ 145.815689][ T1035] lag2.100: left allmulticast mode [ 145.815946][ T1035] lag2: left allmulticast mode [ 145.816166][ T1035] veth1: left allmulticast mode [ 145.816394][ T1035] veth7: left allmulticast mode [ 145.816625][ T1035] lag2.100: left promiscuous mode [ 145.816849][ T1035] lag2: left promiscuous mode [ 145.817067][ T1035] veth1: left promiscuous mode [ 145.817580][ T1035] veth7: left promiscuous mode [ 145.820189][ T1035] br1: port 1(lag2.100) entered disabled state [ 146.181486][ T1039] lag2: Port device veth7 removed [ 146.255347][ T1040] lag2: Port device veth1 removed [ 147.637460][ T1057] lag4: Port device veth5 removed [ 147.695587][ T1058] lag4: Port device veth3 removed [ 149.043927][ T1073] lag1: Port device veth6 removed [ 149.102401][ T1074] lag1: Port device veth0 removed