[ 844.792533][T10998] lag1: Mode changed to "loadbalance" [ 844.890669][T11000] 8021q: adding VLAN 0 to HW filter on device lag1 [ 845.353504][T11008] [ 845.353676][T11008] ====================================================== [ 845.353902][T11008] WARNING: possible circular locking dependency detected [ 845.354151][T11008] 6.14.0-rc2-virtme #1 Not tainted [ 845.354308][T11008] ------------------------------------------------------ [ 845.354571][T11008] ip/11008 is trying to acquire lock: [ 845.354816][T11008] ffff8880089b1cb0 (&dev->lock){+.+.}-{4:4}, at: __dev_open+0x57e/0x710 [ 845.355075][T11008] [ 845.355075][T11008] but task is already holding lock: [ 845.355341][T11008] ffff88800b86ad80 (team->team_lock_key){+.+.}-{4:4}, at: team_add_slave+0x33/0x80 [ 845.355679][T11008] [ 845.355679][T11008] which lock already depends on the new lock. [ 845.355679][T11008] [ 845.355982][T11008] [ 845.355982][T11008] the existing dependency chain (in reverse order) is: [ 845.356247][T11008] [ 845.356247][T11008] -> #1 (team->team_lock_key){+.+.}-{4:4}: [ 845.356535][T11008] __lock_acquire+0xb9a/0x1680 [ 845.356737][T11008] lock_acquire.part.0+0xeb/0x330 [ 845.356886][T11008] __mutex_lock+0x169/0xc40 [ 845.357040][T11008] team_set_mac_address+0x120/0x250 [ 845.357248][T11008] netif_set_mac_address+0x28a/0x410 [ 845.357450][T11008] do_setlink.constprop.0+0x1980/0x25f0 [ 845.357701][T11008] rtnl_setlink+0x2b5/0x5a0 [ 845.357854][T11008] rtnetlink_rcv_msg+0x712/0xc10 [ 845.358058][T11008] netlink_rcv_skb+0x130/0x360 [ 845.358228][T11008] netlink_unicast+0x44b/0x710 [ 845.358418][T11008] netlink_sendmsg+0x723/0xbe0 [ 845.358661][T11008] __sys_sendto+0x3c3/0x450 [ 845.358856][T11008] __x64_sys_sendto+0xe0/0x1c0 [ 845.359054][T11008] do_syscall_64+0xc1/0x1d0 [ 845.359206][T11008] entry_SYSCALL_64_after_hwframe+0x77/0x7f [ 845.359406][T11008] [ 845.359406][T11008] -> #0 (&dev->lock){+.+.}-{4:4}: [ 845.359650][T11008] check_prev_add+0x182/0xce0 [ 845.359803][T11008] validate_chain+0x7bd/0xae0 [ 845.359953][T11008] __lock_acquire+0xb9a/0x1680 [ 845.360109][T11008] lock_acquire.part.0+0xeb/0x330 [ 845.360263][T11008] __mutex_lock+0x169/0xc40 [ 845.360409][T11008] __dev_open+0x57e/0x710 [ 845.360573][T11008] netif_open+0xc9/0x130 [ 845.360734][T11008] dev_open+0x9f/0x1a0 [ 845.360891][T11008] team_port_add+0x90a/0x1ce0 [ 845.361050][T11008] team_add_slave+0x41/0x80 [ 845.361203][T11008] do_set_master+0x302/0x610 [ 845.361368][T11008] do_setlink.constprop.0+0x87b/0x25f0 [ 845.361581][T11008] rtnl_newlink+0x69c/0xa70 [ 845.361734][T11008] rtnetlink_rcv_msg+0x712/0xc10 [ 845.361898][T11008] netlink_rcv_skb+0x130/0x360 [ 845.362050][T11008] netlink_unicast+0x44b/0x710 [ 845.362198][T11008] netlink_sendmsg+0x723/0xbe0 [ 845.362411][T11008] ____sys_sendmsg+0x7ac/0xa10 [ 845.362579][T11008] ___sys_sendmsg+0xee/0x170 [ 845.362760][T11008] __sys_sendmsg+0x109/0x1a0 [ 845.362930][T11008] do_syscall_64+0xc1/0x1d0 [ 845.363083][T11008] entry_SYSCALL_64_after_hwframe+0x77/0x7f [ 845.363271][T11008] [ 845.363271][T11008] other info that might help us debug this: [ 845.363271][T11008] [ 845.363582][T11008] Possible unsafe locking scenario: [ 845.363582][T11008] [ 845.363809][T11008] CPU0 CPU1 [ 845.363955][T11008] ---- ---- [ 845.364112][T11008] lock(team->team_lock_key); [ 845.364260][T11008] lock(&dev->lock); [ 845.364466][T11008] lock(team->team_lock_key); [ 845.364692][T11008] lock(&dev->lock); [ 845.364805][T11008] [ 845.364805][T11008] *** DEADLOCK *** [ 845.364805][T11008] [ 845.365023][T11008] 2 locks held by ip/11008: [ 845.365172][T11008] #0: ffffffffa79eeb48 (rtnl_mutex){+.+.}-{4:4}, at: rtnl_newlink+0x653/0xa70 [ 845.365451][T11008] #1: ffff88800b86ad80 (team->team_lock_key){+.+.}-{4:4}, at: team_add_slave+0x33/0x80 [ 845.365744][T11008] [ 845.365744][T11008] stack backtrace: [ 845.365927][T11008] CPU: 0 UID: 0 PID: 11008 Comm: ip Not tainted 6.14.0-rc2-virtme #1 [ 845.365932][T11008] Hardware name: Bochs Bochs, BIOS Bochs 01/01/2011 [ 845.365935][T11008] Call Trace: [ 845.365937][T11008] [ 845.365939][T11008] dump_stack_lvl+0x82/0xd0 [ 845.365949][T11008] print_circular_bug+0x1b7/0x240 [ 845.365954][T11008] check_noncircular+0x2f9/0x3e0 [ 845.365958][T11008] ? __pfx_check_noncircular+0x10/0x10 [ 845.365962][T11008] ? find_held_lock+0x2c/0x110 [ 845.365968][T11008] ? alloc_chain_hlocks+0x4e0/0x720 [ 845.365972][T11008] check_prev_add+0x182/0xce0 [ 845.365977][T11008] validate_chain+0x7bd/0xae0 [ 845.365982][T11008] ? stack_depot_save_flags+0x27e/0x440 [ 845.365989][T11008] ? __pfx_validate_chain+0x10/0x10 [ 845.365993][T11008] ? hlock_class+0x4e/0x130 [ 845.365995][T11008] ? mark_lock+0x38/0x3e0 [ 845.365999][T11008] __lock_acquire+0xb9a/0x1680 [ 845.366005][T11008] lock_acquire.part.0+0xeb/0x330 [ 845.366008][T11008] ? __dev_open+0x57e/0x710 [ 845.366012][T11008] ? __pfx_lock_acquire.part.0+0x10/0x10 [ 845.366015][T11008] ? trace_lock_acquire+0x14c/0x1f0 [ 845.366019][T11008] ? lock_acquire+0x32/0xc0 [ 845.366021][T11008] ? __dev_open+0x57e/0x710 [ 845.366024][T11008] __mutex_lock+0x169/0xc40 [ 845.366027][T11008] ? __dev_open+0x57e/0x710 [ 845.366034][T11008] ? __dev_open+0x57e/0x710 [ 845.366037][T11008] ? __pfx___mutex_lock+0x10/0x10 [ 845.366041][T11008] ? packet_notifier+0x3b0/0x810 [ 845.366050][T11008] ? __dev_open+0x57e/0x710 [ 845.366052][T11008] __dev_open+0x57e/0x710 [ 845.366054][T11008] ? mark_held_locks+0x9e/0xe0 [ 845.366057][T11008] ? __pfx___dev_open+0x10/0x10 [ 845.366060][T11008] ? trace_percpu_alloc_percpu+0x149/0x1d0 [ 845.366067][T11008] netif_open+0xc9/0x130 [ 845.366070][T11008] ? __pfx_netif_open+0x10/0x10 [ 845.366074][T11008] dev_open+0x9f/0x1a0 [ 845.366079][T11008] team_port_add+0x90a/0x1ce0 [ 845.366084][T11008] ? __pfx_team_port_add+0x10/0x10 [ 845.366088][T11008] ? rcu_read_lock_any_held+0x43/0xb0 [ 845.366094][T11008] ? validate_chain+0x1fe/0xae0 [ 845.366099][T11008] team_add_slave+0x41/0x80 [ 845.366103][T11008] do_set_master+0x302/0x610 [ 845.366108][T11008] do_setlink.constprop.0+0x87b/0x25f0 [ 845.366113][T11008] ? __pfx_do_setlink.constprop.0+0x10/0x10 [ 845.366117][T11008] ? lock_acquire.part.0+0xeb/0x330 [ 845.366120][T11008] ? rtnl_newlink+0x653/0xa70 [ 845.366123][T11008] ? rtnl_newlink+0xb9/0xa70 [ 845.366126][T11008] ? rtnetlink_rcv_msg+0x712/0xc10 [ 845.366129][T11008] ? __mutex_trylock_common+0xfa/0x260 [ 845.366133][T11008] ? __pfx___mutex_trylock_common+0x10/0x10 [ 845.366137][T11008] ? lock_acquire+0x32/0xc0 [ 845.366140][T11008] ? trace_contention_end+0xef/0x150 [ 845.366143][T11008] ? __mutex_lock+0x190/0xc40 [ 845.366146][T11008] ? __pfx_do_raw_spin_lock+0x10/0x10 [ 845.366150][T11008] ? rtnl_newlink+0x653/0xa70 [ 845.366153][T11008] ? __pfx___mutex_lock+0x10/0x10 [ 845.366155][T11008] ? __pfx_validate_nla+0x10/0x10 [ 845.366160][T11008] ? trace_cap_capable+0x11e/0x1a0 [ 845.366167][T11008] ? __rtnl_newlink+0x3f2/0xa80 [ 845.366172][T11008] rtnl_newlink+0x69c/0xa70 [ 845.366176][T11008] ? __pfx_rtnl_newlink+0x10/0x10 [ 845.366179][T11008] ? find_held_lock+0x2c/0x110 [ 845.366184][T11008] ? __pfx___lock_release+0x10/0x10 [ 845.366187][T11008] ? __pfx_lock_acquire.part.0+0x10/0x10 [ 845.366192][T11008] ? rtnetlink_rcv_msg+0x6ef/0xc10 [ 845.366195][T11008] ? __pfx_rtnl_newlink+0x10/0x10 [ 845.366198][T11008] rtnetlink_rcv_msg+0x712/0xc10 [ 845.366202][T11008] ? __pfx_rtnetlink_rcv_msg+0x10/0x10 [ 845.366205][T11008] ? hlock_class+0x4e/0x130 [ 845.366207][T11008] ? mark_lock+0x38/0x3e0 [ 845.366212][T11008] ? __lock_acquire+0xb9a/0x1680 [ 845.366215][T11008] netlink_rcv_skb+0x130/0x360 [ 845.366220][T11008] ? __pfx_rtnetlink_rcv_msg+0x10/0x10 [ 845.366224][T11008] ? __pfx_netlink_rcv_skb+0x10/0x10 [ 845.366231][T11008] ? netlink_deliver_tap+0x13e/0x340 [ 845.366235][T11008] ? netlink_deliver_tap+0xc3/0x340 [ 845.366238][T11008] netlink_unicast+0x44b/0x710 [ 845.366242][T11008] ? __pfx_netlink_unicast+0x10/0x10 [ 845.366246][T11008] ? find_held_lock+0x2c/0x110 [ 845.366251][T11008] netlink_sendmsg+0x723/0xbe0 [ 845.366255][T11008] ? __pfx_netlink_sendmsg+0x10/0x10 [ 845.366261][T11008] ____sys_sendmsg+0x7ac/0xa10 [ 845.366266][T11008] ? __pfx_____sys_sendmsg+0x10/0x10 [ 845.366268][T11008] ? __pfx_copy_msghdr_from_user+0x10/0x10 [ 845.366275][T11008] ___sys_sendmsg+0xee/0x170 [ 845.366279][T11008] ? __pfx____sys_sendmsg+0x10/0x10 [ 845.366284][T11008] ? kasan_save_stack+0x34/0x50 [ 845.366288][T11008] ? kasan_save_stack+0x24/0x50 [ 845.366291][T11008] ? kasan_record_aux_stack+0x8c/0xa0 [ 845.366295][T11008] ? __call_rcu_common.constprop.0+0xa1/0x5b0 [ 845.366298][T11008] ? __x64_sys_close+0x7c/0xd0 [ 845.366302][T11008] ? do_syscall_64+0xc1/0x1d0 [ 845.366305][T11008] ? entry_SYSCALL_64_after_hwframe+0x77/0x7f [ 845.366308][T11008] ? __lock_acquire+0xb9a/0x1680 [ 845.366311][T11008] ? lock_downgrade+0x52/0x110 [ 845.366315][T11008] ? __pfx_do_raw_spin_lock+0x10/0x10 [ 845.366318][T11008] ? find_held_lock+0x2c/0x110 [ 845.366322][T11008] ? __lock_release+0x103/0x460 [ 845.366325][T11008] ? __virt_addr_valid+0x22b/0x430 [ 845.366331][T11008] ? __pfx___lock_release+0x10/0x10 [ 845.366333][T11008] ? __pfx_lock_acquire.part.0+0x10/0x10 [ 845.366338][T11008] __sys_sendmsg+0x109/0x1a0 [ 845.366342][T11008] ? __pfx___sys_sendmsg+0x10/0x10 [ 845.366350][T11008] do_syscall_64+0xc1/0x1d0 [ 845.366353][T11008] entry_SYSCALL_64_after_hwframe+0x77/0x7f [ 845.366356][T11008] RIP: 0033:0x7fc2f2bf99a7 [ 845.366362][T11008] Code: 0a 00 f7 d8 64 89 02 48 c7 c0 ff ff ff ff eb b9 0f 1f 00 f3 0f 1e fa 64 8b 04 25 18 00 00 00 85 c0 75 10 b8 2e 00 00 00 0f 05 <48> 3d 00 f0 ff ff 77 51 c3 48 83 ec 28 89 54 24 1c 48 89 74 24 10 [ 845.366365][T11008] RSP: 002b:00007ffe2b3a7918 EFLAGS: 00000246 ORIG_RAX: 000000000000002e [ 845.366369][T11008] RAX: ffffffffffffffda RBX: 00007ffe2b3a8040 RCX: 00007fc2f2bf99a7 [ 845.366371][T11008] RDX: 0000000000000000 RSI: 00007ffe2b3a7980 RDI: 0000000000000005 [ 845.366373][T11008] RBP: 0000000000000004 R08: 0000000000000003 R09: 0000000000000000 [ 845.366374][T11008] R10: 0000000003c20910 R11: 0000000000000246 R12: 0000000000000004 [ 845.366376][T11008] R13: 0000000067b3007f R14: 0000000000498600 R15: 0000000000000000 [ 845.366381][T11008] [ 845.386614][T11008] lag1: Port device veth0 added [ 845.393694][T11008] ip (11008) used greatest stack depth: 23544 bytes left [ 845.464485][T11009] lag1: Port device veth6 added [ 845.833189][T11015] 8021q: adding VLAN 0 to HW filter on device lag1 [ 847.180943][T11038] lag4: Mode changed to "loadbalance" [ 847.254668][T11040] 8021q: adding VLAN 0 to HW filter on device lag4 [ 847.574769][T11046] lag4: Port device veth3 added [ 847.644510][T11047] lag4: Port device veth5 added [ 848.041178][T11053] 8021q: adding VLAN 0 to HW filter on device lag4 [ 849.433849][T11076] lag2: Mode changed to "loadbalance" [ 849.522026][T11078] 8021q: adding VLAN 0 to HW filter on device lag2 [ 849.883459][T11084] lag2: Port device veth1 added [ 849.969792][T11085] lag2: Port device veth7 added [ 850.644545][T11099] br1: port 1(lag2.100) entered blocking state [ 850.644825][T11099] br1: port 1(lag2.100) entered disabled state [ 850.645091][T11099] lag2.100: entered allmulticast mode [ 850.645250][T11099] lag2: entered allmulticast mode [ 850.645426][T11099] veth1: entered allmulticast mode [ 850.645625][T11099] veth7: entered allmulticast mode [ 850.647586][T11099] lag2.100: entered promiscuous mode [ 850.647784][T11099] lag2: entered promiscuous mode [ 850.647948][T11099] veth1: entered promiscuous mode [ 850.648353][T11099] veth7: entered promiscuous mode [ 850.954887][T11106] br2: port 1(lag2.200) entered blocking state [ 850.955612][T11106] br2: port 1(lag2.200) entered disabled state [ 850.955896][T11106] lag2.200: entered allmulticast mode [ 850.957548][T11106] lag2.200: entered promiscuous mode [ 851.183035][T11109] br1: port 1(lag2.100) entered blocking state [ 851.183326][T11109] br1: port 1(lag2.100) entered forwarding state [ 851.245361][T11110] br2: port 1(lag2.200) entered blocking state [ 851.245636][T11110] br2: port 1(lag2.200) entered forwarding state [ 851.640256][T11118] lag3: Mode changed to "loadbalance" [ 851.703229][T11120] 8021q: adding VLAN 0 to HW filter on device lag3 [ 851.998460][T11126] lag3: Port device veth2 added [ 852.070260][T11127] lag3: Port device veth4 added [ 863.676415][T11208] veth7: left promiscuous mode [ 863.676857][T11208] veth7: left allmulticast mode [ 863.678230][T11208] lag2: Port device veth7 removed [ 879.085911][T11269] veth7: entered promiscuous mode [ 879.086722][T11269] veth7: entered allmulticast mode [ 879.088118][T11269] lag2: Port device veth7 added [ 879.358629][T11280] veth1: left promiscuous mode [ 879.359244][T11280] veth1: left allmulticast mode [ 879.361655][T11280] lag2: Port device veth1 removed [ 894.513858][ T461] br1: port 1(lag2.100) entered disabled state [ 894.515045][ T461] br2: port 1(lag2.200) entered disabled state [ 894.565083][T11341] veth7: left promiscuous mode [ 894.565505][T11341] veth7: left allmulticast mode [ 894.566748][T11341] lag2: Port device veth7 removed [ 894.813885][T11352] veth1: entered promiscuous mode [ 894.814681][T11352] veth1: entered allmulticast mode [ 894.815465][T11352] lag2: Port device veth1 added [ 894.831045][ T151] br1: port 1(lag2.100) entered blocking state [ 894.831298][ T151] br1: port 1(lag2.100) entered forwarding state [ 894.832155][ T151] br2: port 1(lag2.200) entered blocking state [ 894.832383][ T151] br2: port 1(lag2.200) entered forwarding state [ 895.078170][T11363] veth7: entered promiscuous mode [ 895.078513][T11363] veth7: entered allmulticast mode [ 895.079240][T11363] lag2: Port device veth7 added [ 910.309187][T11423] lag2.200: left allmulticast mode [ 910.309416][T11423] lag2.200: left promiscuous mode [ 910.310054][T11423] br2: port 1(lag2.200) entered disabled state [ 910.364640][T11424] lag2.100: left allmulticast mode [ 910.364958][T11424] lag2: left allmulticast mode [ 910.365742][T11424] veth1: left allmulticast mode [ 910.366033][T11424] veth7: left allmulticast mode [ 910.366244][T11424] lag2.100: left promiscuous mode [ 910.366423][T11424] lag2: left promiscuous mode [ 910.366641][T11424] veth1: left promiscuous mode [ 910.367187][T11424] veth7: left promiscuous mode [ 910.367958][T11424] br1: port 1(lag2.100) entered disabled state [ 912.499674][T11426] br1: port 1(lag2.100) entered blocking state [ 912.500002][T11426] br1: port 1(lag2.100) entered disabled state [ 912.500301][T11426] lag2.100: entered allmulticast mode [ 912.500512][T11426] lag2: entered allmulticast mode [ 912.500697][T11426] veth1: entered allmulticast mode [ 912.500927][T11426] veth7: entered allmulticast mode [ 912.503329][T11426] lag2.100: entered promiscuous mode [ 912.503552][T11426] lag2: entered promiscuous mode [ 912.503750][T11426] veth1: entered promiscuous mode [ 912.504178][T11426] veth7: entered promiscuous mode [ 912.505631][T11426] br1: port 1(lag2.100) entered blocking state [ 912.505882][T11426] br1: port 1(lag2.100) entered forwarding state [ 912.598403][T11427] br2: port 1(lag2.200) entered blocking state [ 912.598676][T11427] br2: port 1(lag2.200) entered disabled state [ 912.598910][T11427] lag2.200: entered allmulticast mode [ 912.602151][T11427] lag2.200: entered promiscuous mode [ 912.603537][T11427] br2: port 1(lag2.200) entered blocking state [ 912.603767][T11427] br2: port 1(lag2.200) entered forwarding state [ 927.789137][T11487] lag3: Port device veth2 removed [ 943.176395][T11548] lag3: Port device veth2 added [ 943.425435][T11559] lag3: Port device veth4 removed [ 958.758852][T11620] lag3: Port device veth2 removed [ 958.982142][T11631] lag3: Port device veth4 added [ 959.259510][T11642] lag3: Port device veth2 added [ 975.014389][T11712] lag3: Port device veth4 removed [ 975.073921][T11713] lag3: Port device veth2 removed [ 975.778151][T11725] br2: port 1(lag2.200) entered disabled state [ 975.829525][T11726] br1: port 1(lag2.100) entered disabled state [ 976.031600][T11729] lag2.200: left allmulticast mode [ 976.031903][T11729] lag2.200: left promiscuous mode [ 976.033175][T11729] br2: port 1(lag2.200) entered disabled state [ 976.229426][T11731] lag2.100: left allmulticast mode [ 976.229630][T11731] lag2: left allmulticast mode [ 976.229801][T11731] veth1: left allmulticast mode [ 976.229986][T11731] veth7: left allmulticast mode [ 976.230172][T11731] lag2.100: left promiscuous mode [ 976.230335][T11731] lag2: left promiscuous mode [ 976.230501][T11731] veth1: left promiscuous mode [ 976.230905][T11731] veth7: left promiscuous mode [ 976.231563][T11731] br1: port 1(lag2.100) entered disabled state [ 976.634649][T11735] lag2: Port device veth7 removed [ 976.692716][T11736] lag2: Port device veth1 removed [ 978.009495][T11751] lag4: Port device veth5 removed [ 978.091178][T11752] lag4: Port device veth3 removed [ 979.456248][T11768] lag1: Port device veth6 removed [ 979.511159][T11769] lag1: Port device veth0 removed