[ 13.696134][ T293] 8021q: 802.1Q VLAN Support v1.8 [ 17.314121][ T341] gre: GRE over IPv4 demultiplexor driver [ 17.327016][ T341] ip_gre: GRE over IPv4 tunneling driver [ 18.400010][ T359] lag: Mode changed to "loadbalance" [ 18.597691][ T362] lag: Port device veth2 added [ 18.598513][ T359] [ 18.598619][ T359] ====================================================== [ 18.598829][ T359] WARNING: possible circular locking dependency detected [ 18.599036][ T359] 6.13.0-rc5-virtme #1 Not tainted [ 18.599197][ T359] ------------------------------------------------------ [ 18.599398][ T359] teamd/359 is trying to acquire lock: [ 18.599560][ T359] ffff88800bacad80 (team->team_lock_key){+.+.}-{4:4}, at: team_port_change_check+0x43/0x130 [ 18.599900][ T359] [ 18.599900][ T359] but task is already holding lock: [ 18.600139][ T359] ffffffff98bd6630 ((netdev_chain).rwsem){++++}-{4:4}, at: blocking_notifier_call_chain+0x50/0x90 [ 18.600470][ T359] [ 18.600470][ T359] which lock already depends on the new lock. [ 18.600470][ T359] [ 18.600787][ T359] [ 18.600787][ T359] the existing dependency chain (in reverse order) is: [ 18.601070][ T359] [ 18.601070][ T359] -> #1 ((netdev_chain).rwsem){++++}-{4:4}: [ 18.601312][ T359] __lock_acquire+0xb9a/0x1680 [ 18.601479][ T359] lock_acquire.part.0+0xeb/0x330 [ 18.601641][ T359] down_read+0x9f/0x340 [ 18.601803][ T359] blocking_notifier_call_chain+0x50/0x90 [ 18.602006][ T359] __dev_open+0x15c/0x430 [ 18.602170][ T359] dev_open+0xc9/0x130 [ 18.602334][ T359] team_port_add+0x90a/0x1ce0 [ 18.602499][ T359] team_add_slave+0x41/0x80 [ 18.602662][ T359] do_setlink.constprop.0+0x80e/0x2300 [ 18.602863][ T359] rtnl_newlink+0x69c/0xa70 [ 18.603026][ T359] rtnetlink_rcv_msg+0x712/0xc10 [ 18.603188][ T359] netlink_rcv_skb+0x130/0x360 [ 18.603351][ T359] netlink_unicast+0x44b/0x710 [ 18.603512][ T359] netlink_sendmsg+0x723/0xbe0 [ 18.603677][ T359] ____sys_sendmsg+0x7ac/0xa10 [ 18.603840][ T359] ___sys_sendmsg+0xee/0x170 [ 18.604004][ T359] __sys_sendmsg+0x109/0x1a0 [ 18.604167][ T359] do_syscall_64+0xc1/0x1d0 [ 18.604329][ T359] entry_SYSCALL_64_after_hwframe+0x77/0x7f [ 18.604535][ T359] [ 18.604535][ T359] -> #0 (team->team_lock_key){+.+.}-{4:4}: [ 18.604763][ T359] check_prev_add+0x182/0xce0 [ 18.604916][ T359] validate_chain+0x7bd/0xae0 [ 18.605075][ T359] __lock_acquire+0xb9a/0x1680 [ 18.605232][ T359] lock_acquire.part.0+0xeb/0x330 [ 18.605387][ T359] __mutex_lock+0x169/0xbc0 [ 18.605551][ T359] team_port_change_check+0x43/0x130 [ 18.605758][ T359] team_device_event+0x3fe/0x520 [ 18.605927][ T359] notifier_call_chain+0xcd/0x150 [ 18.606098][ T359] blocking_notifier_call_chain+0x66/0x90 [ 18.606302][ T359] netdev_state_change+0xf5/0x120 [ 18.606470][ T359] linkwatch_do_dev+0xd2/0x100 [ 18.606650][ T359] rtnl_getlink+0x4cf/0x970 [ 18.606819][ T359] rtnetlink_rcv_msg+0x2fb/0xc10 [ 18.606991][ T359] netlink_rcv_skb+0x130/0x360 [ 18.607157][ T359] netlink_unicast+0x44b/0x710 [ 18.607321][ T359] netlink_sendmsg+0x723/0xbe0 [ 18.607490][ T359] ____sys_sendmsg+0x7ac/0xa10 [ 18.607653][ T359] ___sys_sendmsg+0xee/0x170 [ 18.607821][ T359] __sys_sendmsg+0x109/0x1a0 [ 18.607988][ T359] do_syscall_64+0xc1/0x1d0 [ 18.608154][ T359] entry_SYSCALL_64_after_hwframe+0x77/0x7f [ 18.608357][ T359] [ 18.608357][ T359] other info that might help us debug this: [ 18.608357][ T359] [ 18.608684][ T359] Possible unsafe locking scenario: [ 18.608684][ T359] [ 18.608927][ T359] CPU0 CPU1 [ 18.609088][ T359] ---- ---- [ 18.609255][ T359] rlock((netdev_chain).rwsem); [ 18.609423][ T359] lock(team->team_lock_key); [ 18.609666][ T359] lock((netdev_chain).rwsem); [ 18.609909][ T359] lock(team->team_lock_key); [ 18.610081][ T359] [ 18.610081][ T359] *** DEADLOCK *** [ 18.610081][ T359] [ 18.610318][ T359] 2 locks held by teamd/359: [ 18.610485][ T359] #0: ffffffff98be7a48 (rtnl_mutex){+.+.}-{4:4}, at: rtnetlink_rcv_msg+0x2af/0xc10 [ 18.610774][ T359] #1: ffffffff98bd6630 ((netdev_chain).rwsem){++++}-{4:4}, at: blocking_notifier_call_chain+0x50/0x90 [ 18.611101][ T359] [ 18.611101][ T359] stack backtrace: [ 18.611310][ T359] CPU: 0 UID: 0 PID: 359 Comm: teamd Not tainted 6.13.0-rc5-virtme #1 [ 18.611554][ T359] Hardware name: Bochs Bochs, BIOS Bochs 01/01/2011 [ 18.611759][ T359] Call Trace: [ 18.611884][ T359] [ 18.611969][ T359] dump_stack_lvl+0x82/0xd0 [ 18.612135][ T359] print_circular_bug+0x2e2/0x4b0 [ 18.612300][ T359] check_noncircular+0x2f9/0x3e0 [ 18.612466][ T359] ? __pfx_check_noncircular+0x10/0x10 [ 18.612638][ T359] ? alloc_chain_hlocks+0x4e0/0x720 [ 18.612804][ T359] ? __lock_release+0x103/0x460 [ 18.612968][ T359] check_prev_add+0x182/0xce0 [ 18.613136][ T359] validate_chain+0x7bd/0xae0 [ 18.613299][ T359] ? __pfx_validate_chain+0x10/0x10 [ 18.613463][ T359] ? hlock_class+0x4e/0x130 [ 18.613628][ T359] ? mark_lock+0x38/0x3e0 [ 18.613754][ T359] __lock_acquire+0xb9a/0x1680 [ 18.613921][ T359] lock_acquire.part.0+0xeb/0x330 [ 18.614089][ T359] ? team_port_change_check+0x43/0x130 [ 18.614250][ T359] ? __pfx___lock_release+0x10/0x10 [ 18.614412][ T359] ? __pfx_lock_acquire.part.0+0x10/0x10 [ 18.614576][ T359] ? trace_lock_acquire+0x14c/0x1f0 [ 18.614739][ T359] ? lock_acquire+0x32/0xc0 [ 18.614899][ T359] ? team_port_change_check+0x43/0x130 [ 18.615062][ T359] __mutex_lock+0x169/0xbc0 [ 18.615209][ T359] ? team_port_change_check+0x43/0x130 [ 18.615356][ T359] ? team_port_change_check+0x43/0x130 [ 18.615515][ T359] ? __pfx___mutex_lock+0x10/0x10 [ 18.615677][ T359] ? lock_acquire.part.0+0xeb/0x330 [ 18.615832][ T359] ? blocking_notifier_call_chain+0x50/0x90 [ 18.616033][ T359] ? ref_tracker_free+0x356/0x910 [ 18.616204][ T359] ? __pfx_fib_sync_up+0x10/0x10 [ 18.616373][ T359] ? __pfx_rwsem_read_trylock+0x10/0x10 [ 18.616539][ T359] ? team_port_change_check+0x43/0x130 [ 18.616725][ T359] team_port_change_check+0x43/0x130 [ 18.616889][ T359] team_device_event+0x3fe/0x520 [ 18.617051][ T359] ? trace_notifier_run+0xe7/0x150 [ 18.617217][ T359] notifier_call_chain+0xcd/0x150 [ 18.617401][ T359] blocking_notifier_call_chain+0x66/0x90 [ 18.617566][ T359] netdev_state_change+0xf5/0x120 [ 18.617731][ T359] ? __pfx_netdev_state_change+0x10/0x10 [ 18.617894][ T359] ? __pfx_dev_activate+0x10/0x10 [ 18.618059][ T359] ? lockdep_hardirqs_on_prepare+0x275/0x410 [ 18.618262][ T359] linkwatch_do_dev+0xd2/0x100 [ 18.618434][ T359] rtnl_getlink+0x4cf/0x970 [ 18.618602][ T359] ? __pfx_rtnl_getlink+0x10/0x10 [ 18.618784][ T359] ? __mutex_lock+0x190/0xbc0 [ 18.618950][ T359] ? rtnetlink_rcv_msg+0x2af/0xc10 [ 18.619115][ T359] ? __pfx___mutex_lock+0x10/0x10 [ 18.619297][ T359] ? __lock_release+0x103/0x460 [ 18.619460][ T359] ? rtnetlink_rcv_msg+0x297/0xc10 [ 18.619622][ T359] ? __pfx___lock_release+0x10/0x10 [ 18.619785][ T359] ? __pfx_lock_acquire.part.0+0x10/0x10 [ 18.619960][ T359] rtnetlink_rcv_msg+0x2fb/0xc10 [ 18.620126][ T359] ? __pfx_rtnetlink_rcv_msg+0x10/0x10 [ 18.620291][ T359] ? hlock_class+0x4e/0x130 [ 18.620455][ T359] ? mark_lock+0x38/0x3e0 [ 18.620587][ T359] ? __lock_acquire+0xb9a/0x1680 [ 18.620749][ T359] netlink_rcv_skb+0x130/0x360 [ 18.620911][ T359] ? __pfx_rtnetlink_rcv_msg+0x10/0x10 [ 18.621082][ T359] ? __pfx_netlink_rcv_skb+0x10/0x10 [ 18.621248][ T359] ? netlink_deliver_tap+0x13e/0x340 [ 18.621414][ T359] ? netlink_deliver_tap+0xc3/0x340 [ 18.621581][ T359] netlink_unicast+0x44b/0x710 [ 18.621748][ T359] ? __pfx_netlink_unicast+0x10/0x10 [ 18.621911][ T359] ? find_held_lock+0x2c/0x110 [ 18.622085][ T359] netlink_sendmsg+0x723/0xbe0 [ 18.622256][ T359] ? __pfx_netlink_sendmsg+0x10/0x10 [ 18.622423][ T359] ____sys_sendmsg+0x7ac/0xa10 [ 18.622586][ T359] ? __pfx_____sys_sendmsg+0x10/0x10 [ 18.622754][ T359] ? __pfx_copy_msghdr_from_user+0x10/0x10 [ 18.622975][ T359] ___sys_sendmsg+0xee/0x170 [ 18.623137][ T359] ? hlock_class+0x4e/0x130 [ 18.623300][ T359] ? __pfx____sys_sendmsg+0x10/0x10 [ 18.623473][ T359] ? usage_skip+0x132/0x190 [ 18.623636][ T359] ? do_user_addr_fault+0x8fd/0xe30 [ 18.623801][ T359] ? do_user_addr_fault+0x8fd/0xe30 [ 18.623962][ T359] ? find_held_lock+0x2c/0x110 [ 18.624129][ T359] ? __lock_release+0x103/0x460 [ 18.624295][ T359] __sys_sendmsg+0x109/0x1a0 [ 18.624462][ T359] ? __pfx___sys_sendmsg+0x10/0x10 [ 18.624722][ T359] ? do_user_addr_fault+0x97c/0xe30 [ 18.624886][ T359] do_syscall_64+0xc1/0x1d0 [ 18.625049][ T359] entry_SYSCALL_64_after_hwframe+0x77/0x7f [ 18.625254][ T359] RIP: 0033:0x7f7d97f519a7 [ 18.625436][ T359] Code: 0a 00 f7 d8 64 89 02 48 c7 c0 ff ff ff ff eb b9 0f 1f 00 f3 0f 1e fa 64 8b 04 25 18 00 00 00 85 c0 75 10 b8 2e 00 00 00 0f 05 <48> 3d 00 f0 ff ff 77 51 c3 48 83 ec 28 89 54 24 1c 48 89 74 24 10 [ 18.626077][ T359] RSP: 002b:00007ffc9a1e9248 EFLAGS: 00000246 ORIG_RAX: 000000000000002e [ 18.626415][ T359] RAX: ffffffffffffffda RBX: 0000000005842320 RCX: 00007f7d97f519a7 [ 18.626689][ T359] RDX: 0000000000000000 RSI: 00007ffc9a1e9280 RDI: 0000000000000005 [ 18.626936][ T359] RBP: 0000000005842230 R08: 0000000000000008 R09: 0000000000000000 [ 18.627281][ T359] R10: 0000000000000400 R11: 0000000000000246 R12: 000000000586a890 [ 18.627537][ T359] R13: 00007ffc9a1e9280 R14: 00007ffc9a1e93b8 R15: 000000000586a358 [ 18.627788][ T359] [ 18.831229][ T366] lag: Port device veth4 added [ 18.948714][ T368] 8021q: adding VLAN 0 to HW filter on device lag [ 19.008068][ T369] br1: port 1(lag) entered blocking state [ 19.008275][ T369] br1: port 1(lag) entered disabled state [ 19.008456][ T369] lag: entered allmulticast mode [ 19.008601][ T369] veth2: entered allmulticast mode [ 19.008751][ T369] veth4: entered allmulticast mode [ 19.010956][ T369] veth2: entered promiscuous mode [ 19.011140][ T369] veth4: entered promiscuous mode [ 19.064977][ T370] br1: port 1(lag) entered blocking state [ 19.065210][ T370] br1: port 1(lag) entered forwarding state [ 19.543947][ T379] GACT probability NOT on [ 25.731058][ T406] Mirror/redirect action on [ 40.123024][ T39] br1: port 1(lag) entered disabled state [ 45.883352][ T39] br1: port 1(lag) entered blocking state [ 45.883597][ T39] br1: port 1(lag) entered forwarding state [ 60.710018][ T37] br1: port 1(lag) entered disabled state [ 66.515618][ T39] br1: port 1(lag) entered blocking state [ 66.515836][ T39] br1: port 1(lag) entered forwarding state [ 67.076724][ T510] lag: left allmulticast mode [ 67.076957][ T510] veth2: left allmulticast mode [ 67.077135][ T510] veth4: left allmulticast mode [ 67.077326][ T510] veth2: left promiscuous mode [ 67.077505][ T510] veth4: left promiscuous mode [ 67.077681][ T510] br1: port 1(lag) entered disabled state [ 67.161955][ T359] lag: Port device veth4 removed [ 67.165321][ T359] lag: Port device veth2 removed [ 67.411568][ T359] teamd (359) used greatest stack depth: 23728 bytes left