[ 15.408894][ T319] br1: port 1(veth1) entered blocking state [ 15.409449][ T319] br1: port 1(veth1) entered disabled state [ 15.409877][ T319] veth1: entered allmulticast mode [ 15.413042][ T319] veth1: entered promiscuous mode [ 15.500126][ T86] br1: port 1(veth1) entered blocking state [ 15.500618][ T86] br1: port 1(veth1) entered forwarding state [ 15.582913][ T321] br1: port 2(veth2) entered blocking state [ 15.583201][ T321] br1: port 2(veth2) entered disabled state [ 15.583528][ T321] veth2: entered allmulticast mode [ 15.585513][ T321] veth2: entered promiscuous mode [ 15.668181][ T37] br1: port 2(veth2) entered blocking state [ 15.668526][ T37] br1: port 2(veth2) entered forwarding state [ 15.961194][ T325] 8021q: 802.1Q VLAN Support v1.8 [ 15.990674][ T324] ip (324) used greatest stack depth: 23984 bytes left [ 16.758238][ T335] GACT probability NOT on [ 24.111614][ T373] Mirror/redirect action on [ 59.520866][ T553] [ 59.521084][ T553] ============================================ [ 59.521278][ T553] WARNING: possible recursive locking detected [ 59.521468][ T553] 6.13.0-rc5-virtme #1 Not tainted [ 59.521617][ T553] -------------------------------------------- [ 59.521804][ T553] ip/553 is trying to acquire lock: [ 59.521965][ T553] ffffffffbcfd6630 ((netdev_chain).rwsem){++++}-{4:4}, at: blocking_notifier_call_chain+0x50/0x90 [ 59.522270][ T553] [ 59.522270][ T553] but task is already holding lock: [ 59.522481][ T553] ffffffffbcfd6630 ((netdev_chain).rwsem){++++}-{4:4}, at: blocking_notifier_call_chain+0x50/0x90 [ 59.522774][ T553] [ 59.522774][ T553] other info that might help us debug this: [ 59.522997][ T553] Possible unsafe locking scenario: [ 59.522997][ T553] [ 59.523210][ T553] CPU0 [ 59.523319][ T553] ---- [ 59.523426][ T553] lock((netdev_chain).rwsem); [ 59.523575][ T553] lock((netdev_chain).rwsem); [ 59.523720][ T553] [ 59.523720][ T553] *** DEADLOCK *** [ 59.523720][ T553] [ 59.523927][ T553] May be due to missing lock nesting notation [ 59.523927][ T553] [ 59.524141][ T553] 2 locks held by ip/553: [ 59.524264][ T553] #0: ffffffffbcfe7a48 (rtnl_mutex){+.+.}-{4:4}, at: rtnl_dellink+0x228/0xa40 [ 59.524529][ T553] #1: ffffffffbcfd6630 ((netdev_chain).rwsem){++++}-{4:4}, at: blocking_notifier_call_chain+0x50/0x90 [ 59.524825][ T553] [ 59.524825][ T553] stack backtrace: [ 59.525003][ T553] CPU: 2 UID: 0 PID: 553 Comm: ip Not tainted 6.13.0-rc5-virtme #1 [ 59.525229][ T553] Hardware name: Bochs Bochs, BIOS Bochs 01/01/2011 [ 59.525419][ T553] Call Trace: [ 59.525530][ T553] [ 59.525607][ T553] dump_stack_lvl+0x82/0xd0 [ 59.525758][ T553] print_deadlock_bug+0x40a/0x650 [ 59.525914][ T553] validate_chain+0x5bf/0xae0 [ 59.526059][ T553] ? __pfx_validate_chain+0x10/0x10 [ 59.526211][ T553] ? hlock_class+0x4e/0x130 [ 59.526355][ T553] ? mark_lock+0x38/0x3e0 [ 59.526467][ T553] __lock_acquire+0xb9a/0x1680 [ 59.526615][ T553] lock_acquire.part.0+0xeb/0x330 [ 59.526758][ T553] ? blocking_notifier_call_chain+0x50/0x90 [ 59.526938][ T553] ? netconsole_process_cleanups_core+0x2e/0x3d0 [ 59.527121][ T553] ? __pfx_lock_acquire.part.0+0x10/0x10 [ 59.527273][ T553] ? trace_lock_acquire+0x14c/0x1f0 [ 59.527419][ T553] ? lock_acquire+0x32/0xc0 [ 59.527564][ T553] ? blocking_notifier_call_chain+0x50/0x90 [ 59.527749][ T553] down_read+0x9f/0x340 [ 59.527860][ T553] ? blocking_notifier_call_chain+0x50/0x90 [ 59.528039][ T553] ? __pfx_down_read+0x10/0x10 [ 59.528184][ T553] ? __lock_release+0x103/0x460 [ 59.528329][ T553] ? netconsole_netdev_event+0x2c1/0x620 [ 59.528486][ T553] ? __pfx___lock_release+0x10/0x10 [ 59.528632][ T553] blocking_notifier_call_chain+0x50/0x90 [ 59.528779][ T553] __netdev_upper_dev_unlink+0xd9/0x220 [ 59.528928][ T553] ? __pfx___netdev_upper_dev_unlink+0x10/0x10 [ 59.529108][ T553] ? __mutex_unlock_slowpath+0x14a/0x3d0 [ 59.529253][ T553] ? __pfx___mutex_unlock_slowpath+0x10/0x10 [ 59.529441][ T553] ? rtnl_is_locked+0x15/0x20 [ 59.529589][ T553] netdev_upper_dev_unlink+0x71/0xa0 [ 59.529732][ T553] ? __pfx_netdev_upper_dev_unlink+0x10/0x10 [ 59.529920][ T553] ? netconsole_netdev_event+0x2c1/0x620 [ 59.530069][ T553] ? mutex_is_locked+0x1c/0x60 [ 59.530216][ T553] ? rtnl_is_locked+0x15/0x20 [ 59.530365][ T553] ? netdev_master_upper_dev_get+0x13/0x120 [ 59.530550][ T553] vrf_device_event+0x78/0xe0 [vrf] [ 59.530700][ T553] notifier_call_chain+0xcd/0x150 [ 59.530847][ T553] blocking_notifier_call_chain+0x66/0x90 [ 59.530994][ T553] unregister_netdevice_many_notify+0xa7b/0x1580 [ 59.531175][ T553] ? __lock_release+0x103/0x460 [ 59.531321][ T553] ? __pfx___lock_release+0x10/0x10 [ 59.531468][ T553] ? __pfx_unregister_netdevice_many_notify+0x10/0x10 [ 59.531654][ T553] ? hlock_class+0x4e/0x130 [ 59.531800][ T553] ? mark_held_locks+0x9e/0xe0 [ 59.531947][ T553] ? __pfx_vlan_info_rcu_free+0x10/0x10 [ 59.532097][ T553] ? lockdep_hardirqs_on_prepare+0x275/0x410 [ 59.532276][ T553] rtnl_dellink+0x348/0xa40 [ 59.532422][ T553] ? __pfx_rtnl_dellink+0x10/0x10 [ 59.532581][ T553] ? find_held_lock+0x2c/0x110 [ 59.532735][ T553] ? __lock_release+0x103/0x460 [ 59.532888][ T553] ? rtnetlink_rcv_msg+0x6ef/0xc10 [ 59.533035][ T553] ? __pfx___lock_release+0x10/0x10 [ 59.533180][ T553] ? __pfx_lock_acquire.part.0+0x10/0x10 [ 59.533329][ T553] ? rtnetlink_rcv_msg+0x6ef/0xc10 [ 59.533473][ T553] ? __pfx_rtnl_dellink+0x10/0x10 [ 59.533620][ T553] rtnetlink_rcv_msg+0x712/0xc10 [ 59.533770][ T553] ? __pfx_rtnetlink_rcv_msg+0x10/0x10 [ 59.533923][ T553] ? hlock_class+0x4e/0x130 [ 59.534155][ T553] ? mark_lock+0x38/0x3e0 [ 59.534278][ T553] ? __lock_acquire+0xb9a/0x1680 [ 59.534424][ T553] netlink_rcv_skb+0x130/0x360 [ 59.534571][ T553] ? __pfx_rtnetlink_rcv_msg+0x10/0x10 [ 59.534719][ T553] ? __pfx_netlink_rcv_skb+0x10/0x10 [ 59.534878][ T553] ? netlink_deliver_tap+0x13e/0x340 [ 59.535026][ T553] ? netlink_deliver_tap+0xc3/0x340 [ 59.535171][ T553] netlink_unicast+0x44b/0x710 [ 59.535317][ T553] ? __pfx_netlink_unicast+0x10/0x10 [ 59.535464][ T553] ? find_held_lock+0x2c/0x110 [ 59.535612][ T553] netlink_sendmsg+0x723/0xbe0 [ 59.535758][ T553] ? __pfx_netlink_sendmsg+0x10/0x10 [ 59.535911][ T553] ____sys_sendmsg+0x7ac/0xa10 [ 59.536059][ T553] ? __pfx_____sys_sendmsg+0x10/0x10 [ 59.536203][ T553] ? __pfx_copy_msghdr_from_user+0x10/0x10 [ 59.536388][ T553] ___sys_sendmsg+0xee/0x170 [ 59.536536][ T553] ? __pfx____sys_sendmsg+0x10/0x10 [ 59.536683][ T553] ? kasan_save_stack+0x34/0x50 [ 59.536830][ T553] ? kasan_save_stack+0x24/0x50 [ 59.536984][ T553] ? __kasan_record_aux_stack+0x8e/0xa0 [ 59.537133][ T553] ? __call_rcu_common.constprop.0+0xa1/0x4b0 [ 59.537316][ T553] ? __x64_sys_close+0x7c/0xd0 [ 59.537462][ T553] ? do_syscall_64+0xc1/0x1d0 [ 59.537608][ T553] ? entry_SYSCALL_64_after_hwframe+0x77/0x7f [ 59.537793][ T553] ? __lock_acquire+0xb9a/0x1680 [ 59.537940][ T553] ? find_held_lock+0x2c/0x110 [ 59.538085][ T553] ? __lock_release+0x103/0x460 [ 59.538232][ T553] ? __virt_addr_valid+0x22b/0x430 [ 59.538383][ T553] ? __pfx___lock_release+0x10/0x10 [ 59.538528][ T553] ? __pfx_lock_acquire.part.0+0x10/0x10 [ 59.538676][ T553] __sys_sendmsg+0x109/0x1a0 [ 59.538822][ T553] ? __pfx___sys_sendmsg+0x10/0x10 [ 59.538968][ T553] ? __pfx_slab_free_after_rcu_debug+0x10/0x10 [ 59.539157][ T553] do_syscall_64+0xc1/0x1d0 [ 59.539302][ T553] entry_SYSCALL_64_after_hwframe+0x77/0x7f [ 59.539484][ T553] RIP: 0033:0x7fd3faccc9a7 [ 59.539650][ T553] Code: 0a 00 f7 d8 64 89 02 48 c7 c0 ff ff ff ff eb b9 0f 1f 00 f3 0f 1e fa 64 8b 04 25 18 00 00 00 85 c0 75 10 b8 2e 00 00 00 0f 05 <48> 3d 00 f0 ff ff 77 51 c3 48 83 ec 28 89 54 24 1c 48 89 74 24 10 [ 59.540162][ T553] RSP: 002b:00007ffc230012b8 EFLAGS: 00000246 ORIG_RAX: 000000000000002e [ 59.540389][ T553] RAX: ffffffffffffffda RBX: 00007ffc230019e0 RCX: 00007fd3faccc9a7 [ 59.540608][ T553] RDX: 0000000000000000 RSI: 00007ffc23001320 RDI: 0000000000000005 [ 59.540827][ T553] RBP: 0000000000000002 R08: 0000000000000003 R09: 0000000000000078 [ 59.541042][ T553] R10: 00007fd3fab8aef8 R11: 0000000000000246 R12: 0000000000000002 [ 59.541263][ T553] R13: 00000000677903e5 R14: 0000000000498600 R15: 0000000000000000 [ 59.541485][ T553] [ 59.560022][ T553] ip (553) used greatest stack depth: 23904 bytes left [ 59.970097][ T560] br1: port 1(veth1) entered disabled state [ 60.026359][ T561] br1: port 2(veth2) entered disabled state [ 60.084706][ T562] veth2: left allmulticast mode [ 60.084939][ T562] veth2: left promiscuous mode [ 60.085236][ T562] br1: port 2(veth2) entered disabled state [ 60.086936][ T562] veth1: left allmulticast mode [ 60.087143][ T562] veth1: left promiscuous mode [ 60.087430][ T562] br1: port 1(veth1) entered disabled state