[ 1277.765705][T17378] br1: port 1(veth1) entered blocking state [ 1277.766232][T17378] br1: port 1(veth1) entered disabled state [ 1277.766877][T17378] veth1: entered allmulticast mode [ 1277.770054][T17378] veth1: entered promiscuous mode [ 1277.865075][T16518] br1: port 1(veth1) entered blocking state [ 1277.865606][T16518] br1: port 1(veth1) entered forwarding state [ 1277.940182][T17380] br1: port 2(veth2) entered blocking state [ 1277.940504][T17380] br1: port 2(veth2) entered disabled state [ 1277.940815][T17380] veth2: entered allmulticast mode [ 1277.942770][T17380] veth2: entered promiscuous mode [ 1278.054177][ T40] br1: port 2(veth2) entered blocking state [ 1278.054557][ T40] br1: port 2(veth2) entered forwarding state [ 1278.358695][T17384] 8021q: 802.1Q VLAN Support v1.8 [ 1279.149694][T17395] GACT probability NOT on [ 1286.558379][T17433] Mirror/redirect action on [ 1322.165216][T17613] [ 1322.165415][T17613] ============================================ [ 1322.165705][T17613] WARNING: possible recursive locking detected [ 1322.165987][T17613] 6.13.0-rc5-virtme #1 Not tainted [ 1322.166220][T17613] -------------------------------------------- [ 1322.166506][T17613] ip/17613 is trying to acquire lock: [ 1322.166739][T17613] ffffffff9f1d6630 ((netdev_chain).rwsem){++++}-{4:4}, at: blocking_notifier_call_chain+0x50/0x90 [ 1322.167238][T17613] [ 1322.167238][T17613] but task is already holding lock: [ 1322.167587][T17613] ffffffff9f1d6630 ((netdev_chain).rwsem){++++}-{4:4}, at: blocking_notifier_call_chain+0x50/0x90 [ 1322.168085][T17613] [ 1322.168085][T17613] other info that might help us debug this: [ 1322.168447][T17613] Possible unsafe locking scenario: [ 1322.168447][T17613] [ 1322.168800][T17613] CPU0 [ 1322.169000][T17613] ---- [ 1322.169175][T17613] lock((netdev_chain).rwsem); [ 1322.169412][T17613] lock((netdev_chain).rwsem); [ 1322.169643][T17613] [ 1322.169643][T17613] *** DEADLOCK *** [ 1322.169643][T17613] [ 1322.169987][T17613] May be due to missing lock nesting notation [ 1322.169987][T17613] [ 1322.170328][T17613] 2 locks held by ip/17613: [ 1322.170574][T17613] #0: ffffffff9f1e7a48 (rtnl_mutex){+.+.}-{4:4}, at: rtnl_dellink+0x228/0xa40 [ 1322.171000][T17613] #1: ffffffff9f1d6630 ((netdev_chain).rwsem){++++}-{4:4}, at: blocking_notifier_call_chain+0x50/0x90 [ 1322.171481][T17613] [ 1322.171481][T17613] stack backtrace: [ 1322.171768][T17613] CPU: 0 UID: 0 PID: 17613 Comm: ip Not tainted 6.13.0-rc5-virtme #1 [ 1322.172112][T17613] Hardware name: Bochs Bochs, BIOS Bochs 01/01/2011 [ 1322.172412][T17613] Call Trace: [ 1322.172592][T17613] [ 1322.172711][T17613] dump_stack_lvl+0x82/0xd0 [ 1322.172954][T17613] print_deadlock_bug+0x40a/0x650 [ 1322.173185][T17613] validate_chain+0x5bf/0xae0 [ 1322.173427][T17613] ? __pfx_validate_chain+0x10/0x10 [ 1322.173669][T17613] ? hlock_class+0x4e/0x130 [ 1322.173908][T17613] ? mark_lock+0x38/0x3e0 [ 1322.174094][T17613] __lock_acquire+0xb9a/0x1680 [ 1322.174335][T17613] lock_acquire.part.0+0xeb/0x330 [ 1322.174565][T17613] ? blocking_notifier_call_chain+0x50/0x90 [ 1322.174858][T17613] ? netconsole_process_cleanups_core+0x2e/0x3d0 [ 1322.175153][T17613] ? __pfx_lock_acquire.part.0+0x10/0x10 [ 1322.175397][T17613] ? trace_lock_acquire+0x14c/0x1f0 [ 1322.175629][T17613] ? lock_acquire+0x32/0xc0 [ 1322.175872][T17613] ? blocking_notifier_call_chain+0x50/0x90 [ 1322.176153][T17613] down_read+0x9f/0x340 [ 1322.176336][T17613] ? blocking_notifier_call_chain+0x50/0x90 [ 1322.176620][T17613] ? __pfx_down_read+0x10/0x10 [ 1322.176850][T17613] ? __lock_release+0x103/0x460 [ 1322.177088][T17613] ? netconsole_netdev_event+0x2c1/0x620 [ 1322.177321][T17613] ? __pfx___lock_release+0x10/0x10 [ 1322.177562][T17613] blocking_notifier_call_chain+0x50/0x90 [ 1322.177796][T17613] __netdev_upper_dev_unlink+0xd9/0x220 [ 1322.178027][T17613] ? __pfx___netdev_upper_dev_unlink+0x10/0x10 [ 1322.178317][T17613] ? __mutex_unlock_slowpath+0x14a/0x3d0 [ 1322.178545][T17613] ? __pfx___mutex_unlock_slowpath+0x10/0x10 [ 1322.178839][T17613] ? rtnl_is_locked+0x15/0x20 [ 1322.179091][T17613] netdev_upper_dev_unlink+0x71/0xa0 [ 1322.179325][T17613] ? __pfx_netdev_upper_dev_unlink+0x10/0x10 [ 1322.179618][T17613] ? netconsole_netdev_event+0x2c1/0x620 [ 1322.179855][T17613] ? mutex_is_locked+0x1c/0x60 [ 1322.180088][T17613] ? rtnl_is_locked+0x15/0x20 [ 1322.180321][T17613] ? netdev_master_upper_dev_get+0x13/0x120 [ 1322.180608][T17613] vrf_device_event+0x78/0xe0 [vrf] [ 1322.180854][T17613] notifier_call_chain+0xcd/0x150 [ 1322.181091][T17613] blocking_notifier_call_chain+0x66/0x90 [ 1322.181329][T17613] unregister_netdevice_many_notify+0xa7b/0x1580 [ 1322.181620][T17613] ? __lock_release+0x103/0x460 [ 1322.181852][T17613] ? __pfx___lock_release+0x10/0x10 [ 1322.182091][T17613] ? __pfx_unregister_netdevice_many_notify+0x10/0x10 [ 1322.182384][T17613] ? hlock_class+0x4e/0x130 [ 1322.182630][T17613] ? mark_held_locks+0x9e/0xe0 [ 1322.182862][T17613] ? __pfx_vlan_info_rcu_free+0x10/0x10 [ 1322.183092][T17613] ? lockdep_hardirqs_on_prepare+0x275/0x410 [ 1322.183380][T17613] rtnl_dellink+0x348/0xa40 [ 1322.183618][T17613] ? __pfx_rtnl_dellink+0x10/0x10 [ 1322.183882][T17613] ? find_held_lock+0x2c/0x110 [ 1322.184122][T17613] ? __lock_release+0x103/0x460 [ 1322.184363][T17613] ? rtnetlink_rcv_msg+0x6ef/0xc10 [ 1322.184598][T17613] ? __pfx___lock_release+0x10/0x10 [ 1322.184819][T17613] ? __pfx_lock_acquire.part.0+0x10/0x10 [ 1322.185053][T17613] ? rtnetlink_rcv_msg+0x6ef/0xc10 [ 1322.185281][T17613] ? __pfx_rtnl_dellink+0x10/0x10 [ 1322.185519][T17613] rtnetlink_rcv_msg+0x712/0xc10 [ 1322.185743][T17613] ? __pfx_rtnetlink_rcv_msg+0x10/0x10 [ 1322.185972][T17613] ? hlock_class+0x4e/0x130 [ 1322.186192][T17613] ? mark_lock+0x38/0x3e0 [ 1322.186370][T17613] ? __lock_acquire+0xb9a/0x1680 [ 1322.186604][T17613] netlink_rcv_skb+0x130/0x360 [ 1322.186836][T17613] ? __pfx_rtnetlink_rcv_msg+0x10/0x10 [ 1322.187066][T17613] ? __pfx_netlink_rcv_skb+0x10/0x10 [ 1322.187410][T17613] ? netlink_deliver_tap+0x13e/0x340 [ 1322.187659][T17613] ? netlink_deliver_tap+0xc3/0x340 [ 1322.187906][T17613] netlink_unicast+0x44b/0x710 [ 1322.188141][T17613] ? __pfx_netlink_unicast+0x10/0x10 [ 1322.188392][T17613] ? find_held_lock+0x2c/0x110 [ 1322.188625][T17613] netlink_sendmsg+0x723/0xbe0 [ 1322.188877][T17613] ? __pfx_netlink_sendmsg+0x10/0x10 [ 1322.189112][T17613] ____sys_sendmsg+0x7ac/0xa10 [ 1322.189349][T17613] ? __pfx_____sys_sendmsg+0x10/0x10 [ 1322.189577][T17613] ? __pfx_copy_msghdr_from_user+0x10/0x10 [ 1322.189864][T17613] ___sys_sendmsg+0xee/0x170 [ 1322.190100][T17613] ? __pfx____sys_sendmsg+0x10/0x10 [ 1322.190335][T17613] ? kasan_save_stack+0x34/0x50 [ 1322.190572][T17613] ? kasan_save_stack+0x24/0x50 [ 1322.190798][T17613] ? __kasan_record_aux_stack+0x8e/0xa0 [ 1322.191016][T17613] ? __call_rcu_common.constprop.0+0xa1/0x4b0 [ 1322.191298][T17613] ? __x64_sys_close+0x7c/0xd0 [ 1322.191517][T17613] ? do_syscall_64+0xc1/0x1d0 [ 1322.191743][T17613] ? entry_SYSCALL_64_after_hwframe+0x77/0x7f [ 1322.192030][T17613] ? __lock_acquire+0xb9a/0x1680 [ 1322.192263][T17613] ? find_held_lock+0x2c/0x110 [ 1322.192495][T17613] ? __lock_release+0x103/0x460 [ 1322.192725][T17613] ? __virt_addr_valid+0x22b/0x430 [ 1322.192969][T17613] ? __pfx___lock_release+0x10/0x10 [ 1322.193196][T17613] ? __pfx_lock_acquire.part.0+0x10/0x10 [ 1322.193433][T17613] __sys_sendmsg+0x109/0x1a0 [ 1322.193650][T17613] ? __pfx___sys_sendmsg+0x10/0x10 [ 1322.193877][T17613] ? __pfx_slab_free_after_rcu_debug+0x10/0x10 [ 1322.194166][T17613] do_syscall_64+0xc1/0x1d0 [ 1322.194397][T17613] entry_SYSCALL_64_after_hwframe+0x77/0x7f [ 1322.194699][T17613] RIP: 0033:0x7fd0c8dd49a7 [ 1322.194949][T17613] Code: 0a 00 f7 d8 64 89 02 48 c7 c0 ff ff ff ff eb b9 0f 1f 00 f3 0f 1e fa 64 8b 04 25 18 00 00 00 85 c0 75 10 b8 2e 00 00 00 0f 05 <48> 3d 00 f0 ff ff 77 51 c3 48 83 ec 28 89 54 24 1c 48 89 74 24 10 [ 1322.195770][T17613] RSP: 002b:00007ffdba031ff8 EFLAGS: 00000246 ORIG_RAX: 000000000000002e [ 1322.196140][T17613] RAX: ffffffffffffffda RBX: 00007ffdba032720 RCX: 00007fd0c8dd49a7 [ 1322.196491][T17613] RDX: 0000000000000000 RSI: 00007ffdba032060 RDI: 0000000000000005 [ 1322.196817][T17613] RBP: 0000000000000002 R08: 0000000000000003 R09: 0000000000000078 [ 1322.197160][T17613] R10: 00007fd0c8c92ef8 R11: 0000000000000246 R12: 0000000000000002 [ 1322.197512][T17613] R13: 000000006779539a R14: 0000000000498600 R15: 0000000000000000 [ 1322.197875][T17613] [ 1322.747801][T17620] br1: port 1(veth1) entered disabled state [ 1322.826099][T17621] br1: port 2(veth2) entered disabled state [ 1322.889172][T17622] veth2: left allmulticast mode [ 1322.889368][T17622] veth2: left promiscuous mode [ 1322.889635][T17622] br1: port 2(veth2) entered disabled state [ 1322.890748][T17622] veth1: left allmulticast mode [ 1322.890919][T17622] veth1: left promiscuous mode [ 1322.891196][T17622] br1: port 1(veth1) entered disabled state