[ 10.665786][ T250] ip (250) used greatest stack depth: 23424 bytes left [ 12.560713][ T287] 8021q: 802.1Q VLAN Support v1.8 [ 15.838297][ T329] br1: port 1(veth1.100) entered blocking state [ 15.838830][ T329] br1: port 1(veth1.100) entered disabled state [ 15.839303][ T329] veth1.100: entered allmulticast mode [ 15.839532][ T329] veth1: entered allmulticast mode [ 15.841680][ T329] veth1.100: entered promiscuous mode [ 15.841942][ T329] veth1: entered promiscuous mode [ 16.089933][ T332] br1: port 1(veth1.100) entered blocking state [ 16.090402][ T332] br1: port 1(veth1.100) entered forwarding state [ 16.735767][ T342] br2: port 1(veth1.200) entered blocking state [ 16.736141][ T342] br2: port 1(veth1.200) entered disabled state [ 16.736464][ T342] veth1.200: entered allmulticast mode [ 16.739226][ T342] veth1.200: entered promiscuous mode [ 16.981670][ T345] br2: port 1(veth1.200) entered blocking state [ 16.982033][ T345] br2: port 1(veth1.200) entered forwarding state [ 28.311800][ T406] veth1.100: left allmulticast mode [ 28.312095][ T406] veth1.100: left promiscuous mode [ 28.312666][ T406] br1: port 1(veth1.100) entered disabled state [ 28.391728][ T407] veth1.200: left allmulticast mode [ 28.391977][ T407] veth1: left allmulticast mode [ 28.392184][ T407] veth1.200: left promiscuous mode [ 28.392386][ T407] veth1: left promiscuous mode [ 28.392910][ T407] br2: port 1(veth1.200) entered disabled state [ 30.522679][ T409] br2: port 1(veth1.200) entered blocking state [ 30.523032][ T409] br2: port 1(veth1.200) entered disabled state [ 30.523352][ T409] veth1.200: entered allmulticast mode [ 30.523933][ T409] veth1: entered allmulticast mode [ 30.525938][ T409] veth1.200: entered promiscuous mode [ 30.526167][ T409] veth1: entered promiscuous mode [ 30.527222][ T409] br2: port 1(veth1.200) entered blocking state [ 30.527520][ T409] br2: port 1(veth1.200) entered forwarding state [ 30.625296][ T410] br1: port 1(veth1.100) entered blocking state [ 30.625603][ T410] br1: port 1(veth1.100) entered disabled state [ 30.635575][ T410] veth1.100: entered allmulticast mode [ 30.644416][ T410] veth1.100: entered promiscuous mode [ 30.645488][ T410] br1: port 1(veth1.100) entered blocking state [ 30.645801][ T410] br1: port 1(veth1.100) entered forwarding state [ 36.528830][ T462] veth1.200: left allmulticast mode [ 36.529091][ T462] veth1.200: left promiscuous mode [ 36.529638][ T462] br2: port 1(veth1.200) entered disabled state [ 37.053456][ T469] veth1.100: left allmulticast mode [ 37.053696][ T469] veth1: left allmulticast mode [ 37.053892][ T469] veth1.100: left promiscuous mode [ 37.054066][ T469] veth1: left promiscuous mode [ 37.054575][ T469] br1: port 1(veth1.100) entered disabled state [ 38.977576][ T490] [ 38.977834][ T490] ============================================ [ 38.978146][ T490] WARNING: possible recursive locking detected [ 38.978448][ T490] 6.13.0-rc5-virtme #1 Not tainted [ 38.978702][ T490] -------------------------------------------- [ 38.979000][ T490] ip/490 is trying to acquire lock: [ 38.979241][ T490] ffffffff9fbd6630 ((netdev_chain).rwsem){++++}-{4:4}, at: blocking_notifier_call_chain+0x50/0x90 [ 38.979745][ T490] [ 38.979745][ T490] but task is already holding lock: [ 38.980103][ T490] ffffffff9fbd6630 ((netdev_chain).rwsem){++++}-{4:4}, at: blocking_notifier_call_chain+0x50/0x90 [ 38.980598][ T490] [ 38.980598][ T490] other info that might help us debug this: [ 38.980956][ T490] Possible unsafe locking scenario: [ 38.980956][ T490] [ 38.981319][ T490] CPU0 [ 38.981505][ T490] ---- [ 38.981703][ T490] lock((netdev_chain).rwsem); [ 38.981962][ T490] lock((netdev_chain).rwsem); [ 38.982206][ T490] [ 38.982206][ T490] *** DEADLOCK *** [ 38.982206][ T490] [ 38.982573][ T490] May be due to missing lock nesting notation [ 38.982573][ T490] [ 38.982929][ T490] 2 locks held by ip/490: [ 38.983115][ T490] #0: ffffffff9fbe7a48 (rtnl_mutex){+.+.}-{4:4}, at: rtnl_dellink+0x228/0xa40 [ 38.983555][ T490] #1: ffffffff9fbd6630 ((netdev_chain).rwsem){++++}-{4:4}, at: blocking_notifier_call_chain+0x50/0x90 [ 38.984061][ T490] [ 38.984061][ T490] stack backtrace: [ 38.984352][ T490] CPU: 0 UID: 0 PID: 490 Comm: ip Not tainted 6.13.0-rc5-virtme #1 [ 38.984728][ T490] Hardware name: Bochs Bochs, BIOS Bochs 01/01/2011 [ 38.985039][ T490] Call Trace: [ 38.985227][ T490] [ 38.985370][ T490] dump_stack_lvl+0x82/0xd0 [ 38.985628][ T490] print_deadlock_bug+0x40a/0x650 [ 38.985880][ T490] validate_chain+0x5bf/0xae0 [ 38.986133][ T490] ? __pfx_validate_chain+0x10/0x10 [ 38.986382][ T490] ? hlock_class+0x4e/0x130 [ 38.986625][ T490] ? mark_lock+0x38/0x3e0 [ 38.986817][ T490] __lock_acquire+0xb9a/0x1680 [ 38.987065][ T490] lock_acquire.part.0+0xeb/0x330 [ 38.987325][ T490] ? blocking_notifier_call_chain+0x50/0x90 [ 38.987636][ T490] ? netconsole_process_cleanups_core+0x2e/0x3d0 [ 38.987940][ T490] ? __pfx_lock_acquire.part.0+0x10/0x10 [ 38.988198][ T490] ? trace_lock_acquire+0x14c/0x1f0 [ 38.988441][ T490] ? lock_acquire+0x32/0xc0 [ 38.988694][ T490] ? blocking_notifier_call_chain+0x50/0x90 [ 38.989021][ T490] down_read+0x9f/0x340 [ 38.989211][ T490] ? blocking_notifier_call_chain+0x50/0x90 [ 38.989625][ T490] ? __pfx_down_read+0x10/0x10 [ 38.989861][ T490] ? __lock_release+0x103/0x460 [ 38.990093][ T490] ? netconsole_netdev_event+0x2c1/0x620 [ 38.990326][ T490] ? __pfx___lock_release+0x10/0x10 [ 38.990561][ T490] blocking_notifier_call_chain+0x50/0x90 [ 38.990808][ T490] __netdev_upper_dev_unlink+0xd9/0x220 [ 38.991029][ T490] ? __pfx___netdev_upper_dev_unlink+0x10/0x10 [ 38.991316][ T490] ? __mutex_unlock_slowpath+0x14a/0x3d0 [ 38.991554][ T490] ? __pfx___mutex_unlock_slowpath+0x10/0x10 [ 38.991872][ T490] ? rtnl_is_locked+0x15/0x20 [ 38.992149][ T490] netdev_upper_dev_unlink+0x71/0xa0 [ 38.992387][ T490] ? __pfx_netdev_upper_dev_unlink+0x10/0x10 [ 38.992690][ T490] ? netconsole_netdev_event+0x2c1/0x620 [ 38.992928][ T490] ? mutex_is_locked+0x1c/0x60 [ 38.993162][ T490] ? rtnl_is_locked+0x15/0x20 [ 38.993400][ T490] ? netdev_master_upper_dev_get+0x13/0x120 [ 38.993693][ T490] vrf_device_event+0x78/0xe0 [vrf] [ 38.993942][ T490] notifier_call_chain+0xcd/0x150 [ 38.994173][ T490] blocking_notifier_call_chain+0x66/0x90 [ 38.994421][ T490] unregister_netdevice_many_notify+0xa7b/0x1580 [ 38.994727][ T490] ? hlock_class+0x4e/0x130 [ 38.994971][ T490] ? mark_lock+0x38/0x3e0 [ 38.995154][ T490] ? __pfx_unregister_netdevice_many_notify+0x10/0x10 [ 38.995452][ T490] ? lockdep_hardirqs_on_prepare+0x275/0x410 [ 38.995756][ T490] ? kfree+0xf3/0x330 [ 38.995943][ T490] ? vlan_vid_del+0x310/0x5e0 [ 38.996197][ T490] ? vlan_vid_del+0x310/0x5e0 [ 38.996440][ T490] rtnl_dellink+0x348/0xa40 [ 38.996676][ T490] ? __pfx_rtnl_dellink+0x10/0x10 [ 38.996942][ T490] ? find_held_lock+0x2c/0x110 [ 38.997176][ T490] ? __lock_release+0x103/0x460 [ 38.997411][ T490] ? rtnetlink_rcv_msg+0x6ef/0xc10 [ 38.997652][ T490] ? __pfx___lock_release+0x10/0x10 [ 38.997890][ T490] ? __pfx_lock_acquire.part.0+0x10/0x10 [ 38.998125][ T490] ? rtnetlink_rcv_msg+0x6ef/0xc10 [ 38.998343][ T490] ? __pfx_rtnl_dellink+0x10/0x10 [ 38.998572][ T490] rtnetlink_rcv_msg+0x712/0xc10 [ 38.998804][ T490] ? __pfx_rtnetlink_rcv_msg+0x10/0x10 [ 38.999029][ T490] ? hlock_class+0x4e/0x130 [ 38.999254][ T490] ? mark_lock+0x38/0x3e0 [ 38.999426][ T490] ? __lock_acquire+0xb9a/0x1680 [ 38.999665][ T490] netlink_rcv_skb+0x130/0x360 [ 38.999896][ T490] ? __pfx_rtnetlink_rcv_msg+0x10/0x10 [ 39.000130][ T490] ? __pfx_netlink_rcv_skb+0x10/0x10 [ 39.000362][ T490] ? netlink_deliver_tap+0x13e/0x340 [ 39.000592][ T490] ? netlink_deliver_tap+0xc3/0x340 [ 39.000819][ T490] netlink_unicast+0x44b/0x710 [ 39.001047][ T490] ? __pfx_netlink_unicast+0x10/0x10 [ 39.001282][ T490] ? find_held_lock+0x2c/0x110 [ 39.001514][ T490] netlink_sendmsg+0x723/0xbe0 [ 39.001756][ T490] ? __pfx_netlink_sendmsg+0x10/0x10 [ 39.001987][ T490] ____sys_sendmsg+0x7ac/0xa10 [ 39.002206][ T490] ? __pfx_____sys_sendmsg+0x10/0x10 [ 39.002430][ T490] ? __pfx_copy_msghdr_from_user+0x10/0x10 [ 39.002720][ T490] ___sys_sendmsg+0xee/0x170 [ 39.002961][ T490] ? __pfx____sys_sendmsg+0x10/0x10 [ 39.003192][ T490] ? kasan_save_stack+0x34/0x50 [ 39.003417][ T490] ? kasan_save_stack+0x24/0x50 [ 39.003642][ T490] ? __kasan_record_aux_stack+0x8e/0xa0 [ 39.003867][ T490] ? __call_rcu_common.constprop.0+0xa1/0x4b0 [ 39.004148][ T490] ? __x64_sys_close+0x7c/0xd0 [ 39.004373][ T490] ? do_syscall_64+0xc1/0x1d0 [ 39.004606][ T490] ? entry_SYSCALL_64_after_hwframe+0x77/0x7f [ 39.004890][ T490] ? __lock_acquire+0xb9a/0x1680 [ 39.005121][ T490] ? find_held_lock+0x2c/0x110 [ 39.005355][ T490] ? __lock_release+0x103/0x460 [ 39.005596][ T490] ? __virt_addr_valid+0x22b/0x430 [ 39.005840][ T490] ? __pfx___lock_release+0x10/0x10 [ 39.006073][ T490] ? __pfx_lock_acquire.part.0+0x10/0x10 [ 39.006321][ T490] __sys_sendmsg+0x109/0x1a0 [ 39.006552][ T490] ? __pfx___sys_sendmsg+0x10/0x10 [ 39.006774][ T490] ? __pfx_slab_free_after_rcu_debug+0x10/0x10 [ 39.007068][ T490] do_syscall_64+0xc1/0x1d0 [ 39.007303][ T490] entry_SYSCALL_64_after_hwframe+0x77/0x7f [ 39.007592][ T490] RIP: 0033:0x7fcd959579a7 [ 39.007833][ T490] Code: 0a 00 f7 d8 64 89 02 48 c7 c0 ff ff ff ff eb b9 0f 1f 00 f3 0f 1e fa 64 8b 04 25 18 00 00 00 85 c0 75 10 b8 2e 00 00 00 0f 05 <48> 3d 00 f0 ff ff 77 51 c3 48 83 ec 28 89 54 24 1c 48 89 74 24 10 [ 39.008631][ T490] RSP: 002b:00007fff4b9e5058 EFLAGS: 00000246 ORIG_RAX: 000000000000002e [ 39.008971][ T490] RAX: ffffffffffffffda RBX: 00007fff4b9e5780 RCX: 00007fcd959579a7 [ 39.009314][ T490] RDX: 0000000000000000 RSI: 00007fff4b9e50c0 RDI: 0000000000000005 [ 39.009668][ T490] RBP: 0000000000000002 R08: 0000000000000003 R09: 0000000000000078 [ 39.010016][ T490] R10: 00007fcd95815ef8 R11: 0000000000000246 R12: 0000000000000002 [ 39.010367][ T490] R13: 000000006779049d R14: 0000000000498600 R15: 0000000000000000 [ 39.010723][ T490]