[ 11.232390][ T249] ip (249) used greatest stack depth: 23248 bytes left [ 14.288690][ T299] 8021q: 802.1Q VLAN Support v1.8 [ 18.236921][ T349] br1: port 1(vx10) entered blocking state [ 18.237460][ T349] br1: port 1(vx10) entered disabled state [ 18.237898][ T349] vx10: entered allmulticast mode [ 18.240025][ T349] vx10: entered promiscuous mode [ 18.241117][ T349] br1: port 1(vx10) entered blocking state [ 18.241532][ T349] br1: port 1(vx10) entered forwarding state [ 18.630529][ T354] br1: port 2(vx20) entered blocking state [ 18.630837][ T354] br1: port 2(vx20) entered disabled state [ 18.631133][ T354] vx20: entered allmulticast mode [ 18.633451][ T354] vx20: entered promiscuous mode [ 18.634325][ T354] br1: port 2(vx20) entered blocking state [ 18.634600][ T354] br1: port 2(vx20) entered forwarding state [ 18.837740][ T356] br1: port 3(veth1) entered blocking state [ 18.838849][ T356] br1: port 3(veth1) entered disabled state [ 18.839369][ T356] veth1: entered allmulticast mode [ 18.842663][ T356] veth1: entered promiscuous mode [ 18.934452][ T40] br1: port 3(veth1) entered blocking state [ 18.935227][ T40] br1: port 3(veth1) entered forwarding state [ 19.354101][ T361] br1: port 4(veth2) entered blocking state [ 19.354470][ T361] br1: port 4(veth2) entered disabled state [ 19.354763][ T361] veth2: entered allmulticast mode [ 19.356982][ T361] veth2: entered promiscuous mode [ 19.443821][ T37] br1: port 4(veth2) entered blocking state [ 19.444115][ T37] br1: port 4(veth2) entered forwarding state [ 22.868127][ T412] br2: port 1(w1) entered blocking state [ 22.868586][ T412] br2: port 1(w1) entered disabled state [ 22.868944][ T412] w1: entered allmulticast mode [ 22.872326][ T412] w1: entered promiscuous mode [ 23.545926][ T420] br2: port 2(vx10) entered blocking state [ 23.546257][ T420] br2: port 2(vx10) entered disabled state [ 23.546552][ T420] vx10: entered allmulticast mode [ 23.548862][ T420] vx10: entered promiscuous mode [ 23.549421][ T420] br2: port 2(vx10) entered blocking state [ 23.549720][ T420] br2: port 2(vx10) entered forwarding state [ 24.174505][ T427] br2: port 3(vx20) entered blocking state [ 24.174827][ T427] br2: port 3(vx20) entered disabled state [ 24.175127][ T427] vx20: entered allmulticast mode [ 24.177067][ T427] vx20: entered promiscuous mode [ 24.178030][ T427] br2: port 3(vx20) entered blocking state [ 24.178312][ T427] br2: port 3(vx20) entered forwarding state [ 24.887325][ T40] br2: port 1(w1) entered blocking state [ 24.887724][ T40] br2: port 1(w1) entered forwarding state [ 27.240108][ T469] br2: port 1(w1) entered blocking state [ 27.240461][ T469] br2: port 1(w1) entered disabled state [ 27.240979][ T469] w1: entered allmulticast mode [ 27.242930][ T469] w1: entered promiscuous mode [ 27.882817][ T477] br2: port 2(vx10) entered blocking state [ 27.883782][ T477] br2: port 2(vx10) entered disabled state [ 27.884226][ T477] vx10: entered allmulticast mode [ 27.888104][ T477] vx10: entered promiscuous mode [ 27.889026][ T477] br2: port 2(vx10) entered blocking state [ 27.889670][ T477] br2: port 2(vx10) entered forwarding state [ 28.438699][ T484] br2: port 3(vx20) entered blocking state [ 28.438998][ T484] br2: port 3(vx20) entered disabled state [ 28.439301][ T484] vx20: entered allmulticast mode [ 28.441212][ T484] vx20: entered promiscuous mode [ 28.441723][ T484] br2: port 3(vx20) entered blocking state [ 28.441960][ T484] br2: port 3(vx20) entered forwarding state [ 29.076890][ T40] br2: port 1(w1) entered blocking state [ 29.077220][ T40] br2: port 1(w1) entered forwarding state [ 39.980377][ T571] GACT probability NOT on [ 133.872226][ T67] vx20: left allmulticast mode [ 133.872540][ T67] vx20: left promiscuous mode [ 133.873023][ T67] br2: port 3(vx20) entered disabled state [ 133.876133][ T67] vx10: left allmulticast mode [ 133.876352][ T67] vx10: left promiscuous mode [ 133.876654][ T67] br2: port 2(vx10) entered disabled state [ 133.878859][ T67] w1: left allmulticast mode [ 133.879083][ T67] w1: left promiscuous mode [ 133.879581][ T67] br2: port 1(w1) entered disabled state [ 134.467843][ T67] vx20: left allmulticast mode [ 134.468141][ T67] vx20: left promiscuous mode [ 134.468498][ T67] br2: port 3(vx20) entered disabled state [ 134.471206][ T67] vx10: left allmulticast mode [ 134.471503][ T67] vx10: left promiscuous mode [ 134.471976][ T67] br2: port 2(vx10) entered disabled state [ 134.474690][ T67] w1: left allmulticast mode [ 134.474880][ T67] w1: left promiscuous mode [ 134.475194][ T67] br2: port 1(w1) entered disabled state [ 136.700782][ T780] br1: port 4(veth2) entered disabled state [ 136.781547][ T781] veth2: left allmulticast mode [ 136.781860][ T781] veth2: left promiscuous mode [ 136.782271][ T781] br1: port 4(veth2) entered disabled state [ 137.137798][ T785] br1: port 3(veth1) entered disabled state [ 137.214271][ T786] veth1: left allmulticast mode [ 137.214513][ T786] veth1: left promiscuous mode [ 137.214841][ T786] br1: port 3(veth1) entered disabled state [ 137.356389][ T788] vx20: left allmulticast mode [ 137.356632][ T788] vx20: left promiscuous mode [ 137.356976][ T788] br1: port 2(vx20) entered disabled state [ 137.724028][ T792] vx10: left allmulticast mode [ 137.724280][ T792] vx10: left promiscuous mode [ 137.724610][ T792] br1: port 1(vx10) entered disabled state [ 138.664273][ T802] [ 138.664437][ T802] ============================================ [ 138.664673][ T802] WARNING: possible recursive locking detected [ 138.664896][ T802] 6.13.0-rc5-virtme #1 Not tainted [ 138.665075][ T802] -------------------------------------------- [ 138.665297][ T802] ip/802 is trying to acquire lock: [ 138.665478][ T802] ffffffffa3bd6630 ((netdev_chain).rwsem){++++}-{4:4}, at: blocking_notifier_call_chain+0x50/0x90 [ 138.665852][ T802] [ 138.665852][ T802] but task is already holding lock: [ 138.666127][ T802] ffffffffa3bd6630 ((netdev_chain).rwsem){++++}-{4:4}, at: blocking_notifier_call_chain+0x50/0x90 [ 138.666500][ T802] [ 138.666500][ T802] other info that might help us debug this: [ 138.666767][ T802] Possible unsafe locking scenario: [ 138.666767][ T802] [ 138.667039][ T802] CPU0 [ 138.667174][ T802] ---- [ 138.667310][ T802] lock((netdev_chain).rwsem); [ 138.667492][ T802] lock((netdev_chain).rwsem); [ 138.667679][ T802] [ 138.667679][ T802] *** DEADLOCK *** [ 138.667679][ T802] [ 138.667944][ T802] May be due to missing lock nesting notation [ 138.667944][ T802] [ 138.668213][ T802] 2 locks held by ip/802: [ 138.668353][ T802] #0: ffffffffa3be7a48 (rtnl_mutex){+.+.}-{4:4}, at: rtnl_dellink+0x228/0xa40 [ 138.668679][ T802] #1: ffffffffa3bd6630 ((netdev_chain).rwsem){++++}-{4:4}, at: blocking_notifier_call_chain+0x50/0x90 [ 138.669043][ T802] [ 138.669043][ T802] stack backtrace: [ 138.669269][ T802] CPU: 2 UID: 0 PID: 802 Comm: ip Not tainted 6.13.0-rc5-virtme #1 [ 138.669552][ T802] Hardware name: Bochs Bochs, BIOS Bochs 01/01/2011 [ 138.669786][ T802] Call Trace: [ 138.669928][ T802] [ 138.670024][ T802] dump_stack_lvl+0x82/0xd0 [ 138.670214][ T802] print_deadlock_bug+0x40a/0x650 [ 138.670401][ T802] validate_chain+0x5bf/0xae0 [ 138.670584][ T802] ? __pfx_validate_chain+0x10/0x10 [ 138.670769][ T802] ? hlock_class+0x4e/0x130 [ 138.670950][ T802] ? mark_lock+0x38/0x3e0 [ 138.671095][ T802] __lock_acquire+0xb9a/0x1680 [ 138.671281][ T802] lock_acquire.part.0+0xeb/0x330 [ 138.671470][ T802] ? blocking_notifier_call_chain+0x50/0x90 [ 138.671706][ T802] ? netconsole_process_cleanups_core+0x2e/0x3d0 [ 138.671933][ T802] ? __pfx_lock_acquire.part.0+0x10/0x10 [ 138.672135][ T802] ? trace_lock_acquire+0x14c/0x1f0 [ 138.672322][ T802] ? lock_acquire+0x32/0xc0 [ 138.672509][ T802] ? blocking_notifier_call_chain+0x50/0x90 [ 138.672737][ T802] down_read+0x9f/0x340 [ 138.672875][ T802] ? blocking_notifier_call_chain+0x50/0x90 [ 138.673106][ T802] ? __pfx_down_read+0x10/0x10 [ 138.673296][ T802] ? __lock_release+0x103/0x460 [ 138.673477][ T802] ? netconsole_netdev_event+0x2c1/0x620 [ 138.673661][ T802] ? __pfx___lock_release+0x10/0x10 [ 138.673860][ T802] blocking_notifier_call_chain+0x50/0x90 [ 138.674043][ T802] __netdev_upper_dev_unlink+0xd9/0x220 [ 138.674224][ T802] ? __pfx___netdev_upper_dev_unlink+0x10/0x10 [ 138.674448][ T802] ? __mutex_unlock_slowpath+0x14a/0x3d0 [ 138.674630][ T802] ? __pfx___mutex_unlock_slowpath+0x10/0x10 [ 138.674855][ T802] ? rtnl_is_locked+0x15/0x20 [ 138.675039][ T802] netdev_upper_dev_unlink+0x71/0xa0 [ 138.675226][ T802] ? __pfx_netdev_upper_dev_unlink+0x10/0x10 [ 138.675451][ T802] ? netconsole_netdev_event+0x2c1/0x620 [ 138.675632][ T802] ? mutex_is_locked+0x1c/0x60 [ 138.675813][ T802] ? rtnl_is_locked+0x15/0x20 [ 138.675992][ T802] ? netdev_master_upper_dev_get+0x13/0x120 [ 138.676220][ T802] vrf_device_event+0x78/0xe0 [vrf] [ 138.676412][ T802] notifier_call_chain+0xcd/0x150 [ 138.676596][ T802] blocking_notifier_call_chain+0x66/0x90 [ 138.676777][ T802] unregister_netdevice_many_notify+0xa7b/0x1580 [ 138.677001][ T802] ? hlock_class+0x4e/0x130 [ 138.677183][ T802] ? mark_lock+0x38/0x3e0 [ 138.677320][ T802] ? __pfx_unregister_netdevice_many_notify+0x10/0x10 [ 138.677545][ T802] ? lockdep_hardirqs_on_prepare+0x275/0x410 [ 138.677774][ T802] ? kfree+0xf3/0x330 [ 138.677913][ T802] ? vlan_vid_del+0x310/0x5e0 [ 138.678097][ T802] ? vlan_vid_del+0x310/0x5e0 [ 138.678278][ T802] rtnl_dellink+0x348/0xa40 [ 138.678461][ T802] ? __pfx_rtnl_dellink+0x10/0x10 [ 138.678657][ T802] ? find_held_lock+0x2c/0x110 [ 138.678840][ T802] ? __lock_release+0x103/0x460 [ 138.679019][ T802] ? rtnetlink_rcv_msg+0x6ef/0xc10 [ 138.679204][ T802] ? __pfx___lock_release+0x10/0x10 [ 138.679384][ T802] ? __pfx_lock_acquire.part.0+0x10/0x10 [ 138.679568][ T802] ? rtnetlink_rcv_msg+0x6ef/0xc10 [ 138.679750][ T802] ? __pfx_rtnl_dellink+0x10/0x10 [ 138.679932][ T802] rtnetlink_rcv_msg+0x712/0xc10 [ 138.680113][ T802] ? __pfx_rtnetlink_rcv_msg+0x10/0x10 [ 138.680293][ T802] ? hlock_class+0x4e/0x130 [ 138.680478][ T802] ? mark_lock+0x38/0x3e0 [ 138.680618][ T802] ? __lock_acquire+0xb9a/0x1680 [ 138.680802][ T802] netlink_rcv_skb+0x130/0x360 [ 138.680983][ T802] ? __pfx_rtnetlink_rcv_msg+0x10/0x10 [ 138.681164][ T802] ? __pfx_netlink_rcv_skb+0x10/0x10 [ 138.681351][ T802] ? netlink_deliver_tap+0x13e/0x340 [ 138.681533][ T802] ? netlink_deliver_tap+0xc3/0x340 [ 138.681717][ T802] netlink_unicast+0x44b/0x710 [ 138.681898][ T802] ? __pfx_netlink_unicast+0x10/0x10 [ 138.682086][ T802] ? find_held_lock+0x2c/0x110 [ 138.682274][ T802] netlink_sendmsg+0x723/0xbe0 [ 138.682456][ T802] ? __pfx_netlink_sendmsg+0x10/0x10 [ 138.682639][ T802] ____sys_sendmsg+0x7ac/0xa10 [ 138.682821][ T802] ? __pfx_____sys_sendmsg+0x10/0x10 [ 138.683007][ T802] ? __pfx_copy_msghdr_from_user+0x10/0x10 [ 138.683236][ T802] ___sys_sendmsg+0xee/0x170 [ 138.683419][ T802] ? __pfx____sys_sendmsg+0x10/0x10 [ 138.683602][ T802] ? kasan_save_stack+0x34/0x50 [ 138.683781][ T802] ? kasan_save_stack+0x24/0x50 [ 138.683960][ T802] ? __kasan_record_aux_stack+0x8e/0xa0 [ 138.684140][ T802] ? __call_rcu_common.constprop.0+0xa1/0x4b0 [ 138.684372][ T802] ? __x64_sys_close+0x7c/0xd0 [ 138.684553][ T802] ? do_syscall_64+0xc1/0x1d0 [ 138.684736][ T802] ? entry_SYSCALL_64_after_hwframe+0x77/0x7f [ 138.684970][ T802] ? __lock_acquire+0xb9a/0x1680 [ 138.685154][ T802] ? find_held_lock+0x2c/0x110 [ 138.685339][ T802] ? __lock_release+0x103/0x460 [ 138.685520][ T802] ? __virt_addr_valid+0x22b/0x430 [ 138.685704][ T802] ? __pfx___lock_release+0x10/0x10 [ 138.685890][ T802] ? __pfx_lock_acquire.part.0+0x10/0x10 [ 138.686072][ T802] __sys_sendmsg+0x109/0x1a0 [ 138.686253][ T802] ? __pfx___sys_sendmsg+0x10/0x10 [ 138.686433][ T802] ? __pfx_slab_free_after_rcu_debug+0x10/0x10 [ 138.686666][ T802] do_syscall_64+0xc1/0x1d0 [ 138.686846][ T802] entry_SYSCALL_64_after_hwframe+0x77/0x7f [ 138.687068][ T802] RIP: 0033:0x7f472e28f9a7 [ 138.687261][ T802] Code: 0a 00 f7 d8 64 89 02 48 c7 c0 ff ff ff ff eb b9 0f 1f 00 f3 0f 1e fa 64 8b 04 25 18 00 00 00 85 c0 75 10 b8 2e 00 00 00 0f 05 <48> 3d 00 f0 ff ff 77 51 c3 48 83 ec 28 89 54 24 1c 48 89 74 24 10 [ 138.687895][ T802] RSP: 002b:00007fffa5f152f8 EFLAGS: 00000246 ORIG_RAX: 000000000000002e [ 138.688169][ T802] RAX: ffffffffffffffda RBX: 00007fffa5f15a20 RCX: 00007f472e28f9a7 [ 138.688441][ T802] RDX: 0000000000000000 RSI: 00007fffa5f15360 RDI: 0000000000000005 [ 138.688714][ T802] RBP: 0000000000000002 R08: 0000000000000003 R09: 0000000000000078 [ 138.688984][ T802] R10: 00007f472e14def8 R11: 0000000000000246 R12: 0000000000000002 [ 138.689255][ T802] R13: 0000000067795616 R14: 0000000000498600 R15: 0000000000000000 [ 138.689537][ T802]