[ 13.917872][ T299] 8021q: 802.1Q VLAN Support v1.8 [ 17.753863][ T350] br1: port 1(vx10) entered blocking state [ 17.754351][ T350] br1: port 1(vx10) entered disabled state [ 17.755012][ T350] vx10: entered allmulticast mode [ 17.757156][ T350] vx10: entered promiscuous mode [ 17.758129][ T350] br1: port 1(vx10) entered blocking state [ 17.758466][ T350] br1: port 1(vx10) entered forwarding state [ 18.093378][ T355] br1: port 2(vx20) entered blocking state [ 18.094012][ T355] br1: port 2(vx20) entered disabled state [ 18.094304][ T355] vx20: entered allmulticast mode [ 18.096275][ T355] vx20: entered promiscuous mode [ 18.097913][ T355] br1: port 2(vx20) entered blocking state [ 18.098179][ T355] br1: port 2(vx20) entered forwarding state [ 18.295723][ T357] br1: port 3(veth1) entered blocking state [ 18.296150][ T357] br1: port 3(veth1) entered disabled state [ 18.296580][ T357] veth1: entered allmulticast mode [ 18.299639][ T357] veth1: entered promiscuous mode [ 18.375101][ T40] br1: port 3(veth1) entered blocking state [ 18.375645][ T40] br1: port 3(veth1) entered forwarding state [ 18.703255][ T363] br1: port 4(veth2) entered blocking state [ 18.703543][ T363] br1: port 4(veth2) entered disabled state [ 18.703860][ T363] veth2: entered allmulticast mode [ 18.705980][ T363] veth2: entered promiscuous mode [ 18.782779][ T40] br1: port 4(veth2) entered blocking state [ 18.783110][ T40] br1: port 4(veth2) entered forwarding state [ 22.361471][ T413] br2: port 1(w1) entered blocking state [ 22.361735][ T413] br2: port 1(w1) entered disabled state [ 22.361981][ T413] w1: entered allmulticast mode [ 22.363984][ T413] w1: entered promiscuous mode [ 23.022609][ T421] br2: port 2(vx10) entered blocking state [ 23.024208][ T421] br2: port 2(vx10) entered disabled state [ 23.024614][ T421] vx10: entered allmulticast mode [ 23.026591][ T421] vx10: entered promiscuous mode [ 23.027989][ T421] br2: port 2(vx10) entered blocking state [ 23.028265][ T421] br2: port 2(vx10) entered forwarding state [ 23.313247][ T424] ip (424) used greatest stack depth: 23648 bytes left [ 23.613656][ T428] br2: port 3(vx20) entered blocking state [ 23.614185][ T428] br2: port 3(vx20) entered disabled state [ 23.614482][ T428] vx20: entered allmulticast mode [ 23.616417][ T428] vx20: entered promiscuous mode [ 23.616973][ T428] br2: port 3(vx20) entered blocking state [ 23.617231][ T428] br2: port 3(vx20) entered forwarding state [ 24.284255][ T39] br2: port 1(w1) entered blocking state [ 24.284774][ T39] br2: port 1(w1) entered forwarding state [ 26.742721][ T470] br2: port 1(w1) entered blocking state [ 26.743028][ T470] br2: port 1(w1) entered disabled state [ 26.743636][ T470] w1: entered allmulticast mode [ 26.745633][ T470] w1: entered promiscuous mode [ 27.382343][ T478] br2: port 2(vx10) entered blocking state [ 27.382734][ T478] br2: port 2(vx10) entered disabled state [ 27.383146][ T478] vx10: entered allmulticast mode [ 27.385663][ T478] vx10: entered promiscuous mode [ 27.386346][ T478] br2: port 2(vx10) entered blocking state [ 27.386697][ T478] br2: port 2(vx10) entered forwarding state [ 27.983239][ T485] br2: port 3(vx20) entered blocking state [ 27.983571][ T485] br2: port 3(vx20) entered disabled state [ 27.984345][ T485] vx20: entered allmulticast mode [ 27.986326][ T485] vx20: entered promiscuous mode [ 27.986898][ T485] br2: port 3(vx20) entered blocking state [ 27.987159][ T485] br2: port 3(vx20) entered forwarding state [ 28.657469][ T37] br2: port 1(w1) entered blocking state [ 28.657758][ T37] br2: port 1(w1) entered forwarding state [ 39.558585][ T572] GACT probability NOT on [ 133.471353][ T65] vx20: left allmulticast mode [ 133.471663][ T65] vx20: left promiscuous mode [ 133.472017][ T65] br2: port 3(vx20) entered disabled state [ 133.474537][ T65] vx10: left allmulticast mode [ 133.474761][ T65] vx10: left promiscuous mode [ 133.475056][ T65] br2: port 2(vx10) entered disabled state [ 133.477145][ T65] w1: left allmulticast mode [ 133.477340][ T65] w1: left promiscuous mode [ 133.477694][ T65] br2: port 1(w1) entered disabled state [ 134.040063][ T65] vx20: left allmulticast mode [ 134.040287][ T65] vx20: left promiscuous mode [ 134.040615][ T65] br2: port 3(vx20) entered disabled state [ 134.043220][ T65] vx10: left allmulticast mode [ 134.043423][ T65] vx10: left promiscuous mode [ 134.043926][ T65] br2: port 2(vx10) entered disabled state [ 134.052024][ T65] w1: left allmulticast mode [ 134.052316][ T65] w1: left promiscuous mode [ 134.052631][ T65] br2: port 1(w1) entered disabled state [ 136.311254][ T781] br1: port 4(veth2) entered disabled state [ 136.401004][ T782] veth2: left allmulticast mode [ 136.401249][ T782] veth2: left promiscuous mode [ 136.401581][ T782] br1: port 4(veth2) entered disabled state [ 136.777662][ T786] br1: port 3(veth1) entered disabled state [ 136.858331][ T788] veth1: left allmulticast mode [ 136.858579][ T788] veth1: left promiscuous mode [ 136.858912][ T788] br1: port 3(veth1) entered disabled state [ 137.020474][ T790] vx20: left allmulticast mode [ 137.020744][ T790] vx20: left promiscuous mode [ 137.021076][ T790] br1: port 2(vx20) entered disabled state [ 137.382616][ T794] vx10: left allmulticast mode [ 137.382873][ T794] vx10: left promiscuous mode [ 137.383193][ T794] br1: port 1(vx10) entered disabled state [ 138.409973][ T804] [ 138.410203][ T804] ============================================ [ 138.410395][ T804] WARNING: possible recursive locking detected [ 138.410592][ T804] 6.13.0-rc5-virtme #1 Not tainted [ 138.410743][ T804] -------------------------------------------- [ 138.410928][ T804] ip/804 is trying to acquire lock: [ 138.411076][ T804] ffffffff8e7d6630 ((netdev_chain).rwsem){++++}-{4:4}, at: blocking_notifier_call_chain+0x50/0x90 [ 138.411389][ T804] [ 138.411389][ T804] but task is already holding lock: [ 138.411615][ T804] ffffffff8e7d6630 ((netdev_chain).rwsem){++++}-{4:4}, at: blocking_notifier_call_chain+0x50/0x90 [ 138.411913][ T804] [ 138.411913][ T804] other info that might help us debug this: [ 138.412137][ T804] Possible unsafe locking scenario: [ 138.412137][ T804] [ 138.412355][ T804] CPU0 [ 138.412476][ T804] ---- [ 138.412588][ T804] lock((netdev_chain).rwsem); [ 138.412736][ T804] lock((netdev_chain).rwsem); [ 138.412887][ T804] [ 138.412887][ T804] *** DEADLOCK *** [ 138.412887][ T804] [ 138.413100][ T804] May be due to missing lock nesting notation [ 138.413100][ T804] [ 138.413335][ T804] 2 locks held by ip/804: [ 138.413480][ T804] #0: ffffffff8e7e7a48 (rtnl_mutex){+.+.}-{4:4}, at: rtnl_dellink+0x228/0xa40 [ 138.413766][ T804] #1: ffffffff8e7d6630 ((netdev_chain).rwsem){++++}-{4:4}, at: blocking_notifier_call_chain+0x50/0x90 [ 138.414077][ T804] [ 138.414077][ T804] stack backtrace: [ 138.414262][ T804] CPU: 3 UID: 0 PID: 804 Comm: ip Not tainted 6.13.0-rc5-virtme #1 [ 138.414501][ T804] Hardware name: Bochs Bochs, BIOS Bochs 01/01/2011 [ 138.414715][ T804] Call Trace: [ 138.414835][ T804] [ 138.414916][ T804] dump_stack_lvl+0x82/0xd0 [ 138.415079][ T804] print_deadlock_bug+0x40a/0x650 [ 138.415237][ T804] validate_chain+0x5bf/0xae0 [ 138.415390][ T804] ? __pfx_validate_chain+0x10/0x10 [ 138.415556][ T804] ? hlock_class+0x4e/0x130 [ 138.415707][ T804] ? mark_lock+0x38/0x3e0 [ 138.415823][ T804] __lock_acquire+0xb9a/0x1680 [ 138.415978][ T804] lock_acquire.part.0+0xeb/0x330 [ 138.416133][ T804] ? blocking_notifier_call_chain+0x50/0x90 [ 138.416321][ T804] ? netconsole_process_cleanups_core+0x2e/0x3d0 [ 138.416515][ T804] ? __pfx_lock_acquire.part.0+0x10/0x10 [ 138.416665][ T804] ? trace_lock_acquire+0x14c/0x1f0 [ 138.416824][ T804] ? lock_acquire+0x32/0xc0 [ 138.416976][ T804] ? blocking_notifier_call_chain+0x50/0x90 [ 138.417161][ T804] down_read+0x9f/0x340 [ 138.417277][ T804] ? blocking_notifier_call_chain+0x50/0x90 [ 138.417508][ T804] ? __pfx_down_read+0x10/0x10 [ 138.417658][ T804] ? __lock_release+0x103/0x460 [ 138.417806][ T804] ? netconsole_netdev_event+0x2c1/0x620 [ 138.417963][ T804] ? __pfx___lock_release+0x10/0x10 [ 138.418125][ T804] blocking_notifier_call_chain+0x50/0x90 [ 138.418272][ T804] __netdev_upper_dev_unlink+0xd9/0x220 [ 138.418431][ T804] ? __pfx___netdev_upper_dev_unlink+0x10/0x10 [ 138.418623][ T804] ? __mutex_unlock_slowpath+0x14a/0x3d0 [ 138.418772][ T804] ? __pfx___mutex_unlock_slowpath+0x10/0x10 [ 138.418956][ T804] ? rtnl_is_locked+0x15/0x20 [ 138.419110][ T804] netdev_upper_dev_unlink+0x71/0xa0 [ 138.419267][ T804] ? __pfx_netdev_upper_dev_unlink+0x10/0x10 [ 138.419461][ T804] ? netconsole_netdev_event+0x2c1/0x620 [ 138.419612][ T804] ? mutex_is_locked+0x1c/0x60 [ 138.419761][ T804] ? rtnl_is_locked+0x15/0x20 [ 138.419911][ T804] ? netdev_master_upper_dev_get+0x13/0x120 [ 138.420098][ T804] vrf_device_event+0x78/0xe0 [vrf] [ 138.420252][ T804] notifier_call_chain+0xcd/0x150 [ 138.420406][ T804] blocking_notifier_call_chain+0x66/0x90 [ 138.420565][ T804] unregister_netdevice_many_notify+0xa7b/0x1580 [ 138.420750][ T804] ? hlock_class+0x4e/0x130 [ 138.420901][ T804] ? mark_lock+0x38/0x3e0 [ 138.421014][ T804] ? __pfx_unregister_netdevice_many_notify+0x10/0x10 [ 138.421202][ T804] ? lockdep_hardirqs_on_prepare+0x275/0x410 [ 138.421432][ T804] ? kfree+0xf3/0x330 [ 138.421564][ T804] ? vlan_vid_del+0x310/0x5e0 [ 138.421721][ T804] ? vlan_vid_del+0x310/0x5e0 [ 138.421870][ T804] rtnl_dellink+0x348/0xa40 [ 138.422020][ T804] ? __pfx_rtnl_dellink+0x10/0x10 [ 138.422183][ T804] ? find_held_lock+0x2c/0x110 [ 138.422333][ T804] ? __lock_release+0x103/0x460 [ 138.422492][ T804] ? rtnetlink_rcv_msg+0x6ef/0xc10 [ 138.422648][ T804] ? __pfx___lock_release+0x10/0x10 [ 138.422795][ T804] ? __pfx_lock_acquire.part.0+0x10/0x10 [ 138.422944][ T804] ? rtnetlink_rcv_msg+0x6ef/0xc10 [ 138.423091][ T804] ? __pfx_rtnl_dellink+0x10/0x10 [ 138.423245][ T804] rtnetlink_rcv_msg+0x712/0xc10 [ 138.423398][ T804] ? __pfx_rtnetlink_rcv_msg+0x10/0x10 [ 138.423555][ T804] ? hlock_class+0x4e/0x130 [ 138.423704][ T804] ? mark_lock+0x38/0x3e0 [ 138.423816][ T804] ? __lock_acquire+0xb9a/0x1680 [ 138.423966][ T804] netlink_rcv_skb+0x130/0x360 [ 138.424114][ T804] ? __pfx_rtnetlink_rcv_msg+0x10/0x10 [ 138.424264][ T804] ? __pfx_netlink_rcv_skb+0x10/0x10 [ 138.424420][ T804] ? netlink_deliver_tap+0x13e/0x340 [ 138.424581][ T804] ? netlink_deliver_tap+0xc3/0x340 [ 138.424752][ T804] netlink_unicast+0x44b/0x710 [ 138.424900][ T804] ? __pfx_netlink_unicast+0x10/0x10 [ 138.425046][ T804] ? find_held_lock+0x2c/0x110 [ 138.425197][ T804] netlink_sendmsg+0x723/0xbe0 [ 138.425375][ T804] ? __pfx_netlink_sendmsg+0x10/0x10 [ 138.425547][ T804] ____sys_sendmsg+0x7ac/0xa10 [ 138.425703][ T804] ? __pfx_____sys_sendmsg+0x10/0x10 [ 138.425851][ T804] ? __pfx_copy_msghdr_from_user+0x10/0x10 [ 138.426052][ T804] ___sys_sendmsg+0xee/0x170 [ 138.426199][ T804] ? __pfx____sys_sendmsg+0x10/0x10 [ 138.426351][ T804] ? kasan_save_stack+0x34/0x50 [ 138.426524][ T804] ? kasan_save_stack+0x24/0x50 [ 138.426673][ T804] ? __kasan_record_aux_stack+0x8e/0xa0 [ 138.426821][ T804] ? __call_rcu_common.constprop.0+0xa1/0x4b0 [ 138.427008][ T804] ? __x64_sys_close+0x7c/0xd0 [ 138.427162][ T804] ? do_syscall_64+0xc1/0x1d0 [ 138.427317][ T804] ? entry_SYSCALL_64_after_hwframe+0x77/0x7f [ 138.427594][ T804] ? __lock_acquire+0xb9a/0x1680 [ 138.427843][ T804] ? find_held_lock+0x2c/0x110 [ 138.428092][ T804] ? __lock_release+0x103/0x460 [ 138.428326][ T804] ? __virt_addr_valid+0x22b/0x430 [ 138.428560][ T804] ? __pfx___lock_release+0x10/0x10 [ 138.428810][ T804] ? __pfx_lock_acquire.part.0+0x10/0x10 [ 138.429060][ T804] __sys_sendmsg+0x109/0x1a0 [ 138.429304][ T804] ? __pfx___sys_sendmsg+0x10/0x10 [ 138.429548][ T804] ? __pfx_slab_free_after_rcu_debug+0x10/0x10 [ 138.429860][ T804] do_syscall_64+0xc1/0x1d0 [ 138.430107][ T804] entry_SYSCALL_64_after_hwframe+0x77/0x7f [ 138.430405][ T804] RIP: 0033:0x7fb54f66c9a7 [ 138.430676][ T804] Code: 0a 00 f7 d8 64 89 02 48 c7 c0 ff ff ff ff eb b9 0f 1f 00 f3 0f 1e fa 64 8b 04 25 18 00 00 00 85 c0 75 10 b8 2e 00 00 00 0f 05 <48> 3d 00 f0 ff ff 77 51 c3 48 83 ec 28 89 54 24 1c 48 89 74 24 10 [ 138.431513][ T804] RSP: 002b:00007ffe2d446a58 EFLAGS: 00000246 ORIG_RAX: 000000000000002e [ 138.431877][ T804] RAX: ffffffffffffffda RBX: 00007ffe2d447180 RCX: 00007fb54f66c9a7 [ 138.432247][ T804] RDX: 0000000000000000 RSI: 00007ffe2d446ac0 RDI: 0000000000000005 [ 138.432602][ T804] RBP: 0000000000000002 R08: 0000000000000003 R09: 0000000000000078 [ 138.432963][ T804] R10: 00007fb54f52aef8 R11: 0000000000000246 R12: 0000000000000002 [ 138.433323][ T804] R13: 0000000067792d6a R14: 0000000000498600 R15: 0000000000000000 [ 138.433693][ T804]