[ 10.471069][ T250] ip (250) used greatest stack depth: 24304 bytes left [ 13.242576][ T300] 8021q: 802.1Q VLAN Support v1.8 [ 17.079411][ T351] br1: port 1(vx10) entered blocking state [ 17.079879][ T351] br1: port 1(vx10) entered disabled state [ 17.080289][ T351] vx10: entered allmulticast mode [ 17.083020][ T351] vx10: entered promiscuous mode [ 17.083971][ T351] br1: port 1(vx10) entered blocking state [ 17.084319][ T351] br1: port 1(vx10) entered forwarding state [ 17.435785][ T356] br1: port 2(vx20) entered blocking state [ 17.436070][ T356] br1: port 2(vx20) entered disabled state [ 17.436343][ T356] vx20: entered allmulticast mode [ 17.439104][ T356] vx20: entered promiscuous mode [ 17.439940][ T356] br1: port 2(vx20) entered blocking state [ 17.440190][ T356] br1: port 2(vx20) entered forwarding state [ 17.618830][ T358] br1: port 3(veth1) entered blocking state [ 17.619148][ T358] br1: port 3(veth1) entered disabled state [ 17.619455][ T358] veth1: entered allmulticast mode [ 17.621601][ T358] veth1: entered promiscuous mode [ 17.708927][ T125] br1: port 3(veth1) entered blocking state [ 17.709532][ T125] br1: port 3(veth1) entered forwarding state [ 18.060696][ T363] br1: port 4(veth2) entered blocking state [ 18.061005][ T363] br1: port 4(veth2) entered disabled state [ 18.061285][ T363] veth2: entered allmulticast mode [ 18.063216][ T363] veth2: entered promiscuous mode [ 18.154443][ T57] br1: port 4(veth2) entered blocking state [ 18.154740][ T57] br1: port 4(veth2) entered forwarding state [ 18.762729][ T371] ip (371) used greatest stack depth: 23616 bytes left [ 21.490964][ T414] br2: port 1(w1) entered blocking state [ 21.491202][ T414] br2: port 1(w1) entered disabled state [ 21.491439][ T414] w1: entered allmulticast mode [ 21.493268][ T414] w1: entered promiscuous mode [ 22.138025][ T422] br2: port 2(vx10) entered blocking state [ 22.138665][ T422] br2: port 2(vx10) entered disabled state [ 22.138939][ T422] vx10: entered allmulticast mode [ 22.140843][ T422] vx10: entered promiscuous mode [ 22.141461][ T422] br2: port 2(vx10) entered blocking state [ 22.141743][ T422] br2: port 2(vx10) entered forwarding state [ 22.729047][ T429] br2: port 3(vx20) entered blocking state [ 22.729344][ T429] br2: port 3(vx20) entered disabled state [ 22.729622][ T429] vx20: entered allmulticast mode [ 22.731781][ T429] vx20: entered promiscuous mode [ 22.732272][ T429] br2: port 3(vx20) entered blocking state [ 22.732517][ T429] br2: port 3(vx20) entered forwarding state [ 23.380860][ T125] br2: port 1(w1) entered blocking state [ 23.381326][ T125] br2: port 1(w1) entered forwarding state [ 25.707683][ T471] br2: port 1(w1) entered blocking state [ 25.707952][ T471] br2: port 1(w1) entered disabled state [ 25.708465][ T471] w1: entered allmulticast mode [ 25.710434][ T471] w1: entered promiscuous mode [ 26.371838][ T479] br2: port 2(vx10) entered blocking state [ 26.372177][ T479] br2: port 2(vx10) entered disabled state [ 26.372568][ T479] vx10: entered allmulticast mode [ 26.374529][ T479] vx10: entered promiscuous mode [ 26.375147][ T479] br2: port 2(vx10) entered blocking state [ 26.375420][ T479] br2: port 2(vx10) entered forwarding state [ 26.989291][ T486] br2: port 3(vx20) entered blocking state [ 26.989645][ T486] br2: port 3(vx20) entered disabled state [ 26.989953][ T486] vx20: entered allmulticast mode [ 26.992415][ T486] vx20: entered promiscuous mode [ 26.992948][ T486] br2: port 3(vx20) entered blocking state [ 26.993226][ T486] br2: port 3(vx20) entered forwarding state [ 27.693133][ T45] br2: port 1(w1) entered blocking state [ 27.693495][ T45] br2: port 1(w1) entered forwarding state [ 38.599069][ T573] GACT probability NOT on [ 132.051135][ T11] vx20: left allmulticast mode [ 132.051634][ T11] vx20: left promiscuous mode [ 132.052194][ T11] br2: port 3(vx20) entered disabled state [ 132.057467][ T11] vx10: left allmulticast mode [ 132.057695][ T11] vx10: left promiscuous mode [ 132.058024][ T11] br2: port 2(vx10) entered disabled state [ 132.060138][ T11] w1: left allmulticast mode [ 132.060382][ T11] w1: left promiscuous mode [ 132.060729][ T11] br2: port 1(w1) entered disabled state [ 132.544848][ T11] vx20: left allmulticast mode [ 132.545061][ T11] vx20: left promiscuous mode [ 132.545386][ T11] br2: port 3(vx20) entered disabled state [ 132.548060][ T11] vx10: left allmulticast mode [ 132.548330][ T11] vx10: left promiscuous mode [ 132.548805][ T11] br2: port 2(vx10) entered disabled state [ 132.551410][ T11] w1: left allmulticast mode [ 132.551653][ T11] w1: left promiscuous mode [ 132.551988][ T11] br2: port 1(w1) entered disabled state [ 134.888042][ T781] br1: port 4(veth2) entered disabled state [ 134.969983][ T782] veth2: left allmulticast mode [ 134.970220][ T782] veth2: left promiscuous mode [ 134.970570][ T782] br1: port 4(veth2) entered disabled state [ 135.332935][ T786] br1: port 3(veth1) entered disabled state [ 135.401524][ T787] veth1: left allmulticast mode [ 135.401759][ T787] veth1: left promiscuous mode [ 135.402073][ T787] br1: port 3(veth1) entered disabled state [ 135.569002][ T789] vx20: left allmulticast mode [ 135.569351][ T789] vx20: left promiscuous mode [ 135.569720][ T789] br1: port 2(vx20) entered disabled state [ 135.911994][ T793] vx10: left allmulticast mode [ 135.912220][ T793] vx10: left promiscuous mode [ 135.912553][ T793] br1: port 1(vx10) entered disabled state [ 136.895546][ T803] [ 136.895751][ T803] ============================================ [ 136.895938][ T803] WARNING: possible recursive locking detected [ 136.896137][ T803] 6.13.0-rc5-virtme #1 Not tainted [ 136.896284][ T803] -------------------------------------------- [ 136.896463][ T803] ip/803 is trying to acquire lock: [ 136.896610][ T803] ffffffffa41d6630 ((netdev_chain).rwsem){++++}-{4:4}, at: blocking_notifier_call_chain+0x50/0x90 [ 136.896911][ T803] [ 136.896911][ T803] but task is already holding lock: [ 136.897129][ T803] ffffffffa41d6630 ((netdev_chain).rwsem){++++}-{4:4}, at: blocking_notifier_call_chain+0x50/0x90 [ 136.897422][ T803] [ 136.897422][ T803] other info that might help us debug this: [ 136.897634][ T803] Possible unsafe locking scenario: [ 136.897634][ T803] [ 136.897994][ T803] CPU0 [ 136.898106][ T803] ---- [ 136.898214][ T803] lock((netdev_chain).rwsem); [ 136.898360][ T803] lock((netdev_chain).rwsem); [ 136.898513][ T803] [ 136.898513][ T803] *** DEADLOCK *** [ 136.898513][ T803] [ 136.898725][ T803] May be due to missing lock nesting notation [ 136.898725][ T803] [ 136.898935][ T803] 2 locks held by ip/803: [ 136.899045][ T803] #0: ffffffffa41e7a48 (rtnl_mutex){+.+.}-{4:4}, at: rtnl_dellink+0x228/0xa40 [ 136.899314][ T803] #1: ffffffffa41d6630 ((netdev_chain).rwsem){++++}-{4:4}, at: blocking_notifier_call_chain+0x50/0x90 [ 136.899611][ T803] [ 136.899611][ T803] stack backtrace: [ 136.899788][ T803] CPU: 3 UID: 0 PID: 803 Comm: ip Not tainted 6.13.0-rc5-virtme #1 [ 136.900009][ T803] Hardware name: Bochs Bochs, BIOS Bochs 01/01/2011 [ 136.900192][ T803] Call Trace: [ 136.900304][ T803] [ 136.900383][ T803] dump_stack_lvl+0x82/0xd0 [ 136.900539][ T803] print_deadlock_bug+0x40a/0x650 [ 136.900693][ T803] validate_chain+0x5bf/0xae0 [ 136.900839][ T803] ? __pfx_validate_chain+0x10/0x10 [ 136.900993][ T803] ? hlock_class+0x4e/0x130 [ 136.901139][ T803] ? mark_lock+0x38/0x3e0 [ 136.901250][ T803] __lock_acquire+0xb9a/0x1680 [ 136.901396][ T803] lock_acquire.part.0+0xeb/0x330 [ 136.901543][ T803] ? blocking_notifier_call_chain+0x50/0x90 [ 136.901726][ T803] ? netconsole_process_cleanups_core+0x2e/0x3d0 [ 136.901919][ T803] ? __pfx_lock_acquire.part.0+0x10/0x10 [ 136.902071][ T803] ? trace_lock_acquire+0x14c/0x1f0 [ 136.902218][ T803] ? lock_acquire+0x32/0xc0 [ 136.902361][ T803] ? blocking_notifier_call_chain+0x50/0x90 [ 136.902550][ T803] down_read+0x9f/0x340 [ 136.902665][ T803] ? blocking_notifier_call_chain+0x50/0x90 [ 136.902846][ T803] ? __pfx_down_read+0x10/0x10 [ 136.902989][ T803] ? __lock_release+0x103/0x460 [ 136.903132][ T803] ? netconsole_netdev_event+0x2c1/0x620 [ 136.903279][ T803] ? __pfx___lock_release+0x10/0x10 [ 136.903425][ T803] blocking_notifier_call_chain+0x50/0x90 [ 136.903576][ T803] __netdev_upper_dev_unlink+0xd9/0x220 [ 136.903725][ T803] ? __pfx___netdev_upper_dev_unlink+0x10/0x10 [ 136.903906][ T803] ? __mutex_unlock_slowpath+0x14a/0x3d0 [ 136.904050][ T803] ? __pfx___mutex_unlock_slowpath+0x10/0x10 [ 136.904229][ T803] ? rtnl_is_locked+0x15/0x20 [ 136.904378][ T803] netdev_upper_dev_unlink+0x71/0xa0 [ 136.904523][ T803] ? __pfx_netdev_upper_dev_unlink+0x10/0x10 [ 136.904708][ T803] ? netconsole_netdev_event+0x2c1/0x620 [ 136.904851][ T803] ? mutex_is_locked+0x1c/0x60 [ 136.904998][ T803] ? rtnl_is_locked+0x15/0x20 [ 136.905141][ T803] ? netdev_master_upper_dev_get+0x13/0x120 [ 136.905320][ T803] vrf_device_event+0x78/0xe0 [vrf] [ 136.905468][ T803] notifier_call_chain+0xcd/0x150 [ 136.905614][ T803] blocking_notifier_call_chain+0x66/0x90 [ 136.905762][ T803] unregister_netdevice_many_notify+0xa7b/0x1580 [ 136.905949][ T803] ? hlock_class+0x4e/0x130 [ 136.906094][ T803] ? mark_lock+0x38/0x3e0 [ 136.906206][ T803] ? __pfx_unregister_netdevice_many_notify+0x10/0x10 [ 136.906385][ T803] ? lockdep_hardirqs_on_prepare+0x275/0x410 [ 136.906566][ T803] ? kfree+0xf3/0x330 [ 136.906677][ T803] ? vlan_vid_del+0x310/0x5e0 [ 136.906831][ T803] ? vlan_vid_del+0x310/0x5e0 [ 136.906975][ T803] rtnl_dellink+0x348/0xa40 [ 136.907121][ T803] ? __pfx_rtnl_dellink+0x10/0x10 [ 136.907280][ T803] ? find_held_lock+0x2c/0x110 [ 136.907427][ T803] ? __lock_release+0x103/0x460 [ 136.907570][ T803] ? rtnetlink_rcv_msg+0x6ef/0xc10 [ 136.907720][ T803] ? __pfx___lock_release+0x10/0x10 [ 136.907874][ T803] ? __pfx_lock_acquire.part.0+0x10/0x10 [ 136.908021][ T803] ? rtnetlink_rcv_msg+0x6ef/0xc10 [ 136.908163][ T803] ? __pfx_rtnl_dellink+0x10/0x10 [ 136.908310][ T803] rtnetlink_rcv_msg+0x712/0xc10 [ 136.908454][ T803] ? __pfx_rtnetlink_rcv_msg+0x10/0x10 [ 136.908599][ T803] ? hlock_class+0x4e/0x130 [ 136.908740][ T803] ? mark_lock+0x38/0x3e0 [ 136.908851][ T803] ? __lock_acquire+0xb9a/0x1680 [ 136.909003][ T803] netlink_rcv_skb+0x130/0x360 [ 136.909152][ T803] ? __pfx_rtnetlink_rcv_msg+0x10/0x10 [ 136.909297][ T803] ? __pfx_netlink_rcv_skb+0x10/0x10 [ 136.909445][ T803] ? netlink_deliver_tap+0x13e/0x340 [ 136.909592][ T803] ? netlink_deliver_tap+0xc3/0x340 [ 136.909747][ T803] netlink_unicast+0x44b/0x710 [ 136.909895][ T803] ? __pfx_netlink_unicast+0x10/0x10 [ 136.910048][ T803] ? find_held_lock+0x2c/0x110 [ 136.910194][ T803] netlink_sendmsg+0x723/0xbe0 [ 136.910340][ T803] ? __pfx_netlink_sendmsg+0x10/0x10 [ 136.910487][ T803] ____sys_sendmsg+0x7ac/0xa10 [ 136.910632][ T803] ? __pfx_____sys_sendmsg+0x10/0x10 [ 136.910776][ T803] ? __pfx_copy_msghdr_from_user+0x10/0x10 [ 136.910964][ T803] ___sys_sendmsg+0xee/0x170 [ 136.911113][ T803] ? __pfx____sys_sendmsg+0x10/0x10 [ 136.911259][ T803] ? kasan_save_stack+0x34/0x50 [ 136.911404][ T803] ? kasan_save_stack+0x24/0x50 [ 136.911548][ T803] ? __kasan_record_aux_stack+0x8e/0xa0 [ 136.911693][ T803] ? __call_rcu_common.constprop.0+0xa1/0x4b0 [ 136.911875][ T803] ? __x64_sys_close+0x7c/0xd0 [ 136.912020][ T803] ? do_syscall_64+0xc1/0x1d0 [ 136.912170][ T803] ? entry_SYSCALL_64_after_hwframe+0x77/0x7f [ 136.912350][ T803] ? __lock_acquire+0xb9a/0x1680 [ 136.912497][ T803] ? find_held_lock+0x2c/0x110 [ 136.912643][ T803] ? __lock_release+0x103/0x460 [ 136.912787][ T803] ? __virt_addr_valid+0x22b/0x430 [ 136.912934][ T803] ? __pfx___lock_release+0x10/0x10 [ 136.913077][ T803] ? __pfx_lock_acquire.part.0+0x10/0x10 [ 136.913230][ T803] __sys_sendmsg+0x109/0x1a0 [ 136.913380][ T803] ? __pfx___sys_sendmsg+0x10/0x10 [ 136.913526][ T803] ? __pfx_slab_free_after_rcu_debug+0x10/0x10 [ 136.913712][ T803] do_syscall_64+0xc1/0x1d0 [ 136.913857][ T803] entry_SYSCALL_64_after_hwframe+0x77/0x7f [ 136.914045][ T803] RIP: 0033:0x7fb0b68f29a7 [ 136.914199][ T803] Code: 0a 00 f7 d8 64 89 02 48 c7 c0 ff ff ff ff eb b9 0f 1f 00 f3 0f 1e fa 64 8b 04 25 18 00 00 00 85 c0 75 10 b8 2e 00 00 00 0f 05 <48> 3d 00 f0 ff ff 77 51 c3 48 83 ec 28 89 54 24 1c 48 89 74 24 10 [ 136.914711][ T803] RSP: 002b:00007ffed5499f08 EFLAGS: 00000246 ORIG_RAX: 000000000000002e [ 136.914931][ T803] RAX: ffffffffffffffda RBX: 00007ffed549a630 RCX: 00007fb0b68f29a7 [ 136.915150][ T803] RDX: 0000000000000000 RSI: 00007ffed5499f70 RDI: 0000000000000005 [ 136.915373][ T803] RBP: 0000000000000002 R08: 0000000000000003 R09: 0000000000000078 [ 136.915590][ T803] R10: 00007fb0b67b0ef8 R11: 0000000000000246 R12: 0000000000000002 [ 136.915806][ T803] R13: 00000000677902e0 R14: 0000000000498600 R15: 0000000000000000 [ 136.916026][ T803]