[ 11.373069][ T249] ip (249) used greatest stack depth: 23984 bytes left [ 13.644913][ T292] 8021q: 802.1Q VLAN Support v1.8 [ 16.391745][ T328] br0: port 1(veth1) entered blocking state [ 16.392296][ T328] br0: port 1(veth1) entered disabled state [ 16.392736][ T328] veth1: entered allmulticast mode [ 16.394835][ T328] veth1: entered promiscuous mode [ 16.499212][ T39] br0: port 1(veth1) entered blocking state [ 16.499684][ T39] br0: port 1(veth1) entered forwarding state [ 16.681507][ T332] br0: port 2(veth2) entered blocking state [ 16.681825][ T332] br0: port 2(veth2) entered disabled state [ 16.682122][ T332] veth2: entered allmulticast mode [ 16.684209][ T332] veth2: entered promiscuous mode [ 16.766283][ T39] br0: port 2(veth2) entered blocking state [ 16.766719][ T39] br0: port 2(veth2) entered forwarding state [ 87.804213][ T2115] br0: port 2(veth2) entered disabled state [ 87.902467][ T2116] veth2: left allmulticast mode [ 87.902756][ T2116] veth2: left promiscuous mode [ 87.903112][ T2116] br0: port 2(veth2) entered disabled state [ 87.981242][ T2117] br0: port 1(veth1) entered disabled state [ 88.072376][ T2118] veth1: left allmulticast mode [ 88.072633][ T2118] veth1: left promiscuous mode [ 88.072959][ T2118] br0: port 1(veth1) entered disabled state [ 88.850332][ T2136] br0: port 1(veth1) entered blocking state [ 88.850616][ T2136] br0: port 1(veth1) entered disabled state [ 88.851119][ T2136] veth1: entered allmulticast mode [ 88.853523][ T2136] veth1: entered promiscuous mode [ 88.942665][ T37] br0: port 1(veth1) entered blocking state [ 88.943080][ T37] br0: port 1(veth1) entered forwarding state [ 89.267041][ T2141] br0: port 2(veth2) entered blocking state [ 89.267422][ T2141] br0: port 2(veth2) entered disabled state [ 89.267687][ T2141] veth2: entered allmulticast mode [ 89.270004][ T2141] veth2: entered promiscuous mode [ 89.358776][ T37] br0: port 2(veth2) entered blocking state [ 89.359054][ T37] br0: port 2(veth2) entered forwarding state [ 160.552689][ T3933] br0: port 2(veth2) entered disabled state [ 160.630242][ T3934] veth2: left allmulticast mode [ 160.630464][ T3934] veth2: left promiscuous mode [ 160.630773][ T3934] br0: port 2(veth2) entered disabled state [ 160.720046][ T3935] br0: port 1(veth1) entered disabled state [ 160.797510][ T3936] veth1: left allmulticast mode [ 160.797805][ T3936] veth1: left promiscuous mode [ 160.798150][ T3936] br0: port 1(veth1) entered disabled state [ 161.509507][ T3954] br0: port 1(veth1) entered blocking state [ 161.509820][ T3954] br0: port 1(veth1) entered disabled state [ 161.510102][ T3954] veth1: entered allmulticast mode [ 161.511983][ T3954] veth1: entered promiscuous mode [ 161.599851][ T40] br0: port 1(veth1) entered blocking state [ 161.600140][ T40] br0: port 1(veth1) entered forwarding state [ 161.916921][ T3959] br0: port 2(veth2) entered blocking state [ 161.917336][ T3959] br0: port 2(veth2) entered disabled state [ 161.917694][ T3959] veth2: entered allmulticast mode [ 161.920078][ T3959] veth2: entered promiscuous mode [ 162.004373][ T37] br0: port 2(veth2) entered blocking state [ 162.004852][ T37] br0: port 2(veth2) entered forwarding state [ 315.079148][ T7808] br0: port 2(veth2) entered disabled state [ 315.162750][ T7809] veth2: left allmulticast mode [ 315.163023][ T7809] veth2: left promiscuous mode [ 315.163407][ T7809] br0: port 2(veth2) entered disabled state [ 315.250036][ T7810] br0: port 1(veth1) entered disabled state [ 315.369427][ T7811] veth1: left allmulticast mode [ 315.369757][ T7811] veth1: left promiscuous mode [ 315.370261][ T7811] br0: port 1(veth1) entered disabled state [ 316.644662][ T7832] [ 316.644812][ T7832] ============================================ [ 316.645018][ T7832] WARNING: possible recursive locking detected [ 316.645207][ T7832] 6.13.0-rc5-virtme #1 Not tainted [ 316.645379][ T7832] -------------------------------------------- [ 316.645608][ T7832] ip/7832 is trying to acquire lock: [ 316.645782][ T7832] ffffffff8f9d6630 ((netdev_chain).rwsem){++++}-{4:4}, at: blocking_notifier_call_chain+0x50/0x90 [ 316.646180][ T7832] [ 316.646180][ T7832] but task is already holding lock: [ 316.646456][ T7832] ffffffff8f9d6630 ((netdev_chain).rwsem){++++}-{4:4}, at: blocking_notifier_call_chain+0x50/0x90 [ 316.646783][ T7832] [ 316.646783][ T7832] other info that might help us debug this: [ 316.647004][ T7832] Possible unsafe locking scenario: [ 316.647004][ T7832] [ 316.647236][ T7832] CPU0 [ 316.647354][ T7832] ---- [ 316.647477][ T7832] lock((netdev_chain).rwsem); [ 316.647629][ T7832] lock((netdev_chain).rwsem); [ 316.647790][ T7832] [ 316.647790][ T7832] *** DEADLOCK *** [ 316.647790][ T7832] [ 316.648035][ T7832] May be due to missing lock nesting notation [ 316.648035][ T7832] [ 316.648267][ T7832] 2 locks held by ip/7832: [ 316.648435][ T7832] #0: ffffffff8f9e7a48 (rtnl_mutex){+.+.}-{4:4}, at: rtnl_dellink+0x228/0xa40 [ 316.648732][ T7832] #1: ffffffff8f9d6630 ((netdev_chain).rwsem){++++}-{4:4}, at: blocking_notifier_call_chain+0x50/0x90 [ 316.649065][ T7832] [ 316.649065][ T7832] stack backtrace: [ 316.649266][ T7832] CPU: 0 UID: 0 PID: 7832 Comm: ip Not tainted 6.13.0-rc5-virtme #1 [ 316.649522][ T7832] Hardware name: Bochs Bochs, BIOS Bochs 01/01/2011 [ 316.649727][ T7832] Call Trace: [ 316.649854][ T7832] [ 316.649942][ T7832] dump_stack_lvl+0x82/0xd0 [ 316.650096][ T7832] print_deadlock_bug+0x40a/0x650 [ 316.650265][ T7832] validate_chain+0x5bf/0xae0 [ 316.650434][ T7832] ? __pfx_validate_chain+0x10/0x10 [ 316.650603][ T7832] ? hlock_class+0x4e/0x130 [ 316.650784][ T7832] ? mark_lock+0x38/0x3e0 [ 316.650919][ T7832] __lock_acquire+0xb9a/0x1680 [ 316.651091][ T7832] lock_acquire.part.0+0xeb/0x330 [ 316.651262][ T7832] ? blocking_notifier_call_chain+0x50/0x90 [ 316.651496][ T7832] ? netconsole_process_cleanups_core+0x2e/0x3d0 [ 316.651711][ T7832] ? __pfx_lock_acquire.part.0+0x10/0x10 [ 316.651875][ T7832] ? trace_lock_acquire+0x14c/0x1f0 [ 316.652036][ T7832] ? lock_acquire+0x32/0xc0 [ 316.652187][ T7832] ? blocking_notifier_call_chain+0x50/0x90 [ 316.652376][ T7832] down_read+0x9f/0x340 [ 316.652502][ T7832] ? blocking_notifier_call_chain+0x50/0x90 [ 316.652703][ T7832] ? __pfx_down_read+0x10/0x10 [ 316.652859][ T7832] ? __lock_release+0x103/0x460 [ 316.653034][ T7832] ? netconsole_netdev_event+0x2c1/0x620 [ 316.653209][ T7832] ? __pfx___lock_release+0x10/0x10 [ 316.653378][ T7832] blocking_notifier_call_chain+0x50/0x90 [ 316.653552][ T7832] __netdev_upper_dev_unlink+0xd9/0x220 [ 316.653704][ T7832] ? __pfx___netdev_upper_dev_unlink+0x10/0x10 [ 316.653919][ T7832] ? __mutex_unlock_slowpath+0x14a/0x3d0 [ 316.654084][ T7832] ? __pfx___mutex_unlock_slowpath+0x10/0x10 [ 316.654357][ T7832] ? rtnl_is_locked+0x15/0x20 [ 316.654549][ T7832] netdev_upper_dev_unlink+0x71/0xa0 [ 316.654725][ T7832] ? __pfx_netdev_upper_dev_unlink+0x10/0x10 [ 316.654918][ T7832] ? netconsole_netdev_event+0x2c1/0x620 [ 316.655090][ T7832] ? mutex_is_locked+0x1c/0x60 [ 316.655261][ T7832] ? rtnl_is_locked+0x15/0x20 [ 316.655444][ T7832] ? netdev_master_upper_dev_get+0x13/0x120 [ 316.655660][ T7832] vrf_device_event+0x78/0xe0 [vrf] [ 316.655829][ T7832] notifier_call_chain+0xcd/0x150 [ 316.655993][ T7832] blocking_notifier_call_chain+0x66/0x90 [ 316.656159][ T7832] unregister_netdevice_many_notify+0xa7b/0x1580 [ 316.656362][ T7832] ? hlock_class+0x4e/0x130 [ 316.656523][ T7832] ? mark_lock+0x38/0x3e0 [ 316.656634][ T7832] ? __pfx_unregister_netdevice_many_notify+0x10/0x10 [ 316.656819][ T7832] ? lockdep_hardirqs_on_prepare+0x275/0x410 [ 316.657013][ T7832] ? kfree+0xf3/0x330 [ 316.657128][ T7832] ? vlan_vid_del+0x310/0x5e0 [ 316.657287][ T7832] ? vlan_vid_del+0x310/0x5e0 [ 316.657451][ T7832] rtnl_dellink+0x348/0xa40 [ 316.657608][ T7832] ? __pfx_rtnl_dellink+0x10/0x10 [ 316.657778][ T7832] ? find_held_lock+0x2c/0x110 [ 316.657938][ T7832] ? __lock_release+0x103/0x460 [ 316.658101][ T7832] ? rtnetlink_rcv_msg+0x6ef/0xc10 [ 316.658260][ T7832] ? __pfx___lock_release+0x10/0x10 [ 316.658418][ T7832] ? __pfx_lock_acquire.part.0+0x10/0x10 [ 316.658583][ T7832] ? rtnetlink_rcv_msg+0x6ef/0xc10 [ 316.658740][ T7832] ? __pfx_rtnl_dellink+0x10/0x10 [ 316.658901][ T7832] rtnetlink_rcv_msg+0x712/0xc10 [ 316.659058][ T7832] ? __pfx_rtnetlink_rcv_msg+0x10/0x10 [ 316.659217][ T7832] ? hlock_class+0x4e/0x130 [ 316.659373][ T7832] ? mark_lock+0x38/0x3e0 [ 316.659493][ T7832] ? __lock_acquire+0xb9a/0x1680 [ 316.659652][ T7832] netlink_rcv_skb+0x130/0x360 [ 316.659807][ T7832] ? __pfx_rtnetlink_rcv_msg+0x10/0x10 [ 316.659975][ T7832] ? __pfx_netlink_rcv_skb+0x10/0x10 [ 316.660135][ T7832] ? netlink_deliver_tap+0x13e/0x340 [ 316.660299][ T7832] ? netlink_deliver_tap+0xc3/0x340 [ 316.660463][ T7832] netlink_unicast+0x44b/0x710 [ 316.660612][ T7832] ? __pfx_netlink_unicast+0x10/0x10 [ 316.660762][ T7832] ? find_held_lock+0x2c/0x110 [ 316.660911][ T7832] netlink_sendmsg+0x723/0xbe0 [ 316.661062][ T7832] ? __pfx_netlink_sendmsg+0x10/0x10 [ 316.661220][ T7832] ____sys_sendmsg+0x7ac/0xa10 [ 316.661374][ T7832] ? __pfx_____sys_sendmsg+0x10/0x10 [ 316.661527][ T7832] ? __pfx_copy_msghdr_from_user+0x10/0x10 [ 316.661725][ T7832] ___sys_sendmsg+0xee/0x170 [ 316.661884][ T7832] ? __pfx____sys_sendmsg+0x10/0x10 [ 316.662036][ T7832] ? kasan_save_stack+0x34/0x50 [ 316.662191][ T7832] ? kasan_save_stack+0x24/0x50 [ 316.662345][ T7832] ? __kasan_record_aux_stack+0x8e/0xa0 [ 316.662502][ T7832] ? __call_rcu_common.constprop.0+0xa1/0x4b0 [ 316.662698][ T7832] ? __x64_sys_close+0x7c/0xd0 [ 316.662853][ T7832] ? do_syscall_64+0xc1/0x1d0 [ 316.663007][ T7832] ? entry_SYSCALL_64_after_hwframe+0x77/0x7f [ 316.663199][ T7832] ? __lock_acquire+0xb9a/0x1680 [ 316.663353][ T7832] ? find_held_lock+0x2c/0x110 [ 316.663508][ T7832] ? __lock_release+0x103/0x460 [ 316.663662][ T7832] ? __virt_addr_valid+0x22b/0x430 [ 316.663816][ T7832] ? __pfx___lock_release+0x10/0x10 [ 316.663964][ T7832] ? __pfx_lock_acquire.part.0+0x10/0x10 [ 316.664123][ T7832] __sys_sendmsg+0x109/0x1a0 [ 316.664275][ T7832] ? __pfx___sys_sendmsg+0x10/0x10 [ 316.664447][ T7832] ? __pfx_slab_free_after_rcu_debug+0x10/0x10 [ 316.664644][ T7832] do_syscall_64+0xc1/0x1d0 [ 316.664795][ T7832] entry_SYSCALL_64_after_hwframe+0x77/0x7f [ 316.664997][ T7832] RIP: 0033:0x7fdb005f19a7 [ 316.665160][ T7832] Code: 0a 00 f7 d8 64 89 02 48 c7 c0 ff ff ff ff eb b9 0f 1f 00 f3 0f 1e fa 64 8b 04 25 18 00 00 00 85 c0 75 10 b8 2e 00 00 00 0f 05 <48> 3d 00 f0 ff ff 77 51 c3 48 83 ec 28 89 54 24 1c 48 89 74 24 10 [ 316.665706][ T7832] RSP: 002b:00007fffb8741b68 EFLAGS: 00000246 ORIG_RAX: 000000000000002e [ 316.665930][ T7832] RAX: ffffffffffffffda RBX: 00007fffb8742290 RCX: 00007fdb005f19a7 [ 316.666163][ T7832] RDX: 0000000000000000 RSI: 00007fffb8741bd0 RDI: 0000000000000005 [ 316.666387][ T7832] RBP: 0000000000000002 R08: 0000000000000003 R09: 0000000000000078 [ 316.666608][ T7832] R10: 00007fdb004afef8 R11: 0000000000000246 R12: 0000000000000002 [ 316.666849][ T7832] R13: 0000000067794fbf R14: 0000000000498600 R15: 0000000000000000 [ 316.667083][ T7832] [ 316.702515][ T7832] ip (7832) used greatest stack depth: 23808 bytes left