[ 11.009056][ T259] ip (259) used greatest stack depth: 23552 bytes left [ 11.371728][ T273] ip (273) used greatest stack depth: 23544 bytes left [ 13.485191][ T301] 8021q: 802.1Q VLAN Support v1.8 [ 16.531073][ T344] br1: port 1(vx100) entered blocking state [ 16.531528][ T344] br1: port 1(vx100) entered disabled state [ 16.531912][ T344] vx100: entered allmulticast mode [ 16.534969][ T344] vx100: entered promiscuous mode [ 16.542712][ T344] br1: port 1(vx100) entered blocking state [ 16.543261][ T344] br1: port 1(vx100) entered forwarding state [ 16.718800][ T346] br1: port 2(veth1) entered blocking state [ 16.719220][ T346] br1: port 2(veth1) entered disabled state [ 16.719688][ T346] veth1: entered allmulticast mode [ 16.722895][ T346] veth1: entered promiscuous mode [ 16.814888][ T40] br1: port 2(veth1) entered blocking state [ 16.815491][ T40] br1: port 2(veth1) entered forwarding state [ 17.173022][ T353] br2: port 1(vx200) entered blocking state [ 17.173391][ T353] br2: port 1(vx200) entered disabled state [ 17.173846][ T353] vx200: entered allmulticast mode [ 17.176036][ T353] vx200: entered promiscuous mode [ 17.177475][ T353] br2: port 1(vx200) entered blocking state [ 17.178027][ T353] br2: port 1(vx200) entered forwarding state [ 17.471498][ T356] br2: port 2(veth2.20) entered blocking state [ 17.471774][ T356] br2: port 2(veth2.20) entered disabled state [ 17.472044][ T356] veth2.20: entered allmulticast mode [ 17.474210][ T356] veth2.20: entered promiscuous mode [ 17.565198][ T358] veth2: entered allmulticast mode [ 17.565511][ T358] veth2: entered promiscuous mode [ 17.567046][ T358] br2: port 2(veth2.20) entered blocking state [ 17.567507][ T358] br2: port 2(veth2.20) entered forwarding state [ 20.609658][ T402] br3: port 1(w1) entered blocking state [ 20.610025][ T402] br3: port 1(w1) entered disabled state [ 20.610275][ T402] w1: entered allmulticast mode [ 20.612651][ T402] w1: entered promiscuous mode [ 21.108562][ T408] br3: port 2(vx100) entered blocking state [ 21.108878][ T408] br3: port 2(vx100) entered disabled state [ 21.109163][ T408] vx100: entered allmulticast mode [ 21.111086][ T408] vx100: entered promiscuous mode [ 21.111927][ T408] br3: port 2(vx100) entered blocking state [ 21.112184][ T408] br3: port 2(vx100) entered forwarding state [ 21.731505][ T70] br3: port 1(w1) entered blocking state [ 21.732094][ T70] br3: port 1(w1) entered forwarding state [ 24.209690][ T456] br3: port 1(w1) entered blocking state [ 24.210190][ T456] br3: port 1(w1) entered disabled state [ 24.210513][ T456] w1: entered allmulticast mode [ 24.212471][ T456] w1: entered promiscuous mode [ 24.665721][ T462] br3: port 2(vx200) entered blocking state [ 24.666075][ T462] br3: port 2(vx200) entered disabled state [ 24.666358][ T462] vx200: entered allmulticast mode [ 24.668215][ T462] vx200: entered promiscuous mode [ 24.669207][ T462] br3: port 2(vx200) entered blocking state [ 24.669493][ T462] br3: port 2(vx200) entered forwarding state [ 25.262003][ T37] br3: port 1(w1) entered blocking state [ 25.262527][ T37] br3: port 1(w1) entered forwarding state [ 26.901880][ T501] br3: port 3(w1.20) entered blocking state [ 26.902168][ T501] br3: port 3(w1.20) entered disabled state [ 26.902883][ T501] w1.20: entered allmulticast mode [ 26.904840][ T501] w1.20: entered promiscuous mode [ 27.504794][ T514] br3: port 3(w1.20) entered blocking state [ 27.505121][ T514] br3: port 3(w1.20) entered forwarding state [ 37.558725][ T11] w1.20: left allmulticast mode [ 37.559027][ T11] w1.20: left promiscuous mode [ 37.559605][ T11] br3: port 3(w1.20) entered disabled state [ 37.562548][ T11] vx200: left allmulticast mode [ 37.562774][ T11] vx200: left promiscuous mode [ 37.563073][ T11] br3: port 2(vx200) entered disabled state [ 37.564863][ T11] w1: left allmulticast mode [ 37.565100][ T11] w1: left promiscuous mode [ 37.565978][ T11] br3: port 1(w1) entered disabled state [ 38.033776][ T11] vx100: left allmulticast mode [ 38.034040][ T11] vx100: left promiscuous mode [ 38.034370][ T11] br3: port 2(vx100) entered disabled state [ 38.035938][ T11] w1: left allmulticast mode [ 38.036150][ T11] w1: left promiscuous mode [ 38.036488][ T11] br3: port 1(w1) entered disabled state [ 39.855094][ T612] vx200: left allmulticast mode [ 39.855560][ T612] vx200: left promiscuous mode [ 39.855885][ T612] br2: port 1(vx200) entered disabled state [ 40.150759][ T615] veth2: left allmulticast mode [ 40.151068][ T615] veth2: left promiscuous mode [ 40.153399][ T615] br2: port 2(veth2.20) entered disabled state [ 40.174564][ T615] veth2.20 (unregistering): left allmulticast mode [ 40.175038][ T615] veth2.20 (unregistering): left promiscuous mode [ 40.175370][ T615] br2: port 2(veth2.20) entered disabled state [ 40.176340][ T615] [ 40.176467][ T615] ============================================ [ 40.176702][ T615] WARNING: possible recursive locking detected [ 40.176931][ T615] 6.13.0-rc5-virtme #1 Not tainted [ 40.177126][ T615] -------------------------------------------- [ 40.177360][ T615] ip/615 is trying to acquire lock: [ 40.177545][ T615] ffffffff95fd6630 ((netdev_chain).rwsem){++++}-{4:4}, at: blocking_notifier_call_chain+0x50/0x90 [ 40.177929][ T615] [ 40.177929][ T615] but task is already holding lock: [ 40.178203][ T615] ffffffff95fd6630 ((netdev_chain).rwsem){++++}-{4:4}, at: blocking_notifier_call_chain+0x50/0x90 [ 40.178577][ T615] [ 40.178577][ T615] other info that might help us debug this: [ 40.178848][ T615] Possible unsafe locking scenario: [ 40.178848][ T615] [ 40.179118][ T615] CPU0 [ 40.179260][ T615] ---- [ 40.179400][ T615] lock((netdev_chain).rwsem); [ 40.179585][ T615] lock((netdev_chain).rwsem); [ 40.179773][ T615] [ 40.179773][ T615] *** DEADLOCK *** [ 40.179773][ T615] [ 40.180045][ T615] May be due to missing lock nesting notation [ 40.180045][ T615] [ 40.180320][ T615] 2 locks held by ip/615: [ 40.180464][ T615] #0: ffffffff95fe7a48 (rtnl_mutex){+.+.}-{4:4}, at: rtnl_dellink+0x228/0xa40 [ 40.180799][ T615] #1: ffffffff95fd6630 ((netdev_chain).rwsem){++++}-{4:4}, at: blocking_notifier_call_chain+0x50/0x90 [ 40.181177][ T615] [ 40.181177][ T615] stack backtrace: [ 40.181413][ T615] CPU: 2 UID: 0 PID: 615 Comm: ip Not tainted 6.13.0-rc5-virtme #1 [ 40.181701][ T615] Hardware name: Bochs Bochs, BIOS Bochs 01/01/2011 [ 40.181939][ T615] Call Trace: [ 40.182083][ T615] [ 40.182180][ T615] dump_stack_lvl+0x82/0xd0 [ 40.182376][ T615] print_deadlock_bug+0x40a/0x650 [ 40.182579][ T615] validate_chain+0x5bf/0xae0 [ 40.182779][ T615] ? __pfx_validate_chain+0x10/0x10 [ 40.182968][ T615] ? hlock_class+0x4e/0x130 [ 40.183156][ T615] ? mark_lock+0x38/0x3e0 [ 40.183299][ T615] __lock_acquire+0xb9a/0x1680 [ 40.183487][ T615] ? hlock_class+0x4e/0x130 [ 40.183675][ T615] lock_acquire.part.0+0xeb/0x330 [ 40.183860][ T615] ? blocking_notifier_call_chain+0x50/0x90 [ 40.184094][ T615] ? validate_chain+0x5a7/0xae0 [ 40.184286][ T615] ? __pfx_lock_acquire.part.0+0x10/0x10 [ 40.184475][ T615] ? trace_lock_acquire+0x14c/0x1f0 [ 40.184664][ T615] ? lock_acquire+0x32/0xc0 [ 40.184849][ T615] ? blocking_notifier_call_chain+0x50/0x90 [ 40.185083][ T615] down_read+0x9f/0x340 [ 40.185226][ T615] ? blocking_notifier_call_chain+0x50/0x90 [ 40.185460][ T615] ? __pfx_down_read+0x10/0x10 [ 40.185652][ T615] ? __lock_acquire+0xb9a/0x1680 [ 40.185842][ T615] blocking_notifier_call_chain+0x50/0x90 [ 40.186035][ T615] __netdev_upper_dev_unlink+0xd9/0x220 [ 40.186223][ T615] ? br_fdb_delete_by_port+0x36/0x260 [bridge] [ 40.186498][ T615] ? __pfx___netdev_upper_dev_unlink+0x10/0x10 [ 40.186734][ T615] ? find_held_lock+0x2c/0x110 [ 40.186925][ T615] ? del_nbp+0x377/0xbe0 [bridge] [ 40.187158][ T615] ? hlock_class+0x4e/0x130 [ 40.187347][ T615] netdev_upper_dev_unlink+0x71/0xa0 [ 40.187536][ T615] ? __pfx_netdev_upper_dev_unlink+0x10/0x10 [ 40.187768][ T615] ? mutex_spin_on_owner+0x1f2/0x2b0 [ 40.187956][ T615] ? rtnl_is_locked+0x15/0x20 [ 40.188147][ T615] del_nbp+0x602/0xbe0 [bridge] [ 40.188372][ T615] br_del_if+0xa3/0x1f0 [bridge] [ 40.188602][ T615] br_device_event+0x53d/0x800 [bridge] [ 40.188823][ T615] ? __pfx_br_device_event+0x10/0x10 [bridge] [ 40.189087][ T615] ? netconsole_netdev_event+0x2c1/0x620 [ 40.189277][ T615] ? packet_notifier+0x3b0/0x810 [ 40.189467][ T615] notifier_call_chain+0xcd/0x150 [ 40.189657][ T615] blocking_notifier_call_chain+0x66/0x90 [ 40.189848][ T615] unregister_netdevice_many_notify+0xa7b/0x1580 [ 40.190085][ T615] ? __lock_release+0x103/0x460 [ 40.190273][ T615] ? __pfx___lock_release+0x10/0x10 [ 40.190460][ T615] ? __pfx_unregister_netdevice_many_notify+0x10/0x10 [ 40.190699][ T615] ? hlock_class+0x4e/0x130 [ 40.190887][ T615] ? mark_held_locks+0x9e/0xe0 [ 40.191075][ T615] ? __pfx_vlan_info_rcu_free+0x10/0x10 [ 40.191264][ T615] ? lockdep_hardirqs_on_prepare+0x275/0x410 [ 40.191505][ T615] rtnl_dellink+0x348/0xa40 [ 40.191699][ T615] ? __pfx_rtnl_dellink+0x10/0x10 [ 40.191898][ T615] ? find_held_lock+0x2c/0x110 [ 40.192086][ T615] ? __lock_release+0x103/0x460 [ 40.192273][ T615] ? rtnetlink_rcv_msg+0x6ef/0xc10 [ 40.192468][ T615] ? __pfx___lock_release+0x10/0x10 [ 40.192656][ T615] ? __pfx_lock_acquire.part.0+0x10/0x10 [ 40.192852][ T615] ? rtnetlink_rcv_msg+0x6ef/0xc10 [ 40.193040][ T615] ? __pfx_rtnl_dellink+0x10/0x10 [ 40.193228][ T615] rtnetlink_rcv_msg+0x712/0xc10 [ 40.193415][ T615] ? __pfx_rtnetlink_rcv_msg+0x10/0x10 [ 40.193603][ T615] ? hlock_class+0x4e/0x130 [ 40.193788][ T615] ? mark_lock+0x38/0x3e0 [ 40.193932][ T615] ? __lock_acquire+0xb9a/0x1680 [ 40.194119][ T615] netlink_rcv_skb+0x130/0x360 [ 40.194318][ T615] ? __pfx_rtnetlink_rcv_msg+0x10/0x10 [ 40.194505][ T615] ? __pfx_netlink_rcv_skb+0x10/0x10 [ 40.194696][ T615] ? netlink_deliver_tap+0x13e/0x340 [ 40.194884][ T615] ? netlink_deliver_tap+0xc3/0x340 [ 40.195074][ T615] netlink_unicast+0x44b/0x710 [ 40.195263][ T615] ? __pfx_netlink_unicast+0x10/0x10 [ 40.195459][ T615] ? find_held_lock+0x2c/0x110 [ 40.195648][ T615] netlink_sendmsg+0x723/0xbe0 [ 40.195839][ T615] ? __pfx_netlink_sendmsg+0x10/0x10 [ 40.196030][ T615] ____sys_sendmsg+0x7ac/0xa10 [ 40.196218][ T615] ? __pfx_____sys_sendmsg+0x10/0x10 [ 40.196406][ T615] ? __pfx_copy_msghdr_from_user+0x10/0x10 [ 40.196644][ T615] ___sys_sendmsg+0xee/0x170 [ 40.196835][ T615] ? __pfx____sys_sendmsg+0x10/0x10 [ 40.197026][ T615] ? kasan_save_stack+0x34/0x50 [ 40.197215][ T615] ? kasan_save_stack+0x24/0x50 [ 40.197404][ T615] ? __kasan_record_aux_stack+0x8e/0xa0 [ 40.197593][ T615] ? __call_rcu_common.constprop.0+0xa1/0x4b0 [ 40.197830][ T615] ? __x64_sys_close+0x7c/0xd0 [ 40.198017][ T615] ? do_syscall_64+0xc1/0x1d0 [ 40.198205][ T615] ? entry_SYSCALL_64_after_hwframe+0x77/0x7f [ 40.198443][ T615] ? __lock_acquire+0xb9a/0x1680 [ 40.198632][ T615] ? find_held_lock+0x2c/0x110 [ 40.198821][ T615] ? __lock_release+0x103/0x460 [ 40.199009][ T615] ? __virt_addr_valid+0x22b/0x430 [ 40.199199][ T615] ? __pfx___lock_release+0x10/0x10 [ 40.199386][ T615] ? __pfx_lock_acquire.part.0+0x10/0x10 [ 40.199578][ T615] __sys_sendmsg+0x109/0x1a0 [ 40.199774][ T615] ? __pfx___sys_sendmsg+0x10/0x10 [ 40.199962][ T615] ? __pfx_slab_free_after_rcu_debug+0x10/0x10 [ 40.200203][ T615] do_syscall_64+0xc1/0x1d0 [ 40.200391][ T615] entry_SYSCALL_64_after_hwframe+0x77/0x7f [ 40.200627][ T615] RIP: 0033:0x7f205b5ff9a7 [ 40.200822][ T615] Code: 0a 00 f7 d8 64 89 02 48 c7 c0 ff ff ff ff eb b9 0f 1f 00 f3 0f 1e fa 64 8b 04 25 18 00 00 00 85 c0 75 10 b8 2e 00 00 00 0f 05 <48> 3d 00 f0 ff ff 77 51 c3 48 83 ec 28 89 54 24 1c 48 89 74 24 10 [ 40.201488][ T615] RSP: 002b:00007ffe5443b8c8 EFLAGS: 00000246 ORIG_RAX: 000000000000002e [ 40.201768][ T615] RAX: ffffffffffffffda RBX: 00007ffe5443bff0 RCX: 00007f205b5ff9a7 [ 40.202048][ T615] RDX: 0000000000000000 RSI: 00007ffe5443b930 RDI: 0000000000000005 [ 40.202329][ T615] RBP: 0000000000000002 R08: 0000000000000003 R09: 0000000000000078 [ 40.202616][ T615] R10: 00007f205b4bdef8 R11: 0000000000000246 R12: 0000000000000002 [ 40.202894][ T615] R13: 0000000067792f36 R14: 0000000000498600 R15: 0000000000000000 [ 40.203177][ T615] [ 40.433620][ T619] br1: port 2(veth1) entered disabled state [ 40.494774][ T620] veth1: left allmulticast mode [ 40.494995][ T620] veth1: left promiscuous mode [ 40.495260][ T620] br1: port 2(veth1) entered disabled state [ 40.550489][ T621] vx100: left allmulticast mode [ 40.550689][ T621] vx100: left promiscuous mode [ 40.550945][ T621] br1: port 1(vx100) entered disabled state