[ 11.039495][ T250] ip (250) used greatest stack depth: 24320 bytes left [ 11.792084][ T273] ip (273) used greatest stack depth: 23136 bytes left [ 14.040821][ T301] 8021q: 802.1Q VLAN Support v1.8 [ 17.324031][ T344] br1: port 1(vx100) entered blocking state [ 17.324801][ T344] br1: port 1(vx100) entered disabled state [ 17.325230][ T344] vx100: entered allmulticast mode [ 17.327502][ T344] vx100: entered promiscuous mode [ 17.328511][ T344] br1: port 1(vx100) entered blocking state [ 17.328873][ T344] br1: port 1(vx100) entered forwarding state [ 17.501759][ T346] br1: port 2(veth1) entered blocking state [ 17.502143][ T346] br1: port 2(veth1) entered disabled state [ 17.502431][ T346] veth1: entered allmulticast mode [ 17.505130][ T346] veth1: entered promiscuous mode [ 17.594706][ T69] br1: port 2(veth1) entered blocking state [ 17.595224][ T69] br1: port 2(veth1) entered forwarding state [ 17.946171][ T352] br2: port 1(vx200) entered blocking state [ 17.946516][ T352] br2: port 1(vx200) entered disabled state [ 17.946862][ T352] vx200: entered allmulticast mode [ 17.948969][ T352] vx200: entered promiscuous mode [ 17.950263][ T352] br2: port 1(vx200) entered blocking state [ 17.950609][ T352] br2: port 1(vx200) entered forwarding state [ 18.220601][ T355] br2: port 2(veth2.20) entered blocking state [ 18.220898][ T355] br2: port 2(veth2.20) entered disabled state [ 18.221198][ T355] veth2.20: entered allmulticast mode [ 18.230287][ T355] veth2.20: entered promiscuous mode [ 18.327396][ T357] veth2: entered allmulticast mode [ 18.327650][ T357] veth2: entered promiscuous mode [ 18.329346][ T357] br2: port 2(veth2.20) entered blocking state [ 18.329758][ T357] br2: port 2(veth2.20) entered forwarding state [ 21.593421][ T402] br3: port 1(w1) entered blocking state [ 21.593687][ T402] br3: port 1(w1) entered disabled state [ 21.593938][ T402] w1: entered allmulticast mode [ 21.595917][ T402] w1: entered promiscuous mode [ 22.099476][ T408] br3: port 2(vx100) entered blocking state [ 22.099821][ T408] br3: port 2(vx100) entered disabled state [ 22.100148][ T408] vx100: entered allmulticast mode [ 22.102451][ T408] vx100: entered promiscuous mode [ 22.103328][ T408] br3: port 2(vx100) entered blocking state [ 22.103618][ T408] br3: port 2(vx100) entered forwarding state [ 22.756802][ T37] br3: port 1(w1) entered blocking state [ 22.757358][ T37] br3: port 1(w1) entered forwarding state [ 25.333542][ T456] br3: port 1(w1) entered blocking state [ 25.333782][ T456] br3: port 1(w1) entered disabled state [ 25.334033][ T456] w1: entered allmulticast mode [ 25.336294][ T456] w1: entered promiscuous mode [ 25.850731][ T462] br3: port 2(vx200) entered blocking state [ 25.851075][ T462] br3: port 2(vx200) entered disabled state [ 25.851415][ T462] vx200: entered allmulticast mode [ 25.853452][ T462] vx200: entered promiscuous mode [ 25.854354][ T462] br3: port 2(vx200) entered blocking state [ 25.854637][ T462] br3: port 2(vx200) entered forwarding state [ 26.488874][ T37] br3: port 1(w1) entered blocking state [ 26.489148][ T37] br3: port 1(w1) entered forwarding state [ 28.200904][ T501] br3: port 3(w1.20) entered blocking state [ 28.201224][ T501] br3: port 3(w1.20) entered disabled state [ 28.201523][ T501] w1.20: entered allmulticast mode [ 28.203499][ T501] w1.20: entered promiscuous mode [ 28.928298][ T514] br3: port 3(w1.20) entered blocking state [ 28.928679][ T514] br3: port 3(w1.20) entered forwarding state [ 39.247675][ T11] w1.20: left allmulticast mode [ 39.248086][ T11] w1.20: left promiscuous mode [ 39.248624][ T11] br3: port 3(w1.20) entered disabled state [ 39.252210][ T11] vx200: left allmulticast mode [ 39.252611][ T11] vx200: left promiscuous mode [ 39.252959][ T11] br3: port 2(vx200) entered disabled state [ 39.255002][ T11] w1: left allmulticast mode [ 39.255247][ T11] w1: left promiscuous mode [ 39.255570][ T11] br3: port 1(w1) entered disabled state [ 39.690631][ T11] vx100: left allmulticast mode [ 39.690856][ T11] vx100: left promiscuous mode [ 39.691189][ T11] br3: port 2(vx100) entered disabled state [ 39.692682][ T11] w1: left allmulticast mode [ 39.692880][ T11] w1: left promiscuous mode [ 39.693200][ T11] br3: port 1(w1) entered disabled state [ 41.728963][ T612] vx200: left allmulticast mode [ 41.729673][ T612] vx200: left promiscuous mode [ 41.730022][ T612] br2: port 1(vx200) entered disabled state [ 42.004339][ T615] veth2: left allmulticast mode [ 42.004691][ T615] veth2: left promiscuous mode [ 42.007914][ T615] br2: port 2(veth2.20) entered disabled state [ 42.032251][ T615] veth2.20 (unregistering): left allmulticast mode [ 42.033075][ T615] veth2.20 (unregistering): left promiscuous mode [ 42.033367][ T615] br2: port 2(veth2.20) entered disabled state [ 42.034321][ T615] [ 42.034419][ T615] ============================================ [ 42.034616][ T615] WARNING: possible recursive locking detected [ 42.034815][ T615] 6.13.0-rc5-virtme #1 Not tainted [ 42.034984][ T615] -------------------------------------------- [ 42.035181][ T615] ip/615 is trying to acquire lock: [ 42.035342][ T615] ffffffffa59d6630 ((netdev_chain).rwsem){++++}-{4:4}, at: blocking_notifier_call_chain+0x50/0x90 [ 42.035673][ T615] [ 42.035673][ T615] but task is already holding lock: [ 42.035913][ T615] ffffffffa59d6630 ((netdev_chain).rwsem){++++}-{4:4}, at: blocking_notifier_call_chain+0x50/0x90 [ 42.036237][ T615] [ 42.036237][ T615] other info that might help us debug this: [ 42.036465][ T615] Possible unsafe locking scenario: [ 42.036465][ T615] [ 42.036692][ T615] CPU0 [ 42.036813][ T615] ---- [ 42.036931][ T615] lock((netdev_chain).rwsem); [ 42.037098][ T615] lock((netdev_chain).rwsem); [ 42.037259][ T615] [ 42.037259][ T615] *** DEADLOCK *** [ 42.037259][ T615] [ 42.037497][ T615] May be due to missing lock nesting notation [ 42.037497][ T615] [ 42.037726][ T615] 2 locks held by ip/615: [ 42.037846][ T615] #0: ffffffffa59e7a48 (rtnl_mutex){+.+.}-{4:4}, at: rtnl_dellink+0x228/0xa40 [ 42.038140][ T615] #1: ffffffffa59d6630 ((netdev_chain).rwsem){++++}-{4:4}, at: blocking_notifier_call_chain+0x50/0x90 [ 42.038463][ T615] [ 42.038463][ T615] stack backtrace: [ 42.038659][ T615] CPU: 2 UID: 0 PID: 615 Comm: ip Not tainted 6.13.0-rc5-virtme #1 [ 42.038905][ T615] Hardware name: Bochs Bochs, BIOS Bochs 01/01/2011 [ 42.039109][ T615] Call Trace: [ 42.039234][ T615] <TASK> [ 42.039325][ T615] dump_stack_lvl+0x82/0xd0 [ 42.039505][ T615] print_deadlock_bug+0x40a/0x650 [ 42.039669][ T615] validate_chain+0x5bf/0xae0 [ 42.039833][ T615] ? __pfx_validate_chain+0x10/0x10 [ 42.039993][ T615] ? hlock_class+0x4e/0x130 [ 42.040153][ T615] ? mark_lock+0x38/0x3e0 [ 42.040281][ T615] __lock_acquire+0xb9a/0x1680 [ 42.040443][ T615] ? hlock_class+0x4e/0x130 [ 42.040602][ T615] lock_acquire.part.0+0xeb/0x330 [ 42.040760][ T615] ? blocking_notifier_call_chain+0x50/0x90 [ 42.040956][ T615] ? validate_chain+0x5a7/0xae0 [ 42.041119][ T615] ? __pfx_lock_acquire.part.0+0x10/0x10 [ 42.041277][ T615] ? trace_lock_acquire+0x14c/0x1f0 [ 42.041436][ T615] ? lock_acquire+0x32/0xc0 [ 42.041592][ T615] ? blocking_notifier_call_chain+0x50/0x90 [ 42.041787][ T615] down_read+0x9f/0x340 [ 42.041906][ T615] ? blocking_notifier_call_chain+0x50/0x90 [ 42.042101][ T615] ? __pfx_down_read+0x10/0x10 [ 42.042267][ T615] ? __lock_acquire+0xb9a/0x1680 [ 42.042425][ T615] blocking_notifier_call_chain+0x50/0x90 [ 42.042583][ T615] __netdev_upper_dev_unlink+0xd9/0x220 [ 42.042743][ T615] ? br_fdb_delete_by_port+0x36/0x260 [bridge] [ 42.042978][ T615] ? __pfx___netdev_upper_dev_unlink+0x10/0x10 [ 42.043173][ T615] ? find_held_lock+0x2c/0x110 [ 42.043334][ T615] ? del_nbp+0x377/0xbe0 [bridge] [ 42.043532][ T615] ? hlock_class+0x4e/0x130 [ 42.043689][ T615] netdev_upper_dev_unlink+0x71/0xa0 [ 42.043846][ T615] ? __pfx_netdev_upper_dev_unlink+0x10/0x10 [ 42.044040][ T615] ? mutex_spin_on_owner+0x1f2/0x2b0 [ 42.044196][ T615] ? rtnl_is_locked+0x15/0x20 [ 42.044358][ T615] del_nbp+0x602/0xbe0 [bridge] [ 42.044551][ T615] br_del_if+0xa3/0x1f0 [bridge] [ 42.044750][ T615] br_device_event+0x53d/0x800 [bridge] [ 42.044949][ T615] ? __pfx_br_device_event+0x10/0x10 [bridge] [ 42.045177][ T615] ? netconsole_netdev_event+0x2c1/0x620 [ 42.045338][ T615] ? packet_notifier+0x3b0/0x810 [ 42.045501][ T615] notifier_call_chain+0xcd/0x150 [ 42.045656][ T615] blocking_notifier_call_chain+0x66/0x90 [ 42.045812][ T615] unregister_netdevice_many_notify+0xa7b/0x1580 [ 42.046013][ T615] ? __lock_release+0x103/0x460 [ 42.046168][ T615] ? __pfx___lock_release+0x10/0x10 [ 42.046325][ T615] ? __pfx_unregister_netdevice_many_notify+0x10/0x10 [ 42.046521][ T615] ? hlock_class+0x4e/0x130 [ 42.046680][ T615] ? mark_held_locks+0x9e/0xe0 [ 42.046838][ T615] ? __pfx_vlan_info_rcu_free+0x10/0x10 [ 42.046996][ T615] ? lockdep_hardirqs_on_prepare+0x275/0x410 [ 42.047200][ T615] rtnl_dellink+0x348/0xa40 [ 42.047359][ T615] ? __pfx_rtnl_dellink+0x10/0x10 [ 42.047531][ T615] ? find_held_lock+0x2c/0x110 [ 42.047689][ T615] ? __lock_release+0x103/0x460 [ 42.047846][ T615] ? rtnetlink_rcv_msg+0x6ef/0xc10 [ 42.048004][ T615] ? __pfx___lock_release+0x10/0x10 [ 42.048159][ T615] ? __pfx_lock_acquire.part.0+0x10/0x10 [ 42.048325][ T615] ? rtnetlink_rcv_msg+0x6ef/0xc10 [ 42.048482][ T615] ? __pfx_rtnl_dellink+0x10/0x10 [ 42.048640][ T615] rtnetlink_rcv_msg+0x712/0xc10 [ 42.048796][ T615] ? __pfx_rtnetlink_rcv_msg+0x10/0x10 [ 42.048953][ T615] ? hlock_class+0x4e/0x130 [ 42.049108][ T615] ? mark_lock+0x38/0x3e0 [ 42.049227][ T615] ? __lock_acquire+0xb9a/0x1680 [ 42.049389][ T615] netlink_rcv_skb+0x130/0x360 [ 42.049547][ T615] ? __pfx_rtnetlink_rcv_msg+0x10/0x10 [ 42.049702][ T615] ? __pfx_netlink_rcv_skb+0x10/0x10 [ 42.049863][ T615] ? netlink_deliver_tap+0x13e/0x340 [ 42.050021][ T615] ? netlink_deliver_tap+0xc3/0x340 [ 42.050178][ T615] netlink_unicast+0x44b/0x710 [ 42.050333][ T615] ? __pfx_netlink_unicast+0x10/0x10 [ 42.050495][ T615] ? find_held_lock+0x2c/0x110 [ 42.050653][ T615] netlink_sendmsg+0x723/0xbe0 [ 42.050812][ T615] ? __pfx_netlink_sendmsg+0x10/0x10 [ 42.050971][ T615] ____sys_sendmsg+0x7ac/0xa10 [ 42.051130][ T615] ? __pfx_____sys_sendmsg+0x10/0x10 [ 42.051284][ T615] ? __pfx_copy_msghdr_from_user+0x10/0x10 [ 42.051480][ T615] ___sys_sendmsg+0xee/0x170 [ 42.051645][ T615] ? __pfx____sys_sendmsg+0x10/0x10 [ 42.051802][ T615] ? kasan_save_stack+0x34/0x50 [ 42.051963][ T615] ? kasan_save_stack+0x24/0x50 [ 42.052117][ T615] ? __kasan_record_aux_stack+0x8e/0xa0 [ 42.052275][ T615] ? __call_rcu_common.constprop.0+0xa1/0x4b0 [ 42.052475][ T615] ? __x64_sys_close+0x7c/0xd0 [ 42.052642][ T615] ? do_syscall_64+0xc1/0x1d0 [ 42.052800][ T615] ? entry_SYSCALL_64_after_hwframe+0x77/0x7f [ 42.053004][ T615] ? __lock_acquire+0xb9a/0x1680 [ 42.053164][ T615] ? find_held_lock+0x2c/0x110 [ 42.053322][ T615] ? __lock_release+0x103/0x460 [ 42.053479][ T615] ? __virt_addr_valid+0x22b/0x430 [ 42.053638][ T615] ? __pfx___lock_release+0x10/0x10 [ 42.053794][ T615] ? __pfx_lock_acquire.part.0+0x10/0x10 [ 42.053952][ T615] __sys_sendmsg+0x109/0x1a0 [ 42.054114][ T615] ? __pfx___sys_sendmsg+0x10/0x10 [ 42.054277][ T615] ? __pfx_slab_free_after_rcu_debug+0x10/0x10 [ 42.054481][ T615] do_syscall_64+0xc1/0x1d0 [ 42.054638][ T615] entry_SYSCALL_64_after_hwframe+0x77/0x7f [ 42.054831][ T615] RIP: 0033:0x7fbb1d4aa9a7 [ 42.055007][ T615] Code: 0a 00 f7 d8 64 89 02 48 c7 c0 ff ff ff ff eb b9 0f 1f 00 f3 0f 1e fa 64 8b 04 25 18 00 00 00 85 c0 75 10 b8 2e 00 00 00 0f 05 <48> 3d 00 f0 ff ff 77 51 c3 48 83 ec 28 89 54 24 1c 48 89 74 24 10 [ 42.055565][ T615] RSP: 002b:00007ffd7cd48568 EFLAGS: 00000246 ORIG_RAX: 000000000000002e [ 42.055803][ T615] RAX: ffffffffffffffda RBX: 00007ffd7cd48c90 RCX: 00007fbb1d4aa9a7 [ 42.056039][ T615] RDX: 0000000000000000 RSI: 00007ffd7cd485d0 RDI: 0000000000000005 [ 42.056288][ T615] RBP: 0000000000000002 R08: 0000000000000003 R09: 0000000000000078 [ 42.056525][ T615] R10: 00007fbb1d368ef8 R11: 0000000000000246 R12: 0000000000000002 [ 42.056760][ T615] R13: 0000000067792b9d R14: 0000000000498600 R15: 0000000000000000 [ 42.056999][ T615] </TASK> [ 42.298259][ T619] br1: port 2(veth1) entered disabled state [ 42.356124][ T621] veth1: left allmulticast mode [ 42.356370][ T621] veth1: left promiscuous mode [ 42.356653][ T621] br1: port 2(veth1) entered disabled state [ 42.419416][ T622] vx100: left allmulticast mode [ 42.419695][ T622] vx100: left promiscuous mode [ 42.419967][ T622] br1: port 1(vx100) entered disabled state