[ 15.083253][ T300] 8021q: 802.1Q VLAN Support v1.8 [ 19.277929][ T350] br1: port 1(vx10) entered blocking state [ 19.278604][ T350] br1: port 1(vx10) entered disabled state [ 19.279184][ T350] vx10: entered allmulticast mode [ 19.282218][ T350] vx10: entered promiscuous mode [ 19.283975][ T350] br1: port 1(vx10) entered blocking state [ 19.284563][ T350] br1: port 1(vx10) entered forwarding state [ 19.687357][ T355] br1: port 2(vx20) entered blocking state [ 19.688989][ T355] br1: port 2(vx20) entered disabled state [ 19.689360][ T355] vx20: entered allmulticast mode [ 19.691485][ T355] vx20: entered promiscuous mode [ 19.692414][ T355] br1: port 2(vx20) entered blocking state [ 19.692744][ T355] br1: port 2(vx20) entered forwarding state [ 19.907124][ T357] br1: port 3(veth1) entered blocking state [ 19.907528][ T357] br1: port 3(veth1) entered disabled state [ 19.907870][ T357] veth1: entered allmulticast mode [ 19.910003][ T357] veth1: entered promiscuous mode [ 19.998162][ T151] br1: port 3(veth1) entered blocking state [ 19.998614][ T151] br1: port 3(veth1) entered forwarding state [ 20.352318][ T362] br1: port 4(veth2) entered blocking state [ 20.353763][ T362] br1: port 4(veth2) entered disabled state [ 20.354082][ T362] veth2: entered allmulticast mode [ 20.356084][ T362] veth2: entered promiscuous mode [ 20.436143][ T69] br1: port 4(veth2) entered blocking state [ 20.436548][ T69] br1: port 4(veth2) entered forwarding state [ 24.263265][ T412] br2: port 1(w1) entered blocking state [ 24.263530][ T412] br2: port 1(w1) entered disabled state [ 24.263771][ T412] w1: entered allmulticast mode [ 24.265767][ T412] w1: entered promiscuous mode [ 25.023104][ T420] br2: port 2(vx10) entered blocking state [ 25.023395][ T420] br2: port 2(vx10) entered disabled state [ 25.023712][ T420] vx10: entered allmulticast mode [ 25.026258][ T420] vx10: entered promiscuous mode [ 25.027190][ T420] br2: port 2(vx10) entered blocking state [ 25.027454][ T420] br2: port 2(vx10) entered forwarding state [ 25.630026][ T427] br2: port 3(vx20) entered blocking state [ 25.630341][ T427] br2: port 3(vx20) entered disabled state [ 25.630656][ T427] vx20: entered allmulticast mode [ 25.632595][ T427] vx20: entered promiscuous mode [ 25.633090][ T427] br2: port 3(vx20) entered blocking state [ 25.633352][ T427] br2: port 3(vx20) entered forwarding state [ 26.400274][ T151] br2: port 1(w1) entered blocking state [ 26.400801][ T151] br2: port 1(w1) entered forwarding state [ 29.033219][ T469] br2: port 1(w1) entered blocking state [ 29.033513][ T469] br2: port 1(w1) entered disabled state [ 29.034098][ T469] w1: entered allmulticast mode [ 29.036854][ T469] w1: entered promiscuous mode [ 29.734558][ T477] br2: port 2(vx10) entered blocking state [ 29.734950][ T477] br2: port 2(vx10) entered disabled state [ 29.735237][ T477] vx10: entered allmulticast mode [ 29.737227][ T477] vx10: entered promiscuous mode [ 29.738237][ T477] br2: port 2(vx10) entered blocking state [ 29.738580][ T477] br2: port 2(vx10) entered forwarding state [ 30.415473][ T484] br2: port 3(vx20) entered blocking state [ 30.415834][ T484] br2: port 3(vx20) entered disabled state [ 30.416186][ T484] vx20: entered allmulticast mode [ 30.418299][ T484] vx20: entered promiscuous mode [ 30.418911][ T484] br2: port 3(vx20) entered blocking state [ 30.419227][ T484] br2: port 3(vx20) entered forwarding state [ 31.174102][ T69] br2: port 1(w1) entered blocking state [ 31.174418][ T69] br2: port 1(w1) entered forwarding state [ 42.272684][ T571] GACT probability NOT on [ 172.781552][ T1372] veth3: entered promiscuous mode [ 228.530304][ T2299] veth3: left promiscuous mode [ 229.131868][ T2305] veth3: entered promiscuous mode [ 285.470185][ T3233] veth3: left promiscuous mode [ 286.523885][ T3251] vx20: left allmulticast mode [ 286.524168][ T3251] vx20: left promiscuous mode [ 286.524829][ T3251] br1: port 2(vx20) entered disabled state [ 286.624311][ T3252] vx10: left allmulticast mode [ 286.624635][ T3252] vx10: left promiscuous mode [ 286.625003][ T3252] br1: port 1(vx10) entered disabled state [ 292.368404][ T3257] br1: port 1(vx10) entered blocking state [ 292.369950][ T3257] br1: port 1(vx10) entered disabled state [ 292.372306][ T3257] vx10: entered allmulticast mode [ 292.378215][ T3257] vx10: entered promiscuous mode [ 292.380186][ T3257] br1: port 1(vx10) entered blocking state [ 292.381017][ T3257] br1: port 1(vx10) entered forwarding state [ 293.129070][ T3259] br1: port 2(vx20) entered blocking state [ 293.131692][ T3259] br1: port 2(vx20) entered disabled state [ 293.132673][ T3259] vx20: entered allmulticast mode [ 293.137130][ T3259] vx20: entered promiscuous mode [ 293.139111][ T3259] br1: port 2(vx20) entered blocking state [ 293.140048][ T3259] br1: port 2(vx20) entered forwarding state [ 435.491691][ T4088] veth3: entered promiscuous mode [ 491.529694][ T5015] veth3: left promiscuous mode [ 492.066820][ T5021] veth3: entered promiscuous mode [ 546.926444][ T5949] veth3: left promiscuous mode [ 893.585418][ T67] vx20: left allmulticast mode [ 893.586061][ T67] vx20: left promiscuous mode [ 893.586555][ T67] br2: port 3(vx20) entered disabled state [ 893.589315][ T67] vx10: left allmulticast mode [ 893.589671][ T67] vx10: left promiscuous mode [ 893.589993][ T67] br2: port 2(vx10) entered disabled state [ 893.592524][ T67] w1: left allmulticast mode [ 893.592790][ T67] w1: left promiscuous mode [ 893.593118][ T67] br2: port 1(w1) entered disabled state [ 894.085711][ T67] vx20: left allmulticast mode [ 894.085944][ T67] vx20: left promiscuous mode [ 894.086302][ T67] br2: port 3(vx20) entered disabled state [ 894.088739][ T67] vx10: left allmulticast mode [ 894.088987][ T67] vx10: left promiscuous mode [ 894.089382][ T67] br2: port 2(vx10) entered disabled state [ 894.093194][ T67] w1: left allmulticast mode [ 894.093911][ T67] w1: left promiscuous mode [ 894.094340][ T67] br2: port 1(w1) entered disabled state [ 896.572463][T10514] br1: port 4(veth2) entered disabled state [ 896.661647][T10515] veth2: left allmulticast mode [ 896.662011][T10515] veth2: left promiscuous mode [ 896.662526][T10515] br1: port 4(veth2) entered disabled state [ 897.090841][T10519] br1: port 3(veth1) entered disabled state [ 897.177503][T10520] veth1: left allmulticast mode [ 897.177791][T10520] veth1: left promiscuous mode [ 897.178129][T10520] br1: port 3(veth1) entered disabled state [ 897.372392][T10522] vx20: left allmulticast mode [ 897.372671][T10522] vx20: left promiscuous mode [ 897.372987][T10522] br1: port 2(vx20) entered disabled state [ 897.763014][T10526] vx10: left allmulticast mode [ 897.763294][T10526] vx10: left promiscuous mode [ 897.764611][T10526] br1: port 1(vx10) entered disabled state [ 898.806587][T10536] [ 898.806779][T10536] ============================================ [ 898.806988][T10536] WARNING: possible recursive locking detected [ 898.807186][T10536] 6.13.0-rc5-virtme #1 Not tainted [ 898.807357][T10536] -------------------------------------------- [ 898.807563][T10536] ip/10536 is trying to acquire lock: [ 898.807731][T10536] ffffffffb1dd6630 ((netdev_chain).rwsem){++++}-{4:4}, at: blocking_notifier_call_chain+0x50/0x90 [ 898.808068][T10536] [ 898.808068][T10536] but task is already holding lock: [ 898.808307][T10536] ffffffffb1dd6630 ((netdev_chain).rwsem){++++}-{4:4}, at: blocking_notifier_call_chain+0x50/0x90 [ 898.808636][T10536] [ 898.808636][T10536] other info that might help us debug this: [ 898.808872][T10536] Possible unsafe locking scenario: [ 898.808872][T10536] [ 898.809113][T10536] CPU0 [ 898.809234][T10536] ---- [ 898.809353][T10536] lock((netdev_chain).rwsem); [ 898.809512][T10536] lock((netdev_chain).rwsem); [ 898.809674][T10536] [ 898.809674][T10536] *** DEADLOCK *** [ 898.809674][T10536] [ 898.809908][T10536] May be due to missing lock nesting notation [ 898.809908][T10536] [ 898.810147][T10536] 2 locks held by ip/10536: [ 898.810309][T10536] #0: ffffffffb1de7a48 (rtnl_mutex){+.+.}-{4:4}, at: rtnl_dellink+0x228/0xa40 [ 898.810602][T10536] #1: ffffffffb1dd6630 ((netdev_chain).rwsem){++++}-{4:4}, at: blocking_notifier_call_chain+0x50/0x90 [ 898.810929][T10536] [ 898.810929][T10536] stack backtrace: [ 898.811132][T10536] CPU: 2 UID: 0 PID: 10536 Comm: ip Not tainted 6.13.0-rc5-virtme #1 [ 898.811377][T10536] Hardware name: Bochs Bochs, BIOS Bochs 01/01/2011 [ 898.811585][T10536] Call Trace: [ 898.811713][T10536] [ 898.811802][T10536] dump_stack_lvl+0x82/0xd0 [ 898.811973][T10536] print_deadlock_bug+0x40a/0x650 [ 898.812137][T10536] validate_chain+0x5bf/0xae0 [ 898.812299][T10536] ? __pfx_validate_chain+0x10/0x10 [ 898.812467][T10536] ? hlock_class+0x4e/0x130 [ 898.812628][T10536] ? mark_lock+0x38/0x3e0 [ 898.812750][T10536] __lock_acquire+0xb9a/0x1680 [ 898.812914][T10536] lock_acquire.part.0+0xeb/0x330 [ 898.813083][T10536] ? blocking_notifier_call_chain+0x50/0x90 [ 898.813286][T10536] ? netconsole_process_cleanups_core+0x2e/0x3d0 [ 898.813492][T10536] ? __pfx_lock_acquire.part.0+0x10/0x10 [ 898.813653][T10536] ? trace_lock_acquire+0x14c/0x1f0 [ 898.813814][T10536] ? lock_acquire+0x32/0xc0 [ 898.813974][T10536] ? blocking_notifier_call_chain+0x50/0x90 [ 898.814184][T10536] down_read+0x9f/0x340 [ 898.814312][T10536] ? blocking_notifier_call_chain+0x50/0x90 [ 898.814521][T10536] ? __pfx_down_read+0x10/0x10 [ 898.814685][T10536] ? __lock_release+0x103/0x460 [ 898.814847][T10536] ? netconsole_netdev_event+0x2c1/0x620 [ 898.815015][T10536] ? __pfx___lock_release+0x10/0x10 [ 898.815182][T10536] blocking_notifier_call_chain+0x50/0x90 [ 898.815342][T10536] __netdev_upper_dev_unlink+0xd9/0x220 [ 898.815531][T10536] ? __pfx___netdev_upper_dev_unlink+0x10/0x10 [ 898.815740][T10536] ? __mutex_unlock_slowpath+0x14a/0x3d0 [ 898.815913][T10536] ? __pfx___mutex_unlock_slowpath+0x10/0x10 [ 898.816120][T10536] ? rtnl_is_locked+0x15/0x20 [ 898.816286][T10536] netdev_upper_dev_unlink+0x71/0xa0 [ 898.816451][T10536] ? __pfx_netdev_upper_dev_unlink+0x10/0x10 [ 898.816656][T10536] ? netconsole_netdev_event+0x2c1/0x620 [ 898.816821][T10536] ? mutex_is_locked+0x1c/0x60 [ 898.816992][T10536] ? rtnl_is_locked+0x15/0x20 [ 898.817158][T10536] ? netdev_master_upper_dev_get+0x13/0x120 [ 898.817365][T10536] vrf_device_event+0x78/0xe0 [vrf] [ 898.817531][T10536] notifier_call_chain+0xcd/0x150 [ 898.817701][T10536] blocking_notifier_call_chain+0x66/0x90 [ 898.817867][T10536] unregister_netdevice_many_notify+0xa7b/0x1580 [ 898.818071][T10536] ? hlock_class+0x4e/0x130 [ 898.818230][T10536] ? mark_lock+0x38/0x3e0 [ 898.818357][T10536] ? __pfx_unregister_netdevice_many_notify+0x10/0x10 [ 898.818579][T10536] ? lockdep_hardirqs_on_prepare+0x275/0x410 [ 898.818783][T10536] ? kfree+0xf3/0x330 [ 898.818910][T10536] ? vlan_vid_del+0x310/0x5e0 [ 898.819075][T10536] ? vlan_vid_del+0x310/0x5e0 [ 898.819238][T10536] rtnl_dellink+0x348/0xa40 [ 898.819400][T10536] ? __pfx_rtnl_dellink+0x10/0x10 [ 898.819576][T10536] ? find_held_lock+0x2c/0x110 [ 898.819740][T10536] ? __lock_release+0x103/0x460 [ 898.819907][T10536] ? rtnetlink_rcv_msg+0x6ef/0xc10 [ 898.820065][T10536] ? __pfx___lock_release+0x10/0x10 [ 898.820227][T10536] ? __pfx_lock_acquire.part.0+0x10/0x10 [ 898.820392][T10536] ? rtnetlink_rcv_msg+0x6ef/0xc10 [ 898.820554][T10536] ? __pfx_rtnl_dellink+0x10/0x10 [ 898.820713][T10536] rtnetlink_rcv_msg+0x712/0xc10 [ 898.820874][T10536] ? __pfx_rtnetlink_rcv_msg+0x10/0x10 [ 898.821044][T10536] ? hlock_class+0x4e/0x130 [ 898.821203][T10536] ? mark_lock+0x38/0x3e0 [ 898.821326][T10536] ? __lock_acquire+0xb9a/0x1680 [ 898.821486][T10536] netlink_rcv_skb+0x130/0x360 [ 898.821649][T10536] ? __pfx_rtnetlink_rcv_msg+0x10/0x10 [ 898.821811][T10536] ? __pfx_netlink_rcv_skb+0x10/0x10 [ 898.821975][T10536] ? netlink_deliver_tap+0x13e/0x340 [ 898.822143][T10536] ? netlink_deliver_tap+0xc3/0x340 [ 898.822306][T10536] netlink_unicast+0x44b/0x710 [ 898.822466][T10536] ? __pfx_netlink_unicast+0x10/0x10 [ 898.822628][T10536] ? find_held_lock+0x2c/0x110 [ 898.822789][T10536] netlink_sendmsg+0x723/0xbe0 [ 898.822951][T10536] ? __pfx_netlink_sendmsg+0x10/0x10 [ 898.823115][T10536] ____sys_sendmsg+0x7ac/0xa10 [ 898.823276][T10536] ? __pfx_____sys_sendmsg+0x10/0x10 [ 898.823444][T10536] ? __pfx_copy_msghdr_from_user+0x10/0x10 [ 898.823648][T10536] ___sys_sendmsg+0xee/0x170 [ 898.823807][T10536] ? __pfx____sys_sendmsg+0x10/0x10 [ 898.823969][T10536] ? kasan_save_stack+0x34/0x50 [ 898.824129][T10536] ? kasan_save_stack+0x24/0x50 [ 898.824291][T10536] ? __kasan_record_aux_stack+0x8e/0xa0 [ 898.824455][T10536] ? __call_rcu_common.constprop.0+0xa1/0x4b0 [ 898.824662][T10536] ? __x64_sys_close+0x7c/0xd0 [ 898.824828][T10536] ? do_syscall_64+0xc1/0x1d0 [ 898.824991][T10536] ? entry_SYSCALL_64_after_hwframe+0x77/0x7f [ 898.825195][T10536] ? __lock_acquire+0xb9a/0x1680 [ 898.825359][T10536] ? find_held_lock+0x2c/0x110 [ 898.825530][T10536] ? __lock_release+0x103/0x460 [ 898.825691][T10536] ? __virt_addr_valid+0x22b/0x430 [ 898.825897][T10536] ? __pfx___lock_release+0x10/0x10 [ 898.826061][T10536] ? __pfx_lock_acquire.part.0+0x10/0x10 [ 898.826229][T10536] __sys_sendmsg+0x109/0x1a0 [ 898.826392][T10536] ? __pfx___sys_sendmsg+0x10/0x10 [ 898.826556][T10536] ? __pfx_slab_free_after_rcu_debug+0x10/0x10 [ 898.826769][T10536] do_syscall_64+0xc1/0x1d0 [ 898.826933][T10536] entry_SYSCALL_64_after_hwframe+0x77/0x7f [ 898.827143][T10536] RIP: 0033:0x7f89720a99a7 [ 898.827322][T10536] Code: 0a 00 f7 d8 64 89 02 48 c7 c0 ff ff ff ff eb b9 0f 1f 00 f3 0f 1e fa 64 8b 04 25 18 00 00 00 85 c0 75 10 b8 2e 00 00 00 0f 05 <48> 3d 00 f0 ff ff 77 51 c3 48 83 ec 28 89 54 24 1c 48 89 74 24 10 [ 898.827901][T10536] RSP: 002b:00007ffd2fce7648 EFLAGS: 00000246 ORIG_RAX: 000000000000002e [ 898.828161][T10536] RAX: ffffffffffffffda RBX: 00007ffd2fce7d70 RCX: 00007f89720a99a7 [ 898.828410][T10536] RDX: 0000000000000000 RSI: 00007ffd2fce76b0 RDI: 0000000000000005 [ 898.828657][T10536] RBP: 0000000000000002 R08: 0000000000000003 R09: 0000000000000078 [ 898.828905][T10536] R10: 00007f8971f67ef8 R11: 0000000000000246 R12: 0000000000000002 [ 898.829152][T10536] R13: 000000006778feb4 R14: 0000000000498600 R15: 0000000000000000 [ 898.829409][T10536]