[ 31.740187][ T532] 8021q: 802.1Q VLAN Support v1.8 [ 34.981156][ T575] ip (575) used greatest stack depth: 23640 bytes left [ 35.071178][ T577] br1: port 1(vx10) entered blocking state [ 35.071658][ T577] br1: port 1(vx10) entered disabled state [ 35.071946][ T577] vx10: entered allmulticast mode [ 35.073933][ T577] vx10: entered promiscuous mode [ 35.074858][ T577] br1: port 1(vx10) entered blocking state [ 35.075187][ T577] br1: port 1(vx10) entered forwarding state [ 35.432887][ T582] br1: port 2(vx20) entered blocking state [ 35.433200][ T582] br1: port 2(vx20) entered disabled state [ 35.433871][ T582] vx20: entered allmulticast mode [ 35.435914][ T582] vx20: entered promiscuous mode [ 35.436779][ T582] br1: port 2(vx20) entered blocking state [ 35.437060][ T582] br1: port 2(vx20) entered forwarding state [ 35.617773][ T584] br1: port 3(veth1) entered blocking state [ 35.618053][ T584] br1: port 3(veth1) entered disabled state [ 35.618692][ T584] veth1: entered allmulticast mode [ 35.621368][ T584] veth1: entered promiscuous mode [ 35.702058][ T37] br1: port 3(veth1) entered blocking state [ 35.702462][ T37] br1: port 3(veth1) entered forwarding state [ 35.940701][ T588] br1: port 4(veth2) entered blocking state [ 35.941046][ T588] br1: port 4(veth2) entered disabled state [ 35.941328][ T588] veth2: entered allmulticast mode [ 35.943232][ T588] veth2: entered promiscuous mode [ 36.028855][ T37] br1: port 4(veth2) entered blocking state [ 36.029128][ T37] br1: port 4(veth2) entered forwarding state [ 39.309184][ T639] br2: port 1(w1) entered blocking state [ 39.309505][ T639] br2: port 1(w1) entered disabled state [ 39.309737][ T639] w1: entered allmulticast mode [ 39.311760][ T639] w1: entered promiscuous mode [ 39.919631][ T647] br2: port 2(vx10) entered blocking state [ 39.919965][ T647] br2: port 2(vx10) entered disabled state [ 39.920281][ T647] vx10: entered allmulticast mode [ 39.922191][ T647] vx10: entered promiscuous mode [ 39.922807][ T647] br2: port 2(vx10) entered blocking state [ 39.923096][ T647] br2: port 2(vx10) entered forwarding state [ 40.476119][ T654] br2: port 3(vx20) entered blocking state [ 40.476460][ T654] br2: port 3(vx20) entered disabled state [ 40.476758][ T654] vx20: entered allmulticast mode [ 40.478713][ T654] vx20: entered promiscuous mode [ 40.479247][ T654] br2: port 3(vx20) entered blocking state [ 40.479502][ T654] br2: port 3(vx20) entered forwarding state [ 41.208539][ T40] br2: port 1(w1) entered blocking state [ 41.209129][ T40] br2: port 1(w1) entered forwarding state [ 43.489484][ T694] br2: port 1(w1) entered blocking state [ 43.489754][ T694] br2: port 1(w1) entered disabled state [ 43.489992][ T694] w1: entered allmulticast mode [ 43.492354][ T694] w1: entered promiscuous mode [ 44.116859][ T702] br2: port 2(vx10) entered blocking state [ 44.117246][ T702] br2: port 2(vx10) entered disabled state [ 44.117534][ T702] vx10: entered allmulticast mode [ 44.120658][ T702] vx10: entered promiscuous mode [ 44.122414][ T702] br2: port 2(vx10) entered blocking state [ 44.122784][ T702] br2: port 2(vx10) entered forwarding state [ 44.751321][ T709] br2: port 3(vx20) entered blocking state [ 44.751637][ T709] br2: port 3(vx20) entered disabled state [ 44.751957][ T709] vx20: entered allmulticast mode [ 44.753830][ T709] vx20: entered promiscuous mode [ 44.754366][ T709] br2: port 3(vx20) entered blocking state [ 44.754619][ T709] br2: port 3(vx20) entered forwarding state [ 45.430984][ T40] br2: port 1(w1) entered blocking state [ 45.431269][ T40] br2: port 1(w1) entered forwarding state [ 91.623843][ T1442] veth3: entered promiscuous mode [ 140.992017][ T2369] veth3: left promiscuous mode [ 141.474575][ T2375] veth3: entered promiscuous mode [ 191.303450][ T3302] veth3: left promiscuous mode [ 192.198380][ T3320] vx20: left allmulticast mode [ 192.198643][ T3320] vx20: left promiscuous mode [ 192.198996][ T3320] br1: port 2(vx20) entered disabled state [ 192.300073][ T3321] vx10: left allmulticast mode [ 192.302065][ T3321] vx10: left promiscuous mode [ 192.302623][ T3321] br1: port 1(vx10) entered disabled state [ 197.707197][ T3326] br1: port 1(vx10) entered blocking state [ 197.707653][ T3326] br1: port 1(vx10) entered disabled state [ 197.707969][ T3326] vx10: entered allmulticast mode [ 197.709996][ T3326] vx10: entered promiscuous mode [ 197.710843][ T3326] br1: port 1(vx10) entered blocking state [ 197.711121][ T3326] br1: port 1(vx10) entered forwarding state [ 197.878793][ T3328] br1: port 2(vx20) entered blocking state [ 197.879117][ T3328] br1: port 2(vx20) entered disabled state [ 197.886731][ T3328] vx20: entered allmulticast mode [ 197.889172][ T3328] vx20: entered promiscuous mode [ 197.889727][ T3328] br1: port 2(vx20) entered blocking state [ 197.890005][ T3328] br1: port 2(vx20) entered forwarding state [ 243.361548][ T4006] veth3: entered promiscuous mode [ 292.640413][ T4933] veth3: left promiscuous mode [ 293.158062][ T4939] veth3: entered promiscuous mode [ 341.911790][ T5866] veth3: left promiscuous mode [ 343.285106][ T5889] vx20: left allmulticast mode [ 343.285589][ T5889] vx20: left promiscuous mode [ 343.286112][ T5889] br1: port 2(vx20) entered disabled state [ 343.368820][ T5890] vx10: left allmulticast mode [ 343.369070][ T5890] vx10: left promiscuous mode [ 343.369417][ T5890] br1: port 1(vx10) entered disabled state [ 348.785531][ T5895] br1: port 1(vx10) entered blocking state [ 348.785839][ T5895] br1: port 1(vx10) entered disabled state [ 348.786125][ T5895] vx10: entered allmulticast mode [ 348.788110][ T5895] vx10: entered promiscuous mode [ 348.788709][ T5895] br1: port 1(vx10) entered blocking state [ 348.788960][ T5895] br1: port 1(vx10) entered forwarding state [ 348.946647][ T5897] br1: port 2(vx20) entered blocking state [ 348.946965][ T5897] br1: port 2(vx20) entered disabled state [ 348.947295][ T5897] vx20: entered allmulticast mode [ 348.949728][ T5897] vx20: entered promiscuous mode [ 348.950280][ T5897] br1: port 2(vx20) entered blocking state [ 348.950551][ T5897] br1: port 2(vx20) entered forwarding state [ 656.377040][ T9172] vx20: left allmulticast mode [ 656.377299][ T9172] vx20: left promiscuous mode [ 656.377621][ T9172] br1: port 2(vx20) entered disabled state [ 656.463108][ T9173] vx10: left allmulticast mode [ 656.463411][ T9173] vx10: left promiscuous mode [ 656.463769][ T9173] br1: port 1(vx10) entered disabled state [ 661.828179][ T9178] br1: port 1(vx10) entered blocking state [ 661.828825][ T9178] br1: port 1(vx10) entered disabled state [ 661.829122][ T9178] vx10: entered allmulticast mode [ 661.831121][ T9178] vx10: entered promiscuous mode [ 661.838858][ T9178] br1: port 1(vx10) entered blocking state [ 661.839154][ T9178] br1: port 1(vx10) entered forwarding state [ 661.989957][ T9180] br1: port 2(vx20) entered blocking state [ 661.990390][ T9180] br1: port 2(vx20) entered disabled state [ 661.990697][ T9180] vx20: entered allmulticast mode [ 661.992718][ T9180] vx20: entered promiscuous mode [ 661.993296][ T9180] br1: port 2(vx20) entered blocking state [ 661.993559][ T9180] br1: port 2(vx20) entered forwarding state [ 766.328652][ T11] vx20: left allmulticast mode [ 766.329031][ T11] vx20: left promiscuous mode [ 766.329747][ T11] br2: port 3(vx20) entered disabled state [ 766.333362][ T11] vx10: left allmulticast mode [ 766.333597][ T11] vx10: left promiscuous mode [ 766.334052][ T11] br2: port 2(vx10) entered disabled state [ 766.336519][ T11] w1: left allmulticast mode [ 766.336755][ T11] w1: left promiscuous mode [ 766.337202][ T11] br2: port 1(w1) entered disabled state [ 766.907901][ T11] vx20: left allmulticast mode [ 766.908145][ T11] vx20: left promiscuous mode [ 766.908526][ T11] br2: port 3(vx20) entered disabled state [ 766.911582][ T11] vx10: left allmulticast mode [ 766.911807][ T11] vx10: left promiscuous mode [ 766.912139][ T11] br2: port 2(vx10) entered disabled state [ 766.915254][ T11] w1: left allmulticast mode [ 766.915498][ T11] w1: left promiscuous mode [ 766.915893][ T11] br2: port 1(w1) entered disabled state [ 769.318575][T11084] br1: port 4(veth2) entered disabled state [ 769.396162][T11085] veth2: left allmulticast mode [ 769.396428][T11085] veth2: left promiscuous mode [ 769.396753][T11085] br1: port 4(veth2) entered disabled state [ 769.640564][T11088] br1: port 3(veth1) entered disabled state [ 769.745297][T11089] veth1: left allmulticast mode [ 769.745669][T11089] veth1: left promiscuous mode [ 769.746750][T11089] br1: port 3(veth1) entered disabled state [ 769.913189][T11091] vx20: left allmulticast mode [ 769.913524][T11091] vx20: left promiscuous mode [ 769.913906][T11091] br1: port 2(vx20) entered disabled state [ 770.300708][T11095] vx10: left allmulticast mode [ 770.300959][T11095] vx10: left promiscuous mode [ 770.301296][T11095] br1: port 1(vx10) entered disabled state [ 771.137997][T11104] [ 771.138140][T11104] ============================================ [ 771.138352][T11104] WARNING: possible recursive locking detected [ 771.138555][T11104] 6.13.0-rc5-virtme #1 Not tainted [ 771.138714][T11104] -------------------------------------------- [ 771.138935][T11104] ip/11104 is trying to acquire lock: [ 771.139099][T11104] ffffffff967d6630 ((netdev_chain).rwsem){++++}-{4:4}, at: blocking_notifier_call_chain+0x50/0x90 [ 771.139429][T11104] [ 771.139429][T11104] but task is already holding lock: [ 771.139663][T11104] ffffffff967d6630 ((netdev_chain).rwsem){++++}-{4:4}, at: blocking_notifier_call_chain+0x50/0x90 [ 771.139986][T11104] [ 771.139986][T11104] other info that might help us debug this: [ 771.140223][T11104] Possible unsafe locking scenario: [ 771.140223][T11104] [ 771.140459][T11104] CPU0 [ 771.140579][T11104] ---- [ 771.140700][T11104] lock((netdev_chain).rwsem); [ 771.140871][T11104] lock((netdev_chain).rwsem); [ 771.141030][T11104] [ 771.141030][T11104] *** DEADLOCK *** [ 771.141030][T11104] [ 771.141261][T11104] May be due to missing lock nesting notation [ 771.141261][T11104] [ 771.141492][T11104] 2 locks held by ip/11104: [ 771.141653][T11104] #0: ffffffff967e7a48 (rtnl_mutex){+.+.}-{4:4}, at: rtnl_dellink+0x228/0xa40 [ 771.141960][T11104] #1: ffffffff967d6630 ((netdev_chain).rwsem){++++}-{4:4}, at: blocking_notifier_call_chain+0x50/0x90 [ 771.142291][T11104] [ 771.142291][T11104] stack backtrace: [ 771.142499][T11104] CPU: 3 UID: 0 PID: 11104 Comm: ip Not tainted 6.13.0-rc5-virtme #1 [ 771.142741][T11104] Hardware name: Bochs Bochs, BIOS Bochs 01/01/2011 [ 771.142947][T11104] Call Trace: [ 771.143075][T11104] [ 771.143162][T11104] dump_stack_lvl+0x82/0xd0 [ 771.143332][T11104] print_deadlock_bug+0x40a/0x650 [ 771.143499][T11104] validate_chain+0x5bf/0xae0 [ 771.143666][T11104] ? __pfx_validate_chain+0x10/0x10 [ 771.143832][T11104] ? hlock_class+0x4e/0x130 [ 771.143990][T11104] ? mark_lock+0x38/0x3e0 [ 771.144118][T11104] __lock_acquire+0xb9a/0x1680 [ 771.144281][T11104] lock_acquire.part.0+0xeb/0x330 [ 771.144443][T11104] ? blocking_notifier_call_chain+0x50/0x90 [ 771.144641][T11104] ? netconsole_process_cleanups_core+0x2e/0x3d0 [ 771.144846][T11104] ? __pfx_lock_acquire.part.0+0x10/0x10 [ 771.145008][T11104] ? trace_lock_acquire+0x14c/0x1f0 [ 771.145171][T11104] ? lock_acquire+0x32/0xc0 [ 771.145337][T11104] ? blocking_notifier_call_chain+0x50/0x90 [ 771.145539][T11104] down_read+0x9f/0x340 [ 771.145668][T11104] ? blocking_notifier_call_chain+0x50/0x90 [ 771.145869][T11104] ? __pfx_down_read+0x10/0x10 [ 771.146032][T11104] ? __lock_release+0x103/0x460 [ 771.146194][T11104] ? netconsole_netdev_event+0x2c1/0x620 [ 771.146361][T11104] ? __pfx___lock_release+0x10/0x10 [ 771.146524][T11104] blocking_notifier_call_chain+0x50/0x90 [ 771.146697][T11104] __netdev_upper_dev_unlink+0xd9/0x220 [ 771.146861][T11104] ? __pfx___netdev_upper_dev_unlink+0x10/0x10 [ 771.147062][T11104] ? __mutex_unlock_slowpath+0x14a/0x3d0 [ 771.147228][T11104] ? __pfx___mutex_unlock_slowpath+0x10/0x10 [ 771.147431][T11104] ? rtnl_is_locked+0x15/0x20 [ 771.147596][T11104] netdev_upper_dev_unlink+0x71/0xa0 [ 771.147765][T11104] ? __pfx_netdev_upper_dev_unlink+0x10/0x10 [ 771.147964][T11104] ? netconsole_netdev_event+0x2c1/0x620 [ 771.148126][T11104] ? mutex_is_locked+0x1c/0x60 [ 771.148286][T11104] ? rtnl_is_locked+0x15/0x20 [ 771.148445][T11104] ? netdev_master_upper_dev_get+0x13/0x120 [ 771.148644][T11104] vrf_device_event+0x78/0xe0 [vrf] [ 771.148808][T11104] notifier_call_chain+0xcd/0x150 [ 771.148975][T11104] blocking_notifier_call_chain+0x66/0x90 [ 771.149139][T11104] unregister_netdevice_many_notify+0xa7b/0x1580 [ 771.149338][T11104] ? hlock_class+0x4e/0x130 [ 771.149497][T11104] ? mark_lock+0x38/0x3e0 [ 771.149623][T11104] ? __pfx_unregister_netdevice_many_notify+0x10/0x10 [ 771.149820][T11104] ? lockdep_hardirqs_on_prepare+0x275/0x410 [ 771.150023][T11104] ? kfree+0xf3/0x330 [ 771.150150][T11104] ? vlan_vid_del+0x310/0x5e0 [ 771.150312][T11104] ? vlan_vid_del+0x310/0x5e0 [ 771.150474][T11104] rtnl_dellink+0x348/0xa40 [ 771.150635][T11104] ? __pfx_rtnl_dellink+0x10/0x10 [ 771.150809][T11104] ? find_held_lock+0x2c/0x110 [ 771.150971][T11104] ? __lock_release+0x103/0x460 [ 771.151130][T11104] ? rtnetlink_rcv_msg+0x6ef/0xc10 [ 771.151288][T11104] ? __pfx___lock_release+0x10/0x10 [ 771.151454][T11104] ? __pfx_lock_acquire.part.0+0x10/0x10 [ 771.151616][T11104] ? rtnetlink_rcv_msg+0x6ef/0xc10 [ 771.151777][T11104] ? __pfx_rtnl_dellink+0x10/0x10 [ 771.151940][T11104] rtnetlink_rcv_msg+0x712/0xc10 [ 771.152101][T11104] ? __pfx_rtnetlink_rcv_msg+0x10/0x10 [ 771.152272][T11104] ? hlock_class+0x4e/0x130 [ 771.152431][T11104] ? mark_lock+0x38/0x3e0 [ 771.152558][T11104] ? __lock_acquire+0xb9a/0x1680 [ 771.152719][T11104] netlink_rcv_skb+0x130/0x360 [ 771.152881][T11104] ? __pfx_rtnetlink_rcv_msg+0x10/0x10 [ 771.153040][T11104] ? __pfx_netlink_rcv_skb+0x10/0x10 [ 771.153204][T11104] ? netlink_deliver_tap+0x13e/0x340 [ 771.153366][T11104] ? netlink_deliver_tap+0xc3/0x340 [ 771.153527][T11104] netlink_unicast+0x44b/0x710 [ 771.153691][T11104] ? __pfx_netlink_unicast+0x10/0x10 [ 771.153851][T11104] ? find_held_lock+0x2c/0x110 [ 771.154013][T11104] netlink_sendmsg+0x723/0xbe0 [ 771.154172][T11104] ? __pfx_netlink_sendmsg+0x10/0x10 [ 771.154337][T11104] ____sys_sendmsg+0x7ac/0xa10 [ 771.154498][T11104] ? __pfx_____sys_sendmsg+0x10/0x10 [ 771.154658][T11104] ? __pfx_copy_msghdr_from_user+0x10/0x10 [ 771.154872][T11104] ___sys_sendmsg+0xee/0x170 [ 771.155029][T11104] ? __pfx____sys_sendmsg+0x10/0x10 [ 771.155188][T11104] ? kasan_save_stack+0x34/0x50 [ 771.155348][T11104] ? kasan_save_stack+0x24/0x50 [ 771.155506][T11104] ? __kasan_record_aux_stack+0x8e/0xa0 [ 771.155664][T11104] ? __call_rcu_common.constprop.0+0xa1/0x4b0 [ 771.155865][T11104] ? __x64_sys_close+0x7c/0xd0 [ 771.156030][T11104] ? do_syscall_64+0xc1/0x1d0 [ 771.156190][T11104] ? entry_SYSCALL_64_after_hwframe+0x77/0x7f [ 771.156390][T11104] ? __lock_acquire+0xb9a/0x1680 [ 771.156553][T11104] ? find_held_lock+0x2c/0x110 [ 771.156714][T11104] ? __lock_release+0x103/0x460 [ 771.156872][T11104] ? __virt_addr_valid+0x22b/0x430 [ 771.157035][T11104] ? __pfx___lock_release+0x10/0x10 [ 771.157197][T11104] ? __pfx_lock_acquire.part.0+0x10/0x10 [ 771.157364][T11104] __sys_sendmsg+0x109/0x1a0 [ 771.157525][T11104] ? __pfx___sys_sendmsg+0x10/0x10 [ 771.157687][T11104] ? __pfx_slab_free_after_rcu_debug+0x10/0x10 [ 771.157890][T11104] do_syscall_64+0xc1/0x1d0 [ 771.158050][T11104] entry_SYSCALL_64_after_hwframe+0x77/0x7f [ 771.158245][T11104] RIP: 0033:0x7f32570159a7 [ 771.158412][T11104] Code: 0a 00 f7 d8 64 89 02 48 c7 c0 ff ff ff ff eb b9 0f 1f 00 f3 0f 1e fa 64 8b 04 25 18 00 00 00 85 c0 75 10 b8 2e 00 00 00 0f 05 <48> 3d 00 f0 ff ff 77 51 c3 48 83 ec 28 89 54 24 1c 48 89 74 24 10 [ 771.158984][T11104] RSP: 002b:00007ffc540b3d78 EFLAGS: 00000246 ORIG_RAX: 000000000000002e [ 771.159224][T11104] RAX: ffffffffffffffda RBX: 00007ffc540b44a0 RCX: 00007f32570159a7 [ 771.159471][T11104] RDX: 0000000000000000 RSI: 00007ffc540b3de0 RDI: 0000000000000005 [ 771.159708][T11104] RBP: 0000000000000002 R08: 0000000000000003 R09: 0000000000000078 [ 771.159948][T11104] R10: 00007f3256ed3ef8 R11: 0000000000000246 R12: 0000000000000002 [ 771.160187][T11104] R13: 0000000067790460 R14: 0000000000498600 R15: 0000000000000000 [ 771.160434][T11104]