[ 33.063054][ T495] 8021q: 802.1Q VLAN Support v1.8 [ 37.016682][ T544] br1: port 1(vx10) entered blocking state [ 37.017197][ T544] br1: port 1(vx10) entered disabled state [ 37.017823][ T544] vx10: entered allmulticast mode [ 37.019846][ T544] vx10: entered promiscuous mode [ 37.020887][ T544] br1: port 1(vx10) entered blocking state [ 37.021529][ T544] br1: port 1(vx10) entered forwarding state [ 37.404360][ T549] br1: port 2(vx20) entered blocking state [ 37.404657][ T549] br1: port 2(vx20) entered disabled state [ 37.404948][ T549] vx20: entered allmulticast mode [ 37.407493][ T549] vx20: entered promiscuous mode [ 37.408526][ T549] br1: port 2(vx20) entered blocking state [ 37.408805][ T549] br1: port 2(vx20) entered forwarding state [ 37.572735][ T551] br1: port 3(veth1) entered blocking state [ 37.573053][ T551] br1: port 3(veth1) entered disabled state [ 37.573401][ T551] veth1: entered allmulticast mode [ 37.575748][ T551] veth1: entered promiscuous mode [ 37.658044][ T37] br1: port 3(veth1) entered blocking state [ 37.658494][ T37] br1: port 3(veth1) entered forwarding state [ 38.035956][ T556] br1: port 4(veth2) entered blocking state [ 38.037745][ T556] br1: port 4(veth2) entered disabled state [ 38.038199][ T556] veth2: entered allmulticast mode [ 38.041599][ T556] veth2: entered promiscuous mode [ 38.129894][ T39] br1: port 4(veth2) entered blocking state [ 38.130369][ T39] br1: port 4(veth2) entered forwarding state [ 41.755928][ T606] br2: port 1(w1) entered blocking state [ 41.756261][ T606] br2: port 1(w1) entered disabled state [ 41.756524][ T606] w1: entered allmulticast mode [ 41.758541][ T606] w1: entered promiscuous mode [ 42.424846][ T614] br2: port 2(vx10) entered blocking state [ 42.425374][ T614] br2: port 2(vx10) entered disabled state [ 42.425902][ T614] vx10: entered allmulticast mode [ 42.427882][ T614] vx10: entered promiscuous mode [ 42.428832][ T614] br2: port 2(vx10) entered blocking state [ 42.429127][ T614] br2: port 2(vx10) entered forwarding state [ 43.077142][ T621] br2: port 3(vx20) entered blocking state [ 43.077491][ T621] br2: port 3(vx20) entered disabled state [ 43.077828][ T621] vx20: entered allmulticast mode [ 43.080024][ T621] vx20: entered promiscuous mode [ 43.080859][ T621] br2: port 3(vx20) entered blocking state [ 43.081130][ T621] br2: port 3(vx20) entered forwarding state [ 43.841784][ T37] br2: port 1(w1) entered blocking state [ 43.842398][ T37] br2: port 1(w1) entered forwarding state [ 46.326678][ T663] br2: port 1(w1) entered blocking state [ 46.327064][ T663] br2: port 1(w1) entered disabled state [ 46.328147][ T663] w1: entered allmulticast mode [ 46.331580][ T663] w1: entered promiscuous mode [ 47.018371][ T671] br2: port 2(vx10) entered blocking state [ 47.018773][ T671] br2: port 2(vx10) entered disabled state [ 47.019188][ T671] vx10: entered allmulticast mode [ 47.021165][ T671] vx10: entered promiscuous mode [ 47.022449][ T671] br2: port 2(vx10) entered blocking state [ 47.022709][ T671] br2: port 2(vx10) entered forwarding state [ 47.658803][ T678] br2: port 3(vx20) entered blocking state [ 47.659165][ T678] br2: port 3(vx20) entered disabled state [ 47.660080][ T678] vx20: entered allmulticast mode [ 47.662050][ T678] vx20: entered promiscuous mode [ 47.662611][ T678] br2: port 3(vx20) entered blocking state [ 47.662857][ T678] br2: port 3(vx20) entered forwarding state [ 48.357206][ T39] br2: port 1(w1) entered blocking state [ 48.357537][ T39] br2: port 1(w1) entered forwarding state [ 59.570522][ T765] GACT probability NOT on [ 184.929647][ T1566] veth3: entered promiscuous mode [ 234.645602][ T2493] veth3: left promiscuous mode [ 235.145006][ T2499] veth3: entered promiscuous mode [ 283.726706][ T3426] veth3: left promiscuous mode [ 284.683139][ T3444] vx20: left allmulticast mode [ 284.683844][ T3444] vx20: left promiscuous mode [ 284.684247][ T3444] br1: port 2(vx20) entered disabled state [ 284.765781][ T3445] vx10: left allmulticast mode [ 284.766045][ T3445] vx10: left promiscuous mode [ 284.766401][ T3445] br1: port 1(vx10) entered disabled state [ 290.150055][ T3450] br1: port 1(vx10) entered blocking state [ 290.150395][ T3450] br1: port 1(vx10) entered disabled state [ 290.150668][ T3450] vx10: entered allmulticast mode [ 290.152670][ T3450] vx10: entered promiscuous mode [ 290.153211][ T3450] br1: port 1(vx10) entered blocking state [ 290.153486][ T3450] br1: port 1(vx10) entered forwarding state [ 290.298974][ T3452] br1: port 2(vx20) entered blocking state [ 290.299397][ T3452] br1: port 2(vx20) entered disabled state [ 290.299699][ T3452] vx20: entered allmulticast mode [ 290.301953][ T3452] vx20: entered promiscuous mode [ 290.302528][ T3452] br1: port 2(vx20) entered blocking state [ 290.302802][ T3452] br1: port 2(vx20) entered forwarding state [ 425.153140][ T4280] veth3: entered promiscuous mode [ 473.115585][ T5207] veth3: left promiscuous mode [ 473.622495][ T5213] veth3: entered promiscuous mode [ 523.690383][ T6140] veth3: left promiscuous mode [ 848.889276][ T67] vx20: left allmulticast mode [ 848.889640][ T67] vx20: left promiscuous mode [ 848.890016][ T67] br2: port 3(vx20) entered disabled state [ 848.893755][ T67] vx10: left allmulticast mode [ 848.893979][ T67] vx10: left promiscuous mode [ 848.894558][ T67] br2: port 2(vx10) entered disabled state [ 848.896401][ T67] w1: left allmulticast mode [ 848.896637][ T67] w1: left promiscuous mode [ 848.896982][ T67] br2: port 1(w1) entered disabled state [ 849.492346][ T67] vx20: left allmulticast mode [ 849.492556][ T67] vx20: left promiscuous mode [ 849.492868][ T67] br2: port 3(vx20) entered disabled state [ 849.498596][ T67] vx10: left allmulticast mode [ 849.498824][ T67] vx10: left promiscuous mode [ 849.499126][ T67] br2: port 2(vx10) entered disabled state [ 849.502829][ T67] w1: left allmulticast mode [ 849.503015][ T67] w1: left promiscuous mode [ 849.503361][ T67] br2: port 1(w1) entered disabled state [ 851.903646][T10705] br1: port 4(veth2) entered disabled state [ 851.982452][T10706] veth2: left allmulticast mode [ 851.982728][T10706] veth2: left promiscuous mode [ 851.983156][T10706] br1: port 4(veth2) entered disabled state [ 852.323268][T10710] br1: port 3(veth1) entered disabled state [ 852.424780][T10711] veth1: left allmulticast mode [ 852.425014][T10711] veth1: left promiscuous mode [ 852.425344][T10711] br1: port 3(veth1) entered disabled state [ 852.593740][T10713] vx20: left allmulticast mode [ 852.593974][T10713] vx20: left promiscuous mode [ 852.594287][T10713] br1: port 2(vx20) entered disabled state [ 852.946881][T10717] vx10: left allmulticast mode [ 852.947109][T10717] vx10: left promiscuous mode [ 852.947442][T10717] br1: port 1(vx10) entered disabled state [ 853.958899][T10727] [ 853.959051][T10727] ============================================ [ 853.959242][T10727] WARNING: possible recursive locking detected [ 853.959441][T10727] 6.13.0-rc5-virtme #1 Not tainted [ 853.959600][T10727] -------------------------------------------- [ 853.959789][T10727] ip/10727 is trying to acquire lock: [ 853.959945][T10727] ffffffff9e7d6630 ((netdev_chain).rwsem){++++}-{4:4}, at: blocking_notifier_call_chain+0x50/0x90 [ 853.960255][T10727] [ 853.960255][T10727] but task is already holding lock: [ 853.960475][T10727] ffffffff9e7d6630 ((netdev_chain).rwsem){++++}-{4:4}, at: blocking_notifier_call_chain+0x50/0x90 [ 853.960777][T10727] [ 853.960777][T10727] other info that might help us debug this: [ 853.961000][T10727] Possible unsafe locking scenario: [ 853.961000][T10727] [ 853.961231][T10727] CPU0 [ 853.961352][T10727] ---- [ 853.961465][T10727] lock((netdev_chain).rwsem); [ 853.961615][T10727] lock((netdev_chain).rwsem); [ 853.961765][T10727] [ 853.961765][T10727] *** DEADLOCK *** [ 853.961765][T10727] [ 853.961982][T10727] May be due to missing lock nesting notation [ 853.961982][T10727] [ 853.962198][T10727] 2 locks held by ip/10727: [ 853.962352][T10727] #0: ffffffff9e7e7a48 (rtnl_mutex){+.+.}-{4:4}, at: rtnl_dellink+0x228/0xa40 [ 853.962673][T10727] #1: ffffffff9e7d6630 ((netdev_chain).rwsem){++++}-{4:4}, at: blocking_notifier_call_chain+0x50/0x90 [ 853.962984][T10727] [ 853.962984][T10727] stack backtrace: [ 853.963170][T10727] CPU: 0 UID: 0 PID: 10727 Comm: ip Not tainted 6.13.0-rc5-virtme #1 [ 853.963394][T10727] Hardware name: Bochs Bochs, BIOS Bochs 01/01/2011 [ 853.963581][T10727] Call Trace: [ 853.963697][T10727] [ 853.963776][T10727] dump_stack_lvl+0x82/0xd0 [ 853.963930][T10727] print_deadlock_bug+0x40a/0x650 [ 853.964079][T10727] validate_chain+0x5bf/0xae0 [ 853.964238][T10727] ? __pfx_validate_chain+0x10/0x10 [ 853.964391][T10727] ? hlock_class+0x4e/0x130 [ 853.964542][T10727] ? mark_lock+0x38/0x3e0 [ 853.964657][T10727] __lock_acquire+0xb9a/0x1680 [ 853.964822][T10727] lock_acquire.part.0+0xeb/0x330 [ 853.964976][T10727] ? blocking_notifier_call_chain+0x50/0x90 [ 853.965165][T10727] ? netconsole_process_cleanups_core+0x2e/0x3d0 [ 853.965357][T10727] ? __pfx_lock_acquire.part.0+0x10/0x10 [ 853.965511][T10727] ? trace_lock_acquire+0x14c/0x1f0 [ 853.965659][T10727] ? lock_acquire+0x32/0xc0 [ 853.965810][T10727] ? blocking_notifier_call_chain+0x50/0x90 [ 853.965996][T10727] down_read+0x9f/0x340 [ 853.966112][T10727] ? blocking_notifier_call_chain+0x50/0x90 [ 853.966297][T10727] ? __pfx_down_read+0x10/0x10 [ 853.966444][T10727] ? __lock_release+0x103/0x460 [ 853.966592][T10727] ? netconsole_netdev_event+0x2c1/0x620 [ 853.966749][T10727] ? __pfx___lock_release+0x10/0x10 [ 853.966913][T10727] blocking_notifier_call_chain+0x50/0x90 [ 853.967065][T10727] __netdev_upper_dev_unlink+0xd9/0x220 [ 853.967215][T10727] ? __pfx___netdev_upper_dev_unlink+0x10/0x10 [ 853.967405][T10727] ? __mutex_unlock_slowpath+0x14a/0x3d0 [ 853.967555][T10727] ? __pfx___mutex_unlock_slowpath+0x10/0x10 [ 853.967744][T10727] ? rtnl_is_locked+0x15/0x20 [ 853.967901][T10727] netdev_upper_dev_unlink+0x71/0xa0 [ 853.968053][T10727] ? __pfx_netdev_upper_dev_unlink+0x10/0x10 [ 853.968235][T10727] ? netconsole_netdev_event+0x2c1/0x620 [ 853.968383][T10727] ? mutex_is_locked+0x1c/0x60 [ 853.968535][T10727] ? rtnl_is_locked+0x15/0x20 [ 853.968683][T10727] ? netdev_master_upper_dev_get+0x13/0x120 [ 853.968894][T10727] vrf_device_event+0x78/0xe0 [vrf] [ 853.969052][T10727] notifier_call_chain+0xcd/0x150 [ 853.969202][T10727] blocking_notifier_call_chain+0x66/0x90 [ 853.969355][T10727] unregister_netdevice_many_notify+0xa7b/0x1580 [ 853.969538][T10727] ? hlock_class+0x4e/0x130 [ 853.969690][T10727] ? mark_lock+0x38/0x3e0 [ 853.969811][T10727] ? __pfx_unregister_netdevice_many_notify+0x10/0x10 [ 853.969999][T10727] ? lockdep_hardirqs_on_prepare+0x275/0x410 [ 853.970188][T10727] ? kfree+0xf3/0x330 [ 853.970301][T10727] ? vlan_vid_del+0x310/0x5e0 [ 853.970459][T10727] ? vlan_vid_del+0x310/0x5e0 [ 853.970609][T10727] rtnl_dellink+0x348/0xa40 [ 853.970765][T10727] ? __pfx_rtnl_dellink+0x10/0x10 [ 853.970934][T10727] ? find_held_lock+0x2c/0x110 [ 853.971085][T10727] ? __lock_release+0x103/0x460 [ 853.971246][T10727] ? rtnetlink_rcv_msg+0x6ef/0xc10 [ 853.971402][T10727] ? __pfx___lock_release+0x10/0x10 [ 853.971552][T10727] ? __pfx_lock_acquire.part.0+0x10/0x10 [ 853.971708][T10727] ? rtnetlink_rcv_msg+0x6ef/0xc10 [ 853.971857][T10727] ? __pfx_rtnl_dellink+0x10/0x10 [ 853.972010][T10727] rtnetlink_rcv_msg+0x712/0xc10 [ 853.972159][T10727] ? __pfx_rtnetlink_rcv_msg+0x10/0x10 [ 853.972310][T10727] ? hlock_class+0x4e/0x130 [ 853.972459][T10727] ? mark_lock+0x38/0x3e0 [ 853.972573][T10727] ? __lock_acquire+0xb9a/0x1680 [ 853.972722][T10727] netlink_rcv_skb+0x130/0x360 [ 853.972871][T10727] ? __pfx_rtnetlink_rcv_msg+0x10/0x10 [ 853.973026][T10727] ? __pfx_netlink_rcv_skb+0x10/0x10 [ 853.973178][T10727] ? netlink_deliver_tap+0x13e/0x340 [ 853.973329][T10727] ? netlink_deliver_tap+0xc3/0x340 [ 853.973480][T10727] netlink_unicast+0x44b/0x710 [ 853.973630][T10727] ? __pfx_netlink_unicast+0x10/0x10 [ 853.973775][T10727] ? find_held_lock+0x2c/0x110 [ 853.973953][T10727] netlink_sendmsg+0x723/0xbe0 [ 853.974110][T10727] ? __pfx_netlink_sendmsg+0x10/0x10 [ 853.974263][T10727] ____sys_sendmsg+0x7ac/0xa10 [ 853.974412][T10727] ? __pfx_____sys_sendmsg+0x10/0x10 [ 853.974560][T10727] ? __pfx_copy_msghdr_from_user+0x10/0x10 [ 853.974748][T10727] ___sys_sendmsg+0xee/0x170 [ 853.974896][T10727] ? __pfx____sys_sendmsg+0x10/0x10 [ 853.975044][T10727] ? kasan_save_stack+0x34/0x50 [ 853.975197][T10727] ? kasan_save_stack+0x24/0x50 [ 853.975343][T10727] ? __kasan_record_aux_stack+0x8e/0xa0 [ 853.975496][T10727] ? __call_rcu_common.constprop.0+0xa1/0x4b0 [ 853.975683][T10727] ? __x64_sys_close+0x7c/0xd0 [ 853.975830][T10727] ? do_syscall_64+0xc1/0x1d0 [ 853.975979][T10727] ? entry_SYSCALL_64_after_hwframe+0x77/0x7f [ 853.976163][T10727] ? __lock_acquire+0xb9a/0x1680 [ 853.976317][T10727] ? find_held_lock+0x2c/0x110 [ 853.976471][T10727] ? __lock_release+0x103/0x460 [ 853.976621][T10727] ? __virt_addr_valid+0x22b/0x430 [ 853.976771][T10727] ? __pfx___lock_release+0x10/0x10 [ 853.976919][T10727] ? __pfx_lock_acquire.part.0+0x10/0x10 [ 853.977073][T10727] __sys_sendmsg+0x109/0x1a0 [ 853.977224][T10727] ? __pfx___sys_sendmsg+0x10/0x10 [ 853.977372][T10727] ? __pfx_slab_free_after_rcu_debug+0x10/0x10 [ 853.977566][T10727] do_syscall_64+0xc1/0x1d0 [ 853.977717][T10727] entry_SYSCALL_64_after_hwframe+0x77/0x7f [ 853.977898][T10727] RIP: 0033:0x7f901e4929a7 [ 853.978059][T10727] Code: 0a 00 f7 d8 64 89 02 48 c7 c0 ff ff ff ff eb b9 0f 1f 00 f3 0f 1e fa 64 8b 04 25 18 00 00 00 85 c0 75 10 b8 2e 00 00 00 0f 05 <48> 3d 00 f0 ff ff 77 51 c3 48 83 ec 28 89 54 24 1c 48 89 74 24 10 [ 853.978581][T10727] RSP: 002b:00007ffdf1b075c8 EFLAGS: 00000246 ORIG_RAX: 000000000000002e [ 853.978805][T10727] RAX: ffffffffffffffda RBX: 00007ffdf1b07cf0 RCX: 00007f901e4929a7 [ 853.979036][T10727] RDX: 0000000000000000 RSI: 00007ffdf1b07630 RDI: 0000000000000005 [ 853.979257][T10727] RBP: 0000000000000002 R08: 0000000000000003 R09: 0000000000000078 [ 853.979494][T10727] R10: 00007f901e350ef8 R11: 0000000000000246 R12: 0000000000000002 [ 853.979721][T10727] R13: 00000000677957fb R14: 0000000000498600 R15: 0000000000000000 [ 853.979949][T10727]