[ 24.547222][ T422] 8021q: 802.1Q VLAN Support v1.8 [ 28.096451][ T470] br1: port 1(vx10) entered blocking state [ 28.096754][ T470] br1: port 1(vx10) entered disabled state [ 28.097066][ T470] vx10: entered allmulticast mode [ 28.099367][ T470] vx10: entered promiscuous mode [ 28.100191][ T470] br1: port 1(vx10) entered blocking state [ 28.100438][ T470] br1: port 1(vx10) entered forwarding state [ 28.414064][ T475] br1: port 2(vx20) entered blocking state [ 28.414384][ T475] br1: port 2(vx20) entered disabled state [ 28.414671][ T475] vx20: entered allmulticast mode [ 28.416567][ T475] vx20: entered promiscuous mode [ 28.417154][ T475] br1: port 2(vx20) entered blocking state [ 28.417421][ T475] br1: port 2(vx20) entered forwarding state [ 28.571468][ T477] br1: port 3(veth1) entered blocking state [ 28.571797][ T477] br1: port 3(veth1) entered disabled state [ 28.572083][ T477] veth1: entered allmulticast mode [ 28.575247][ T477] veth1: entered promiscuous mode [ 28.689904][ T40] br1: port 3(veth1) entered blocking state [ 28.690272][ T40] br1: port 3(veth1) entered forwarding state [ 29.027213][ T482] br1: port 4(veth2) entered blocking state [ 29.027498][ T482] br1: port 4(veth2) entered disabled state [ 29.027851][ T482] veth2: entered allmulticast mode [ 29.030129][ T482] veth2: entered promiscuous mode [ 29.102232][ T45] br1: port 4(veth2) entered blocking state [ 29.102556][ T45] br1: port 4(veth2) entered forwarding state [ 32.468684][ T532] br2: port 1(w1) entered blocking state [ 32.468920][ T532] br2: port 1(w1) entered disabled state [ 32.469181][ T532] w1: entered allmulticast mode [ 32.471199][ T532] w1: entered promiscuous mode [ 33.114409][ T540] br2: port 2(vx10) entered blocking state [ 33.114722][ T540] br2: port 2(vx10) entered disabled state [ 33.115011][ T540] vx10: entered allmulticast mode [ 33.117882][ T540] vx10: entered promiscuous mode [ 33.118456][ T540] br2: port 2(vx10) entered blocking state [ 33.118718][ T540] br2: port 2(vx10) entered forwarding state [ 33.715781][ T547] br2: port 3(vx20) entered blocking state [ 33.716081][ T547] br2: port 3(vx20) entered disabled state [ 33.716364][ T547] vx20: entered allmulticast mode [ 33.718245][ T547] vx20: entered promiscuous mode [ 33.718743][ T547] br2: port 3(vx20) entered blocking state [ 33.719001][ T547] br2: port 3(vx20) entered forwarding state [ 34.364330][ T68] br2: port 1(w1) entered blocking state [ 34.364718][ T68] br2: port 1(w1) entered forwarding state [ 36.699676][ T590] br2: port 1(w1) entered blocking state [ 36.700046][ T590] br2: port 1(w1) entered disabled state [ 36.700627][ T590] w1: entered allmulticast mode [ 36.702724][ T590] w1: entered promiscuous mode [ 37.304240][ T598] br2: port 2(vx10) entered blocking state [ 37.304584][ T598] br2: port 2(vx10) entered disabled state [ 37.304837][ T598] vx10: entered allmulticast mode [ 37.307611][ T598] vx10: entered promiscuous mode [ 37.308139][ T598] br2: port 2(vx10) entered blocking state [ 37.308376][ T598] br2: port 2(vx10) entered forwarding state [ 37.907283][ T605] br2: port 3(vx20) entered blocking state [ 37.907606][ T605] br2: port 3(vx20) entered disabled state [ 37.907908][ T605] vx20: entered allmulticast mode [ 37.911366][ T605] vx20: entered promiscuous mode [ 37.911977][ T605] br2: port 3(vx20) entered blocking state [ 37.912282][ T605] br2: port 3(vx20) entered forwarding state [ 38.619249][ T40] br2: port 1(w1) entered blocking state [ 38.619530][ T40] br2: port 1(w1) entered forwarding state [ 49.539870][ T692] GACT probability NOT on [ 173.788991][ T1493] veth3: entered promiscuous mode [ 222.068666][ T2420] veth3: left promiscuous mode [ 222.587684][ T2426] veth3: entered promiscuous mode [ 271.659333][ T3353] veth3: left promiscuous mode [ 272.514351][ T3371] vx20: left allmulticast mode [ 272.514855][ T3371] vx20: left promiscuous mode [ 272.515215][ T3371] br1: port 2(vx20) entered disabled state [ 272.601502][ T3372] vx10: left allmulticast mode [ 272.601755][ T3372] vx10: left promiscuous mode [ 272.602093][ T3372] br1: port 1(vx10) entered disabled state [ 277.993871][ T3377] br1: port 1(vx10) entered blocking state [ 277.994260][ T3377] br1: port 1(vx10) entered disabled state [ 277.994533][ T3377] vx10: entered allmulticast mode [ 277.996864][ T3377] vx10: entered promiscuous mode [ 277.998737][ T3377] br1: port 1(vx10) entered blocking state [ 277.999038][ T3377] br1: port 1(vx10) entered forwarding state [ 278.164486][ T3379] br1: port 2(vx20) entered blocking state [ 278.164763][ T3379] br1: port 2(vx20) entered disabled state [ 278.165027][ T3379] vx20: entered allmulticast mode [ 278.167063][ T3379] vx20: entered promiscuous mode [ 278.167576][ T3379] br1: port 2(vx20) entered blocking state [ 278.167811][ T3379] br1: port 2(vx20) entered forwarding state [ 413.315397][ T4207] veth3: entered promiscuous mode [ 462.720516][ T5134] veth3: left promiscuous mode [ 463.191480][ T5140] veth3: entered promiscuous mode [ 513.257470][ T6067] veth3: left promiscuous mode [ 829.685589][ T11] vx20: left allmulticast mode [ 829.685937][ T11] vx20: left promiscuous mode [ 829.686433][ T11] br2: port 3(vx20) entered disabled state [ 829.689149][ T11] vx10: left allmulticast mode [ 829.689396][ T11] vx10: left promiscuous mode [ 829.689733][ T11] br2: port 2(vx10) entered disabled state [ 829.691921][ T11] w1: left allmulticast mode [ 829.692187][ T11] w1: left promiscuous mode [ 829.692526][ T11] br2: port 1(w1) entered disabled state [ 830.331517][ T11] vx20: left allmulticast mode [ 830.331763][ T11] vx20: left promiscuous mode [ 830.332123][ T11] br2: port 3(vx20) entered disabled state [ 830.333698][ T11] vx10: left allmulticast mode [ 830.333884][ T11] vx10: left promiscuous mode [ 830.334191][ T11] br2: port 2(vx10) entered disabled state [ 830.336968][ T11] w1: left allmulticast mode [ 830.337175][ T11] w1: left promiscuous mode [ 830.337469][ T11] br2: port 1(w1) entered disabled state [ 832.703893][T10633] br1: port 4(veth2) entered disabled state [ 832.774531][T10634] veth2: left allmulticast mode [ 832.774758][T10634] veth2: left promiscuous mode [ 832.775079][T10634] br1: port 4(veth2) entered disabled state [ 833.146295][T10638] br1: port 3(veth1) entered disabled state [ 833.231824][T10639] veth1: left allmulticast mode [ 833.232096][T10639] veth1: left promiscuous mode [ 833.232461][T10639] br1: port 3(veth1) entered disabled state [ 833.380637][T10641] vx20: left allmulticast mode [ 833.380921][T10641] vx20: left promiscuous mode [ 833.381755][T10641] br1: port 2(vx20) entered disabled state [ 833.736837][T10645] vx10: left allmulticast mode [ 833.737145][T10645] vx10: left promiscuous mode [ 833.737496][T10645] br1: port 1(vx10) entered disabled state [ 834.709737][T10655] [ 834.709878][T10655] ============================================ [ 834.710160][T10655] WARNING: possible recursive locking detected [ 834.710437][T10655] 6.13.0-rc5-virtme #1 Not tainted [ 834.710668][T10655] -------------------------------------------- [ 834.710942][T10655] ip/10655 is trying to acquire lock: [ 834.711168][T10655] ffffffff9bbd6630 ((netdev_chain).rwsem){++++}-{4:4}, at: blocking_notifier_call_chain+0x50/0x90 [ 834.711634][T10655] [ 834.711634][T10655] but task is already holding lock: [ 834.711966][T10655] ffffffff9bbd6630 ((netdev_chain).rwsem){++++}-{4:4}, at: blocking_notifier_call_chain+0x50/0x90 [ 834.712423][T10655] [ 834.712423][T10655] other info that might help us debug this: [ 834.712755][T10655] Possible unsafe locking scenario: [ 834.712755][T10655] [ 834.713088][T10655] CPU0 [ 834.713265][T10655] ---- [ 834.713431][T10655] lock((netdev_chain).rwsem); [ 834.713658][T10655] lock((netdev_chain).rwsem); [ 834.713883][T10655] [ 834.713883][T10655] *** DEADLOCK *** [ 834.713883][T10655] [ 834.714217][T10655] May be due to missing lock nesting notation [ 834.714217][T10655] [ 834.714547][T10655] 2 locks held by ip/10655: [ 834.714771][T10655] #0: ffffffff9bbe7a48 (rtnl_mutex){+.+.}-{4:4}, at: rtnl_dellink+0x228/0xa40 [ 834.715169][T10655] #1: ffffffff9bbd6630 ((netdev_chain).rwsem){++++}-{4:4}, at: blocking_notifier_call_chain+0x50/0x90 [ 834.715464][T10655] [ 834.715464][T10655] stack backtrace: [ 834.715649][T10655] CPU: 0 UID: 0 PID: 10655 Comm: ip Not tainted 6.13.0-rc5-virtme #1 [ 834.715869][T10655] Hardware name: Bochs Bochs, BIOS Bochs 01/01/2011 [ 834.716057][T10655] Call Trace: [ 834.716169][T10655] [ 834.716247][T10655] dump_stack_lvl+0x82/0xd0 [ 834.716404][T10655] print_deadlock_bug+0x40a/0x650 [ 834.716560][T10655] validate_chain+0x5bf/0xae0 [ 834.716707][T10655] ? __pfx_validate_chain+0x10/0x10 [ 834.716857][T10655] ? hlock_class+0x4e/0x130 [ 834.717003][T10655] ? mark_lock+0x38/0x3e0 [ 834.717116][T10655] __lock_acquire+0xb9a/0x1680 [ 834.717267][T10655] lock_acquire.part.0+0xeb/0x330 [ 834.717415][T10655] ? blocking_notifier_call_chain+0x50/0x90 [ 834.717607][T10655] ? netconsole_process_cleanups_core+0x2e/0x3d0 [ 834.717793][T10655] ? __pfx_lock_acquire.part.0+0x10/0x10 [ 834.717941][T10655] ? trace_lock_acquire+0x14c/0x1f0 [ 834.718090][T10655] ? lock_acquire+0x32/0xc0 [ 834.718242][T10655] ? blocking_notifier_call_chain+0x50/0x90 [ 834.718428][T10655] down_read+0x9f/0x340 [ 834.718547][T10655] ? blocking_notifier_call_chain+0x50/0x90 [ 834.718732][T10655] ? __pfx_down_read+0x10/0x10 [ 834.718878][T10655] ? __lock_release+0x103/0x460 [ 834.719025][T10655] ? netconsole_netdev_event+0x2c1/0x620 [ 834.719171][T10655] ? __pfx___lock_release+0x10/0x10 [ 834.719322][T10655] blocking_notifier_call_chain+0x50/0x90 [ 834.719472][T10655] __netdev_upper_dev_unlink+0xd9/0x220 [ 834.719629][T10655] ? __pfx___netdev_upper_dev_unlink+0x10/0x10 [ 834.719814][T10655] ? __mutex_unlock_slowpath+0x14a/0x3d0 [ 834.719962][T10655] ? __pfx___mutex_unlock_slowpath+0x10/0x10 [ 834.720144][T10655] ? rtnl_is_locked+0x15/0x20 [ 834.720296][T10655] netdev_upper_dev_unlink+0x71/0xa0 [ 834.720443][T10655] ? __pfx_netdev_upper_dev_unlink+0x10/0x10 [ 834.720633][T10655] ? netconsole_netdev_event+0x2c1/0x620 [ 834.720782][T10655] ? mutex_is_locked+0x1c/0x60 [ 834.720936][T10655] ? rtnl_is_locked+0x15/0x20 [ 834.721085][T10655] ? netdev_master_upper_dev_get+0x13/0x120 [ 834.721267][T10655] vrf_device_event+0x78/0xe0 [vrf] [ 834.721420][T10655] notifier_call_chain+0xcd/0x150 [ 834.721581][T10655] blocking_notifier_call_chain+0x66/0x90 [ 834.721733][T10655] unregister_netdevice_many_notify+0xa7b/0x1580 [ 834.721915][T10655] ? hlock_class+0x4e/0x130 [ 834.722066][T10655] ? mark_lock+0x38/0x3e0 [ 834.722187][T10655] ? __pfx_unregister_netdevice_many_notify+0x10/0x10 [ 834.722375][T10655] ? lockdep_hardirqs_on_prepare+0x275/0x410 [ 834.722561][T10655] ? kfree+0xf3/0x330 [ 834.722678][T10655] ? vlan_vid_del+0x310/0x5e0 [ 834.722828][T10655] ? vlan_vid_del+0x310/0x5e0 [ 834.722980][T10655] rtnl_dellink+0x348/0xa40 [ 834.723129][T10655] ? __pfx_rtnl_dellink+0x10/0x10 [ 834.723291][T10655] ? find_held_lock+0x2c/0x110 [ 834.723440][T10655] ? __lock_release+0x103/0x460 [ 834.723593][T10655] ? rtnetlink_rcv_msg+0x6ef/0xc10 [ 834.723749][T10655] ? __pfx___lock_release+0x10/0x10 [ 834.723896][T10655] ? __pfx_lock_acquire.part.0+0x10/0x10 [ 834.724045][T10655] ? rtnetlink_rcv_msg+0x6ef/0xc10 [ 834.724192][T10655] ? __pfx_rtnl_dellink+0x10/0x10 [ 834.724339][T10655] rtnetlink_rcv_msg+0x712/0xc10 [ 834.724484][T10655] ? __pfx_rtnetlink_rcv_msg+0x10/0x10 [ 834.724633][T10655] ? hlock_class+0x4e/0x130 [ 834.724784][T10655] ? mark_lock+0x38/0x3e0 [ 834.724898][T10655] ? __lock_acquire+0xb9a/0x1680 [ 834.725053][T10655] netlink_rcv_skb+0x130/0x360 [ 834.725198][T10655] ? __pfx_rtnetlink_rcv_msg+0x10/0x10 [ 834.725345][T10655] ? __pfx_netlink_rcv_skb+0x10/0x10 [ 834.725497][T10655] ? netlink_deliver_tap+0x13e/0x340 [ 834.725654][T10655] ? netlink_deliver_tap+0xc3/0x340 [ 834.725808][T10655] netlink_unicast+0x44b/0x710 [ 834.725957][T10655] ? __pfx_netlink_unicast+0x10/0x10 [ 834.726104][T10655] ? find_held_lock+0x2c/0x110 [ 834.726253][T10655] netlink_sendmsg+0x723/0xbe0 [ 834.726400][T10655] ? __pfx_netlink_sendmsg+0x10/0x10 [ 834.726551][T10655] ____sys_sendmsg+0x7ac/0xa10 [ 834.726697][T10655] ? __pfx_____sys_sendmsg+0x10/0x10 [ 834.726843][T10655] ? __pfx_copy_msghdr_from_user+0x10/0x10 [ 834.727032][T10655] ___sys_sendmsg+0xee/0x170 [ 834.727179][T10655] ? __pfx____sys_sendmsg+0x10/0x10 [ 834.727327][T10655] ? kasan_save_stack+0x34/0x50 [ 834.727472][T10655] ? kasan_save_stack+0x24/0x50 [ 834.727622][T10655] ? __kasan_record_aux_stack+0x8e/0xa0 [ 834.727770][T10655] ? __call_rcu_common.constprop.0+0xa1/0x4b0 [ 834.727956][T10655] ? __x64_sys_close+0x7c/0xd0 [ 834.728107][T10655] ? do_syscall_64+0xc1/0x1d0 [ 834.728256][T10655] ? entry_SYSCALL_64_after_hwframe+0x77/0x7f [ 834.728438][T10655] ? __lock_acquire+0xb9a/0x1680 [ 834.728588][T10655] ? find_held_lock+0x2c/0x110 [ 834.728738][T10655] ? __lock_release+0x103/0x460 [ 834.728885][T10655] ? __virt_addr_valid+0x22b/0x430 [ 834.729033][T10655] ? __pfx___lock_release+0x10/0x10 [ 834.729186][T10655] ? __pfx_lock_acquire.part.0+0x10/0x10 [ 834.729338][T10655] __sys_sendmsg+0x109/0x1a0 [ 834.729490][T10655] ? __pfx___sys_sendmsg+0x10/0x10 [ 834.729638][T10655] ? __pfx_slab_free_after_rcu_debug+0x10/0x10 [ 834.729828][T10655] do_syscall_64+0xc1/0x1d0 [ 834.729974][T10655] entry_SYSCALL_64_after_hwframe+0x77/0x7f [ 834.730155][T10655] RIP: 0033:0x7f25738dd9a7 [ 834.730316][T10655] Code: 0a 00 f7 d8 64 89 02 48 c7 c0 ff ff ff ff eb b9 0f 1f 00 f3 0f 1e fa 64 8b 04 25 18 00 00 00 85 c0 75 10 b8 2e 00 00 00 0f 05 <48> 3d 00 f0 ff ff 77 51 c3 48 83 ec 28 89 54 24 1c 48 89 74 24 10 [ 834.730834][T10655] RSP: 002b:00007fffc127bc58 EFLAGS: 00000246 ORIG_RAX: 000000000000002e [ 834.731056][T10655] RAX: ffffffffffffffda RBX: 00007fffc127c380 RCX: 00007f25738dd9a7 [ 834.731288][T10655] RDX: 0000000000000000 RSI: 00007fffc127bcc0 RDI: 0000000000000005 [ 834.731506][T10655] RBP: 0000000000000002 R08: 0000000000000003 R09: 0000000000000078 [ 834.731731][T10655] R10: 00007f257379bef8 R11: 0000000000000246 R12: 0000000000000002 [ 834.731962][T10655] R13: 00000000677904aa R14: 0000000000498600 R15: 0000000000000000 [ 834.732192][T10655]