[ 13.220515][ T250] IPVS: Registered protocols (TCP) [ 13.221907][ T250] IPVS: Connection hash table configured (size=4096, memory=32Kbytes) [ 13.223387][ T250] IPVS: ipvs loaded. [ 14.673024][ T267] ip (267) used greatest stack depth: 23616 bytes left [ 15.679501][ T274] bridge: filtering via arp/ip/ip6tables is no longer available by default. Update your scripts to load br_netfilter if you need this. [ 15.932931][ T275] br0: port 1(veth01) entered blocking state [ 15.933607][ T275] br0: port 1(veth01) entered disabled state [ 15.934122][ T275] veth01: entered allmulticast mode [ 15.937087][ T275] veth01: entered promiscuous mode [ 16.146947][ T276] br0: port 2(veth02) entered blocking state [ 16.147402][ T276] br0: port 2(veth02) entered disabled state [ 16.148315][ T276] veth02: entered allmulticast mode [ 16.150704][ T276] veth02: entered promiscuous mode [ 16.700550][ T40] br0: port 1(veth01) entered blocking state [ 16.700947][ T40] br0: port 1(veth01) entered forwarding state [ 17.871999][ T39] br0: port 2(veth02) entered blocking state [ 17.872442][ T39] br0: port 2(veth02) entered forwarding state [ 19.796881][ T292] IPVS: [rr] scheduler registered. [ 19.805865][ T57] IPVS: starting estimator thread 0... [ 19.868569][ T293] IPVS: using max 28 ests per chain, 67200 per kthread [ 21.733061][ T11] veth02: left allmulticast mode [ 21.733439][ T11] veth02: left promiscuous mode [ 21.733800][ T11] br0: port 2(veth02) entered disabled state [ 21.737001][ T11] veth01: left allmulticast mode [ 21.737219][ T11] veth01: left promiscuous mode [ 21.737549][ T11] br0: port 1(veth01) entered disabled state [ 22.229451][ T11] IPVS: stop unused estimator thread 0... [ 25.498881][ T353] br0: port 1(veth01) entered blocking state [ 25.499213][ T353] br0: port 1(veth01) entered disabled state [ 25.499492][ T353] veth01: entered allmulticast mode [ 25.501485][ T353] veth01: entered promiscuous mode [ 25.675163][ T354] br0: port 2(veth02) entered blocking state [ 25.675463][ T354] br0: port 2(veth02) entered disabled state [ 25.675732][ T354] veth02: entered allmulticast mode [ 25.677642][ T354] veth02: entered promiscuous mode [ 26.268342][ T39] br0: port 1(veth01) entered blocking state [ 26.268947][ T39] br0: port 1(veth01) entered forwarding state [ 27.477904][ T46] br0: port 2(veth02) entered blocking state [ 27.478211][ T46] br0: port 2(veth02) entered forwarding state [ 29.351574][ T63] IPVS: starting estimator thread 0... [ 29.416512][ T370] IPVS: using max 44 ests per chain, 105600 per kthread [ 29.888204][ T373] br0: port 2(veth02) entered disabled state [ 30.041248][ T373] veth02 (unregistering): left allmulticast mode [ 30.041761][ T373] veth02 (unregistering): left promiscuous mode [ 30.042017][ T373] br0: port 2(veth02) entered disabled state [ 30.043196][ T373] [ 30.043299][ T373] ============================================ [ 30.043499][ T373] WARNING: possible recursive locking detected [ 30.043694][ T373] 6.13.0-rc5-virtme #1 Not tainted [ 30.043851][ T373] -------------------------------------------- [ 30.044042][ T373] ip/373 is trying to acquire lock: [ 30.044197][ T373] ffffffffbd5dfc30 ((netdev_chain).rwsem){++++}-{4:4}, at: blocking_notifier_call_chain+0x50/0x90 [ 30.044535][ T373] [ 30.044535][ T373] but task is already holding lock: [ 30.044760][ T373] ffffffffbd5dfc30 ((netdev_chain).rwsem){++++}-{4:4}, at: blocking_notifier_call_chain+0x50/0x90 [ 30.045069][ T373] [ 30.045069][ T373] other info that might help us debug this: [ 30.045292][ T373] Possible unsafe locking scenario: [ 30.045292][ T373] [ 30.045518][ T373] CPU0 [ 30.045636][ T373] ---- [ 30.045750][ T373] lock((netdev_chain).rwsem); [ 30.045908][ T373] lock((netdev_chain).rwsem); [ 30.046068][ T373] [ 30.046068][ T373] *** DEADLOCK *** [ 30.046068][ T373] [ 30.046294][ T373] May be due to missing lock nesting notation [ 30.046294][ T373] [ 30.046521][ T373] 2 locks held by ip/373: [ 30.046641][ T373] #0: ffffffffbd5f1088 (rtnl_mutex){+.+.}-{4:4}, at: rtnl_dellink+0x228/0xa40 [ 30.046927][ T373] #1: ffffffffbd5dfc30 ((netdev_chain).rwsem){++++}-{4:4}, at: blocking_notifier_call_chain+0x50/0x90 [ 30.047240][ T373] [ 30.047240][ T373] stack backtrace: [ 30.047431][ T373] CPU: 3 UID: 0 PID: 373 Comm: ip Not tainted 6.13.0-rc5-virtme #1 [ 30.047662][ T373] Hardware name: Bochs Bochs, BIOS Bochs 01/01/2011 [ 30.047862][ T373] Call Trace: [ 30.047982][ T373] [ 30.048062][ T373] dump_stack_lvl+0x82/0xd0 [ 30.048220][ T373] print_deadlock_bug+0x40a/0x650 [ 30.048376][ T373] validate_chain+0x5bf/0xae0 [ 30.048535][ T373] ? __pfx_validate_chain+0x10/0x10 [ 30.048693][ T373] ? hlock_class+0x4e/0x130 [ 30.048849][ T373] ? mark_lock+0x38/0x3e0 [ 30.048975][ T373] __lock_acquire+0xb9a/0x1680 [ 30.049133][ T373] lock_acquire.part.0+0xeb/0x330 [ 30.049290][ T373] ? blocking_notifier_call_chain+0x50/0x90 [ 30.049485][ T373] ? __pfx_lock_acquire.part.0+0x10/0x10 [ 30.049640][ T373] ? trace_lock_acquire+0x14c/0x1f0 [ 30.049798][ T373] ? lock_acquire+0x32/0xc0 [ 30.049956][ T373] ? blocking_notifier_call_chain+0x50/0x90 [ 30.050155][ T373] down_read+0x9f/0x340 [ 30.050274][ T373] ? blocking_notifier_call_chain+0x50/0x90 [ 30.050467][ T373] ? __pfx_down_read+0x10/0x10 [ 30.050622][ T373] ? __pfx___lock_release+0x10/0x10 [ 30.050778][ T373] blocking_notifier_call_chain+0x50/0x90 [ 30.050941][ T373] __netdev_upper_dev_unlink+0xd9/0x220 [ 30.051099][ T373] ? __pfx___netdev_upper_dev_unlink+0x10/0x10 [ 30.051297][ T373] ? find_held_lock+0x2c/0x110 [ 30.051454][ T373] ? del_nbp+0x377/0xbe0 [bridge] [ 30.051657][ T373] ? hlock_class+0x4e/0x130 [ 30.051814][ T373] netdev_upper_dev_unlink+0x71/0xa0 [ 30.051976][ T373] ? __pfx_netdev_upper_dev_unlink+0x10/0x10 [ 30.052167][ T373] ? mutex_spin_on_owner+0x282/0x2b0 [ 30.052326][ T373] ? rtnl_is_locked+0x15/0x20 [ 30.052492][ T373] del_nbp+0x602/0xbe0 [bridge] [ 30.052683][ T373] br_del_if+0xa3/0x1f0 [bridge] [ 30.052901][ T373] br_device_event+0x53d/0x800 [bridge] [ 30.053088][ T373] ? __pfx_br_device_event+0x10/0x10 [bridge] [ 30.053313][ T373] ? ip_vs_dst_event+0x3d/0x770 [ip_vs] [ 30.053484][ T373] ? packet_notifier+0x3b0/0x810 [ 30.053641][ T373] notifier_call_chain+0xcd/0x150 [ 30.053805][ T373] blocking_notifier_call_chain+0x66/0x90 [ 30.053964][ T373] unregister_netdevice_many_notify+0xa7b/0x1580 [ 30.054158][ T373] ? __pfx_unregister_netdevice_many_notify+0x10/0x10 [ 30.054353][ T373] ? rtnl_dellink+0x228/0xa40 [ 30.054508][ T373] ? sock_def_readable+0x11e/0x300 [ 30.054666][ T373] ? rtnl_is_locked+0x15/0x20 [ 30.054821][ T373] ? unregister_netdevice_queue+0x70/0x410 [ 30.055022][ T373] ? __pfx_unregister_netdevice_queue+0x10/0x10 [ 30.055214][ T373] ? __pfx_unregister_netdevice_queue+0x10/0x10 [ 30.055406][ T373] rtnl_dellink+0x348/0xa40 [ 30.055564][ T373] ? __pfx_rtnl_dellink+0x10/0x10 [ 30.055738][ T373] ? find_held_lock+0x2c/0x110 [ 30.055899][ T373] ? __lock_release+0x103/0x460 [ 30.056052][ T373] ? rtnetlink_rcv_msg+0x6ef/0xc10 [ 30.056214][ T373] ? __pfx___lock_release+0x10/0x10 [ 30.056369][ T373] ? __pfx_lock_acquire.part.0+0x10/0x10 [ 30.056532][ T373] ? rtnetlink_rcv_msg+0x6ef/0xc10 [ 30.056688][ T373] ? __pfx_rtnl_dellink+0x10/0x10 [ 30.056847][ T373] rtnetlink_rcv_msg+0x712/0xc10 [ 30.057005][ T373] ? __pfx_rtnetlink_rcv_msg+0x10/0x10 [ 30.057158][ T373] ? hlock_class+0x4e/0x130 [ 30.057317][ T373] ? mark_lock+0x38/0x3e0 [ 30.057435][ T373] ? __lock_acquire+0xb9a/0x1680 [ 30.057596][ T373] netlink_rcv_skb+0x130/0x360 [ 30.057753][ T373] ? __pfx_rtnetlink_rcv_msg+0x10/0x10 [ 30.057911][ T373] ? __pfx_netlink_rcv_skb+0x10/0x10 [ 30.058069][ T373] ? netlink_deliver_tap+0x13e/0x340 [ 30.058226][ T373] ? netlink_deliver_tap+0xc3/0x340 [ 30.058384][ T373] netlink_unicast+0x44b/0x710 [ 30.058539][ T373] ? __pfx_netlink_unicast+0x10/0x10 [ 30.058692][ T373] ? find_held_lock+0x2c/0x110 [ 30.058850][ T373] ? __lock_release+0x103/0x460 [ 30.059011][ T373] netlink_sendmsg+0x723/0xbe0 [ 30.059166][ T373] ? __pfx_netlink_sendmsg+0x10/0x10 [ 30.059320][ T373] ? _copy_from_user+0x55/0xa0 [ 30.059478][ T373] ____sys_sendmsg+0x7ac/0xa10 [ 30.059637][ T373] ? __pfx_____sys_sendmsg+0x10/0x10 [ 30.059791][ T373] ? __pfx_copy_msghdr_from_user+0x10/0x10 [ 30.059997][ T373] ___sys_sendmsg+0xee/0x170 [ 30.060152][ T373] ? __pfx____sys_sendmsg+0x10/0x10 [ 30.060306][ T373] ? kasan_save_stack+0x34/0x50 [ 30.060465][ T373] ? kasan_save_stack+0x24/0x50 [ 30.060622][ T373] ? __kasan_record_aux_stack+0x8e/0xa0 [ 30.060783][ T373] ? __call_rcu_common.constprop.0+0xa1/0x4b0 [ 30.060981][ T373] ? __x64_sys_close+0x7c/0xd0 [ 30.061139][ T373] ? do_syscall_64+0xc1/0x1d0 [ 30.061296][ T373] ? entry_SYSCALL_64_after_hwframe+0x77/0x7f [ 30.061486][ T373] ? __lock_acquire+0xb9a/0x1680 [ 30.061644][ T373] ? find_held_lock+0x2c/0x110 [ 30.061800][ T373] ? __lock_release+0x103/0x460 [ 30.061966][ T373] ? __virt_addr_valid+0x22b/0x430 [ 30.062120][ T373] ? __pfx___lock_release+0x10/0x10 [ 30.062274][ T373] ? __pfx_lock_acquire.part.0+0x10/0x10 [ 30.062431][ T373] __sys_sendmsg+0x109/0x1a0 [ 30.062585][ T373] ? __pfx___sys_sendmsg+0x10/0x10 [ 30.062740][ T373] ? __pfx_slab_free_after_rcu_debug+0x10/0x10 [ 30.062958][ T373] do_syscall_64+0xc1/0x1d0 [ 30.063122][ T373] entry_SYSCALL_64_after_hwframe+0x77/0x7f [ 30.063315][ T373] RIP: 0033:0x7f8ada5039a7 [ 30.063478][ T373] Code: 0a 00 f7 d8 64 89 02 48 c7 c0 ff ff ff ff eb b9 0f 1f 00 f3 0f 1e fa 64 8b 04 25 18 00 00 00 85 c0 75 10 b8 2e 00 00 00 0f 05 <48> 3d 00 f0 ff ff 77 51 c3 48 83 ec 28 89 54 24 1c 48 89 74 24 10 [ 30.064028][ T373] RSP: 002b:00007fff1d6e2898 EFLAGS: 00000246 ORIG_RAX: 000000000000002e [ 30.064263][ T373] RAX: ffffffffffffffda RBX: 00007fff1d6e2fc0 RCX: 00007f8ada5039a7 [ 30.064498][ T373] RDX: 0000000000000000 RSI: 00007fff1d6e2900 RDI: 0000000000000005 [ 30.064729][ T373] RBP: 0000000000000001 R08: 0000000000000003 R09: 0000000000000078 [ 30.064967][ T373] R10: 00007f8ada3c1ef8 R11: 0000000000000246 R12: 0000000000000001 [ 30.065207][ T373] R13: 0000000067792f28 R14: 0000000000498600 R15: 0000000000000000 [ 30.065443][ T373] [ 30.830999][ T11] veth01: left allmulticast mode [ 30.831210][ T11] veth01: left promiscuous mode [ 30.831492][ T11] br0: port 1(veth01) entered disabled state [ 31.216090][ T11] IPVS: stop unused estimator thread 0... [ 33.558975][ T426] br0: port 1(veth01) entered blocking state [ 33.559200][ T426] br0: port 1(veth01) entered disabled state [ 33.559404][ T426] veth01: entered allmulticast mode [ 33.560886][ T426] veth01: entered promiscuous mode [ 33.686239][ T427] br0: port 2(veth02) entered blocking state [ 33.686492][ T427] br0: port 2(veth02) entered disabled state [ 33.686721][ T427] veth02: entered allmulticast mode [ 33.688307][ T427] veth02: entered promiscuous mode [ 34.070057][ T40] br0: port 1(veth01) entered blocking state [ 34.070326][ T40] br0: port 1(veth01) entered forwarding state [ 34.820456][ T40] br0: port 2(veth02) entered blocking state [ 34.820693][ T40] br0: port 2(veth02) entered forwarding state [ 36.378066][ T441] ipip: IPv4 and MPLS over IPv4 tunneling driver [ 36.945522][ T231] IPVS: starting estimator thread 0... [ 37.008538][ T450] IPVS: using max 92 ests per chain, 220800 per kthread [ 38.745985][ T11] veth02: left allmulticast mode [ 38.746253][ T11] veth02: left promiscuous mode [ 38.746579][ T11] br0: port 2(veth02) entered disabled state [ 38.747932][ T11] veth01: left allmulticast mode [ 38.748096][ T11] veth01: left promiscuous mode [ 38.748369][ T11] br0: port 1(veth01) entered disabled state [ 39.159496][ T11] IPVS: stop unused estimator thread 0...