[ 100.680456][ T1022] IPVS: Registered protocols (TCP) [ 100.681808][ T1022] IPVS: Connection hash table configured (size=4096, memory=32Kbytes) [ 100.683233][ T1022] IPVS: ipvs loaded. [ 102.985780][ T1043] bridge: filtering via arp/ip/ip6tables is no longer available by default. Update your scripts to load br_netfilter if you need this. [ 103.193337][ T1044] br0: port 1(veth01) entered blocking state [ 103.193822][ T1044] br0: port 1(veth01) entered disabled state [ 103.194107][ T1044] veth01: entered allmulticast mode [ 103.196228][ T1044] veth01: entered promiscuous mode [ 103.393919][ T1045] br0: port 2(veth02) entered blocking state [ 103.394238][ T1045] br0: port 2(veth02) entered disabled state [ 103.395512][ T1045] veth02: entered allmulticast mode [ 103.397478][ T1045] veth02: entered promiscuous mode [ 103.959132][ T37] br0: port 1(veth01) entered blocking state [ 103.959627][ T37] br0: port 1(veth01) entered forwarding state [ 105.058499][ T69] br0: port 2(veth02) entered blocking state [ 105.058830][ T69] br0: port 2(veth02) entered forwarding state [ 106.944016][ T1061] IPVS: [rr] scheduler registered. [ 106.963855][ T61] IPVS: starting estimator thread 0... [ 107.027438][ T1062] IPVS: using max 43 ests per chain, 103200 per kthread [ 108.852872][ T67] veth02: left allmulticast mode [ 108.853285][ T67] veth02: left promiscuous mode [ 108.853662][ T67] br0: port 2(veth02) entered disabled state [ 108.862677][ T67] veth01: left allmulticast mode [ 108.862891][ T67] veth01: left promiscuous mode [ 108.863189][ T67] br0: port 1(veth01) entered disabled state [ 109.417285][ T67] IPVS: stop unused estimator thread 0... [ 112.645741][ T1121] br0: port 1(veth01) entered blocking state [ 112.646096][ T1121] br0: port 1(veth01) entered disabled state [ 112.646446][ T1121] veth01: entered allmulticast mode [ 112.648432][ T1121] veth01: entered promiscuous mode [ 112.807786][ T1122] br0: port 2(veth02) entered blocking state [ 112.808144][ T1122] br0: port 2(veth02) entered disabled state [ 112.808954][ T1122] veth02: entered allmulticast mode [ 112.819088][ T1122] veth02: entered promiscuous mode [ 113.347750][ T69] br0: port 1(veth01) entered blocking state [ 113.348044][ T69] br0: port 1(veth01) entered forwarding state [ 114.410721][ T69] br0: port 2(veth02) entered blocking state [ 114.411231][ T69] br0: port 2(veth02) entered forwarding state [ 116.196700][ T51] IPVS: starting estimator thread 0... [ 116.259360][ T1138] IPVS: using max 42 ests per chain, 100800 per kthread [ 116.763206][ T1141] br0: port 2(veth02) entered disabled state [ 116.910242][ T1141] veth02 (unregistering): left allmulticast mode [ 116.910784][ T1141] veth02 (unregistering): left promiscuous mode [ 116.911055][ T1141] br0: port 2(veth02) entered disabled state [ 116.912658][ T1141] [ 116.912792][ T1141] ============================================ [ 116.913014][ T1141] WARNING: possible recursive locking detected [ 116.913235][ T1141] 6.13.0-rc5-virtme #1 Not tainted [ 116.913416][ T1141] -------------------------------------------- [ 116.913635][ T1141] ip/1141 is trying to acquire lock: [ 116.913821][ T1141] ffffffff875dfc30 ((netdev_chain).rwsem){++++}-{4:4}, at: blocking_notifier_call_chain+0x50/0x90 [ 116.914160][ T1141] [ 116.914160][ T1141] but task is already holding lock: [ 116.914432][ T1141] ffffffff875dfc30 ((netdev_chain).rwsem){++++}-{4:4}, at: blocking_notifier_call_chain+0x50/0x90 [ 116.914795][ T1141] [ 116.914795][ T1141] other info that might help us debug this: [ 116.915031][ T1141] Possible unsafe locking scenario: [ 116.915031][ T1141] [ 116.915254][ T1141] CPU0 [ 116.915405][ T1141] ---- [ 116.915523][ T1141] lock((netdev_chain).rwsem); [ 116.915680][ T1141] lock((netdev_chain).rwsem); [ 116.915833][ T1141] [ 116.915833][ T1141] *** DEADLOCK *** [ 116.915833][ T1141] [ 116.916064][ T1141] May be due to missing lock nesting notation [ 116.916064][ T1141] [ 116.916289][ T1141] 2 locks held by ip/1141: [ 116.916442][ T1141] #0: ffffffff875f1088 (rtnl_mutex){+.+.}-{4:4}, at: rtnl_dellink+0x228/0xa40 [ 116.916728][ T1141] #1: ffffffff875dfc30 ((netdev_chain).rwsem){++++}-{4:4}, at: blocking_notifier_call_chain+0x50/0x90 [ 116.917038][ T1141] [ 116.917038][ T1141] stack backtrace: [ 116.917236][ T1141] CPU: 3 UID: 0 PID: 1141 Comm: ip Not tainted 6.13.0-rc5-virtme #1 [ 116.917473][ T1141] Hardware name: Bochs Bochs, BIOS Bochs 01/01/2011 [ 116.917665][ T1141] Call Trace: [ 116.917791][ T1141] [ 116.917875][ T1141] dump_stack_lvl+0x82/0xd0 [ 116.918039][ T1141] print_deadlock_bug+0x40a/0x650 [ 116.918199][ T1141] validate_chain+0x5bf/0xae0 [ 116.918379][ T1141] ? __pfx_validate_chain+0x10/0x10 [ 116.918545][ T1141] ? hlock_class+0x4e/0x130 [ 116.918709][ T1141] ? mark_lock+0x38/0x3e0 [ 116.918828][ T1141] __lock_acquire+0xb9a/0x1680 [ 116.918987][ T1141] lock_acquire.part.0+0xeb/0x330 [ 116.919142][ T1141] ? blocking_notifier_call_chain+0x50/0x90 [ 116.919336][ T1141] ? find_held_lock+0x2c/0x110 [ 116.919499][ T1141] ? __pfx_lock_acquire.part.0+0x10/0x10 [ 116.919679][ T1141] ? trace_lock_acquire+0x14c/0x1f0 [ 116.919849][ T1141] ? lock_acquire+0x32/0xc0 [ 116.920006][ T1141] ? blocking_notifier_call_chain+0x50/0x90 [ 116.920201][ T1141] down_read+0x9f/0x340 [ 116.920319][ T1141] ? blocking_notifier_call_chain+0x50/0x90 [ 116.920521][ T1141] ? __pfx_down_read+0x10/0x10 [ 116.920683][ T1141] ? lockdep_hardirqs_on_prepare+0x12b/0x410 [ 116.920877][ T1141] blocking_notifier_call_chain+0x50/0x90 [ 116.921033][ T1141] __netdev_upper_dev_unlink+0xd9/0x220 [ 116.921191][ T1141] ? __pfx___netdev_upper_dev_unlink+0x10/0x10 [ 116.921390][ T1141] ? find_held_lock+0x2c/0x110 [ 116.921546][ T1141] ? del_nbp+0x377/0xbe0 [bridge] [ 116.921749][ T1141] ? hlock_class+0x4e/0x130 [ 116.921920][ T1141] netdev_upper_dev_unlink+0x71/0xa0 [ 116.922082][ T1141] ? __pfx_netdev_upper_dev_unlink+0x10/0x10 [ 116.922278][ T1141] ? mutex_spin_on_owner+0x282/0x2b0 [ 116.922439][ T1141] ? rtnl_is_locked+0x15/0x20 [ 116.922604][ T1141] del_nbp+0x602/0xbe0 [bridge] [ 116.922802][ T1141] br_del_if+0xa3/0x1f0 [bridge] [ 116.922995][ T1141] br_device_event+0x53d/0x800 [bridge] [ 116.923191][ T1141] ? __pfx_br_device_event+0x10/0x10 [bridge] [ 116.923420][ T1141] ? ip_vs_dst_event+0x3d/0x770 [ip_vs] [ 116.923595][ T1141] ? nft_offload_netdev_event+0x158/0x3b0 [nf_tables] [ 116.923834][ T1141] notifier_call_chain+0xcd/0x150 [ 116.923991][ T1141] blocking_notifier_call_chain+0x66/0x90 [ 116.924150][ T1141] unregister_netdevice_many_notify+0xa7b/0x1580 [ 116.924348][ T1141] ? __pfx_unregister_netdevice_many_notify+0x10/0x10 [ 116.924552][ T1141] ? rtnl_dellink+0x228/0xa40 [ 116.924710][ T1141] ? sock_def_readable+0x11e/0x300 [ 116.924870][ T1141] ? rtnl_is_locked+0x15/0x20 [ 116.925028][ T1141] ? unregister_netdevice_queue+0x70/0x410 [ 116.925222][ T1141] ? __pfx_unregister_netdevice_queue+0x10/0x10 [ 116.925418][ T1141] ? __pfx_unregister_netdevice_queue+0x10/0x10 [ 116.925621][ T1141] rtnl_dellink+0x348/0xa40 [ 116.925781][ T1141] ? __pfx_rtnl_dellink+0x10/0x10 [ 116.925953][ T1141] ? find_held_lock+0x2c/0x110 [ 116.926113][ T1141] ? __lock_release+0x103/0x460 [ 116.926271][ T1141] ? rtnetlink_rcv_msg+0x6ef/0xc10 [ 116.926432][ T1141] ? __pfx___lock_release+0x10/0x10 [ 116.926590][ T1141] ? __pfx_lock_acquire.part.0+0x10/0x10 [ 116.926755][ T1141] ? rtnetlink_rcv_msg+0x6ef/0xc10 [ 116.926914][ T1141] ? __pfx_rtnl_dellink+0x10/0x10 [ 116.927076][ T1141] rtnetlink_rcv_msg+0x712/0xc10 [ 116.927233][ T1141] ? __pfx_rtnetlink_rcv_msg+0x10/0x10 [ 116.927389][ T1141] ? hlock_class+0x4e/0x130 [ 116.927558][ T1141] ? mark_lock+0x38/0x3e0 [ 116.927680][ T1141] ? __lock_acquire+0xb9a/0x1680 [ 116.927838][ T1141] netlink_rcv_skb+0x130/0x360 [ 116.928000][ T1141] ? __pfx_rtnetlink_rcv_msg+0x10/0x10 [ 116.928157][ T1141] ? __pfx_netlink_rcv_skb+0x10/0x10 [ 116.928327][ T1141] ? netlink_deliver_tap+0x13e/0x340 [ 116.928490][ T1141] ? netlink_deliver_tap+0xc3/0x340 [ 116.928650][ T1141] netlink_unicast+0x44b/0x710 [ 116.928816][ T1141] ? __pfx_netlink_unicast+0x10/0x10 [ 116.928973][ T1141] ? find_held_lock+0x2c/0x110 [ 116.929139][ T1141] ? __lock_release+0x103/0x460 [ 116.929298][ T1141] netlink_sendmsg+0x723/0xbe0 [ 116.929455][ T1141] ? __pfx_netlink_sendmsg+0x10/0x10 [ 116.929613][ T1141] ? _copy_from_user+0x55/0xa0 [ 116.929780][ T1141] ____sys_sendmsg+0x7ac/0xa10 [ 116.929940][ T1141] ? __pfx_____sys_sendmsg+0x10/0x10 [ 116.930094][ T1141] ? __pfx_copy_msghdr_from_user+0x10/0x10 [ 116.930299][ T1141] ___sys_sendmsg+0xee/0x170 [ 116.930462][ T1141] ? __pfx____sys_sendmsg+0x10/0x10 [ 116.930620][ T1141] ? kasan_save_stack+0x34/0x50 [ 116.930781][ T1141] ? kasan_save_stack+0x24/0x50 [ 116.930941][ T1141] ? __kasan_record_aux_stack+0x8e/0xa0 [ 116.931098][ T1141] ? __call_rcu_common.constprop.0+0xa1/0x4b0 [ 116.931294][ T1141] ? __x64_sys_close+0x7c/0xd0 [ 116.931465][ T1141] ? do_syscall_64+0xc1/0x1d0 [ 116.931626][ T1141] ? entry_SYSCALL_64_after_hwframe+0x77/0x7f [ 116.931818][ T1141] ? __lock_acquire+0xb9a/0x1680 [ 116.931978][ T1141] ? find_held_lock+0x2c/0x110 [ 116.932137][ T1141] ? __lock_release+0x103/0x460 [ 116.932294][ T1141] ? __virt_addr_valid+0x22b/0x430 [ 116.932454][ T1141] ? __pfx___lock_release+0x10/0x10 [ 116.932610][ T1141] ? __pfx_lock_acquire.part.0+0x10/0x10 [ 116.932772][ T1141] __sys_sendmsg+0x109/0x1a0 [ 116.932928][ T1141] ? __pfx___sys_sendmsg+0x10/0x10 [ 116.933083][ T1141] ? __pfx_slab_free_after_rcu_debug+0x10/0x10 [ 116.933282][ T1141] do_syscall_64+0xc1/0x1d0 [ 116.933439][ T1141] entry_SYSCALL_64_after_hwframe+0x77/0x7f [ 116.933633][ T1141] RIP: 0033:0x7fbcef3fd9a7 [ 116.933797][ T1141] Code: 0a 00 f7 d8 64 89 02 48 c7 c0 ff ff ff ff eb b9 0f 1f 00 f3 0f 1e fa 64 8b 04 25 18 00 00 00 85 c0 75 10 b8 2e 00 00 00 0f 05 <48> 3d 00 f0 ff ff 77 51 c3 48 83 ec 28 89 54 24 1c 48 89 74 24 10 [ 116.934352][ T1141] RSP: 002b:00007ffcdb7b3928 EFLAGS: 00000246 ORIG_RAX: 000000000000002e [ 116.934597][ T1141] RAX: ffffffffffffffda RBX: 00007ffcdb7b4050 RCX: 00007fbcef3fd9a7 [ 116.934837][ T1141] RDX: 0000000000000000 RSI: 00007ffcdb7b3990 RDI: 0000000000000005 [ 116.935075][ T1141] RBP: 0000000000000001 R08: 0000000000000003 R09: 0000000000000078 [ 116.935307][ T1141] R10: 00007fbcef2bbef8 R11: 0000000000000246 R12: 0000000000000001 [ 116.935539][ T1141] R13: 00000000677901d4 R14: 0000000000498600 R15: 0000000000000000 [ 116.935776][ T1141] [ 116.967165][ T1141] ip (1141) used greatest stack depth: 23448 bytes left [ 117.759500][ T67] veth01: left allmulticast mode [ 117.759688][ T67] veth01: left promiscuous mode [ 117.759996][ T67] br0: port 1(veth01) entered disabled state [ 118.186661][ T67] IPVS: stop unused estimator thread 0... [ 120.530509][ T1193] br0: port 1(veth01) entered blocking state [ 120.530772][ T1193] br0: port 1(veth01) entered disabled state [ 120.531043][ T1193] veth01: entered allmulticast mode [ 120.532920][ T1193] veth01: entered promiscuous mode [ 120.666000][ T1194] br0: port 2(veth02) entered blocking state [ 120.666256][ T1194] br0: port 2(veth02) entered disabled state [ 120.667113][ T1194] veth02: entered allmulticast mode [ 120.668729][ T1194] veth02: entered promiscuous mode [ 121.082650][ T480] br0: port 1(veth01) entered blocking state [ 121.082915][ T480] br0: port 1(veth01) entered forwarding state [ 121.882052][ T480] br0: port 2(veth02) entered blocking state [ 121.882358][ T480] br0: port 2(veth02) entered forwarding state [ 123.369775][ T1208] ipip: IPv4 and MPLS over IPv4 tunneling driver [ 124.011990][ T51] IPVS: starting estimator thread 0... [ 124.075349][ T1217] IPVS: using max 93 ests per chain, 223200 per kthread [ 126.042430][ T67] veth02: left allmulticast mode [ 126.042632][ T67] veth02: left promiscuous mode [ 126.042956][ T67] br0: port 2(veth02) entered disabled state [ 126.044030][ T67] veth01: left allmulticast mode [ 126.044202][ T67] veth01: left promiscuous mode [ 126.044695][ T67] br0: port 1(veth01) entered disabled state [ 126.534276][ T67] IPVS: stop unused estimator thread 0...