[ 128.689883][ T1463] IPVS: Registered protocols (TCP) [ 128.691139][ T1463] IPVS: Connection hash table configured (size=4096, memory=32Kbytes) [ 128.692497][ T1463] IPVS: ipvs loaded. [ 130.863853][ T1484] bridge: filtering via arp/ip/ip6tables is no longer available by default. Update your scripts to load br_netfilter if you need this. [ 131.088423][ T1486] br0: port 1(veth01) entered blocking state [ 131.088858][ T1486] br0: port 1(veth01) entered disabled state [ 131.089124][ T1486] veth01: entered allmulticast mode [ 131.091099][ T1486] veth01: entered promiscuous mode [ 131.272927][ T1487] br0: port 2(veth02) entered blocking state [ 131.273209][ T1487] br0: port 2(veth02) entered disabled state [ 131.273492][ T1487] veth02: entered allmulticast mode [ 131.275375][ T1487] veth02: entered promiscuous mode [ 131.822563][ T37] br0: port 1(veth01) entered blocking state [ 131.823219][ T37] br0: port 1(veth01) entered forwarding state [ 132.924492][ T40] br0: port 2(veth02) entered blocking state [ 132.924929][ T40] br0: port 2(veth02) entered forwarding state [ 134.829680][ T1503] IPVS: [rr] scheduler registered. [ 134.838772][ T60] IPVS: starting estimator thread 0... [ 134.901388][ T1504] IPVS: using max 44 ests per chain, 105600 per kthread [ 136.808087][ T67] veth02: left allmulticast mode [ 136.808489][ T67] veth02: left promiscuous mode [ 136.808828][ T67] br0: port 2(veth02) entered disabled state [ 136.811728][ T67] veth01: left allmulticast mode [ 136.811912][ T67] veth01: left promiscuous mode [ 136.812203][ T67] br0: port 1(veth01) entered disabled state [ 137.297195][ T67] IPVS: stop unused estimator thread 0... [ 140.641209][ T1563] br0: port 1(veth01) entered blocking state [ 140.641519][ T1563] br0: port 1(veth01) entered disabled state [ 140.641797][ T1563] veth01: entered allmulticast mode [ 140.643721][ T1563] veth01: entered promiscuous mode [ 140.817654][ T1564] br0: port 2(veth02) entered blocking state [ 140.817961][ T1564] br0: port 2(veth02) entered disabled state [ 140.818246][ T1564] veth02: entered allmulticast mode [ 140.820164][ T1564] veth02: entered promiscuous mode [ 141.356109][ T40] br0: port 1(veth01) entered blocking state [ 141.356422][ T40] br0: port 1(veth01) entered forwarding state [ 142.410775][ T46] br0: port 2(veth02) entered blocking state [ 142.411056][ T46] br0: port 2(veth02) entered forwarding state [ 144.182662][ T60] IPVS: starting estimator thread 0... [ 144.245374][ T1580] IPVS: using max 24 ests per chain, 57600 per kthread [ 144.676433][ T1583] br0: port 2(veth02) entered disabled state [ 144.835131][ T1583] veth02 (unregistering): left allmulticast mode [ 144.835705][ T1583] veth02 (unregistering): left promiscuous mode [ 144.836089][ T1583] br0: port 2(veth02) entered disabled state [ 144.837846][ T1583] [ 144.837989][ T1583] ============================================ [ 144.838301][ T1583] WARNING: possible recursive locking detected [ 144.838633][ T1583] 6.13.0-rc5-virtme #1 Not tainted [ 144.838891][ T1583] -------------------------------------------- [ 144.839209][ T1583] ip/1583 is trying to acquire lock: [ 144.839467][ T1583] ffffffffb81dfc30 ((netdev_chain).rwsem){++++}-{4:4}, at: blocking_notifier_call_chain+0x50/0x90 [ 144.839993][ T1583] [ 144.839993][ T1583] but task is already holding lock: [ 144.840377][ T1583] ffffffffb81dfc30 ((netdev_chain).rwsem){++++}-{4:4}, at: blocking_notifier_call_chain+0x50/0x90 [ 144.840899][ T1583] [ 144.840899][ T1583] other info that might help us debug this: [ 144.841279][ T1583] Possible unsafe locking scenario: [ 144.841279][ T1583] [ 144.841665][ T1583] CPU0 [ 144.841875][ T1583] ---- [ 144.842067][ T1583] lock((netdev_chain).rwsem); [ 144.842332][ T1583] lock((netdev_chain).rwsem); [ 144.842592][ T1583] [ 144.842592][ T1583] *** DEADLOCK *** [ 144.842592][ T1583] [ 144.842970][ T1583] May be due to missing lock nesting notation [ 144.842970][ T1583] [ 144.843345][ T1583] 2 locks held by ip/1583: [ 144.843607][ T1583] #0: ffffffffb81f1088 (rtnl_mutex){+.+.}-{4:4}, at: rtnl_dellink+0x228/0xa40 [ 144.844079][ T1583] #1: ffffffffb81dfc30 ((netdev_chain).rwsem){++++}-{4:4}, at: blocking_notifier_call_chain+0x50/0x90 [ 144.844615][ T1583] [ 144.844615][ T1583] stack backtrace: [ 144.844935][ T1583] CPU: 0 UID: 0 PID: 1583 Comm: ip Not tainted 6.13.0-rc5-virtme #1 [ 144.845326][ T1583] Hardware name: Bochs Bochs, BIOS Bochs 01/01/2011 [ 144.845655][ T1583] Call Trace: [ 144.845854][ T1583] [ 144.845999][ T1583] dump_stack_lvl+0x82/0xd0 [ 144.846269][ T1583] print_deadlock_bug+0x40a/0x650 [ 144.846545][ T1583] validate_chain+0x5bf/0xae0 [ 144.846813][ T1583] ? __pfx_validate_chain+0x10/0x10 [ 144.847074][ T1583] ? hlock_class+0x4e/0x130 [ 144.847322][ T1583] ? mark_lock+0x38/0x3e0 [ 144.847511][ T1583] __lock_acquire+0xb9a/0x1680 [ 144.847762][ T1583] lock_acquire.part.0+0xeb/0x330 [ 144.848020][ T1583] ? blocking_notifier_call_chain+0x50/0x90 [ 144.848327][ T1583] ? find_held_lock+0x2c/0x110 [ 144.848572][ T1583] ? __pfx_lock_acquire.part.0+0x10/0x10 [ 144.848820][ T1583] ? trace_lock_acquire+0x14c/0x1f0 [ 144.849067][ T1583] ? lock_acquire+0x32/0xc0 [ 144.849313][ T1583] ? blocking_notifier_call_chain+0x50/0x90 [ 144.849623][ T1583] down_read+0x9f/0x340 [ 144.849820][ T1583] ? blocking_notifier_call_chain+0x50/0x90 [ 144.850124][ T1583] ? __pfx_down_read+0x10/0x10 [ 144.850366][ T1583] ? lockdep_hardirqs_on_prepare+0x12b/0x410 [ 144.850676][ T1583] blocking_notifier_call_chain+0x50/0x90 [ 144.850922][ T1583] __netdev_upper_dev_unlink+0xd9/0x220 [ 144.851167][ T1583] ? __pfx___netdev_upper_dev_unlink+0x10/0x10 [ 144.851469][ T1583] ? find_held_lock+0x2c/0x110 [ 144.851738][ T1583] ? del_nbp+0x377/0xbe0 [bridge] [ 144.852067][ T1583] ? hlock_class+0x4e/0x130 [ 144.852320][ T1583] netdev_upper_dev_unlink+0x71/0xa0 [ 144.852568][ T1583] ? __pfx_netdev_upper_dev_unlink+0x10/0x10 [ 144.852869][ T1583] ? mutex_spin_on_owner+0x282/0x2b0 [ 144.853116][ T1583] ? rtnl_is_locked+0x15/0x20 [ 144.853361][ T1583] del_nbp+0x602/0xbe0 [bridge] [ 144.853681][ T1583] br_del_if+0xa3/0x1f0 [bridge] [ 144.853990][ T1583] br_device_event+0x53d/0x800 [bridge] [ 144.854297][ T1583] ? __pfx_br_device_event+0x10/0x10 [bridge] [ 144.854662][ T1583] ? ip_vs_dst_event+0x3d/0x770 [ip_vs] [ 144.854933][ T1583] ? nft_offload_netdev_event+0x158/0x3b0 [nf_tables] [ 144.855292][ T1583] notifier_call_chain+0xcd/0x150 [ 144.855547][ T1583] blocking_notifier_call_chain+0x66/0x90 [ 144.855817][ T1583] unregister_netdevice_many_notify+0xa7b/0x1580 [ 144.856149][ T1583] ? __pfx_unregister_netdevice_many_notify+0x10/0x10 [ 144.856476][ T1583] ? rtnl_dellink+0x228/0xa40 [ 144.856740][ T1583] ? sock_def_readable+0x11e/0x300 [ 144.857006][ T1583] ? rtnl_is_locked+0x15/0x20 [ 144.857263][ T1583] ? unregister_netdevice_queue+0x70/0x410 [ 144.857588][ T1583] ? __pfx_unregister_netdevice_queue+0x10/0x10 [ 144.857894][ T1583] ? __pfx_unregister_netdevice_queue+0x10/0x10 [ 144.858201][ T1583] rtnl_dellink+0x348/0xa40 [ 144.858440][ T1583] ? __pfx_rtnl_dellink+0x10/0x10 [ 144.858715][ T1583] ? find_held_lock+0x2c/0x110 [ 144.858967][ T1583] ? __lock_release+0x103/0x460 [ 144.859211][ T1583] ? rtnetlink_rcv_msg+0x6ef/0xc10 [ 144.859463][ T1583] ? __pfx___lock_release+0x10/0x10 [ 144.859711][ T1583] ? __pfx_lock_acquire.part.0+0x10/0x10 [ 144.859962][ T1583] ? rtnetlink_rcv_msg+0x6ef/0xc10 [ 144.860206][ T1583] ? __pfx_rtnl_dellink+0x10/0x10 [ 144.860452][ T1583] rtnetlink_rcv_msg+0x712/0xc10 [ 144.860710][ T1583] ? __pfx_rtnetlink_rcv_msg+0x10/0x10 [ 144.860955][ T1583] ? hlock_class+0x4e/0x130 [ 144.861206][ T1583] ? mark_lock+0x38/0x3e0 [ 144.861392][ T1583] ? __lock_acquire+0xb9a/0x1680 [ 144.861640][ T1583] netlink_rcv_skb+0x130/0x360 [ 144.861885][ T1583] ? __pfx_rtnetlink_rcv_msg+0x10/0x10 [ 144.862130][ T1583] ? __pfx_netlink_rcv_skb+0x10/0x10 [ 144.862380][ T1583] ? netlink_deliver_tap+0x13e/0x340 [ 144.862630][ T1583] ? netlink_deliver_tap+0xc3/0x340 [ 144.862883][ T1583] netlink_unicast+0x44b/0x710 [ 144.863131][ T1583] ? __pfx_netlink_unicast+0x10/0x10 [ 144.863378][ T1583] netlink_sendmsg+0x723/0xbe0 [ 144.863628][ T1583] ? __pfx_netlink_sendmsg+0x10/0x10 [ 144.863875][ T1583] ? __kasan_kmalloc+0x7f/0x90 [ 144.864127][ T1583] ____sys_sendmsg+0x7ac/0xa10 [ 144.864374][ T1583] ? __pfx_____sys_sendmsg+0x10/0x10 [ 144.864619][ T1583] ? __pfx_copy_msghdr_from_user+0x10/0x10 [ 144.864942][ T1583] ___sys_sendmsg+0xee/0x170 [ 144.865190][ T1583] ? __pfx____sys_sendmsg+0x10/0x10 [ 144.865444][ T1583] ? kasan_save_stack+0x34/0x50 [ 144.865702][ T1583] ? kasan_save_stack+0x24/0x50 [ 144.865959][ T1583] ? __kasan_record_aux_stack+0x8e/0xa0 [ 144.866209][ T1583] ? __call_rcu_common.constprop.0+0xa1/0x4b0 [ 144.866525][ T1583] ? __x64_sys_close+0x7c/0xd0 [ 144.866779][ T1583] ? do_syscall_64+0xc1/0x1d0 [ 144.867028][ T1583] ? entry_SYSCALL_64_after_hwframe+0x77/0x7f [ 144.867332][ T1583] ? __lock_acquire+0xb9a/0x1680 [ 144.867585][ T1583] ? find_held_lock+0x2c/0x110 [ 144.867833][ T1583] ? __lock_release+0x103/0x460 [ 144.868082][ T1583] ? __virt_addr_valid+0x22b/0x430 [ 144.868329][ T1583] ? __pfx___lock_release+0x10/0x10 [ 144.868582][ T1583] ? __pfx_lock_acquire.part.0+0x10/0x10 [ 144.868832][ T1583] __sys_sendmsg+0x109/0x1a0 [ 144.869079][ T1583] ? __pfx___sys_sendmsg+0x10/0x10 [ 144.869322][ T1583] ? kfree+0x2d/0x330 [ 144.869517][ T1583] do_syscall_64+0xc1/0x1d0 [ 144.869768][ T1583] entry_SYSCALL_64_after_hwframe+0x77/0x7f [ 144.870071][ T1583] RIP: 0033:0x7fcb4226d9a7 [ 144.870331][ T1583] Code: 0a 00 f7 d8 64 89 02 48 c7 c0 ff ff ff ff eb b9 0f 1f 00 f3 0f 1e fa 64 8b 04 25 18 00 00 00 85 c0 75 10 b8 2e 00 00 00 0f 05 <48> 3d 00 f0 ff ff 77 51 c3 48 83 ec 28 89 54 24 1c 48 89 74 24 10 [ 144.871203][ T1583] RSP: 002b:00007ffd7528b138 EFLAGS: 00000246 ORIG_RAX: 000000000000002e [ 144.871578][ T1583] RAX: ffffffffffffffda RBX: 00007ffd7528b860 RCX: 00007fcb4226d9a7 [ 144.871959][ T1583] RDX: 0000000000000000 RSI: 00007ffd7528b1a0 RDI: 0000000000000005 [ 144.872325][ T1583] RBP: 0000000000000001 R08: 0000000000000003 R09: 0000000000000078 [ 144.872690][ T1583] R10: 00007fcb4212bef8 R11: 0000000000000246 R12: 0000000000000001 [ 144.873056][ T1583] R13: 0000000067795616 R14: 0000000000498600 R15: 0000000000000000 [ 144.873438][ T1583] [ 145.659134][ T67] veth01: left allmulticast mode [ 145.659326][ T67] veth01: left promiscuous mode [ 145.659614][ T67] br0: port 1(veth01) entered disabled state [ 146.065889][ T67] IPVS: stop unused estimator thread 0... [ 148.394606][ T1635] br0: port 1(veth01) entered blocking state [ 148.394833][ T1635] br0: port 1(veth01) entered disabled state [ 148.395047][ T1635] veth01: entered allmulticast mode [ 148.397164][ T1635] veth01: entered promiscuous mode [ 148.528617][ T1636] br0: port 2(veth02) entered blocking state [ 148.528983][ T1636] br0: port 2(veth02) entered disabled state [ 148.529806][ T1636] veth02: entered allmulticast mode [ 148.532351][ T1636] veth02: entered promiscuous mode [ 148.958957][ T37] br0: port 1(veth01) entered blocking state [ 148.959255][ T37] br0: port 1(veth01) entered forwarding state [ 149.741415][ T46] br0: port 2(veth02) entered blocking state [ 149.741651][ T46] br0: port 2(veth02) entered forwarding state [ 151.255644][ T1650] ipip: IPv4 and MPLS over IPv4 tunneling driver [ 151.833631][ T60] IPVS: starting estimator thread 0... [ 151.897354][ T1659] IPVS: using max 92 ests per chain, 220800 per kthread [ 153.653594][ T67] veth02: left allmulticast mode [ 153.653813][ T67] veth02: left promiscuous mode [ 153.654089][ T67] br0: port 2(veth02) entered disabled state [ 153.656697][ T67] veth01: left allmulticast mode [ 153.656861][ T67] veth01: left promiscuous mode [ 153.657113][ T67] br0: port 1(veth01) entered disabled state [ 154.091381][ T67] IPVS: stop unused estimator thread 0...