[ 247.218448][ T44] INFO: task ip:450 blocked for more than 122 seconds. [ 247.218986][ T44] Not tainted 6.15.0-rc3-virtme #1 [ 247.219258][ T44] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. [ 247.219663][ T44] task:ip state:D stack:25208 pid:450 tgid:450 ppid:332 task_flags:0x400100 flags:0x00004006 [ 247.220199][ T44] Call Trace: [ 247.220405][ T44] [ 247.220552][ T44] __schedule+0x851/0x1b00 [ 247.220815][ T44] ? __pfx___schedule+0x10/0x10 [ 247.221070][ T44] ? schedule+0x1ee/0x270 [ 247.221268][ T44] ? __lock_release+0x5d/0x170 [ 247.221521][ T44] schedule+0xe2/0x270 [ 247.221711][ T44] schedule_preempt_disabled+0x18/0x30 [ 247.221954][ T44] __mutex_lock+0x77f/0xcd0 [ 247.222199][ T44] ? kasan_quarantine_put+0x10d/0x230 [ 247.222454][ T44] ? xdp_features_set_redirect_target+0x23/0x80 [ 247.222753][ T44] ? __pfx___mutex_lock+0x10/0x10 [ 247.223000][ T44] ? kfree+0x144/0x320 [ 247.223187][ T44] ? virtnet_set_guest_offloads+0xf6/0x1b0 [ 247.223512][ T44] ? virtnet_set_guest_offloads+0xf6/0x1b0 [ 247.223811][ T44] ? __pfx_virtnet_set_guest_offloads+0x10/0x10 [ 247.224114][ T44] ? xdp_features_set_redirect_target+0x23/0x80 [ 247.224416][ T44] xdp_features_set_redirect_target+0x23/0x80 [ 247.224713][ T44] virtnet_xdp_set+0x1440/0x1a70 [ 247.224964][ T44] dev_xdp_install+0x41e/0x8c0 [ 247.225208][ T44] ? __pfx_virtnet_xdp+0x10/0x10 [ 247.225454][ T44] ? __pfx_dev_xdp_install+0x10/0x10 [ 247.225709][ T44] dev_xdp_attach+0x53e/0x1120 [ 247.225955][ T44] ? __bpf_prog_get+0x1ac/0x230 [ 247.226350][ T44] ? __pfx_dev_xdp_attach+0x10/0x10 [ 247.226594][ T44] ? __pfx_bpf_prog_get_type_dev+0x10/0x10 [ 247.226900][ T44] dev_change_xdp_fd+0x244/0x290 [ 247.227148][ T44] do_setlink.constprop.0+0x1ffd/0x2790 [ 247.227415][ T44] ? rcu_read_lock_any_held+0x43/0xb0 [ 247.227676][ T44] ? validate_chain+0x166/0x4d0 [ 247.227938][ T44] ? __pfx_do_setlink.constprop.0+0x10/0x10 [ 247.228250][ T44] ? __lock_acquire+0x591/0x9d0 [ 247.228492][ T44] ? mn_hlist_invalidate_range_start+0x1e9/0x590 [ 247.228796][ T44] ? __mutex_trylock_common+0xf9/0x260 [ 247.229044][ T44] ? __pfx___mutex_trylock_common+0x10/0x10 [ 247.229350][ T44] ? rcu_is_watching+0x12/0xc0 [ 247.229601][ T44] ? rcu_is_watching+0x12/0xc0 [ 247.229845][ T44] ? find_held_lock+0x2b/0x80 [ 247.230087][ T44] ? rtnl_newlink+0x651/0xa60 [ 247.230336][ T44] ? __create_object+0x5e/0xb0 [ 247.230586][ T44] ? __pfx___mutex_lock+0x10/0x10 [ 247.230834][ T44] ? trace_cap_capable+0x10f/0x190 [ 247.231077][ T44] ? _raw_spin_unlock_irqrestore+0x46/0x80 [ 247.231394][ T44] ? __rtnl_newlink+0x40e/0xa40 [ 247.231645][ T44] rtnl_newlink+0x69a/0xa60 [ 247.231890][ T44] ? __pfx_rtnl_newlink+0x10/0x10 [ 247.232133][ T44] ? __lock_acquire+0x591/0x9d0 [ 247.232388][ T44] ? find_held_lock+0x2b/0x80 [ 247.232631][ T44] ? rtnetlink_rcv_msg+0x6ed/0xc00 [ 247.232868][ T44] ? __lock_release+0x5d/0x170 [ 247.233114][ T44] ? __pfx_rtnl_newlink+0x10/0x10 [ 247.233366][ T44] rtnetlink_rcv_msg+0x710/0xc00 [ 247.233608][ T44] ? __pfx_rtnetlink_rcv_msg+0x10/0x10 [ 247.233850][ T44] ? validate_chain+0x166/0x4d0 [ 247.234107][ T44] netlink_rcv_skb+0x12f/0x360 [ 247.234361][ T44] ? __pfx_rtnetlink_rcv_msg+0x10/0x10 [ 247.234608][ T44] ? __pfx_netlink_rcv_skb+0x10/0x10 [ 247.234851][ T44] ? __lock_acquire+0x591/0x9d0 [ 247.235100][ T44] ? netlink_deliver_tap+0x13e/0x340 [ 247.235368][ T44] ? __lock_release+0x5d/0x170 [ 247.235617][ T44] ? netlink_deliver_tap+0x13e/0x340 [ 247.235861][ T44] ? netlink_deliver_tap+0xc3/0x340 [ 247.236104][ T44] netlink_unicast+0x449/0x710 [ 247.236362][ T44] ? __pfx_netlink_unicast+0x10/0x10 [ 247.236604][ T44] ? __lock_acquire+0x591/0x9d0 [ 247.236852][ T44] netlink_sendmsg+0x721/0xbe0 [ 247.237097][ T44] ? __pfx_netlink_sendmsg+0x10/0x10 [ 247.237348][ T44] ? __import_iovec+0x238/0x3c0 [ 247.237602][ T44] ____sys_sendmsg+0x7aa/0xa10 [ 247.237854][ T44] ? __pfx_____sys_sendmsg+0x10/0x10 [ 247.238095][ T44] ? __pfx_copy_msghdr_from_user+0x10/0x10 [ 247.238420][ T44] ___sys_sendmsg+0xed/0x170 [ 247.238659][ T44] ? kasan_record_aux_stack+0x8c/0xa0 [ 247.238896][ T44] ? __call_rcu_common.constprop.0+0xa1/0x610 [ 247.239200][ T44] ? __pfx____sys_sendmsg+0x10/0x10 [ 247.239462][ T44] ? find_held_lock+0x2b/0x80 [ 247.239707][ T44] ? __lock_acquire+0x591/0x9d0 [ 247.239953][ T44] ? find_held_lock+0x2b/0x80 [ 247.240198][ T44] ? __virt_addr_valid+0x22e/0x450 [ 247.240448][ T44] ? __lock_release+0x5d/0x170 [ 247.240701][ T44] __sys_sendmsg+0x108/0x1a0 [ 247.240941][ T44] ? __pfx___sys_sendmsg+0x10/0x10 [ 247.241200][ T44] ? rcu_is_watching+0x12/0xc0 [ 247.241450][ T44] do_syscall_64+0xc1/0x1d0 [ 247.241693][ T44] entry_SYSCALL_64_after_hwframe+0x77/0x7f [ 247.241989][ T44] RIP: 0033:0x7f711c7849a7 [ 247.242254][ T44] RSP: 002b:00007ffc87f0aa08 EFLAGS: 00000246 ORIG_RAX: 000000000000002e [ 247.242613][ T44] RAX: ffffffffffffffda RBX: 00007ffc87f0b138 RCX: 00007f711c7849a7 [ 247.242972][ T44] RDX: 0000000000000000 RSI: 00007ffc87f0aa70 RDI: 0000000000000003 [ 247.243350][ T44] RBP: 0000000000000009 R08: 00000000ffff7fff R09: 0000000000000000 [ 247.243708][ T44] R10: 00007f711c6424a8 R11: 0000000000000246 R12: 0000000000000009 [ 247.244119][ T44] R13: 00000000680c93fe R14: 0000000000498600 R15: 0000000000000000 [ 247.244497][ T44] [ 247.244689][ T44] INFO: task ip:450 is blocked on a mutex likely owned by task kworker/0:1:10. [ 247.245092][ T44] task:kworker/0:1 state:D stack:28368 pid:10 tgid:10 ppid:2 task_flags:0x4208060 flags:0x00004000 [ 247.245619][ T44] Workqueue: events refill_work [ 247.245877][ T44] Call Trace: [ 247.246063][ T44] [ 247.246199][ T44] __schedule+0x851/0x1b00 [ 247.246462][ T44] ? __pfx___schedule+0x10/0x10 [ 247.246714][ T44] ? schedule+0x1ee/0x270 [ 247.246897][ T44] ? __lock_release+0x5d/0x170 [ 247.247141][ T44] schedule+0xe2/0x270 [ 247.247346][ T44] schedule_hrtimeout_range_clock+0x100/0x2d0 [ 247.247656][ T44] ? __pfx_schedule_hrtimeout_range_clock+0x10/0x10 [ 247.247951][ T44] ? find_held_lock+0x2b/0x80 [ 247.248194][ T44] ? __pfx_hrtimer_wakeup+0x10/0x10 [ 247.248448][ T44] ? ktime_get+0x178/0x200 [ 247.248689][ T44] ? kvm_clock_get_cycles+0x18/0x30 [ 247.248930][ T44] ? ktime_get+0xb8/0x200 [ 247.249116][ T44] usleep_range_state+0x141/0x1d0 [ 247.249378][ T44] ? __pfx_usleep_range_state+0x10/0x10 [ 247.249626][ T44] napi_disable_locked+0x3c0/0x5d0 [ 247.249869][ T44] ? __pfx_napi_disable_locked+0x10/0x10 [ 247.250118][ T44] napi_disable+0x43/0x80 [ 247.250320][ T44] refill_work+0x129/0x220 [ 247.250563][ T44] ? refill_work+0x9b/0x220 [ 247.250806][ T44] process_one_work+0xe40/0x1690 [ 247.251062][ T44] ? __pfx_process_one_work+0x10/0x10 [ 247.251336][ T44] ? assign_work+0x16c/0x240 [ 247.251580][ T44] worker_thread+0x591/0xcf0 [ 247.251830][ T44] ? __pfx_worker_thread+0x10/0x10 [ 247.252079][ T44] kthread+0x37b/0x600 [ 247.252273][ T44] ? __pfx_kthread+0x10/0x10 [ 247.252512][ T44] ? ret_from_fork+0x1b/0x70 [ 247.252746][ T44] ? __lock_release+0x5d/0x170 [ 247.252995][ T44] ? rcu_is_watching+0x12/0xc0 [ 247.253242][ T44] ? __pfx_kthread+0x10/0x10 [ 247.253487][ T44] ret_from_fork+0x31/0x70 [ 247.253725][ T44] ? __pfx_kthread+0x10/0x10 [ 247.253966][ T44] ret_from_fork_asm+0x1a/0x30 [ 247.254220][ T44] [ 247.254419][ T44] [ 247.254419][ T44] Showing all locks held in the system: [ 247.254759][ T44] 3 locks held by kworker/0:1/10: [ 247.255001][ T44] #0: ffff8880010a1548 ((wq_completion)events){+.+.}-{0:0}, at: process_one_work+0x7e5/0x1690 [ 247.255476][ T44] #1: ffffc900000a7d40 ((work_completion)(&(&vi->refill)->work)){+.+.}-{0:0}, at: process_one_work+0xdf6/0x1690 [ 247.255993][ T44] #2: ffff888008d4ac90 (&dev->lock){+.+.}-{4:4}, at: napi_disable+0x3b/0x80 [ 247.256483][ T44] 3 locks held by kworker/u16:0/12: [ 247.256721][ T44] #0: ffff8880091e6548 ((wq_completion)ipv6_addrconf){+.+.}-{0:0}, at: process_one_work+0x7e5/0x1690 [ 247.257179][ T44] #1: ffffc900000c7d40 ((work_completion)(&(&net->ipv6.addr_chk_work)->work)){+.+.}-{0:0}, at: process_one_work+0xdf6/0x1690 [ 247.257759][ T44] #2: ffffffffbb63ca48 (rtnl_mutex){+.+.}-{4:4}, at: addrconf_verify_work+0x12/0x30 [ 247.258174][ T44] 1 lock held by khungtaskd/44: [ 247.258420][ T44] #0: ffffffffbad745a0 (rcu_read_lock){....}-{1:3}, at: debug_show_all_locks+0x36/0x260 [ 247.258840][ T44] 2 locks held by ip/450: [ 247.259016][ T44] #0: ffffffffbb63ca48 (rtnl_mutex){+.+.}-{4:4}, at: rtnl_newlink+0x651/0xa60 [ 247.259436][ T44] #1: ffff888008d4ac90 (&dev->lock){+.+.}-{4:4}, at: xdp_features_set_redirect_target+0x23/0x80 [ 247.259901][ T44] 2 locks held by ethtool/455: [ 247.260131][ T44] #0: ffffffffbb694af0 (cb_lock){++++}-{4:4}, at: genl_rcv+0x19/0x40 [ 247.260487][ T44] #1: ffffffffbb63ca48 (rtnl_mutex){+.+.}-{4:4}, at: ethnl_default_doit+0x31b/0xe60 [ 247.260891][ T44] [ 247.261017][ T44] ============================================= [ 247.261017][ T44]