WAIT TIMEOUT stderr
Ctrl-C stderr
Ctrl-C stderr
[ 247.174650][ T44] INFO: task ip:689 blocked for more than 122 seconds.
[ 247.175206][ T44] Not tainted 6.15.0-rc3-virtme #1
[ 247.175436][ T44] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[ 247.175850][ T44] task:ip state:D stack:25648 pid:689 tgid:689 ppid:571 task_flags:0x400100 flags:0x00004006
[ 247.176351][ T44] Call Trace:
[ 247.176525][ T44]
[ 247.176666][ T44] __schedule+0x851/0x1b00
[ 247.176951][ T44] ? __pfx___schedule+0x10/0x10
[ 247.177187][ T44] ? schedule+0x1ee/0x270
[ 247.177361][ T44] ? __lock_release+0x5d/0x170
[ 247.177624][ T44] schedule+0xe2/0x270
[ 247.177804][ T44] schedule_preempt_disabled+0x18/0x30
[ 247.178044][ T44] __mutex_lock+0x77f/0xcd0
[ 247.178287][ T44] ? kasan_quarantine_put+0x10d/0x230
[ 247.178528][ T44] ? xdp_features_set_redirect_target+0x23/0x80
[ 247.178827][ T44] ? __pfx___mutex_lock+0x10/0x10
[ 247.179063][ T44] ? kfree+0x144/0x320
[ 247.179236][ T44] ? virtnet_set_guest_offloads+0xf6/0x1b0
[ 247.179541][ T44] ? virtnet_set_guest_offloads+0xf6/0x1b0
[ 247.179848][ T44] ? __pfx_virtnet_set_guest_offloads+0x10/0x10
[ 247.180148][ T44] ? xdp_features_set_redirect_target+0x23/0x80
[ 247.180432][ T44] xdp_features_set_redirect_target+0x23/0x80
[ 247.180728][ T44] virtnet_xdp_set+0x1440/0x1a70
[ 247.180978][ T44] dev_xdp_install+0x41e/0x8c0
[ 247.181218][ T44] ? __pfx_virtnet_xdp+0x10/0x10
[ 247.181452][ T44] ? __pfx_dev_xdp_install+0x10/0x10
[ 247.181916][ T44] dev_xdp_attach+0x53e/0x1120
[ 247.182164][ T44] ? __bpf_prog_get+0x1ac/0x230
[ 247.182407][ T44] ? __pfx_dev_xdp_attach+0x10/0x10
[ 247.182655][ T44] ? __pfx_bpf_prog_get_type_dev+0x10/0x10
[ 247.182946][ T44] dev_change_xdp_fd+0x244/0x290
[ 247.183214][ T44] do_setlink.constprop.0+0x1ffd/0x2790
[ 247.183473][ T44] ? rcu_read_lock_any_held+0x43/0xb0
[ 247.183740][ T44] ? validate_chain+0x166/0x4d0
[ 247.184004][ T44] ? __pfx_do_setlink.constprop.0+0x10/0x10
[ 247.184316][ T44] ? __lock_acquire+0x591/0x9d0
[ 247.184550][ T44] ? mn_hlist_invalidate_range_start+0x1e9/0x590
[ 247.184853][ T44] ? __mutex_trylock_common+0xf9/0x260
[ 247.185099][ T44] ? __pfx___mutex_trylock_common+0x10/0x10
[ 247.185385][ T44] ? rcu_is_watching+0x12/0xc0
[ 247.185632][ T44] ? rcu_is_watching+0x12/0xc0
[ 247.185876][ T44] ? find_held_lock+0x2b/0x80
[ 247.186118][ T44] ? rtnl_newlink+0x651/0xa60
[ 247.186351][ T44] ? __create_object+0x5e/0xb0
[ 247.186596][ T44] ? __pfx___mutex_lock+0x10/0x10
[ 247.186840][ T44] ? trace_cap_capable+0x10f/0x190
[ 247.187085][ T44] ? _raw_spin_unlock_irqrestore+0x46/0x80
[ 247.187371][ T44] ? __rtnl_newlink+0x40e/0xa40
[ 247.187637][ T44] rtnl_newlink+0x69a/0xa60
[ 247.187887][ T44] ? __pfx_rtnl_newlink+0x10/0x10
[ 247.188126][ T44] ? __lock_acquire+0x591/0x9d0
[ 247.188369][ T44] ? find_held_lock+0x2b/0x80
[ 247.188613][ T44] ? rtnetlink_rcv_msg+0x6ed/0xc00
[ 247.188850][ T44] ? __lock_release+0x5d/0x170
[ 247.189092][ T44] ? __pfx_rtnl_newlink+0x10/0x10
[ 247.189327][ T44] rtnetlink_rcv_msg+0x710/0xc00
[ 247.189615][ T44] ? __pfx_rtnetlink_rcv_msg+0x10/0x10
[ 247.189864][ T44] ? validate_chain+0x166/0x4d0
[ 247.190129][ T44] netlink_rcv_skb+0x12f/0x360
[ 247.190382][ T44] ? __pfx_rtnetlink_rcv_msg+0x10/0x10
[ 247.190647][ T44] ? __pfx_netlink_rcv_skb+0x10/0x10
[ 247.190884][ T44] ? __lock_acquire+0x591/0x9d0
[ 247.191137][ T44] ? netlink_deliver_tap+0x13e/0x340
[ 247.191372][ T44] ? __lock_release+0x5d/0x170
[ 247.191624][ T44] ? netlink_deliver_tap+0x13e/0x340
[ 247.191867][ T44] ? netlink_deliver_tap+0xc3/0x340
[ 247.192107][ T44] netlink_unicast+0x449/0x710
[ 247.192346][ T44] ? __pfx_netlink_unicast+0x10/0x10
[ 247.192589][ T44] ? __lock_acquire+0x591/0x9d0
[ 247.192835][ T44] netlink_sendmsg+0x721/0xbe0
[ 247.193076][ T44] ? __pfx_netlink_sendmsg+0x10/0x10
[ 247.193314][ T44] ? __import_iovec+0x238/0x3c0
[ 247.193566][ T44] ____sys_sendmsg+0x7aa/0xa10
[ 247.193825][ T44] ? __pfx_____sys_sendmsg+0x10/0x10
[ 247.194073][ T44] ? __pfx_copy_msghdr_from_user+0x10/0x10
[ 247.194390][ T44] ___sys_sendmsg+0xed/0x170
[ 247.194648][ T44] ? kasan_record_aux_stack+0x8c/0xa0
[ 247.194897][ T44] ? __call_rcu_common.constprop.0+0xa1/0x610
[ 247.195206][ T44] ? __pfx____sys_sendmsg+0x10/0x10
[ 247.195444][ T44] ? find_held_lock+0x2b/0x80
[ 247.195700][ T44] ? __lock_acquire+0x591/0x9d0
[ 247.195955][ T44] ? find_held_lock+0x2b/0x80
[ 247.196209][ T44] ? __virt_addr_valid+0x22e/0x450
[ 247.196466][ T44] ? __lock_release+0x5d/0x170
[ 247.196729][ T44] __sys_sendmsg+0x108/0x1a0
[ 247.196976][ T44] ? __pfx___sys_sendmsg+0x10/0x10
[ 247.197239][ T44] ? rcu_is_watching+0x12/0xc0
[ 247.197491][ T44] do_syscall_64+0xc1/0x1d0
[ 247.197737][ T44] entry_SYSCALL_64_after_hwframe+0x77/0x7f
[ 247.198024][ T44] RIP: 0033:0x7f3b3d09e9a7
[ 247.198272][ T44] RSP: 002b:00007fff46f5eb38 EFLAGS: 00000246 ORIG_RAX: 000000000000002e
[ 247.198633][ T44] RAX: ffffffffffffffda RBX: 00007fff46f5f268 RCX: 00007f3b3d09e9a7
[ 247.198978][ T44] RDX: 0000000000000000 RSI: 00007fff46f5eba0 RDI: 0000000000000003
[ 247.199318][ T44] RBP: 0000000000000009 R08: 00000000ffff7fff R09: 0000000000000000
[ 247.199688][ T44] R10: 00007f3b3cf5c4a8 R11: 0000000000000246 R12: 0000000000000009
[ 247.200046][ T44] R13: 00000000680f371f R14: 0000000000498600 R15: 0000000000000000
[ 247.200401][ T44]
[ 247.200616][ T44] INFO: task ip:689 is blocked on a mutex likely owned by task kworker/2:1:62.
[ 247.200996][ T44] task:kworker/2:1 state:D stack:26976 pid:62 tgid:62 ppid:2 task_flags:0x4208060 flags:0x00004000
[ 247.201493][ T44] Workqueue: events refill_work
[ 247.201746][ T44] Call Trace:
[ 247.201918][ T44]
[ 247.202062][ T44] __schedule+0x851/0x1b00
[ 247.202308][ T44] ? __pfx___schedule+0x10/0x10
[ 247.202566][ T44] ? schedule+0x1ee/0x270
[ 247.202739][ T44] ? __lock_release+0x5d/0x170
[ 247.202979][ T44] schedule+0xe2/0x270
[ 247.203154][ T44] schedule_hrtimeout_range_clock+0x100/0x2d0
[ 247.203445][ T44] ? __pfx_schedule_hrtimeout_range_clock+0x10/0x10
[ 247.203746][ T44] ? find_held_lock+0x2b/0x80
[ 247.203988][ T44] ? __pfx_hrtimer_wakeup+0x10/0x10
[ 247.204226][ T44] ? ktime_get+0x178/0x200
[ 247.204465][ T44] ? kvm_clock_get_cycles+0x18/0x30
[ 247.204707][ T44] ? ktime_get+0xb8/0x200
[ 247.204900][ T44] usleep_range_state+0x141/0x1d0
[ 247.205158][ T44] ? __pfx_usleep_range_state+0x10/0x10
[ 247.205410][ T44] ? usleep_range_state+0x171/0x1d0
[ 247.205667][ T44] napi_disable_locked+0x3c0/0x5d0
[ 247.205919][ T44] ? __pfx_napi_disable_locked+0x10/0x10
[ 247.206177][ T44] napi_disable+0x43/0x80
[ 247.206357][ T44] refill_work+0x129/0x220
[ 247.206617][ T44] ? refill_work+0x9b/0x220
[ 247.206884][ T44] process_one_work+0xe40/0x1690
[ 247.207154][ T44] ? __pfx_process_one_work+0x10/0x10
[ 247.207396][ T44] ? assign_work+0x16c/0x240
[ 247.207655][ T44] worker_thread+0x591/0xcf0
[ 247.207911][ T44] ? __pfx_worker_thread+0x10/0x10
[ 247.208148][ T44] kthread+0x37b/0x600
[ 247.208324][ T44] ? __pfx_kthread+0x10/0x10
[ 247.208564][ T44] ? ret_from_fork+0x1b/0x70
[ 247.208805][ T44] ? __lock_release+0x5d/0x170
[ 247.209042][ T44] ? rcu_is_watching+0x12/0xc0
[ 247.209279][ T44] ? __pfx_kthread+0x10/0x10
[ 247.209553][ T44] ret_from_fork+0x31/0x70
[ 247.209808][ T44] ? __pfx_kthread+0x10/0x10
[ 247.210059][ T44] ret_from_fork_asm+0x1a/0x30
[ 247.210326][ T44]
[ 247.210532][ T44]
[ 247.210532][ T44] Showing all locks held in the system:
[ 247.211006][ T44] 3 locks held by kworker/u16:0/12:
[ 247.211243][ T44] #0: ffff8880090e6548 ((wq_completion)ipv6_addrconf){+.+.}-{0:0}, at: process_one_work+0x7e5/0x1690
[ 247.211692][ T44] #1: ffffc900000c7d40 ((work_completion)(&(&net->ipv6.addr_chk_work)->work)){+.+.}-{0:0}, at: process_one_work+0xdf6/0x1690
[ 247.212225][ T44] #2: ffffffff8c63ca48 (rtnl_mutex){+.+.}-{4:4}, at: addrconf_verify_work+0x12/0x30
[ 247.212634][ T44] 1 lock held by khungtaskd/44:
[ 247.212865][ T44] #0: ffffffff8bd745a0 (rcu_read_lock){....}-{1:3}, at: debug_show_all_locks+0x36/0x260
[ 247.213252][ T44] 3 locks held by kworker/2:1/62:
[ 247.213480][ T44] #0: ffff8880010a1548 ((wq_completion)events){+.+.}-{0:0}, at: process_one_work+0x7e5/0x1690
[ 247.213922][ T44] #1: ffffc90000437d40 ((work_completion)(&(&vi->refill)->work)){+.+.}-{0:0}, at: process_one_work+0xdf6/0x1690
[ 247.214402][ T44] #2: ffff888008c82c90 (&dev->lock){+.+.}-{4:4}, at: napi_disable+0x3b/0x80
[ 247.214814][ T44] 2 locks held by ip/689:
[ 247.214983][ T44] #0: ffffffff8c63ca48 (rtnl_mutex){+.+.}-{4:4}, at: rtnl_newlink+0x651/0xa60
[ 247.215367][ T44] #1: ffff888008c82c90 (&dev->lock){+.+.}-{4:4}, at: xdp_features_set_redirect_target+0x23/0x80
[ 247.215814][ T44] 2 locks held by ethtool/695:
[ 247.216041][ T44] #0: ffffffff8c694a70 (cb_lock){++++}-{4:4}, at: genl_rcv+0x19/0x40
[ 247.216375][ T44] #1: ffffffff8c63ca48 (rtnl_mutex){+.+.}-{4:4}, at: ethnl_default_doit+0x31b/0xe60
[ 247.216768][ T44]
[ 247.216888][ T44] =============================================
[ 247.216888][ T44]
WAIT TIMEOUT stderr