[ 4261.047742][T31907] pktgen: Packet Generator for packet performance testing. Version: 2.75 [ 4425.290395][ T44] INFO: task modprobe:31907 blocked for more than 122 seconds. [ 4425.290976][ T44] Not tainted 6.18.0-rc2-virtme #1 [ 4425.291203][ T44] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. [ 4425.291615][ T44] task:modprobe state:D stack:25664 pid:31907 tgid:31907 ppid:1 task_flags:0x400100 flags:0x00080003 [ 4425.292183][ T44] Call Trace: [ 4425.292384][ T44] [ 4425.292528][ T44] __schedule+0x880/0x1b10 [ 4425.292795][ T44] ? io_schedule_timeout+0x150/0x150 [ 4425.293064][ T44] ? schedule+0x1ee/0x270 [ 4425.293239][ T44] ? __lock_release+0x5d/0x170 [ 4425.293492][ T44] schedule+0xe2/0x270 [ 4425.293673][ T44] schedule_preempt_disabled+0x18/0x30 [ 4425.293909][ T44] rwsem_down_write_slowpath+0x3c5/0xc60 [ 4425.294148][ T44] ? __lock_acquire+0x449/0x7e0 [ 4425.294393][ T44] ? rwsem_down_read_slowpath+0xd80/0xd80 [ 4425.294648][ T44] ? rcu_is_watching+0x12/0xb0 [ 4425.294894][ T44] ? lock_acquire+0x104/0x160 [ 4425.295132][ T44] ? register_pernet_subsys+0x1a/0x40 [ 4425.295390][ T44] down_write+0x1d6/0x200 [ 4425.295572][ T44] ? rwsem_down_write_slowpath+0xc60/0xc60 [ 4425.295884][ T44] ? find_held_lock+0x2b/0x80 [ 4425.296127][ T44] ? __create_object+0x5e/0xb0 [ 4425.296382][ T44] ? _sub_I_65535_1+0x5c0/0x5c0 [nsh] [ 4425.296674][ T44] register_pernet_subsys+0x1a/0x40 [ 4425.296914][ T44] pg_init+0x2c/0x1000 [pktgen] [ 4425.297158][ T44] ? _sub_I_65535_1+0x5c0/0x5c0 [nsh] [ 4425.297403][ T44] do_one_initcall+0x8c/0x1d0 [ 4425.297642][ T44] ? trace_initcall_start+0x130/0x130 [ 4425.297879][ T44] ? rcu_is_watching+0x12/0xb0 [ 4425.298491][ T44] ? trace_kmalloc+0x26/0xd0 [ 4425.298741][ T44] ? __kmalloc_cache_noprof+0x2e4/0x690 [ 4425.298977][ T44] ? debug_objects_oom+0x160/0x160 [ 4425.299216][ T44] ? kasan_unpoison+0x27/0x50 [ 4425.299472][ T44] ? __asan_register_globals+0x5d/0x70 [ 4425.299710][ T44] do_init_module+0x27a/0x860 [ 4425.299970][ T44] ? free_module+0x2f0/0x2f0 [ 4425.300206][ T44] ? kfree+0x21d/0x540 [ 4425.300403][ T44] ? load_module+0x20c5/0x2dd0 [ 4425.300647][ T44] load_module+0x1f40/0x2dd0 [ 4425.300897][ T44] ? layout_and_allocate.constprop.0+0x630/0x630 [ 4425.301191][ T44] ? kernel_read_file+0x3ed/0x550 [ 4425.301446][ T44] ? kernel_read_file+0x3c8/0x550 [ 4425.301687][ T44] ? __ia32_sys_fsconfig+0x150/0x150 [ 4425.301930][ T44] ? init_module_from_file+0xe9/0x150 [ 4425.302173][ T44] init_module_from_file+0xe9/0x150 [ 4425.302419][ T44] ? __do_sys_init_module+0x1e0/0x1e0 [ 4425.302670][ T44] ? idempotent_init_module+0x316/0x610 [ 4425.302906][ T44] ? __lock_release+0x5d/0x170 [ 4425.303146][ T44] ? do_raw_spin_unlock+0x58/0x220 [ 4425.303395][ T44] idempotent_init_module+0x331/0x610 [ 4425.303634][ T44] ? init_module_from_file+0x150/0x150 [ 4425.303890][ T44] ? cap_capable+0x94/0x230 [ 4425.304133][ T44] __x64_sys_finit_module+0xca/0x150 [ 4425.304378][ T44] ? do_syscall_64+0x85/0xfd0 [ 4425.304620][ T44] do_syscall_64+0xc1/0xfd0 [ 4425.304860][ T44] entry_SYSCALL_64_after_hwframe+0x4b/0x53 [ 4425.305153][ T44] RIP: 0033:0x7f11f314a92d [ 4425.305413][ T44] RSP: 002b:00007ffd4ecb67e8 EFLAGS: 00000246 ORIG_RAX: 0000000000000139 [ 4425.305792][ T44] RAX: ffffffffffffffda RBX: 000055aab0152ac0 RCX: 00007f11f314a92d [ 4425.306152][ T44] RDX: 0000000000000000 RSI: 000055aa8ae1ea2a RDI: 0000000000000005 [ 4425.306523][ T44] RBP: 0000000000040000 R08: 0000000000000000 R09: 00007ffd4ecb6920 [ 4425.306884][ T44] R10: 0000000000000005 R11: 0000000000000246 R12: 000055aa8ae1ea2a [ 4425.307255][ T44] R13: 000055aab0152b60 R14: 000055aab0152da0 R15: 0000000000000000 [ 4425.307641][ T44] [ 4425.307830][ T44] INFO: task modprobe:31907 blocked on an rw-semaphore likely owned by task kworker/u16:0:12 [ 4425.308342][ T44] task:kworker/u16:0 state:R running task stack:24656 pid:12 tgid:12 ppid:2 task_flags:0x4208060 flags:0x00080000 [ 4425.308941][ T44] Workqueue: netns cleanup_net [ 4425.309174][ T44] Call Trace: [ 4425.309351][ T44] [ 4425.309472][ T44] ? __mutex_trylock_common+0xf9/0x260 [ 4425.309698][ T44] ? __mutex_handoff+0x2b0/0x2b0 [ 4425.309922][ T44] ? rcu_is_watching+0x12/0xb0 [ 4425.310151][ T44] ? rcu_is_watching+0x12/0xb0 [ 4425.310378][ T44] ? trace_contention_end+0xd8/0x140 [ 4425.310604][ T44] ? sched_clock+0x10/0x20 [ 4425.310826][ T44] ? __mutex_lock+0x19f/0x1190 [ 4425.311052][ T44] ? trace_sched_exit_tp+0xcd/0x130 [ 4425.311278][ T44] ? nf_ct_iterate_cleanup+0x4a/0x340 [nf_conntrack] [ 4425.311579][ T44] ? ww_mutex_lock+0x160/0x160 [ 4425.311821][ T44] ? nf_ct_iterate_cleanup+0x4a/0x340 [nf_conntrack] [ 4425.312111][ T44] ? nf_ct_iterate_cleanup+0x4a/0x340 [nf_conntrack] [ 4425.312407][ T44] ? __nf_conntrack_hash_insert+0x200/0x200 [nf_conntrack] [ 4425.312754][ T44] ? nf_conntrack_cleanup_net_list+0x19a/0x4b0 [nf_conntrack] [ 4425.313115][ T44] ? nf_conntrack_cleanup_net_list+0x10e/0x4b0 [nf_conntrack] [ 4425.313484][ T44] ? nf_conntrack_cleanup_end+0x70/0x70 [nf_conntrack] [ 4425.313797][ T44] ? nf_conntrack_pernet_exit+0xa5/0x140 [nf_conntrack] [ 4425.314098][ T44] ? _raw_spin_unlock+0x2d/0x50 [ 4425.314359][ T44] ? ops_undo_list+0x2bf/0x890 [ 4425.314586][ T44] ? rtnl_net_dumpid_one+0x270/0x270 [ 4425.314810][ T44] ? cleanup_net+0x2d6/0x8b0 [ 4425.315037][ T44] ? cleanup_net+0x3b2/0x8b0 [ 4425.315262][ T44] ? net_passive_dec+0x190/0x190 [ 4425.315495][ T44] ? rcu_is_watching+0x12/0xb0 [ 4425.315737][ T44] ? process_one_work+0xe35/0x1650 [ 4425.315970][ T44] ? pwq_dec_nr_in_flight+0x550/0x550 [ 4425.316197][ T44] ? assign_work+0x168/0x240 [ 4425.316427][ T44] ? worker_thread+0x591/0xcf0 [ 4425.316656][ T44] ? rescuer_thread+0xd10/0xd10 [ 4425.316880][ T44] ? kthread+0x37b/0x5f0 [ 4425.317051][ T44] ? kthread_is_per_cpu+0xc0/0xc0 [ 4425.317277][ T44] ? ret_from_fork+0x1b/0x270 [ 4425.317506][ T44] ? __lock_release+0x5d/0x170 [ 4425.317727][ T44] ? rcu_is_watching+0x12/0xb0 [ 4425.317963][ T44] ? kthread_is_per_cpu+0xc0/0xc0 [ 4425.318199][ T44] ? ret_from_fork+0x1db/0x270 [ 4425.318424][ T44] ? kthread_is_per_cpu+0xc0/0xc0 [ 4425.318647][ T44] ? ret_from_fork_asm+0x11/0x20 [ 4425.318885][ T44] [ 4425.319058][ T44] [ 4425.319058][ T44] Showing all locks held in the system: [ 4425.319389][ T44] 4 locks held by kworker/u16:0/12: [ 4425.319620][ T44] 1 lock held by khungtaskd/44: [ 4425.319837][ T44] #0: ffffffff97d881c0 (rcu_read_lock){....}-{1:3}, at: debug_show_all_locks+0x36/0x260 [ 4425.320249][ T44] 1 lock held by modprobe/31907: [ 4425.320470][ T44] #0: ffffffff9847fb10 (pernet_ops_rwsem){++++}-{4:4}, at: register_pernet_subsys+0x1a/0x40 [ 4425.320911][ T44] [ 4425.321021][ T44] ============================================= [ 4425.321021][ T44] [ 4548.170374][ T44] INFO: task modprobe:31907 blocked for more than 245 seconds. [ 4548.170847][ T44] Not tainted 6.18.0-rc2-virtme #1 [ 4548.171068][ T44] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. [ 4548.171467][ T44] task:modprobe state:D stack:25664 pid:31907 tgid:31907 ppid:1 task_flags:0x400100 flags:0x00080003 [ 4548.171957][ T44] Call Trace: [ 4548.172124][ T44] [ 4548.172243][ T44] __schedule+0x880/0x1b10 [ 4548.172505][ T44] ? io_schedule_timeout+0x150/0x150 [ 4548.172739][ T44] ? schedule+0x1ee/0x270 [ 4548.172915][ T44] ? __lock_release+0x5d/0x170 [ 4548.173143][ T44] schedule+0xe2/0x270 [ 4548.173319][ T44] schedule_preempt_disabled+0x18/0x30 [ 4548.173540][ T44] rwsem_down_write_slowpath+0x3c5/0xc60 [ 4548.173764][ T44] ? __lock_acquire+0x449/0x7e0 [ 4548.173984][ T44] ? rwsem_down_read_slowpath+0xd80/0xd80 [ 4548.174215][ T44] ? rcu_is_watching+0x12/0xb0 [ 4548.174448][ T44] ? lock_acquire+0x104/0x160 [ 4548.174674][ T44] ? register_pernet_subsys+0x1a/0x40 [ 4548.174903][ T44] down_write+0x1d6/0x200 [ 4548.175080][ T44] ? rwsem_down_write_slowpath+0xc60/0xc60 [ 4548.175363][ T44] ? find_held_lock+0x2b/0x80 [ 4548.175586][ T44] ? __create_object+0x5e/0xb0 [ 4548.175809][ T44] ? _sub_I_65535_1+0x5c0/0x5c0 [nsh] [ 4548.176030][ T44] register_pernet_subsys+0x1a/0x40 [ 4548.176259][ T44] pg_init+0x2c/0x1000 [pktgen] [ 4548.176493][ T44] ? _sub_I_65535_1+0x5c0/0x5c0 [nsh] [ 4548.176716][ T44] do_one_initcall+0x8c/0x1d0 [ 4548.176938][ T44] ? trace_initcall_start+0x130/0x130 [ 4548.177159][ T44] ? rcu_is_watching+0x12/0xb0 [ 4548.177390][ T44] ? trace_kmalloc+0x26/0xd0 [ 4548.177608][ T44] ? __kmalloc_cache_noprof+0x2e4/0x690 [ 4548.177834][ T44] ? debug_objects_oom+0x160/0x160 [ 4548.178056][ T44] ? kasan_unpoison+0x27/0x50 [ 4548.178279][ T44] ? __asan_register_globals+0x5d/0x70 [ 4548.178513][ T44] do_init_module+0x27a/0x860 [ 4548.178740][ T44] ? free_module+0x2f0/0x2f0 [ 4548.178958][ T44] ? kfree+0x21d/0x540 [ 4548.179124][ T44] ? load_module+0x20c5/0x2dd0 [ 4548.179359][ T44] load_module+0x1f40/0x2dd0 [ 4548.179595][ T44] ? layout_and_allocate.constprop.0+0x630/0x630 [ 4548.179871][ T44] ? kernel_read_file+0x3ed/0x550 [ 4548.180095][ T44] ? kernel_read_file+0x3c8/0x550 [ 4548.180326][ T44] ? __ia32_sys_fsconfig+0x150/0x150 [ 4548.180554][ T44] ? init_module_from_file+0xe9/0x150 [ 4548.180772][ T44] init_module_from_file+0xe9/0x150 [ 4548.180993][ T44] ? __do_sys_init_module+0x1e0/0x1e0 [ 4548.181235][ T44] ? idempotent_init_module+0x316/0x610 [ 4548.181463][ T44] ? __lock_release+0x5d/0x170 [ 4548.181686][ T44] ? do_raw_spin_unlock+0x58/0x220 [ 4548.181910][ T44] idempotent_init_module+0x331/0x610 [ 4548.182133][ T44] ? init_module_from_file+0x150/0x150 [ 4548.182373][ T44] ? cap_capable+0x94/0x230 [ 4548.182608][ T44] __x64_sys_finit_module+0xca/0x150 [ 4548.182826][ T44] ? do_syscall_64+0x85/0xfd0 [ 4548.183048][ T44] do_syscall_64+0xc1/0xfd0 [ 4548.183269][ T44] entry_SYSCALL_64_after_hwframe+0x4b/0x53 [ 4548.183550][ T44] RIP: 0033:0x7f11f314a92d [ 4548.183783][ T44] RSP: 002b:00007ffd4ecb67e8 EFLAGS: 00000246 ORIG_RAX: 0000000000000139 [ 4548.184117][ T44] RAX: ffffffffffffffda RBX: 000055aab0152ac0 RCX: 00007f11f314a92d [ 4548.184461][ T44] RDX: 0000000000000000 RSI: 000055aa8ae1ea2a RDI: 0000000000000005 [ 4548.184792][ T44] RBP: 0000000000040000 R08: 0000000000000000 R09: 00007ffd4ecb6920 [ 4548.185130][ T44] R10: 0000000000000005 R11: 0000000000000246 R12: 000055aa8ae1ea2a [ 4548.185469][ T44] R13: 000055aab0152b60 R14: 000055aab0152da0 R15: 0000000000000000 [ 4548.185820][ T44] [ 4548.185990][ T44] INFO: task modprobe:31907 blocked on an rw-semaphore likely owned by task kworker/u16:0:12 [ 4548.186462][ T44] task:kworker/u16:0 state:R running task stack:24656 pid:12 tgid:12 ppid:2 task_flags:0x4208060 flags:0x00080000 [ 4548.187003][ T44] Workqueue: netns cleanup_net [ 4548.187230][ T44] Call Trace: [ 4548.187404][ T44] [ 4548.187523][ T44] ? __mutex_trylock_common+0xf9/0x260 [ 4548.187745][ T44] ? __mutex_handoff+0x2b0/0x2b0 [ 4548.187965][ T44] ? rcu_is_watching+0x12/0xb0 [ 4548.188210][ T44] ? rcu_is_watching+0x12/0xb0 [ 4548.188440][ T44] ? trace_contention_end+0xd8/0x140 [ 4548.188665][ T44] ? __mutex_lock+0x19f/0x1190 [ 4548.188890][ T44] ? trace_sched_exit_tp+0xcd/0x130 [ 4548.189115][ T44] ? nf_ct_iterate_cleanup+0x4a/0x340 [nf_conntrack] [ 4548.189411][ T44] ? ww_mutex_lock+0x160/0x160 [ 4548.189651][ T44] ? nf_ct_iterate_cleanup+0x4a/0x340 [nf_conntrack] [ 4548.189933][ T44] ? nf_ct_iterate_cleanup+0x4a/0x340 [nf_conntrack] [ 4548.190215][ T44] ? __nf_conntrack_hash_insert+0x200/0x200 [nf_conntrack] [ 4548.190569][ T44] ? nf_conntrack_cleanup_net_list+0x19a/0x4b0 [nf_conntrack] [ 4548.190906][ T44] ? nf_conntrack_cleanup_net_list+0x10e/0x4b0 [nf_conntrack] [ 4548.191245][ T44] ? nf_conntrack_cleanup_end+0x70/0x70 [nf_conntrack] [ 4548.191538][ T44] ? nf_conntrack_pernet_exit+0xa5/0x140 [nf_conntrack] [ 4548.191819][ T44] ? _raw_spin_unlock+0x2d/0x50 [ 4548.192050][ T44] ? ops_undo_list+0x2bf/0x890 [ 4548.192275][ T44] ? rtnl_net_dumpid_one+0x270/0x270 [ 4548.192504][ T44] ? cleanup_net+0x2d6/0x8b0 [ 4548.192736][ T44] ? cleanup_net+0x3b2/0x8b0 [ 4548.192960][ T44] ? net_passive_dec+0x190/0x190 [ 4548.193182][ T44] ? rcu_is_watching+0x12/0xb0 [ 4548.193416][ T44] ? process_one_work+0xe35/0x1650 [ 4548.193649][ T44] ? pwq_dec_nr_in_flight+0x550/0x550 [ 4548.193882][ T44] ? assign_work+0x168/0x240 [ 4548.194106][ T44] ? worker_thread+0x591/0xcf0 [ 4548.194341][ T44] ? rescuer_thread+0xd10/0xd10 [ 4548.194565][ T44] ? kthread+0x37b/0x5f0 [ 4548.194732][ T44] ? kthread_is_per_cpu+0xc0/0xc0 [ 4548.194949][ T44] ? ret_from_fork+0x1b/0x270 [ 4548.195179][ T44] ? __lock_release+0x5d/0x170 [ 4548.195408][ T44] ? rcu_is_watching+0x12/0xb0 [ 4548.195638][ T44] ? kthread_is_per_cpu+0xc0/0xc0 [ 4548.195858][ T44] ? ret_from_fork+0x1db/0x270 [ 4548.196083][ T44] ? kthread_is_per_cpu+0xc0/0xc0 [ 4548.196310][ T44] ? ret_from_fork_asm+0x11/0x20 [ 4548.196544][ T44] [ 4548.196721][ T44] [ 4548.196721][ T44] Showing all locks held in the system: [ 4548.197033][ T44] 4 locks held by kworker/u16:0/12: [ 4548.197255][ T44] 1 lock held by khungtaskd/44: [ 4548.197619][ T44] #0: ffffffff97d881c0 (rcu_read_lock){....}-{1:3}, at: debug_show_all_locks+0x36/0x260 [ 4548.198016][ T44] 1 lock held by modprobe/31907: [ 4548.198227][ T44] #0: ffffffff9847fb10 (pernet_ops_rwsem){++++}-{4:4}, at: register_pernet_subsys+0x1a/0x40 [ 4548.198666][ T44] [ 4548.198774][ T44] ============================================= [ 4548.198774][ T44] [ 4671.050381][ T44] INFO: task modprobe:31907 blocked for more than 368 seconds. [ 4671.050813][ T44] Not tainted 6.18.0-rc2-virtme #1 [ 4671.051039][ T44] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. [ 4671.051448][ T44] task:modprobe state:D stack:25664 pid:31907 tgid:31907 ppid:1 task_flags:0x400100 flags:0x00080003 [ 4671.051973][ T44] Call Trace: [ 4671.052144][ T44] [ 4671.052264][ T44] __schedule+0x880/0x1b10 [ 4671.052544][ T44] ? io_schedule_timeout+0x150/0x150 [ 4671.052782][ T44] ? schedule+0x1ee/0x270 [ 4671.052949][ T44] ? __lock_release+0x5d/0x170 [ 4671.053178][ T44] schedule+0xe2/0x270 [ 4671.053361][ T44] schedule_preempt_disabled+0x18/0x30 [ 4671.053599][ T44] rwsem_down_write_slowpath+0x3c5/0xc60 [ 4671.053824][ T44] ? __lock_acquire+0x449/0x7e0 [ 4671.054044][ T44] ? rwsem_down_read_slowpath+0xd80/0xd80 [ 4671.054284][ T44] ? rcu_is_watching+0x12/0xb0 [ 4671.054527][ T44] ? lock_acquire+0x104/0x160 [ 4671.054751][ T44] ? register_pernet_subsys+0x1a/0x40 [ 4671.054984][ T44] down_write+0x1d6/0x200 [ 4671.055152][ T44] ? rwsem_down_write_slowpath+0xc60/0xc60 [ 4671.055453][ T44] ? find_held_lock+0x2b/0x80 [ 4671.055684][ T44] ? __create_object+0x5e/0xb0 [ 4671.055910][ T44] ? _sub_I_65535_1+0x5c0/0x5c0 [nsh] [ 4671.056134][ T44] register_pernet_subsys+0x1a/0x40 [ 4671.056367][ T44] pg_init+0x2c/0x1000 [pktgen] [ 4671.056599][ T44] ? _sub_I_65535_1+0x5c0/0x5c0 [nsh] [ 4671.056820][ T44] do_one_initcall+0x8c/0x1d0 [ 4671.057053][ T44] ? trace_initcall_start+0x130/0x130 [ 4671.057275][ T44] ? rcu_is_watching+0x12/0xb0 [ 4671.057516][ T44] ? trace_kmalloc+0x26/0xd0 [ 4671.057749][ T44] ? __kmalloc_cache_noprof+0x2e4/0x690 [ 4671.057971][ T44] ? debug_objects_oom+0x160/0x160 [ 4671.058198][ T44] ? kasan_unpoison+0x27/0x50 [ 4671.058438][ T44] ? __asan_register_globals+0x5d/0x70 [ 4671.058673][ T44] do_init_module+0x27a/0x860 [ 4671.058901][ T44] ? free_module+0x2f0/0x2f0 [ 4671.059124][ T44] ? kfree+0x21d/0x540 [ 4671.059301][ T44] ? load_module+0x20c5/0x2dd0 [ 4671.059581][ T44] load_module+0x1f40/0x2dd0 [ 4671.059815][ T44] ? layout_and_allocate.constprop.0+0x630/0x630 [ 4671.060090][ T44] ? kernel_read_file+0x3ed/0x550 [ 4671.060322][ T44] ? kernel_read_file+0x3c8/0x550 [ 4671.060554][ T44] ? __ia32_sys_fsconfig+0x150/0x150 [ 4671.060785][ T44] ? init_module_from_file+0xe9/0x150 [ 4671.061005][ T44] init_module_from_file+0xe9/0x150 [ 4671.061230][ T44] ? __do_sys_init_module+0x1e0/0x1e0 [ 4671.061478][ T44] ? idempotent_init_module+0x316/0x610 [ 4671.061699][ T44] ? __lock_release+0x5d/0x170 [ 4671.061931][ T44] ? do_raw_spin_unlock+0x58/0x220 [ 4671.062156][ T44] idempotent_init_module+0x331/0x610 [ 4671.062392][ T44] ? init_module_from_file+0x150/0x150 [ 4671.062625][ T44] ? cap_capable+0x94/0x230 [ 4671.062857][ T44] __x64_sys_finit_module+0xca/0x150 [ 4671.063076][ T44] ? do_syscall_64+0x85/0xfd0 [ 4671.063308][ T44] do_syscall_64+0xc1/0xfd0 [ 4671.063538][ T44] entry_SYSCALL_64_after_hwframe+0x4b/0x53 [ 4671.063814][ T44] RIP: 0033:0x7f11f314a92d [ 4671.064050][ T44] RSP: 002b:00007ffd4ecb67e8 EFLAGS: 00000246 ORIG_RAX: 0000000000000139 [ 4671.064405][ T44] RAX: ffffffffffffffda RBX: 000055aab0152ac0 RCX: 00007f11f314a92d [ 4671.064740][ T44] RDX: 0000000000000000 RSI: 000055aa8ae1ea2a RDI: 0000000000000005 [ 4671.065084][ T44] RBP: 0000000000040000 R08: 0000000000000000 R09: 00007ffd4ecb6920 [ 4671.065426][ T44] R10: 0000000000000005 R11: 0000000000000246 R12: 000055aa8ae1ea2a [ 4671.065760][ T44] R13: 000055aab0152b60 R14: 000055aab0152da0 R15: 0000000000000000 [ 4671.066112][ T44] [ 4671.066284][ T44] INFO: task modprobe:31907 blocked on an rw-semaphore likely owned by task kworker/u16:0:12 [ 4671.066784][ T44] task:kworker/u16:0 state:R running task stack:24656 pid:12 tgid:12 ppid:2 task_flags:0x4208060 flags:0x00080000 [ 4671.067343][ T44] Workqueue: netns cleanup_net [ 4671.067568][ T44] Call Trace: [ 4671.067735][ T44] [ 4671.067846][ T44] ? __mutex_lock+0x19f/0x1190 [ 4671.068075][ T44] ? __schedule+0x153/0x1b10 [ 4671.068316][ T44] ? nf_ct_iterate_cleanup+0x4a/0x340 [nf_conntrack] [ 4671.068610][ T44] ? ww_mutex_lock+0x160/0x160 [ 4671.068848][ T44] ? nf_ct_iterate_cleanup+0x4a/0x340 [nf_conntrack] [ 4671.069137][ T44] ? nf_ct_iterate_cleanup+0x4a/0x340 [nf_conntrack] [ 4671.069427][ T44] ? __nf_conntrack_hash_insert+0x200/0x200 [nf_conntrack] [ 4671.069776][ T44] ? nf_conntrack_cleanup_net_list+0x19a/0x4b0 [nf_conntrack] [ 4671.070131][ T44] ? nf_conntrack_cleanup_net_list+0x10e/0x4b0 [nf_conntrack] [ 4671.070483][ T44] ? nf_conntrack_cleanup_end+0x70/0x70 [nf_conntrack] [ 4671.070773][ T44] ? nf_conntrack_pernet_exit+0xa5/0x140 [nf_conntrack] [ 4671.071060][ T44] ? _raw_spin_unlock+0x2d/0x50 [ 4671.071286][ T44] ? ops_undo_list+0x2bf/0x890 [ 4671.071533][ T44] ? rtnl_net_dumpid_one+0x270/0x270 [ 4671.071757][ T44] ? cleanup_net+0x2d6/0x8b0 [ 4671.071990][ T44] ? cleanup_net+0x3b2/0x8b0 [ 4671.072214][ T44] ? net_passive_dec+0x190/0x190 [ 4671.072450][ T44] ? rcu_is_watching+0x12/0xb0 [ 4671.072677][ T44] ? process_one_work+0xe35/0x1650 [ 4671.072911][ T44] ? pwq_dec_nr_in_flight+0x550/0x550 [ 4671.073144][ T44] ? assign_work+0x168/0x240 [ 4671.073387][ T44] ? worker_thread+0x591/0xcf0 [ 4671.073620][ T44] ? rescuer_thread+0xd10/0xd10 [ 4671.073843][ T44] ? kthread+0x37b/0x5f0 [ 4671.074015][ T44] ? kthread_is_per_cpu+0xc0/0xc0 [ 4671.074242][ T44] ? ret_from_fork+0x1b/0x270 [ 4671.074481][ T44] ? __lock_release+0x5d/0x170 [ 4671.074705][ T44] ? rcu_is_watching+0x12/0xb0 [ 4671.074930][ T44] ? kthread_is_per_cpu+0xc0/0xc0 [ 4671.075165][ T44] ? ret_from_fork+0x1db/0x270 [ 4671.075395][ T44] ? kthread_is_per_cpu+0xc0/0xc0 [ 4671.075617][ T44] ? ret_from_fork_asm+0x11/0x20 [ 4671.075851][ T44] [ 4671.076046][ T44] [ 4671.076046][ T44] Showing all locks held in the system: [ 4671.076514][ T44] 4 locks held by kworker/u16:0/12: [ 4671.076745][ T44] 1 lock held by khungtaskd/44: [ 4671.076957][ T44] #0: ffffffff97d881c0 (rcu_read_lock){....}-{1:3}, at: debug_show_all_locks+0x36/0x260 [ 4671.077384][ T44] 1 lock held by modprobe/31907: [ 4671.077598][ T44] #0: ffffffff9847fb10 (pernet_ops_rwsem){++++}-{4:4}, at: register_pernet_subsys+0x1a/0x40 [ 4671.078031][ T44] [ 4671.078137][ T44] ============================================= [ 4671.078137][ T44] [ 4793.930377][ T44] INFO: task modprobe:31907 blocked for more than 491 seconds. [ 4793.930852][ T44] Not tainted 6.18.0-rc2-virtme #1 [ 4793.931070][ T44] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. [ 4793.931468][ T44] task:modprobe state:D stack:25664 pid:31907 tgid:31907 ppid:1 task_flags:0x400100 flags:0x00080003 [ 4793.931961][ T44] Call Trace: [ 4793.932126][ T44] [ 4793.932244][ T44] __schedule+0x880/0x1b10 [ 4793.932520][ T44] ? io_schedule_timeout+0x150/0x150 [ 4793.932759][ T44] ? schedule+0x1ee/0x270 [ 4793.932931][ T44] ? __lock_release+0x5d/0x170 [ 4793.933157][ T44] schedule+0xe2/0x270 [ 4793.933341][ T44] schedule_preempt_disabled+0x18/0x30 [ 4793.933570][ T44] rwsem_down_write_slowpath+0x3c5/0xc60 [ 4793.933795][ T44] ? __lock_acquire+0x449/0x7e0 [ 4793.934017][ T44] ? rwsem_down_read_slowpath+0xd80/0xd80 [ 4793.934247][ T44] ? rcu_is_watching+0x12/0xb0 [ 4793.934488][ T44] ? lock_acquire+0x104/0x160 [ 4793.934708][ T44] ? register_pernet_subsys+0x1a/0x40 [ 4793.934937][ T44] down_write+0x1d6/0x200 [ 4793.935106][ T44] ? rwsem_down_write_slowpath+0xc60/0xc60 [ 4793.935388][ T44] ? find_held_lock+0x2b/0x80 [ 4793.935612][ T44] ? __create_object+0x5e/0xb0 [ 4793.935838][ T44] ? _sub_I_65535_1+0x5c0/0x5c0 [nsh] [ 4793.936067][ T44] register_pernet_subsys+0x1a/0x40 [ 4793.936317][ T44] pg_init+0x2c/0x1000 [pktgen] [ 4793.936549][ T44] ? _sub_I_65535_1+0x5c0/0x5c0 [nsh] [ 4793.936768][ T44] do_one_initcall+0x8c/0x1d0 [ 4793.936994][ T44] ? trace_initcall_start+0x130/0x130 [ 4793.937213][ T44] ? rcu_is_watching+0x12/0xb0 [ 4793.937447][ T44] ? trace_kmalloc+0x26/0xd0 [ 4793.937670][ T44] ? __kmalloc_cache_noprof+0x2e4/0x690 [ 4793.937894][ T44] ? debug_objects_oom+0x160/0x160 [ 4793.938116][ T44] ? kasan_unpoison+0x27/0x50 [ 4793.938351][ T44] ? __asan_register_globals+0x5d/0x70 [ 4793.938577][ T44] do_init_module+0x27a/0x860 [ 4793.938806][ T44] ? free_module+0x2f0/0x2f0 [ 4793.939025][ T44] ? kfree+0x21d/0x540 [ 4793.939189][ T44] ? load_module+0x20c5/0x2dd0 [ 4793.939432][ T44] load_module+0x1f40/0x2dd0 [ 4793.939665][ T44] ? layout_and_allocate.constprop.0+0x630/0x630 [ 4793.939937][ T44] ? kernel_read_file+0x3ed/0x550 [ 4793.940162][ T44] ? kernel_read_file+0x3c8/0x550 [ 4793.940409][ T44] ? __ia32_sys_fsconfig+0x150/0x150 [ 4793.940638][ T44] ? init_module_from_file+0xe9/0x150 [ 4793.940857][ T44] init_module_from_file+0xe9/0x150 [ 4793.941084][ T44] ? __do_sys_init_module+0x1e0/0x1e0 [ 4793.941329][ T44] ? idempotent_init_module+0x316/0x610 [ 4793.941549][ T44] ? __lock_release+0x5d/0x170 [ 4793.941771][ T44] ? do_raw_spin_unlock+0x58/0x220 [ 4793.941994][ T44] idempotent_init_module+0x331/0x610 [ 4793.942220][ T44] ? init_module_from_file+0x150/0x150 [ 4793.942461][ T44] ? cap_capable+0x94/0x230 [ 4793.942697][ T44] __x64_sys_finit_module+0xca/0x150 [ 4793.942918][ T44] ? do_syscall_64+0x85/0xfd0 [ 4793.943141][ T44] do_syscall_64+0xc1/0xfd0 [ 4793.943369][ T44] entry_SYSCALL_64_after_hwframe+0x4b/0x53 [ 4793.943659][ T44] RIP: 0033:0x7f11f314a92d [ 4793.943895][ T44] RSP: 002b:00007ffd4ecb67e8 EFLAGS: 00000246 ORIG_RAX: 0000000000000139 [ 4793.944236][ T44] RAX: ffffffffffffffda RBX: 000055aab0152ac0 RCX: 00007f11f314a92d [ 4793.944589][ T44] RDX: 0000000000000000 RSI: 000055aa8ae1ea2a RDI: 0000000000000005 [ 4793.944921][ T44] RBP: 0000000000040000 R08: 0000000000000000 R09: 00007ffd4ecb6920 [ 4793.945249][ T44] R10: 0000000000000005 R11: 0000000000000246 R12: 000055aa8ae1ea2a [ 4793.945585][ T44] R13: 000055aab0152b60 R14: 000055aab0152da0 R15: 0000000000000000 [ 4793.945938][ T44] [ 4793.946112][ T44] INFO: task modprobe:31907 blocked on an rw-semaphore likely owned by task kworker/u16:0:12 [ 4793.946586][ T44] task:kworker/u16:0 state:R running task stack:24656 pid:12 tgid:12 ppid:2 task_flags:0x4208060 flags:0x00080000 [ 4793.947129][ T44] Workqueue: netns cleanup_net [ 4793.947363][ T44] Call Trace: [ 4793.947533][ T44] [ 4793.947650][ T44] ? __mutex_trylock_common+0xf9/0x260 [ 4793.947875][ T44] ? __mutex_handoff+0x2b0/0x2b0 [ 4793.948095][ T44] ? rcu_is_watching+0x12/0xb0 [ 4793.948346][ T44] ? rcu_is_watching+0x12/0xb0 [ 4793.948569][ T44] ? trace_contention_end+0xd8/0x140 [ 4793.948789][ T44] ? lock_pin_lock+0xb5/0x140 [ 4793.949018][ T44] ? __mutex_lock+0x19f/0x1190 [ 4793.949241][ T44] ? trace_sched_exit_tp+0xcd/0x130 [ 4793.949477][ T44] ? nf_ct_iterate_cleanup+0x4a/0x340 [nf_conntrack] [ 4793.949768][ T44] ? ww_mutex_lock+0x160/0x160 [ 4793.950006][ T44] ? nf_ct_iterate_cleanup+0x4a/0x340 [nf_conntrack] [ 4793.950300][ T44] ? __nf_conntrack_hash_insert+0x200/0x200 [nf_conntrack] [ 4793.950649][ T44] ? nf_conntrack_cleanup_net_list+0x19a/0x4b0 [nf_conntrack] [ 4793.950988][ T44] ? nf_conntrack_cleanup_net_list+0x10e/0x4b0 [nf_conntrack] [ 4793.951340][ T44] ? nf_conntrack_cleanup_end+0x70/0x70 [nf_conntrack] [ 4793.951630][ T44] ? nf_conntrack_pernet_exit+0xa5/0x140 [nf_conntrack] [ 4793.951912][ T44] ? _raw_spin_unlock+0x2d/0x50 [ 4793.952139][ T44] ? ops_undo_list+0x2bf/0x890 [ 4793.952387][ T44] ? rtnl_net_dumpid_one+0x270/0x270 [ 4793.952607][ T44] ? cleanup_net+0x2d6/0x8b0 [ 4793.952843][ T44] ? cleanup_net+0x3b2/0x8b0 [ 4793.953063][ T44] ? net_passive_dec+0x190/0x190 [ 4793.953288][ T44] ? rcu_is_watching+0x12/0xb0 [ 4793.953522][ T44] ? process_one_work+0xe35/0x1650 [ 4793.953768][ T44] ? pwq_dec_nr_in_flight+0x550/0x550 [ 4793.954004][ T44] ? assign_work+0x168/0x240 [ 4793.954227][ T44] ? worker_thread+0x591/0xcf0 [ 4793.954465][ T44] ? rescuer_thread+0xd10/0xd10 [ 4793.954689][ T44] ? kthread+0x37b/0x5f0 [ 4793.954860][ T44] ? kthread_is_per_cpu+0xc0/0xc0 [ 4793.955077][ T44] ? ret_from_fork+0x1b/0x270 [ 4793.955305][ T44] ? __lock_release+0x5d/0x170 [ 4793.955528][ T44] ? rcu_is_watching+0x12/0xb0 [ 4793.955754][ T44] ? kthread_is_per_cpu+0xc0/0xc0 [ 4793.955976][ T44] ? ret_from_fork+0x1db/0x270 [ 4793.956194][ T44] ? kthread_is_per_cpu+0xc0/0xc0 [ 4793.956433][ T44] ? ret_from_fork_asm+0x11/0x20 [ 4793.956668][ T44] [ 4793.956860][ T44] [ 4793.956860][ T44] Showing all locks held in the system: [ 4793.957170][ T44] 4 locks held by kworker/u16:0/12: [ 4793.957674][ T44] 1 lock held by khungtaskd/44: [ 4793.957894][ T44] #0: ffffffff97d881c0 (rcu_read_lock){....}-{1:3}, at: debug_show_all_locks+0x36/0x260 [ 4793.958288][ T44] 1 lock held by modprobe/31907: [ 4793.958506][ T44] #0: ffffffff9847fb10 (pernet_ops_rwsem){++++}-{4:4}, at: register_pernet_subsys+0x1a/0x40 [ 4793.958935][ T44] [ 4793.959040][ T44] ============================================= [ 4793.959040][ T44] [ 4916.810395][ T44] INFO: task modprobe:31907 blocked for more than 614 seconds. [ 4916.810799][ T44] Not tainted 6.18.0-rc2-virtme #1 [ 4916.811053][ T44] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. [ 4916.811441][ T44] task:modprobe state:D stack:25664 pid:31907 tgid:31907 ppid:1 task_flags:0x400100 flags:0x00080003 [ 4916.811944][ T44] Call Trace: [ 4916.812112][ T44] [ 4916.812231][ T44] __schedule+0x880/0x1b10 [ 4916.812510][ T44] ? io_schedule_timeout+0x150/0x150 [ 4916.812746][ T44] ? schedule+0x1ee/0x270 [ 4916.812919][ T44] ? __lock_release+0x5d/0x170 [ 4916.813149][ T44] schedule+0xe2/0x270 [ 4916.813325][ T44] schedule_preempt_disabled+0x18/0x30 [ 4916.813545][ T44] rwsem_down_write_slowpath+0x3c5/0xc60 [ 4916.813768][ T44] ? __lock_acquire+0x449/0x7e0 [ 4916.813987][ T44] ? rwsem_down_read_slowpath+0xd80/0xd80 [ 4916.814225][ T44] ? rcu_is_watching+0x12/0xb0 [ 4916.814456][ T44] ? lock_acquire+0x104/0x160 [ 4916.814674][ T44] ? register_pernet_subsys+0x1a/0x40 [ 4916.814903][ T44] down_write+0x1d6/0x200 [ 4916.815081][ T44] ? rwsem_down_write_slowpath+0xc60/0xc60 [ 4916.815363][ T44] ? find_held_lock+0x2b/0x80 [ 4916.815585][ T44] ? __create_object+0x5e/0xb0 [ 4916.815817][ T44] ? _sub_I_65535_1+0x5c0/0x5c0 [nsh] [ 4916.816037][ T44] register_pernet_subsys+0x1a/0x40 [ 4916.816259][ T44] pg_init+0x2c/0x1000 [pktgen] [ 4916.816494][ T44] ? _sub_I_65535_1+0x5c0/0x5c0 [nsh] [ 4916.816715][ T44] do_one_initcall+0x8c/0x1d0 [ 4916.816939][ T44] ? trace_initcall_start+0x130/0x130 [ 4916.817162][ T44] ? rcu_is_watching+0x12/0xb0 [ 4916.817400][ T44] ? trace_kmalloc+0x26/0xd0 [ 4916.817620][ T44] ? __kmalloc_cache_noprof+0x2e4/0x690 [ 4916.817842][ T44] ? debug_objects_oom+0x160/0x160 [ 4916.818066][ T44] ? kasan_unpoison+0x27/0x50 [ 4916.818289][ T44] ? __asan_register_globals+0x5d/0x70 [ 4916.818529][ T44] do_init_module+0x27a/0x860 [ 4916.818759][ T44] ? free_module+0x2f0/0x2f0 [ 4916.818985][ T44] ? kfree+0x21d/0x540 [ 4916.819151][ T44] ? load_module+0x20c5/0x2dd0 [ 4916.819385][ T44] load_module+0x1f40/0x2dd0 [ 4916.819616][ T44] ? layout_and_allocate.constprop.0+0x630/0x630 [ 4916.819892][ T44] ? kernel_read_file+0x3ed/0x550 [ 4916.820117][ T44] ? kernel_read_file+0x3c8/0x550 [ 4916.820348][ T44] ? __ia32_sys_fsconfig+0x150/0x150 [ 4916.820575][ T44] ? init_module_from_file+0xe9/0x150 [ 4916.820802][ T44] init_module_from_file+0xe9/0x150 [ 4916.821027][ T44] ? __do_sys_init_module+0x1e0/0x1e0 [ 4916.821263][ T44] ? idempotent_init_module+0x316/0x610 [ 4916.821491][ T44] ? __lock_release+0x5d/0x170 [ 4916.821714][ T44] ? do_raw_spin_unlock+0x58/0x220 [ 4916.821936][ T44] idempotent_init_module+0x331/0x610 [ 4916.822168][ T44] ? init_module_from_file+0x150/0x150 [ 4916.822410][ T44] ? cap_capable+0x94/0x230 [ 4916.822638][ T44] __x64_sys_finit_module+0xca/0x150 [ 4916.822859][ T44] ? do_syscall_64+0x85/0xfd0 [ 4916.823093][ T44] do_syscall_64+0xc1/0xfd0 [ 4916.823325][ T44] entry_SYSCALL_64_after_hwframe+0x4b/0x53 [ 4916.823604][ T44] RIP: 0033:0x7f11f314a92d [ 4916.823833][ T44] RSP: 002b:00007ffd4ecb67e8 EFLAGS: 00000246 ORIG_RAX: 0000000000000139 [ 4916.824175][ T44] RAX: ffffffffffffffda RBX: 000055aab0152ac0 RCX: 00007f11f314a92d [ 4916.824516][ T44] RDX: 0000000000000000 RSI: 000055aa8ae1ea2a RDI: 0000000000000005 [ 4916.824845][ T44] RBP: 0000000000040000 R08: 0000000000000000 R09: 00007ffd4ecb6920 [ 4916.825179][ T44] R10: 0000000000000005 R11: 0000000000000246 R12: 000055aa8ae1ea2a [ 4916.825525][ T44] R13: 000055aab0152b60 R14: 000055aab0152da0 R15: 0000000000000000 [ 4916.825872][ T44] [ 4916.826043][ T44] INFO: task modprobe:31907 blocked on an rw-semaphore likely owned by task kworker/u16:0:12 [ 4916.826516][ T44] task:kworker/u16:0 state:R running task stack:24656 pid:12 tgid:12 ppid:2 task_flags:0x4208060 flags:0x00080000 [ 4916.827066][ T44] Workqueue: netns cleanup_net [ 4916.827301][ T44] Call Trace: [ 4916.827472][ T44] [ 4916.827590][ T44] ? __mutex_trylock_common+0xf9/0x260 [ 4916.827814][ T44] ? __mutex_handoff+0x2b0/0x2b0 [ 4916.828041][ T44] ? rcu_is_watching+0x12/0xb0 [ 4916.828266][ T44] ? rcu_is_watching+0x12/0xb0 [ 4916.828506][ T44] ? trace_contention_end+0xd8/0x140 [ 4916.828728][ T44] ? __mutex_lock+0x19f/0x1190 [ 4916.828950][ T44] ? trace_sched_exit_tp+0xcd/0x130 [ 4916.829174][ T44] ? nf_ct_iterate_cleanup+0x4a/0x340 [nf_conntrack] [ 4916.829468][ T44] ? ww_mutex_lock+0x160/0x160 [ 4916.829708][ T44] ? nf_ct_iterate_cleanup+0x4a/0x340 [nf_conntrack] [ 4916.829992][ T44] ? __nf_conntrack_hash_insert+0x200/0x200 [nf_conntrack] [ 4916.830349][ T44] ? nf_conntrack_cleanup_net_list+0x19a/0x4b0 [nf_conntrack] [ 4916.830689][ T44] ? nf_conntrack_cleanup_net_list+0x10e/0x4b0 [nf_conntrack] [ 4916.831032][ T44] ? nf_conntrack_cleanup_end+0x70/0x70 [nf_conntrack] [ 4916.831326][ T44] ? nf_conntrack_pernet_exit+0xa5/0x140 [nf_conntrack] [ 4916.831610][ T44] ? _raw_spin_unlock+0x2d/0x50 [ 4916.831840][ T44] ? ops_undo_list+0x2bf/0x890 [ 4916.832068][ T44] ? rtnl_net_dumpid_one+0x270/0x270 [ 4916.832285][ T44] ? cleanup_net+0x2d6/0x8b0 [ 4916.832524][ T44] ? cleanup_net+0x3b2/0x8b0 [ 4916.832744][ T44] ? net_passive_dec+0x190/0x190 [ 4916.832977][ T44] ? rcu_is_watching+0x12/0xb0 [ 4916.833200][ T44] ? process_one_work+0xe35/0x1650 [ 4916.833446][ T44] ? pwq_dec_nr_in_flight+0x550/0x550 [ 4916.833676][ T44] ? assign_work+0x168/0x240 [ 4916.833900][ T44] ? worker_thread+0x591/0xcf0 [ 4916.834134][ T44] ? rescuer_thread+0xd10/0xd10 [ 4916.834366][ T44] ? kthread+0x37b/0x5f0 [ 4916.834536][ T44] ? kthread_is_per_cpu+0xc0/0xc0 [ 4916.834753][ T44] ? ret_from_fork+0x1b/0x270 [ 4916.834975][ T44] ? __lock_release+0x5d/0x170 [ 4916.835202][ T44] ? rcu_is_watching+0x12/0xb0 [ 4916.835429][ T44] ? kthread_is_per_cpu+0xc0/0xc0 [ 4916.835650][ T44] ? ret_from_fork+0x1db/0x270 [ 4916.835866][ T44] ? kthread_is_per_cpu+0xc0/0xc0 [ 4916.836085][ T44] ? ret_from_fork_asm+0x11/0x20 [ 4916.836327][ T44] [ 4916.836515][ T44] [ 4916.836515][ T44] Showing all locks held in the system: [ 4916.836829][ T44] 4 locks held by kworker/u16:0/12: [ 4916.837061][ T44] 1 lock held by khungtaskd/44: [ 4916.837270][ T44] #0: ffffffff97d881c0 (rcu_read_lock){....}-{1:3}, at: debug_show_all_locks+0x36/0x260 [ 4916.837817][ T44] 1 lock held by modprobe/31907: [ 4916.838031][ T44] #0: ffffffff9847fb10 (pernet_ops_rwsem){++++}-{4:4}, at: register_pernet_subsys+0x1a/0x40 [ 4916.838471][ T44] [ 4916.838577][ T44] ============================================= [ 4916.838577][ T44] [ 5039.690387][ T44] INFO: task modprobe:31907 blocked for more than 737 seconds. [ 5039.690799][ T44] Not tainted 6.18.0-rc2-virtme #1 [ 5039.691020][ T44] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. [ 5039.691419][ T44] task:modprobe state:D stack:25664 pid:31907 tgid:31907 ppid:1 task_flags:0x400100 flags:0x00080003 [ 5039.691927][ T44] Call Trace: [ 5039.692093][ T44] [ 5039.692220][ T44] __schedule+0x880/0x1b10 [ 5039.692499][ T44] ? io_schedule_timeout+0x150/0x150 [ 5039.692736][ T44] ? schedule+0x1ee/0x270 [ 5039.692904][ T44] ? __lock_release+0x5d/0x170 [ 5039.693133][ T44] schedule+0xe2/0x270 [ 5039.693316][ T44] schedule_preempt_disabled+0x18/0x30 [ 5039.693538][ T44] rwsem_down_write_slowpath+0x3c5/0xc60 [ 5039.693762][ T44] ? __lock_acquire+0x449/0x7e0 [ 5039.693984][ T44] ? rwsem_down_read_slowpath+0xd80/0xd80 [ 5039.694212][ T44] ? rcu_is_watching+0x12/0xb0 [ 5039.694442][ T44] ? lock_acquire+0x104/0x160 [ 5039.694666][ T44] ? register_pernet_subsys+0x1a/0x40 [ 5039.694902][ T44] down_write+0x1d6/0x200 [ 5039.695071][ T44] ? rwsem_down_write_slowpath+0xc60/0xc60 [ 5039.695355][ T44] ? find_held_lock+0x2b/0x80 [ 5039.695586][ T44] ? __create_object+0x5e/0xb0 [ 5039.695812][ T44] ? _sub_I_65535_1+0x5c0/0x5c0 [nsh] [ 5039.696033][ T44] register_pernet_subsys+0x1a/0x40 [ 5039.696255][ T44] pg_init+0x2c/0x1000 [pktgen] [ 5039.696514][ T44] ? _sub_I_65535_1+0x5c0/0x5c0 [nsh] [ 5039.696738][ T44] do_one_initcall+0x8c/0x1d0 [ 5039.696961][ T44] ? trace_initcall_start+0x130/0x130 [ 5039.697181][ T44] ? rcu_is_watching+0x12/0xb0 [ 5039.697410][ T44] ? trace_kmalloc+0x26/0xd0 [ 5039.697632][ T44] ? __kmalloc_cache_noprof+0x2e4/0x690 [ 5039.697851][ T44] ? debug_objects_oom+0x160/0x160 [ 5039.698083][ T44] ? kasan_unpoison+0x27/0x50 [ 5039.698314][ T44] ? __asan_register_globals+0x5d/0x70 [ 5039.698540][ T44] do_init_module+0x27a/0x860 [ 5039.698766][ T44] ? free_module+0x2f0/0x2f0 [ 5039.698986][ T44] ? kfree+0x21d/0x540 [ 5039.699151][ T44] ? load_module+0x20c5/0x2dd0 [ 5039.699385][ T44] load_module+0x1f40/0x2dd0 [ 5039.699625][ T44] ? layout_and_allocate.constprop.0+0x630/0x630 [ 5039.699901][ T44] ? kernel_read_file+0x3ed/0x550 [ 5039.700125][ T44] ? kernel_read_file+0x3c8/0x550 [ 5039.700371][ T44] ? __ia32_sys_fsconfig+0x150/0x150 [ 5039.700599][ T44] ? init_module_from_file+0xe9/0x150 [ 5039.700818][ T44] init_module_from_file+0xe9/0x150 [ 5039.701037][ T44] ? __do_sys_init_module+0x1e0/0x1e0 [ 5039.701273][ T44] ? idempotent_init_module+0x316/0x610 [ 5039.701510][ T44] ? __lock_release+0x5d/0x170 [ 5039.701734][ T44] ? do_raw_spin_unlock+0x58/0x220 [ 5039.701954][ T44] idempotent_init_module+0x331/0x610 [ 5039.702178][ T44] ? init_module_from_file+0x150/0x150 [ 5039.702428][ T44] ? cap_capable+0x94/0x230 [ 5039.702657][ T44] __x64_sys_finit_module+0xca/0x150 [ 5039.702880][ T44] ? do_syscall_64+0x85/0xfd0 [ 5039.703108][ T44] do_syscall_64+0xc1/0xfd0 [ 5039.703341][ T44] entry_SYSCALL_64_after_hwframe+0x4b/0x53 [ 5039.703619][ T44] RIP: 0033:0x7f11f314a92d [ 5039.703848][ T44] RSP: 002b:00007ffd4ecb67e8 EFLAGS: 00000246 ORIG_RAX: 0000000000000139 [ 5039.704182][ T44] RAX: ffffffffffffffda RBX: 000055aab0152ac0 RCX: 00007f11f314a92d [ 5039.704549][ T44] RDX: 0000000000000000 RSI: 000055aa8ae1ea2a RDI: 0000000000000005 [ 5039.704883][ T44] RBP: 0000000000040000 R08: 0000000000000000 R09: 00007ffd4ecb6920 [ 5039.705212][ T44] R10: 0000000000000005 R11: 0000000000000246 R12: 000055aa8ae1ea2a [ 5039.705553][ T44] R13: 000055aab0152b60 R14: 000055aab0152da0 R15: 0000000000000000 [ 5039.705900][ T44] [ 5039.706074][ T44] INFO: task modprobe:31907 blocked on an rw-semaphore likely owned by task kworker/u16:0:12 [ 5039.706556][ T44] task:kworker/u16:0 state:R running task stack:24656 pid:12 tgid:12 ppid:2 task_flags:0x4208060 flags:0x00080000 [ 5039.707098][ T44] Workqueue: netns cleanup_net [ 5039.707331][ T44] Call Trace: [ 5039.707500][ T44] [ 5039.707624][ T44] ? __mutex_trylock_common+0xf9/0x260 [ 5039.707847][ T44] ? __mutex_handoff+0x2b0/0x2b0 [ 5039.708070][ T44] ? rcu_is_watching+0x12/0xb0 [ 5039.708317][ T44] ? rcu_is_watching+0x12/0xb0 [ 5039.708544][ T44] ? trace_contention_end+0xd8/0x140 [ 5039.708763][ T44] ? __mutex_lock+0x19f/0x1190 [ 5039.708983][ T44] ? trace_sched_exit_tp+0xcd/0x130 [ 5039.709211][ T44] ? nf_ct_iterate_cleanup+0x4a/0x340 [nf_conntrack] [ 5039.709514][ T44] ? ww_mutex_lock+0x160/0x160 [ 5039.709753][ T44] ? nf_ct_iterate_cleanup+0x4a/0x340 [nf_conntrack] [ 5039.710035][ T44] ? nf_ct_iterate_cleanup+0x4a/0x340 [nf_conntrack] [ 5039.710322][ T44] ? __nf_conntrack_hash_insert+0x200/0x200 [nf_conntrack] [ 5039.710671][ T44] ? nf_conntrack_cleanup_net_list+0x19a/0x4b0 [nf_conntrack] [ 5039.711017][ T44] ? nf_conntrack_cleanup_net_list+0x10e/0x4b0 [nf_conntrack] [ 5039.711367][ T44] ? nf_conntrack_cleanup_end+0x70/0x70 [nf_conntrack] [ 5039.711656][ T44] ? nf_conntrack_pernet_exit+0xa5/0x140 [nf_conntrack] [ 5039.711944][ T44] ? _raw_spin_unlock+0x2d/0x50 [ 5039.712173][ T44] ? ops_undo_list+0x2bf/0x890 [ 5039.712430][ T44] ? rtnl_net_dumpid_one+0x270/0x270 [ 5039.712660][ T44] ? cleanup_net+0x2d6/0x8b0 [ 5039.712886][ T44] ? cleanup_net+0x3b2/0x8b0 [ 5039.713110][ T44] ? net_passive_dec+0x190/0x190 [ 5039.713343][ T44] ? rcu_is_watching+0x12/0xb0 [ 5039.713574][ T44] ? process_one_work+0xe35/0x1650 [ 5039.713804][ T44] ? pwq_dec_nr_in_flight+0x550/0x550 [ 5039.714032][ T44] ? assign_work+0x168/0x240 [ 5039.714266][ T44] ? worker_thread+0x591/0xcf0 [ 5039.714503][ T44] ? rescuer_thread+0xd10/0xd10 [ 5039.714730][ T44] ? kthread+0x37b/0x5f0 [ 5039.714899][ T44] ? kthread_is_per_cpu+0xc0/0xc0 [ 5039.715117][ T44] ? ret_from_fork+0x1b/0x270 [ 5039.715343][ T44] ? __lock_release+0x5d/0x170 [ 5039.715570][ T44] ? rcu_is_watching+0x12/0xb0 [ 5039.715788][ T44] ? kthread_is_per_cpu+0xc0/0xc0 [ 5039.716015][ T44] ? ret_from_fork+0x1db/0x270 [ 5039.716231][ T44] ? kthread_is_per_cpu+0xc0/0xc0 [ 5039.716472][ T44] ? ret_from_fork_asm+0x11/0x20 [ 5039.716711][ T44] [ 5039.716906][ T44] [ 5039.716906][ T44] Showing all locks held in the system: [ 5039.717219][ T44] 4 locks held by kworker/u16:0/12: [ 5039.717722][ T44] 1 lock held by khungtaskd/44: [ 5039.717936][ T44] #0: ffffffff97d881c0 (rcu_read_lock){....}-{1:3}, at: debug_show_all_locks+0x36/0x260 [ 5039.718345][ T44] 1 lock held by modprobe/31907: [ 5039.718557][ T44] #0: ffffffff9847fb10 (pernet_ops_rwsem){++++}-{4:4}, at: register_pernet_subsys+0x1a/0x40 [ 5039.718987][ T44] [ 5039.719091][ T44] ============================================= [ 5039.719091][ T44] [ 5162.570356][ T44] INFO: task modprobe:31907 blocked for more than 860 seconds. [ 5162.570720][ T44] Not tainted 6.18.0-rc2-virtme #1 [ 5162.570928][ T44] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. [ 5162.571286][ T44] task:modprobe state:D stack:25664 pid:31907 tgid:31907 ppid:1 task_flags:0x400100 flags:0x00080003 [ 5162.571812][ T44] Call Trace: [ 5162.571981][ T44] [ 5162.572099][ T44] __schedule+0x880/0x1b10 [ 5162.572346][ T44] ? io_schedule_timeout+0x150/0x150 [ 5162.572582][ T44] ? schedule+0x1ee/0x270 [ 5162.572748][ T44] ? __lock_release+0x5d/0x170 [ 5162.572983][ T44] schedule+0xe2/0x270 [ 5162.573151][ T44] schedule_preempt_disabled+0x18/0x30 [ 5162.573411][ T44] rwsem_down_write_slowpath+0x3c5/0xc60 [ 5162.573636][ T44] ? __lock_acquire+0x449/0x7e0 [ 5162.573853][ T44] ? rwsem_down_read_slowpath+0xd80/0xd80 [ 5162.574088][ T44] ? rcu_is_watching+0x12/0xb0 [ 5162.574317][ T44] ? lock_acquire+0x104/0x160 [ 5162.574537][ T44] ? register_pernet_subsys+0x1a/0x40 [ 5162.574764][ T44] down_write+0x1d6/0x200 [ 5162.574931][ T44] ? rwsem_down_write_slowpath+0xc60/0xc60 [ 5162.575206][ T44] ? find_held_lock+0x2b/0x80 [ 5162.575442][ T44] ? __create_object+0x5e/0xb0 [ 5162.575671][ T44] ? _sub_I_65535_1+0x5c0/0x5c0 [nsh] [ 5162.575897][ T44] register_pernet_subsys+0x1a/0x40 [ 5162.576122][ T44] pg_init+0x2c/0x1000 [pktgen] [ 5162.576361][ T44] ? _sub_I_65535_1+0x5c0/0x5c0 [nsh] [ 5162.576580][ T44] do_one_initcall+0x8c/0x1d0 [ 5162.576801][ T44] ? trace_initcall_start+0x130/0x130 [ 5162.577024][ T44] ? rcu_is_watching+0x12/0xb0 [ 5162.577245][ T44] ? trace_kmalloc+0x26/0xd0 [ 5162.577489][ T44] ? __kmalloc_cache_noprof+0x2e4/0x690 [ 5162.577710][ T44] ? debug_objects_oom+0x160/0x160 [ 5162.577936][ T44] ? kasan_unpoison+0x27/0x50 [ 5162.578161][ T44] ? __asan_register_globals+0x5d/0x70 [ 5162.578392][ T44] do_init_module+0x27a/0x860 [ 5162.578620][ T44] ? free_module+0x2f0/0x2f0 [ 5162.578838][ T44] ? kfree+0x21d/0x540 [ 5162.579005][ T44] ? load_module+0x20c5/0x2dd0 [ 5162.579229][ T44] load_module+0x1f40/0x2dd0 [ 5162.579470][ T44] ? layout_and_allocate.constprop.0+0x630/0x630 [ 5162.579747][ T44] ? kernel_read_file+0x3ed/0x550 [ 5162.579974][ T44] ? kernel_read_file+0x3c8/0x550 [ 5162.580195][ T44] ? __ia32_sys_fsconfig+0x150/0x150 [ 5162.580427][ T44] ? init_module_from_file+0xe9/0x150 [ 5162.580647][ T44] init_module_from_file+0xe9/0x150 [ 5162.580864][ T44] ? __do_sys_init_module+0x1e0/0x1e0 [ 5162.581103][ T44] ? idempotent_init_module+0x316/0x610 [ 5162.581343][ T44] ? __lock_release+0x5d/0x170 [ 5162.581573][ T44] ? do_raw_spin_unlock+0x58/0x220 [ 5162.581794][ T44] idempotent_init_module+0x331/0x610 [ 5162.582018][ T44] ? init_module_from_file+0x150/0x150 [ 5162.582250][ T44] ? cap_capable+0x94/0x230 [ 5162.582484][ T44] __x64_sys_finit_module+0xca/0x150 [ 5162.582708][ T44] ? do_syscall_64+0x85/0xfd0 [ 5162.582930][ T44] do_syscall_64+0xc1/0xfd0 [ 5162.583155][ T44] entry_SYSCALL_64_after_hwframe+0x4b/0x53 [ 5162.583436][ T44] RIP: 0033:0x7f11f314a92d [ 5162.583664][ T44] RSP: 002b:00007ffd4ecb67e8 EFLAGS: 00000246 ORIG_RAX: 0000000000000139 [ 5162.583997][ T44] RAX: ffffffffffffffda RBX: 000055aab0152ac0 RCX: 00007f11f314a92d [ 5162.584339][ T44] RDX: 0000000000000000 RSI: 000055aa8ae1ea2a RDI: 0000000000000005 [ 5162.584668][ T44] RBP: 0000000000040000 R08: 0000000000000000 R09: 00007ffd4ecb6920 [ 5162.584999][ T44] R10: 0000000000000005 R11: 0000000000000246 R12: 000055aa8ae1ea2a [ 5162.585351][ T44] R13: 000055aab0152b60 R14: 000055aab0152da0 R15: 0000000000000000 [ 5162.585700][ T44] [ 5162.585868][ T44] INFO: task modprobe:31907 blocked on an rw-semaphore likely owned by task kworker/u16:0:12 [ 5162.586341][ T44] task:kworker/u16:0 state:R running task stack:24656 pid:12 tgid:12 ppid:2 task_flags:0x4208060 flags:0x00080000 [ 5162.586876][ T44] Workqueue: netns cleanup_net [ 5162.587099][ T44] Call Trace: [ 5162.587266][ T44] [ 5162.587393][ T44] ? __mutex_trylock_common+0xf9/0x260 [ 5162.587613][ T44] ? prepare_to_wait+0x2f0/0x360 [ 5162.587833][ T44] ? __mutex_handoff+0x2b0/0x2b0 [ 5162.588052][ T44] ? find_held_lock+0x2b/0x80 [ 5162.588270][ T44] ? rcu_is_watching+0x12/0xb0 [ 5162.588500][ T44] ? nf_ct_iterate_cleanup+0x4a/0x340 [nf_conntrack] [ 5162.588786][ T44] ? rcu_is_watching+0x12/0xb0 [ 5162.589010][ T44] ? trace_contention_end+0xd8/0x140 [ 5162.589230][ T44] ? __mutex_lock+0x19f/0x1190 [ 5162.589472][ T44] ? trace_sched_exit_tp+0xcd/0x130 [ 5162.589693][ T44] ? __schedule+0x121/0x1b10 [ 5162.589913][ T44] ? nf_ct_iterate_cleanup+0x4a/0x340 [nf_conntrack] [ 5162.590215][ T44] ? nf_ct_iterate_cleanup+0x4a/0x340 [nf_conntrack] [ 5162.590510][ T44] ? nf_ct_iterate_cleanup+0x4a/0x340 [nf_conntrack] [ 5162.590793][ T44] ? __nf_conntrack_hash_insert+0x200/0x200 [nf_conntrack] [ 5162.591138][ T44] ? nf_conntrack_cleanup_net_list+0x19a/0x4b0 [nf_conntrack] [ 5162.591481][ T44] ? nf_conntrack_cleanup_net_list+0x10e/0x4b0 [nf_conntrack] [ 5162.591838][ T44] ? nf_conntrack_cleanup_end+0x70/0x70 [nf_conntrack] [ 5162.592131][ T44] ? nf_conntrack_pernet_exit+0xa5/0x140 [nf_conntrack] [ 5162.592421][ T44] ? _raw_spin_unlock+0x2d/0x50 [ 5162.592646][ T44] ? ops_undo_list+0x2bf/0x890 [ 5162.592870][ T44] ? rtnl_net_dumpid_one+0x270/0x270 [ 5162.593092][ T44] ? cleanup_net+0x2d6/0x8b0 [ 5162.593339][ T44] ? cleanup_net+0x3b2/0x8b0 [ 5162.593565][ T44] ? net_passive_dec+0x190/0x190 [ 5162.593787][ T44] ? rcu_is_watching+0x12/0xb0 [ 5162.594018][ T44] ? process_one_work+0xe35/0x1650 [ 5162.594248][ T44] ? pwq_dec_nr_in_flight+0x550/0x550 [ 5162.594482][ T44] ? assign_work+0x168/0x240 [ 5162.594706][ T44] ? worker_thread+0x591/0xcf0 [ 5162.594932][ T44] ? rescuer_thread+0xd10/0xd10 [ 5162.595158][ T44] ? kthread+0x37b/0x5f0 [ 5162.595334][ T44] ? kthread_is_per_cpu+0xc0/0xc0 [ 5162.595556][ T44] ? ret_from_fork+0x1b/0x270 [ 5162.595773][ T44] ? __lock_release+0x5d/0x170 [ 5162.595994][ T44] ? rcu_is_watching+0x12/0xb0 [ 5162.596213][ T44] ? kthread_is_per_cpu+0xc0/0xc0 [ 5162.596442][ T44] ? ret_from_fork+0x1db/0x270 [ 5162.596658][ T44] ? kthread_is_per_cpu+0xc0/0xc0 [ 5162.596877][ T44] ? ret_from_fork_asm+0x11/0x20 [ 5162.597110][ T44] [ 5162.597325][ T44] [ 5162.597325][ T44] Showing all locks held in the system: [ 5162.597641][ T44] 4 locks held by kworker/u16:0/12: [ 5162.597858][ T44] 1 lock held by khungtaskd/44: [ 5162.598070][ T44] #0: ffffffff97d881c0 (rcu_read_lock){....}-{1:3}, at: debug_show_all_locks+0x36/0x260 [ 5162.598862][ T44] 1 lock held by modprobe/31907: [ 5162.599074][ T44] #0: ffffffff9847fb10 (pernet_ops_rwsem){++++}-{4:4}, at: register_pernet_subsys+0x1a/0x40 [ 5162.599516][ T44] [ 5162.599622][ T44] ============================================= [ 5162.599622][ T44] [ 5285.450431][ T44] INFO: task modprobe:31907 blocked for more than 983 seconds. [ 5285.451050][ T44] Not tainted 6.18.0-rc2-virtme #1 [ 5285.451265][ T44] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. [ 5285.451664][ T44] task:modprobe state:D stack:25664 pid:31907 tgid:31907 ppid:1 task_flags:0x400100 flags:0x00080003 [ 5285.452169][ T44] Call Trace: [ 5285.452351][ T44] [ 5285.452470][ T44] __schedule+0x880/0x1b10 [ 5285.452705][ T44] ? io_schedule_timeout+0x150/0x150 [ 5285.452940][ T44] ? schedule+0x1ee/0x270 [ 5285.453108][ T44] ? __lock_release+0x5d/0x170 [ 5285.453375][ T44] schedule+0xe2/0x270 [ 5285.453547][ T44] schedule_preempt_disabled+0x18/0x30 [ 5285.453774][ T44] rwsem_down_write_slowpath+0x3c5/0xc60 [ 5285.453999][ T44] ? __lock_acquire+0x449/0x7e0 [ 5285.454237][ T44] ? rwsem_down_read_slowpath+0xd80/0xd80 [ 5285.454483][ T44] ? rcu_is_watching+0x12/0xb0 [ 5285.454707][ T44] ? lock_acquire+0x104/0x160 [ 5285.454927][ T44] ? register_pernet_subsys+0x1a/0x40 [ 5285.455158][ T44] down_write+0x1d6/0x200 [ 5285.455347][ T44] ? rwsem_down_write_slowpath+0xc60/0xc60 [ 5285.455624][ T44] ? find_held_lock+0x2b/0x80 [ 5285.455850][ T44] ? __create_object+0x5e/0xb0 [ 5285.456083][ T44] ? _sub_I_65535_1+0x5c0/0x5c0 [nsh] [ 5285.456315][ T44] register_pernet_subsys+0x1a/0x40 [ 5285.456539][ T44] pg_init+0x2c/0x1000 [pktgen] [ 5285.456766][ T44] ? _sub_I_65535_1+0x5c0/0x5c0 [nsh] [ 5285.456992][ T44] do_one_initcall+0x8c/0x1d0 [ 5285.457216][ T44] ? trace_initcall_start+0x130/0x130 [ 5285.457470][ T44] ? rcu_is_watching+0x12/0xb0 [ 5285.457696][ T44] ? trace_kmalloc+0x26/0xd0 [ 5285.457918][ T44] ? __kmalloc_cache_noprof+0x2e4/0x690 [ 5285.458141][ T44] ? debug_objects_oom+0x160/0x160 [ 5285.458376][ T44] ? kasan_unpoison+0x27/0x50 [ 5285.458606][ T44] ? __asan_register_globals+0x5d/0x70 [ 5285.458831][ T44] do_init_module+0x27a/0x860 [ 5285.459061][ T44] ? free_module+0x2f0/0x2f0 [ 5285.459281][ T44] ? kfree+0x21d/0x540 [ 5285.459460][ T44] ? load_module+0x20c5/0x2dd0 [ 5285.459691][ T44] load_module+0x1f40/0x2dd0 [ 5285.459927][ T44] ? layout_and_allocate.constprop.0+0x630/0x630 [ 5285.460206][ T44] ? kernel_read_file+0x3ed/0x550 [ 5285.460446][ T44] ? kernel_read_file+0x3c8/0x550 [ 5285.460673][ T44] ? __ia32_sys_fsconfig+0x150/0x150 [ 5285.460915][ T44] ? init_module_from_file+0xe9/0x150 [ 5285.461138][ T44] init_module_from_file+0xe9/0x150 [ 5285.461385][ T44] ? __do_sys_init_module+0x1e0/0x1e0 [ 5285.461624][ T44] ? idempotent_init_module+0x316/0x610 [ 5285.461850][ T44] ? __lock_release+0x5d/0x170 [ 5285.462079][ T44] ? do_raw_spin_unlock+0x58/0x220 [ 5285.462314][ T44] idempotent_init_module+0x331/0x610 [ 5285.462540][ T44] ? init_module_from_file+0x150/0x150 [ 5285.462774][ T44] ? cap_capable+0x94/0x230 [ 5285.463006][ T44] __x64_sys_finit_module+0xca/0x150 [ 5285.463230][ T44] ? do_syscall_64+0x85/0xfd0 [ 5285.463471][ T44] do_syscall_64+0xc1/0xfd0 [ 5285.463694][ T44] entry_SYSCALL_64_after_hwframe+0x4b/0x53 [ 5285.463975][ T44] RIP: 0033:0x7f11f314a92d [ 5285.464210][ T44] RSP: 002b:00007ffd4ecb67e8 EFLAGS: 00000246 ORIG_RAX: 0000000000000139 [ 5285.464558][ T44] RAX: ffffffffffffffda RBX: 000055aab0152ac0 RCX: 00007f11f314a92d [ 5285.464896][ T44] RDX: 0000000000000000 RSI: 000055aa8ae1ea2a RDI: 0000000000000005 [ 5285.465237][ T44] RBP: 0000000000040000 R08: 0000000000000000 R09: 00007ffd4ecb6920 [ 5285.465594][ T44] R10: 0000000000000005 R11: 0000000000000246 R12: 000055aa8ae1ea2a [ 5285.465928][ T44] R13: 000055aab0152b60 R14: 000055aab0152da0 R15: 0000000000000000 [ 5285.466281][ T44] [ 5285.466461][ T44] INFO: task modprobe:31907 blocked on an rw-semaphore likely owned by task kworker/u16:0:12 [ 5285.466938][ T44] task:kworker/u16:0 state:R running task stack:24656 pid:12 tgid:12 ppid:2 task_flags:0x4208060 flags:0x00080000 [ 5285.467493][ T44] Workqueue: netns cleanup_net [ 5285.467721][ T44] Call Trace: [ 5285.467889][ T44] [ 5285.468009][ T44] ? __mutex_trylock_common+0xf9/0x260 [ 5285.468241][ T44] ? __mutex_handoff+0x2b0/0x2b0 [ 5285.468477][ T44] ? rcu_is_watching+0x12/0xb0 [ 5285.468704][ T44] ? rcu_is_watching+0x12/0xb0 [ 5285.468927][ T44] ? trace_contention_end+0xd8/0x140 [ 5285.469149][ T44] ? __mutex_lock+0x19f/0x1190 [ 5285.469398][ T44] ? trace_sched_exit_tp+0xcd/0x130 [ 5285.469628][ T44] ? nf_ct_iterate_cleanup+0x4a/0x340 [nf_conntrack] [ 5285.469924][ T44] ? ww_mutex_lock+0x160/0x160 [ 5285.470167][ T44] ? nf_ct_iterate_cleanup+0x4a/0x340 [nf_conntrack] [ 5285.470465][ T44] ? nf_ct_iterate_cleanup+0x4a/0x340 [nf_conntrack] [ 5285.470750][ T44] ? __nf_conntrack_hash_insert+0x200/0x200 [nf_conntrack] [ 5285.471100][ T44] ? nf_conntrack_cleanup_net_list+0x19a/0x4b0 [nf_conntrack] [ 5285.471459][ T44] ? nf_conntrack_cleanup_net_list+0x10e/0x4b0 [nf_conntrack] [ 5285.471805][ T44] ? nf_conntrack_cleanup_end+0x70/0x70 [nf_conntrack] [ 5285.472095][ T44] ? nf_conntrack_pernet_exit+0xa5/0x140 [nf_conntrack] [ 5285.472391][ T44] ? _raw_spin_unlock+0x2d/0x50 [ 5285.472620][ T44] ? ops_undo_list+0x2bf/0x890 [ 5285.472849][ T44] ? rtnl_net_dumpid_one+0x270/0x270 [ 5285.473082][ T44] ? cleanup_net+0x2d6/0x8b0 [ 5285.473336][ T44] ? cleanup_net+0x3b2/0x8b0 [ 5285.473564][ T44] ? net_passive_dec+0x190/0x190 [ 5285.473794][ T44] ? rcu_is_watching+0x12/0xb0 [ 5285.474021][ T44] ? process_one_work+0xe35/0x1650 [ 5285.474255][ T44] ? pwq_dec_nr_in_flight+0x550/0x550 [ 5285.474494][ T44] ? assign_work+0x168/0x240 [ 5285.474723][ T44] ? worker_thread+0x591/0xcf0 [ 5285.474960][ T44] ? rescuer_thread+0xd10/0xd10 [ 5285.475188][ T44] ? kthread+0x37b/0x5f0 [ 5285.475367][ T44] ? kthread_is_per_cpu+0xc0/0xc0 [ 5285.475590][ T44] ? ret_from_fork+0x1b/0x270 [ 5285.475811][ T44] ? __lock_release+0x5d/0x170 [ 5285.476035][ T44] ? rcu_is_watching+0x12/0xb0 [ 5285.476258][ T44] ? kthread_is_per_cpu+0xc0/0xc0 [ 5285.476491][ T44] ? ret_from_fork+0x1db/0x270 [ 5285.476717][ T44] ? kthread_is_per_cpu+0xc0/0xc0 [ 5285.476939][ T44] ? ret_from_fork_asm+0x11/0x20 [ 5285.477178][ T44] [ 5285.477399][ T44] [ 5285.477399][ T44] Showing all locks held in the system: [ 5285.477718][ T44] 4 locks held by kworker/u16:0/12: [ 5285.477943][ T44] 1 lock held by khungtaskd/44: [ 5285.478156][ T44] #0: ffffffff97d881c0 (rcu_read_lock){....}-{1:3}, at: debug_show_all_locks+0x36/0x260 [ 5285.478919][ T44] 1 lock held by modprobe/31907: [ 5285.479134][ T44] #0: ffffffff9847fb10 (pernet_ops_rwsem){++++}-{4:4}, at: register_pernet_subsys+0x1a/0x40 [ 5285.479584][ T44] [ 5285.479692][ T44] ============================================= [ 5285.479692][ T44] [ 5408.330379][ T44] INFO: task modprobe:31907 blocked for more than 1105 seconds. [ 5408.330973][ T44] Not tainted 6.18.0-rc2-virtme #1 [ 5408.331230][ T44] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. [ 5408.331643][ T44] task:modprobe state:D stack:25664 pid:31907 tgid:31907 ppid:1 task_flags:0x400100 flags:0x00080003 [ 5408.332197][ T44] Call Trace: [ 5408.332460][ T44] [ 5408.332623][ T44] __schedule+0x880/0x1b10 [ 5408.332943][ T44] ? io_schedule_timeout+0x150/0x150 [ 5408.333256][ T44] ? schedule+0x1ee/0x270 [ 5408.333495][ T44] ? __lock_release+0x5d/0x170 [ 5408.333810][ T44] schedule+0xe2/0x270 [ 5408.334001][ T44] schedule_preempt_disabled+0x18/0x30 [ 5408.334312][ T44] rwsem_down_write_slowpath+0x3c5/0xc60 [ 5408.334627][ T44] ? __lock_acquire+0x449/0x7e0 [ 5408.334875][ T44] ? rwsem_down_read_slowpath+0xd80/0xd80 [ 5408.335120][ T44] ? rcu_is_watching+0x12/0xb0 [ 5408.335378][ T44] ? lock_acquire+0x104/0x160 [ 5408.335640][ T44] ? register_pernet_subsys+0x1a/0x40 [ 5408.335884][ T44] down_write+0x1d6/0x200 [ 5408.336065][ T44] ? rwsem_down_write_slowpath+0xc60/0xc60 [ 5408.336454][ T44] ? find_held_lock+0x2b/0x80 [ 5408.336728][ T44] ? __create_object+0x5e/0xb0 [ 5408.336975][ T44] ? _sub_I_65535_1+0x5c0/0x5c0 [nsh] [ 5408.337241][ T44] register_pernet_subsys+0x1a/0x40 [ 5408.337548][ T44] pg_init+0x2c/0x1000 [pktgen] [ 5408.337842][ T44] ? _sub_I_65535_1+0x5c0/0x5c0 [nsh] [ 5408.338089][ T44] do_one_initcall+0x8c/0x1d0 [ 5408.338397][ T44] ? trace_initcall_start+0x130/0x130 [ 5408.338703][ T44] ? rcu_is_watching+0x12/0xb0 [ 5408.339005][ T44] ? trace_kmalloc+0x26/0xd0 [ 5408.339243][ T44] ? __kmalloc_cache_noprof+0x2e4/0x690 [ 5408.339492][ T44] ? debug_objects_oom+0x160/0x160 [ 5408.339733][ T44] ? kasan_unpoison+0x27/0x50 [ 5408.339984][ T44] ? __asan_register_globals+0x5d/0x70 [ 5408.340246][ T44] do_init_module+0x27a/0x860 [ 5408.340566][ T44] ? free_module+0x2f0/0x2f0 [ 5408.340867][ T44] ? kfree+0x21d/0x540 [ 5408.341091][ T44] ? load_module+0x20c5/0x2dd0 [ 5408.341404][ T44] load_module+0x1f40/0x2dd0 [ 5408.341698][ T44] ? layout_and_allocate.constprop.0+0x630/0x630 [ 5408.341988][ T44] ? kernel_read_file+0x3ed/0x550 [ 5408.342312][ T44] ? kernel_read_file+0x3c8/0x550 [ 5408.342613][ T44] ? __ia32_sys_fsconfig+0x150/0x150 [ 5408.342930][ T44] ? init_module_from_file+0xe9/0x150 [ 5408.343162][ T44] init_module_from_file+0xe9/0x150 [ 5408.343420][ T44] ? __do_sys_init_module+0x1e0/0x1e0 [ 5408.343708][ T44] ? idempotent_init_module+0x316/0x610 [ 5408.344002][ T44] ? __lock_release+0x5d/0x170 [ 5408.344313][ T44] ? do_raw_spin_unlock+0x58/0x220 [ 5408.344616][ T44] idempotent_init_module+0x331/0x610 [ 5408.344917][ T44] ? init_module_from_file+0x150/0x150 [ 5408.345177][ T44] ? cap_capable+0x94/0x230 [ 5408.345428][ T44] __x64_sys_finit_module+0xca/0x150 [ 5408.345679][ T44] ? do_syscall_64+0x85/0xfd0 [ 5408.345977][ T44] do_syscall_64+0xc1/0xfd0 [ 5408.346277][ T44] entry_SYSCALL_64_after_hwframe+0x4b/0x53 [ 5408.346626][ T44] RIP: 0033:0x7f11f314a92d [ 5408.346873][ T44] RSP: 002b:00007ffd4ecb67e8 EFLAGS: 00000246 ORIG_RAX: 0000000000000139 [ 5408.347342][ T44] RAX: ffffffffffffffda RBX: 000055aab0152ac0 RCX: 00007f11f314a92d [ 5408.347724][ T44] RDX: 0000000000000000 RSI: 000055aa8ae1ea2a RDI: 0000000000000005 [ 5408.348107][ T44] RBP: 0000000000040000 R08: 0000000000000000 R09: 00007ffd4ecb6920 [ 5408.348564][ T44] R10: 0000000000000005 R11: 0000000000000246 R12: 000055aa8ae1ea2a [ 5408.349011][ T44] R13: 000055aab0152b60 R14: 000055aab0152da0 R15: 0000000000000000 [ 5408.349384][ T44] [ 5408.349564][ T44] INFO: task modprobe:31907 blocked on an rw-semaphore likely owned by task kworker/u16:0:12 [ 5408.350060][ T44] task:kworker/u16:0 state:R running task stack:24656 pid:12 tgid:12 ppid:2 task_flags:0x4208060 flags:0x00080000 [ 5408.350654][ T44] Workqueue: netns cleanup_net [ 5408.350893][ T44] Call Trace: [ 5408.351070][ T44] [ 5408.351194][ T44] ? __mutex_trylock_common+0xf9/0x260 [ 5408.351494][ T44] ? __mutex_handoff+0x2b0/0x2b0 [ 5408.351800][ T44] ? rcu_is_watching+0x12/0xb0 [ 5408.352097][ T44] ? trace_contention_end+0xd8/0x140 [ 5408.352342][ T44] ? __mutex_lock+0x19f/0x1190 [ 5408.352578][ T44] ? trace_sched_exit_tp+0xcd/0x130 [ 5408.352816][ T44] ? nf_ct_iterate_cleanup+0x4a/0x340 [nf_conntrack] [ 5408.353215][ T44] ? ww_mutex_lock+0x160/0x160 [ 5408.353542][ T44] ? nf_ct_iterate_cleanup+0x4a/0x340 [nf_conntrack] [ 5408.353879][ T44] ? nf_ct_iterate_cleanup+0x4a/0x340 [nf_conntrack] [ 5408.354179][ T44] ? __nf_conntrack_hash_insert+0x200/0x200 [nf_conntrack] [ 5408.354549][ T44] ? nf_conntrack_cleanup_net_list+0x19a/0x4b0 [nf_conntrack] [ 5408.354907][ T44] ? nf_conntrack_cleanup_net_list+0x10e/0x4b0 [nf_conntrack] [ 5408.355378][ T44] ? nf_conntrack_cleanup_end+0x70/0x70 [nf_conntrack] [ 5408.355760][ T44] ? nf_conntrack_pernet_exit+0xa5/0x140 [nf_conntrack] [ 5408.356061][ T44] ? _raw_spin_unlock+0x2d/0x50 [ 5408.356308][ T44] ? ops_undo_list+0x2bf/0x890 [ 5408.356564][ T44] ? rtnl_net_dumpid_one+0x270/0x270 [ 5408.356865][ T44] ? cleanup_net+0x2d6/0x8b0 [ 5408.357171][ T44] ? cleanup_net+0x3b2/0x8b0 [ 5408.357477][ T44] ? net_passive_dec+0x190/0x190 [ 5408.357782][ T44] ? rcu_is_watching+0x12/0xb0 [ 5408.358041][ T44] ? process_one_work+0xe35/0x1650 [ 5408.358299][ T44] ? pwq_dec_nr_in_flight+0x550/0x550 [ 5408.358542][ T44] ? assign_work+0x168/0x240 [ 5408.358828][ T44] ? worker_thread+0x591/0xcf0 [ 5408.359137][ T44] ? rescuer_thread+0xd10/0xd10 [ 5408.359442][ T44] ? kthread+0x37b/0x5f0 [ 5408.359668][ T44] ? kthread_is_per_cpu+0xc0/0xc0 [ 5408.359925][ T44] ? ret_from_fork+0x1b/0x270 [ 5408.360220][ T44] ? __lock_release+0x5d/0x170 [ 5408.360525][ T44] ? rcu_is_watching+0x12/0xb0 [ 5408.360805][ T44] ? kthread_is_per_cpu+0xc0/0xc0 [ 5408.361041][ T44] ? ret_from_fork+0x1db/0x270 [ 5408.361271][ T44] ? kthread_is_per_cpu+0xc0/0xc0 [ 5408.361578][ T44] ? ret_from_fork_asm+0x11/0x20 [ 5408.361890][ T44] [ 5408.362112][ T44] [ 5408.362112][ T44] Showing all locks held in the system: [ 5408.362597][ T44] 4 locks held by kworker/u16:0/12: [ 5408.362873][ T44] 1 lock held by khungtaskd/44: [ 5408.363158][ T44] #0: ffffffff97d881c0 (rcu_read_lock){....}-{1:3}, at: debug_show_all_locks+0x36/0x260 [ 5408.363704][ T44] 1 lock held by modprobe/31907: [ 5408.363952][ T44] #0: ffffffff9847fb10 (pernet_ops_rwsem){++++}-{4:4}, at: register_pernet_subsys+0x1a/0x40 [ 5408.364428][ T44] [ 5408.364541][ T44] ============================================= [ 5408.364541][ T44] [ 5531.210375][ T44] INFO: task modprobe:31907 blocked for more than 1228 seconds. [ 5531.210936][ T44] Not tainted 6.18.0-rc2-virtme #1 [ 5531.211161][ T44] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. [ 5531.211573][ T44] task:modprobe state:D stack:25664 pid:31907 tgid:31907 ppid:1 task_flags:0x400100 flags:0x00080003 [ 5531.212254][ T44] Call Trace: [ 5531.212516][ T44] [ 5531.212675][ T44] __schedule+0x880/0x1b10 [ 5531.212943][ T44] ? io_schedule_timeout+0x150/0x150 [ 5531.213205][ T44] ? schedule+0x1ee/0x270 [ 5531.213401][ T44] ? __lock_release+0x5d/0x170 [ 5531.213667][ T44] schedule+0xe2/0x270 [ 5531.213897][ T44] schedule_preempt_disabled+0x18/0x30 [ 5531.214225][ T44] rwsem_down_write_slowpath+0x3c5/0xc60 [ 5531.214540][ T44] ? __lock_acquire+0x449/0x7e0 [ 5531.214816][ T44] ? rwsem_down_read_slowpath+0xd80/0xd80 [ 5531.215060][ T44] ? rcu_is_watching+0x12/0xb0 [ 5531.215310][ T44] ? lock_acquire+0x104/0x160 [ 5531.215541][ T44] ? register_pernet_subsys+0x1a/0x40 [ 5531.215854][ T44] down_write+0x1d6/0x200 [ 5531.216086][ T44] ? rwsem_down_write_slowpath+0xc60/0xc60 [ 5531.216420][ T44] ? find_held_lock+0x2b/0x80 [ 5531.216663][ T44] ? __create_object+0x5e/0xb0 [ 5531.216950][ T44] ? _sub_I_65535_1+0x5c0/0x5c0 [nsh] [ 5531.217248][ T44] register_pernet_subsys+0x1a/0x40 [ 5531.217556][ T44] pg_init+0x2c/0x1000 [pktgen] [ 5531.217863][ T44] ? _sub_I_65535_1+0x5c0/0x5c0 [nsh] [ 5531.218109][ T44] do_one_initcall+0x8c/0x1d0 [ 5531.218430][ T44] ? trace_initcall_start+0x130/0x130 [ 5531.218728][ T44] ? rcu_is_watching+0x12/0xb0 [ 5531.219027][ T44] ? trace_kmalloc+0x26/0xd0 [ 5531.219260][ T44] ? __kmalloc_cache_noprof+0x2e4/0x690 [ 5531.219511][ T44] ? debug_objects_oom+0x160/0x160 [ 5531.219748][ T44] ? kasan_unpoison+0x27/0x50 [ 5531.219990][ T44] ? __asan_register_globals+0x5d/0x70 [ 5531.220243][ T44] do_init_module+0x27a/0x860 [ 5531.220575][ T44] ? free_module+0x2f0/0x2f0 [ 5531.220874][ T44] ? kfree+0x21d/0x540 [ 5531.221075][ T44] ? load_module+0x20c5/0x2dd0 [ 5531.221324][ T44] load_module+0x1f40/0x2dd0 [ 5531.221572][ T44] ? layout_and_allocate.constprop.0+0x630/0x630 [ 5531.221860][ T44] ? kernel_read_file+0x3ed/0x550 [ 5531.222102][ T44] ? kernel_read_file+0x3c8/0x550 [ 5531.222380][ T44] ? __ia32_sys_fsconfig+0x150/0x150 [ 5531.222686][ T44] ? init_module_from_file+0xe9/0x150 [ 5531.222952][ T44] init_module_from_file+0xe9/0x150 [ 5531.223186][ T44] ? __do_sys_init_module+0x1e0/0x1e0 [ 5531.223446][ T44] ? idempotent_init_module+0x316/0x610 [ 5531.223681][ T44] ? __lock_release+0x5d/0x170 [ 5531.223949][ T44] ? do_raw_spin_unlock+0x58/0x220 [ 5531.224248][ T44] idempotent_init_module+0x331/0x610 [ 5531.224574][ T44] ? init_module_from_file+0x150/0x150 [ 5531.224849][ T44] ? cap_capable+0x94/0x230 [ 5531.225163][ T44] __x64_sys_finit_module+0xca/0x150 [ 5531.225470][ T44] ? do_syscall_64+0x85/0xfd0 [ 5531.225769][ T44] do_syscall_64+0xc1/0xfd0 [ 5531.226004][ T44] entry_SYSCALL_64_after_hwframe+0x4b/0x53 [ 5531.226372][ T44] RIP: 0033:0x7f11f314a92d [ 5531.226683][ T44] RSP: 002b:00007ffd4ecb67e8 EFLAGS: 00000246 ORIG_RAX: 0000000000000139 [ 5531.227070][ T44] RAX: ffffffffffffffda RBX: 000055aab0152ac0 RCX: 00007f11f314a92d [ 5531.227429][ T44] RDX: 0000000000000000 RSI: 000055aa8ae1ea2a RDI: 0000000000000005 [ 5531.227789][ T44] RBP: 0000000000040000 R08: 0000000000000000 R09: 00007ffd4ecb6920 [ 5531.228142][ T44] R10: 0000000000000005 R11: 0000000000000246 R12: 000055aa8ae1ea2a [ 5531.228518][ T44] R13: 000055aab0152b60 R14: 000055aab0152da0 R15: 0000000000000000 [ 5531.228889][ T44] [ 5531.229070][ T44] INFO: task modprobe:31907 blocked on an rw-semaphore likely owned by task kworker/u16:0:12 [ 5531.229593][ T44] task:kworker/u16:0 state:R running task stack:24656 pid:12 tgid:12 ppid:2 task_flags:0x4208060 flags:0x00080000 [ 5531.230186][ T44] Workqueue: netns cleanup_net [ 5531.230498][ T44] Call Trace: [ 5531.230708][ T44] [ 5531.230833][ T44] ? __mutex_trylock_common+0xf9/0x260 [ 5531.231074][ T44] ? __mutex_handoff+0x2b0/0x2b0 [ 5531.231316][ T44] ? rcu_is_watching+0x12/0xb0 [ 5531.231607][ T44] ? rcu_is_watching+0x12/0xb0 [ 5531.231847][ T44] ? trace_contention_end+0xd8/0x140 [ 5531.232082][ T44] ? __mutex_lock+0x19f/0x1190 [ 5531.232411][ T44] ? trace_sched_exit_tp+0xcd/0x130 [ 5531.232710][ T44] ? __schedule+0x3d9/0x1b10 [ 5531.232950][ T44] ? nf_ct_iterate_cleanup+0x4a/0x340 [nf_conntrack] [ 5531.233259][ T44] ? ww_mutex_lock+0x160/0x160 [ 5531.233593][ T44] ? nf_ct_iterate_cleanup+0x4a/0x340 [nf_conntrack] [ 5531.233940][ T44] ? nf_ct_iterate_cleanup+0x4a/0x340 [nf_conntrack] [ 5531.234241][ T44] ? __nf_conntrack_hash_insert+0x200/0x200 [nf_conntrack] [ 5531.234609][ T44] ? nf_conntrack_cleanup_net_list+0x19a/0x4b0 [nf_conntrack] [ 5531.234975][ T44] ? nf_conntrack_cleanup_net_list+0x10e/0x4b0 [nf_conntrack] [ 5531.235342][ T44] ? nf_conntrack_cleanup_end+0x70/0x70 [nf_conntrack] [ 5531.235649][ T44] ? nf_conntrack_pernet_exit+0xa5/0x140 [nf_conntrack] [ 5531.235947][ T44] ? _raw_spin_unlock+0x2d/0x50 [ 5531.236183][ T44] ? ops_undo_list+0x2bf/0x890 [ 5531.236509][ T44] ? rtnl_net_dumpid_one+0x270/0x270 [ 5531.236811][ T44] ? cleanup_net+0x2d6/0x8b0 [ 5531.237053][ T44] ? cleanup_net+0x3b2/0x8b0 [ 5531.237288][ T44] ? net_passive_dec+0x190/0x190 [ 5531.237565][ T44] ? rcu_is_watching+0x12/0xb0 [ 5531.237865][ T44] ? process_one_work+0xe35/0x1650 [ 5531.238166][ T44] ? pwq_dec_nr_in_flight+0x550/0x550 [ 5531.238414][ T44] ? assign_work+0x168/0x240 [ 5531.238660][ T44] ? worker_thread+0x591/0xcf0 [ 5531.238982][ T44] ? rescuer_thread+0xd10/0xd10 [ 5531.239281][ T44] ? kthread+0x37b/0x5f0 [ 5531.239524][ T44] ? kthread_is_per_cpu+0xc0/0xc0 [ 5531.239803][ T44] ? ret_from_fork+0x1b/0x270 [ 5531.240035][ T44] ? __lock_release+0x5d/0x170 [ 5531.240270][ T44] ? rcu_is_watching+0x12/0xb0 [ 5531.240527][ T44] ? kthread_is_per_cpu+0xc0/0xc0 [ 5531.240769][ T44] ? ret_from_fork+0x1db/0x270 [ 5531.241000][ T44] ? kthread_is_per_cpu+0xc0/0xc0 [ 5531.241283][ T44] ? ret_from_fork_asm+0x11/0x20 [ 5531.241603][ T44] [ 5531.241824][ T44] Future hung task reports are suppressed, see sysctl kernel.hung_task_warnings [ 5531.242270][ T44] [ 5531.242270][ T44] Showing all locks held in the system: [ 5531.242934][ T44] 4 locks held by kworker/u16:0/12: [ 5531.243178][ T44] 1 lock held by khungtaskd/44: [ 5531.243410][ T44] #0: ffffffff97d881c0 (rcu_read_lock){....}-{1:3}, at: debug_show_all_locks+0x36/0x260 [ 5531.243832][ T44] 1 lock held by modprobe/31907: [ 5531.244131][ T44] #0: ffffffff9847fb10 (pernet_ops_rwsem){++++}-{4:4}, at: register_pernet_subsys+0x1a/0x40 [ 5531.244737][ T44] [ 5531.244881][ T44] ============================================= [ 5531.244881][ T44] WAIT TIMEOUT stderr Ctrl-C stderr Ctrl-C stderr