[ 4253.775149][ T412] pktgen: Packet Generator for packet performance testing. Version: 2.75
[ 4425.342359][ T44] INFO: task modprobe:412 blocked for more than 122 seconds.
[ 4425.343050][ T44] Not tainted 6.18.0-rc6-virtme #1
[ 4425.343319][ T44] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[ 4425.343725][ T44] task:modprobe state:D stack:25664 pid:412 tgid:412 ppid:1 task_flags:0x400100 flags:0x00080003
[ 4425.344311][ T44] Call Trace:
[ 4425.344505][ T44]
[ 4425.344638][ T44] __schedule+0x880/0x1b10
[ 4425.344888][ T44] ? io_schedule_timeout+0x150/0x150
[ 4425.345138][ T44] ? schedule+0x1ee/0x270
[ 4425.345332][ T44] ? __lock_release+0x5d/0x170
[ 4425.345574][ T44] schedule+0xe2/0x270
[ 4425.345754][ T44] schedule_preempt_disabled+0x18/0x30
[ 4425.346045][ T44] rwsem_down_write_slowpath+0x3c5/0xc60
[ 4425.346277][ T44] ? __lock_acquire+0x449/0x7e0
[ 4425.346510][ T44] ? rwsem_down_read_slowpath+0xd80/0xd80
[ 4425.346779][ T44] ? rcu_is_watching+0x12/0xb0
[ 4425.347006][ T44] ? lock_acquire+0x104/0x160
[ 4425.347285][ T44] ? register_pernet_subsys+0x1a/0x40
[ 4425.347666][ T44] down_write+0x1d6/0x200
[ 4425.347960][ T44] ? rwsem_down_write_slowpath+0xc60/0xc60
[ 4425.348420][ T44] ? find_held_lock+0x2b/0x80
[ 4425.348652][ T44] ? __create_object+0x5e/0xb0
[ 4425.348882][ T44] ? _sub_I_65535_1+0xcf0/0xcf0 [xt_conntrack]
[ 4425.349158][ T44] register_pernet_subsys+0x1a/0x40
[ 4425.349475][ T44] pg_init+0x2c/0x1000 [pktgen]
[ 4425.349766][ T44] ? _sub_I_65535_1+0xcf0/0xcf0 [xt_conntrack]
[ 4425.350050][ T44] do_one_initcall+0x8c/0x1d0
[ 4425.350293][ T44] ? trace_initcall_start+0x130/0x130
[ 4425.350521][ T44] ? rcu_is_watching+0x12/0xb0
[ 4425.351137][ T44] ? trace_kmalloc+0x26/0xd0
[ 4425.351391][ T44] ? __kmalloc_cache_noprof+0x2e4/0x690
[ 4425.351629][ T44] ? debug_objects_oom+0x160/0x160
[ 4425.351867][ T44] ? kasan_unpoison+0x27/0x50
[ 4425.352116][ T44] ? __asan_register_globals+0x5d/0x70
[ 4425.352391][ T44] do_init_module+0x27a/0x860
[ 4425.352639][ T44] ? free_module+0x2f0/0x2f0
[ 4425.352873][ T44] ? kfree+0x21d/0x540
[ 4425.353065][ T44] ? load_module+0x20c5/0x2dd0
[ 4425.353456][ T44] load_module+0x1f40/0x2dd0
[ 4425.353837][ T44] ? layout_and_allocate.constprop.0+0x630/0x630
[ 4425.354316][ T44] ? kernel_read_file+0x3ed/0x550
[ 4425.354717][ T44] ? kernel_read_file+0x3c8/0x550
[ 4425.355099][ T44] ? __ia32_sys_fsconfig+0x150/0x150
[ 4425.355499][ T44] ? init_module_from_file+0xe9/0x150
[ 4425.355876][ T44] init_module_from_file+0xe9/0x150
[ 4425.356268][ T44] ? __do_sys_init_module+0x1e0/0x1e0
[ 4425.356689][ T44] ? idempotent_init_module+0x316/0x610
[ 4425.357082][ T44] ? __lock_release+0x5d/0x170
[ 4425.357499][ T44] ? do_raw_spin_unlock+0x58/0x220
[ 4425.357884][ T44] idempotent_init_module+0x331/0x610
[ 4425.358294][ T44] ? init_module_from_file+0x150/0x150
[ 4425.358712][ T44] ? cap_capable+0x94/0x230
[ 4425.359119][ T44] __x64_sys_finit_module+0xca/0x150
[ 4425.359527][ T44] ? do_syscall_64+0x85/0xfd0
[ 4425.359921][ T44] do_syscall_64+0xc1/0xfd0
[ 4425.360340][ T44] entry_SYSCALL_64_after_hwframe+0x4b/0x53
[ 4425.360831][ T44] RIP: 0033:0x7fc863d8b92d
[ 4425.361163][ T44] RSP: 002b:00007ffe23105c28 EFLAGS: 00000246 ORIG_RAX: 0000000000000139
[ 4425.361704][ T44] RAX: ffffffffffffffda RBX: 000055ab78f71ac0 RCX: 00007fc863d8b92d
[ 4425.362261][ T44] RDX: 0000000000000000 RSI: 000055ab485c4a2a RDI: 0000000000000005
[ 4425.362639][ T44] RBP: 0000000000040000 R08: 0000000000000000 R09: 00007ffe23105d60
[ 4425.362997][ T44] R10: 0000000000000005 R11: 0000000000000246 R12: 000055ab485c4a2a
[ 4425.363372][ T44] R13: 000055ab78f71b60 R14: 000055ab78f71da0 R15: 0000000000000000
[ 4425.363746][ T44]
[ 4425.363948][ T44] INFO: task modprobe:412 blocked on an rw-semaphore likely owned by task kworker/u16:1:69
[ 4425.364488][ T44] task:kworker/u16:1 state:R running task stack:24368 pid:69 tgid:69 ppid:2 task_flags:0x4208060 flags:0x00080000
[ 4425.365081][ T44] Workqueue: netns cleanup_net
[ 4425.365350][ T44] Call Trace:
[ 4425.365528][ T44]
[ 4425.365650][ T44] ? __mutex_trylock_common+0xf9/0x260
[ 4425.365885][ T44] ? __mutex_handoff+0x2b0/0x2b0
[ 4425.366117][ T44] ? rcu_is_watching+0x12/0xb0
[ 4425.366401][ T44] ? rcu_is_watching+0x12/0xb0
[ 4425.366779][ T44] ? trace_contention_end+0xd8/0x140
[ 4425.367160][ T44] ? __mutex_lock+0x19f/0x1190
[ 4425.367560][ T44] ? trace_sched_exit_tp+0xcd/0x130
[ 4425.367956][ T44] ? nf_ct_iterate_cleanup+0x4a/0x340 [nf_conntrack]
[ 4425.368476][ T44] ? ww_mutex_lock+0x160/0x160
[ 4425.368894][ T44] ? nf_ct_iterate_cleanup+0x4a/0x340 [nf_conntrack]
[ 4425.369324][ T44] ? nf_ct_iterate_cleanup+0x4a/0x340 [nf_conntrack]
[ 4425.369620][ T44] ? __nf_conntrack_hash_insert+0x200/0x200 [nf_conntrack]
[ 4425.370012][ T44] ? nf_conntrack_cleanup_net_list+0x19a/0x4b0 [nf_conntrack]
[ 4425.370392][ T44] ? nf_conntrack_cleanup_net_list+0x10e/0x4b0 [nf_conntrack]
[ 4425.370745][ T44] ? nf_conntrack_cleanup_end+0x70/0x70 [nf_conntrack]
[ 4425.371091][ T44] ? nf_conntrack_pernet_exit+0xa5/0x140 [nf_conntrack]
[ 4425.371451][ T44] ? _raw_spin_unlock+0x2d/0x50
[ 4425.371832][ T44] ? ops_undo_list+0x2bf/0x890
[ 4425.372177][ T44] ? rtnl_net_dumpid_one+0x270/0x270
[ 4425.372474][ T44] ? cleanup_net+0x2d6/0x8b0
[ 4425.372758][ T44] ? cleanup_net+0x3b2/0x8b0
[ 4425.372991][ T44] ? net_passive_dec+0x190/0x190
[ 4425.373218][ T44] ? rcu_is_watching+0x12/0xb0
[ 4425.373466][ T44] ? process_one_work+0xe35/0x1650
[ 4425.373699][ T44] ? pwq_dec_nr_in_flight+0x550/0x550
[ 4425.373924][ T44] ? assign_work+0x168/0x240
[ 4425.374148][ T44] ? worker_thread+0x591/0xcf0
[ 4425.374418][ T44] ? rescuer_thread+0xd10/0xd10
[ 4425.374643][ T44] ? kthread+0x37b/0x5f0
[ 4425.374813][ T44] ? kthread_is_per_cpu+0xc0/0xc0
[ 4425.375032][ T44] ? ret_from_fork+0x1b/0x270
[ 4425.375258][ T44] ? __lock_release+0x5d/0x170
[ 4425.375619][ T44] ? rcu_is_watching+0x12/0xb0
[ 4425.375983][ T44] ? kthread_is_per_cpu+0xc0/0xc0
[ 4425.376206][ T44] ? ret_from_fork+0x1db/0x270
[ 4425.376441][ T44] ? kthread_is_per_cpu+0xc0/0xc0
[ 4425.376669][ T44] ? ret_from_fork_asm+0x11/0x20
[ 4425.376904][ T44]
[ 4425.377079][ T44]
[ 4425.377079][ T44] Showing all locks held in the system:
[ 4425.377427][ T44] 1 lock held by khungtaskd/44:
[ 4425.377644][ T44] #0: ffffffffb2788580 (rcu_read_lock){....}-{1:3}, at: debug_show_all_locks+0x36/0x260
[ 4425.378026][ T44] 4 locks held by kworker/u16:1/69:
[ 4425.378273][ T44] 1 lock held by modprobe/412:
[ 4425.378505][ T44] #0: ffffffffb2e800d0 (pernet_ops_rwsem){++++}-{4:4}, at: register_pernet_subsys+0x1a/0x40
[ 4425.378934][ T44]
[ 4425.379044][ T44] =============================================
[ 4425.379044][ T44]
[ 4548.222368][ T44] INFO: task modprobe:412 blocked for more than 245 seconds.
[ 4548.222818][ T44] Not tainted 6.18.0-rc6-virtme #1
[ 4548.223035][ T44] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[ 4548.223432][ T44] task:modprobe state:D stack:25664 pid:412 tgid:412 ppid:1 task_flags:0x400100 flags:0x00080003
[ 4548.223935][ T44] Call Trace:
[ 4548.224105][ T44]
[ 4548.224241][ T44] __schedule+0x880/0x1b10
[ 4548.224499][ T44] ? io_schedule_timeout+0x150/0x150
[ 4548.224735][ T44] ? schedule+0x1ee/0x270
[ 4548.224905][ T44] ? __lock_release+0x5d/0x170
[ 4548.225136][ T44] schedule+0xe2/0x270
[ 4548.225325][ T44] schedule_preempt_disabled+0x18/0x30
[ 4548.225548][ T44] rwsem_down_write_slowpath+0x3c5/0xc60
[ 4548.225773][ T44] ? __lock_acquire+0x449/0x7e0
[ 4548.226004][ T44] ? rwsem_down_read_slowpath+0xd80/0xd80
[ 4548.226242][ T44] ? rcu_is_watching+0x12/0xb0
[ 4548.226480][ T44] ? lock_acquire+0x104/0x160
[ 4548.226704][ T44] ? register_pernet_subsys+0x1a/0x40
[ 4548.226945][ T44] down_write+0x1d6/0x200
[ 4548.227126][ T44] ? rwsem_down_write_slowpath+0xc60/0xc60
[ 4548.227413][ T44] ? find_held_lock+0x2b/0x80
[ 4548.227661][ T44] ? __create_object+0x5e/0xb0
[ 4548.227891][ T44] ? _sub_I_65535_1+0xcf0/0xcf0 [xt_conntrack]
[ 4548.228171][ T44] register_pernet_subsys+0x1a/0x40
[ 4548.228408][ T44] pg_init+0x2c/0x1000 [pktgen]
[ 4548.228644][ T44] ? _sub_I_65535_1+0xcf0/0xcf0 [xt_conntrack]
[ 4548.228920][ T44] do_one_initcall+0x8c/0x1d0
[ 4548.229143][ T44] ? trace_initcall_start+0x130/0x130
[ 4548.229395][ T44] ? rcu_is_watching+0x12/0xb0
[ 4548.229624][ T44] ? trace_kmalloc+0x26/0xd0
[ 4548.229850][ T44] ? __kmalloc_cache_noprof+0x2e4/0x690
[ 4548.230073][ T44] ? debug_objects_oom+0x160/0x160
[ 4548.230326][ T44] ? kasan_unpoison+0x27/0x50
[ 4548.230553][ T44] ? __asan_register_globals+0x5d/0x70
[ 4548.230778][ T44] do_init_module+0x27a/0x860
[ 4548.231028][ T44] ? free_module+0x2f0/0x2f0
[ 4548.231260][ T44] ? kfree+0x21d/0x540
[ 4548.231429][ T44] ? load_module+0x20c5/0x2dd0
[ 4548.231655][ T44] load_module+0x1f40/0x2dd0
[ 4548.231892][ T44] ? layout_and_allocate.constprop.0+0x630/0x630
[ 4548.232169][ T44] ? kernel_read_file+0x3ed/0x550
[ 4548.232410][ T44] ? kernel_read_file+0x3c8/0x550
[ 4548.232638][ T44] ? __ia32_sys_fsconfig+0x150/0x150
[ 4548.232877][ T44] ? init_module_from_file+0xe9/0x150
[ 4548.233097][ T44] init_module_from_file+0xe9/0x150
[ 4548.233332][ T44] ? __do_sys_init_module+0x1e0/0x1e0
[ 4548.233570][ T44] ? idempotent_init_module+0x316/0x610
[ 4548.233796][ T44] ? __lock_release+0x5d/0x170
[ 4548.234021][ T44] ? do_raw_spin_unlock+0x58/0x220
[ 4548.234248][ T44] idempotent_init_module+0x331/0x610
[ 4548.234497][ T44] ? init_module_from_file+0x150/0x150
[ 4548.234732][ T44] ? cap_capable+0x94/0x230
[ 4548.234968][ T44] __x64_sys_finit_module+0xca/0x150
[ 4548.235187][ T44] ? do_syscall_64+0x85/0xfd0
[ 4548.235421][ T44] do_syscall_64+0xc1/0xfd0
[ 4548.235646][ T44] entry_SYSCALL_64_after_hwframe+0x4b/0x53
[ 4548.235919][ T44] RIP: 0033:0x7fc863d8b92d
[ 4548.236243][ T44] RSP: 002b:00007ffe23105c28 EFLAGS: 00000246 ORIG_RAX: 0000000000000139
[ 4548.236605][ T44] RAX: ffffffffffffffda RBX: 000055ab78f71ac0 RCX: 00007fc863d8b92d
[ 4548.236944][ T44] RDX: 0000000000000000 RSI: 000055ab485c4a2a RDI: 0000000000000005
[ 4548.237289][ T44] RBP: 0000000000040000 R08: 0000000000000000 R09: 00007ffe23105d60
[ 4548.237634][ T44] R10: 0000000000000005 R11: 0000000000000246 R12: 000055ab485c4a2a
[ 4548.237970][ T44] R13: 000055ab78f71b60 R14: 000055ab78f71da0 R15: 0000000000000000
[ 4548.238329][ T44]
[ 4548.238531][ T44] INFO: task modprobe:412 blocked on an rw-semaphore likely owned by task kworker/u16:1:69
[ 4548.239004][ T44] task:kworker/u16:1 state:R running task stack:24368 pid:69 tgid:69 ppid:2 task_flags:0x4208060 flags:0x00080000
[ 4548.239579][ T44] Workqueue: netns cleanup_net
[ 4548.239814][ T44] Call Trace:
[ 4548.239983][ T44]
[ 4548.240094][ T44] ? __mutex_lock+0x19f/0x1190
[ 4548.240342][ T44] ? trace_sched_exit_tp+0xcd/0x130
[ 4548.240572][ T44] ? nf_ct_iterate_cleanup+0x4a/0x340 [nf_conntrack]
[ 4548.240866][ T44] ? ww_mutex_lock+0x160/0x160
[ 4548.241105][ T44] ? nf_ct_iterate_cleanup+0x4a/0x340 [nf_conntrack]
[ 4548.241399][ T44] ? nf_ct_iterate_cleanup+0x4a/0x340 [nf_conntrack]
[ 4548.241687][ T44] ? __nf_conntrack_hash_insert+0x200/0x200 [nf_conntrack]
[ 4548.242031][ T44] ? nf_conntrack_cleanup_net_list+0x19a/0x4b0 [nf_conntrack]
[ 4548.242390][ T44] ? nf_conntrack_cleanup_net_list+0x10e/0x4b0 [nf_conntrack]
[ 4548.242736][ T44] ? nf_conntrack_cleanup_end+0x70/0x70 [nf_conntrack]
[ 4548.243023][ T44] ? nf_conntrack_pernet_exit+0xa5/0x140 [nf_conntrack]
[ 4548.243317][ T44] ? _raw_spin_unlock+0x2d/0x50
[ 4548.243548][ T44] ? ops_undo_list+0x2bf/0x890
[ 4548.243784][ T44] ? rtnl_net_dumpid_one+0x270/0x270
[ 4548.244015][ T44] ? cleanup_net+0x2d6/0x8b0
[ 4548.244249][ T44] ? cleanup_net+0x3b2/0x8b0
[ 4548.244492][ T44] ? net_passive_dec+0x190/0x190
[ 4548.244723][ T44] ? rcu_is_watching+0x12/0xb0
[ 4548.244946][ T44] ? process_one_work+0xe35/0x1650
[ 4548.245180][ T44] ? pwq_dec_nr_in_flight+0x550/0x550
[ 4548.245416][ T44] ? assign_work+0x168/0x240
[ 4548.245647][ T44] ? worker_thread+0x591/0xcf0
[ 4548.245889][ T44] ? rescuer_thread+0xd10/0xd10
[ 4548.246112][ T44] ? kthread+0x37b/0x5f0
[ 4548.246291][ T44] ? kthread_is_per_cpu+0xc0/0xc0
[ 4548.246512][ T44] ? ret_from_fork+0x1b/0x270
[ 4548.246734][ T44] ? __lock_release+0x5d/0x170
[ 4548.246956][ T44] ? rcu_is_watching+0x12/0xb0
[ 4548.247178][ T44] ? kthread_is_per_cpu+0xc0/0xc0
[ 4548.247410][ T44] ? ret_from_fork+0x1db/0x270
[ 4548.247647][ T44] ? kthread_is_per_cpu+0xc0/0xc0
[ 4548.247867][ T44] ? ret_from_fork_asm+0x11/0x20
[ 4548.248110][ T44]
[ 4548.248305][ T44]
[ 4548.248305][ T44] Showing all locks held in the system:
[ 4548.248632][ T44] 1 lock held by khungtaskd/44:
[ 4548.248852][ T44] #0: ffffffffb2788580 (rcu_read_lock){....}-{1:3}, at: debug_show_all_locks+0x36/0x260
[ 4548.249238][ T44] 4 locks held by kworker/u16:1/69:
[ 4548.249485][ T44] 1 lock held by modprobe/412:
[ 4548.249698][ T44] #0: ffffffffb2e800d0 (pernet_ops_rwsem){++++}-{4:4}, at: register_pernet_subsys+0x1a/0x40
[ 4548.250131][ T44]
[ 4548.250246][ T44] =============================================
[ 4548.250246][ T44]
[ 4671.102341][ T44] INFO: task modprobe:412 blocked for more than 368 seconds.
[ 4671.102826][ T44] Not tainted 6.18.0-rc6-virtme #1
[ 4671.103049][ T44] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[ 4671.103452][ T44] task:modprobe state:D stack:25664 pid:412 tgid:412 ppid:1 task_flags:0x400100 flags:0x00080003
[ 4671.103985][ T44] Call Trace:
[ 4671.104162][ T44]
[ 4671.104298][ T44] __schedule+0x880/0x1b10
[ 4671.104539][ T44] ? io_schedule_timeout+0x150/0x150
[ 4671.104775][ T44] ? schedule+0x1ee/0x270
[ 4671.104943][ T44] ? __lock_release+0x5d/0x170
[ 4671.105173][ T44] schedule+0xe2/0x270
[ 4671.105354][ T44] schedule_preempt_disabled+0x18/0x30
[ 4671.105581][ T44] rwsem_down_write_slowpath+0x3c5/0xc60
[ 4671.105808][ T44] ? __lock_acquire+0x449/0x7e0
[ 4671.106031][ T44] ? rwsem_down_read_slowpath+0xd80/0xd80
[ 4671.106273][ T44] ? rcu_is_watching+0x12/0xb0
[ 4671.106497][ T44] ? lock_acquire+0x104/0x160
[ 4671.106722][ T44] ? register_pernet_subsys+0x1a/0x40
[ 4671.106957][ T44] down_write+0x1d6/0x200
[ 4671.107128][ T44] ? rwsem_down_write_slowpath+0xc60/0xc60
[ 4671.107422][ T44] ? find_held_lock+0x2b/0x80
[ 4671.107645][ T44] ? __create_object+0x5e/0xb0
[ 4671.107873][ T44] ? _sub_I_65535_1+0xcf0/0xcf0 [xt_conntrack]
[ 4671.108149][ T44] register_pernet_subsys+0x1a/0x40
[ 4671.108381][ T44] pg_init+0x2c/0x1000 [pktgen]
[ 4671.108609][ T44] ? _sub_I_65535_1+0xcf0/0xcf0 [xt_conntrack]
[ 4671.108883][ T44] do_one_initcall+0x8c/0x1d0
[ 4671.109116][ T44] ? trace_initcall_start+0x130/0x130
[ 4671.109347][ T44] ? rcu_is_watching+0x12/0xb0
[ 4671.109578][ T44] ? trace_kmalloc+0x26/0xd0
[ 4671.109799][ T44] ? __kmalloc_cache_noprof+0x2e4/0x690
[ 4671.110020][ T44] ? debug_objects_oom+0x160/0x160
[ 4671.110246][ T44] ? kasan_unpoison+0x27/0x50
[ 4671.110480][ T44] ? __asan_register_globals+0x5d/0x70
[ 4671.110711][ T44] do_init_module+0x27a/0x860
[ 4671.110942][ T44] ? free_module+0x2f0/0x2f0
[ 4671.111180][ T44] ? kfree+0x21d/0x540
[ 4671.111369][ T44] ? load_module+0x20c5/0x2dd0
[ 4671.111603][ T44] load_module+0x1f40/0x2dd0
[ 4671.111837][ T44] ? layout_and_allocate.constprop.0+0x630/0x630
[ 4671.112111][ T44] ? kernel_read_file+0x3ed/0x550
[ 4671.112351][ T44] ? kernel_read_file+0x3c8/0x550
[ 4671.112581][ T44] ? __ia32_sys_fsconfig+0x150/0x150
[ 4671.112810][ T44] ? init_module_from_file+0xe9/0x150
[ 4671.113030][ T44] init_module_from_file+0xe9/0x150
[ 4671.113259][ T44] ? __do_sys_init_module+0x1e0/0x1e0
[ 4671.113499][ T44] ? idempotent_init_module+0x316/0x610
[ 4671.113718][ T44] ? __lock_release+0x5d/0x170
[ 4671.113948][ T44] ? do_raw_spin_unlock+0x58/0x220
[ 4671.114170][ T44] idempotent_init_module+0x331/0x610
[ 4671.114407][ T44] ? init_module_from_file+0x150/0x150
[ 4671.114643][ T44] ? cap_capable+0x94/0x230
[ 4671.114872][ T44] __x64_sys_finit_module+0xca/0x150
[ 4671.115094][ T44] ? do_syscall_64+0x85/0xfd0
[ 4671.115326][ T44] do_syscall_64+0xc1/0xfd0
[ 4671.115558][ T44] entry_SYSCALL_64_after_hwframe+0x4b/0x53
[ 4671.115833][ T44] RIP: 0033:0x7fc863d8b92d
[ 4671.116069][ T44] RSP: 002b:00007ffe23105c28 EFLAGS: 00000246 ORIG_RAX: 0000000000000139
[ 4671.116418][ T44] RAX: ffffffffffffffda RBX: 000055ab78f71ac0 RCX: 00007fc863d8b92d
[ 4671.116750][ T44] RDX: 0000000000000000 RSI: 000055ab485c4a2a RDI: 0000000000000005
[ 4671.117090][ T44] RBP: 0000000000040000 R08: 0000000000000000 R09: 00007ffe23105d60
[ 4671.117431][ T44] R10: 0000000000000005 R11: 0000000000000246 R12: 000055ab485c4a2a
[ 4671.117763][ T44] R13: 000055ab78f71b60 R14: 000055ab78f71da0 R15: 0000000000000000
[ 4671.118114][ T44]
[ 4671.118305][ T44] INFO: task modprobe:412 blocked on an rw-semaphore likely owned by task kworker/u16:1:69
[ 4671.118782][ T44] task:kworker/u16:1 state:R running task stack:24368 pid:69 tgid:69 ppid:2 task_flags:0x4208060 flags:0x00080000
[ 4671.119329][ T44] Workqueue: netns cleanup_net
[ 4671.119574][ T44] Call Trace:
[ 4671.119740][ T44]
[ 4671.119854][ T44] ? __mutex_lock+0x19f/0x1190
[ 4671.120080][ T44] ? trace_sched_exit_tp+0xcd/0x130
[ 4671.120323][ T44] ? nf_ct_iterate_cleanup+0x4a/0x340 [nf_conntrack]
[ 4671.120610][ T44] ? find_held_lock+0x2b/0x80
[ 4671.120832][ T44] ? ww_mutex_lock+0x160/0x160
[ 4671.121073][ T44] ? nf_ct_iterate_cleanup+0x4a/0x340 [nf_conntrack]
[ 4671.121366][ T44] ? nf_ct_iterate_cleanup+0x4a/0x340 [nf_conntrack]
[ 4671.121654][ T44] ? __nf_conntrack_hash_insert+0x200/0x200 [nf_conntrack]
[ 4671.122003][ T44] ? nf_conntrack_cleanup_net_list+0x19a/0x4b0 [nf_conntrack]
[ 4671.122351][ T44] ? nf_conntrack_cleanup_net_list+0x10e/0x4b0 [nf_conntrack]
[ 4671.122690][ T44] ? nf_conntrack_cleanup_end+0x70/0x70 [nf_conntrack]
[ 4671.122980][ T44] ? nf_conntrack_pernet_exit+0xa5/0x140 [nf_conntrack]
[ 4671.123271][ T44] ? _raw_spin_unlock+0x2d/0x50
[ 4671.123506][ T44] ? ops_undo_list+0x2bf/0x890
[ 4671.123733][ T44] ? rtnl_net_dumpid_one+0x270/0x270
[ 4671.123953][ T44] ? cleanup_net+0x2d6/0x8b0
[ 4671.124185][ T44] ? cleanup_net+0x3b2/0x8b0
[ 4671.124416][ T44] ? net_passive_dec+0x190/0x190
[ 4671.124641][ T44] ? rcu_is_watching+0x12/0xb0
[ 4671.124866][ T44] ? process_one_work+0xe35/0x1650
[ 4671.125099][ T44] ? pwq_dec_nr_in_flight+0x550/0x550
[ 4671.125340][ T44] ? assign_work+0x168/0x240
[ 4671.125567][ T44] ? worker_thread+0x591/0xcf0
[ 4671.125799][ T44] ? rescuer_thread+0xd10/0xd10
[ 4671.126025][ T44] ? kthread+0x37b/0x5f0
[ 4671.126198][ T44] ? kthread_is_per_cpu+0xc0/0xc0
[ 4671.126426][ T44] ? ret_from_fork+0x1b/0x270
[ 4671.126648][ T44] ? __lock_release+0x5d/0x170
[ 4671.126871][ T44] ? rcu_is_watching+0x12/0xb0
[ 4671.127097][ T44] ? kthread_is_per_cpu+0xc0/0xc0
[ 4671.127328][ T44] ? ret_from_fork+0x1db/0x270
[ 4671.127547][ T44] ? kthread_is_per_cpu+0xc0/0xc0
[ 4671.127768][ T44] ? ret_from_fork_asm+0x11/0x20
[ 4671.128004][ T44]
[ 4671.128183][ T44]
[ 4671.128183][ T44] Showing all locks held in the system:
[ 4671.128513][ T44] 1 lock held by khungtaskd/44:
[ 4671.128727][ T44] #0: ffffffffb2788580 (rcu_read_lock){....}-{1:3}, at: debug_show_all_locks+0x36/0x260
[ 4671.129116][ T44] 4 locks held by kworker/u16:1/69:
[ 4671.129348][ T44] 1 lock held by modprobe/412:
[ 4671.129567][ T44] #0: ffffffffb2e800d0 (pernet_ops_rwsem){++++}-{4:4}, at: register_pernet_subsys+0x1a/0x40
[ 4671.130002][ T44]
[ 4671.130109][ T44] =============================================
[ 4671.130109][ T44]
[ 4793.982342][ T44] INFO: task modprobe:412 blocked for more than 491 seconds.
[ 4793.982836][ T44] Not tainted 6.18.0-rc6-virtme #1
[ 4793.983049][ T44] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[ 4793.983461][ T44] task:modprobe state:D stack:25664 pid:412 tgid:412 ppid:1 task_flags:0x400100 flags:0x00080003
[ 4793.983951][ T44] Call Trace:
[ 4793.984120][ T44]
[ 4793.984241][ T44] __schedule+0x880/0x1b10
[ 4793.984496][ T44] ? io_schedule_timeout+0x150/0x150
[ 4793.984732][ T44] ? schedule+0x1ee/0x270
[ 4793.984900][ T44] ? __lock_release+0x5d/0x170
[ 4793.985142][ T44] schedule+0xe2/0x270
[ 4793.985349][ T44] schedule_preempt_disabled+0x18/0x30
[ 4793.985573][ T44] rwsem_down_write_slowpath+0x3c5/0xc60
[ 4793.985799][ T44] ? __lock_acquire+0x449/0x7e0
[ 4793.986021][ T44] ? rwsem_down_read_slowpath+0xd80/0xd80
[ 4793.986269][ T44] ? rcu_is_watching+0x12/0xb0
[ 4793.986495][ T44] ? lock_acquire+0x104/0x160
[ 4793.986717][ T44] ? register_pernet_subsys+0x1a/0x40
[ 4793.986956][ T44] down_write+0x1d6/0x200
[ 4793.987127][ T44] ? rwsem_down_write_slowpath+0xc60/0xc60
[ 4793.987418][ T44] ? find_held_lock+0x2b/0x80
[ 4793.987643][ T44] ? __create_object+0x5e/0xb0
[ 4793.987871][ T44] ? _sub_I_65535_1+0xcf0/0xcf0 [xt_conntrack]
[ 4793.988144][ T44] register_pernet_subsys+0x1a/0x40
[ 4793.988378][ T44] pg_init+0x2c/0x1000 [pktgen]
[ 4793.988613][ T44] ? _sub_I_65535_1+0xcf0/0xcf0 [xt_conntrack]
[ 4793.988900][ T44] do_one_initcall+0x8c/0x1d0
[ 4793.989127][ T44] ? trace_initcall_start+0x130/0x130
[ 4793.989379][ T44] ? rcu_is_watching+0x12/0xb0
[ 4793.989603][ T44] ? trace_kmalloc+0x26/0xd0
[ 4793.989824][ T44] ? __kmalloc_cache_noprof+0x2e4/0x690
[ 4793.990045][ T44] ? debug_objects_oom+0x160/0x160
[ 4793.990288][ T44] ? kasan_unpoison+0x27/0x50
[ 4793.990509][ T44] ? __asan_register_globals+0x5d/0x70
[ 4793.990736][ T44] do_init_module+0x27a/0x860
[ 4793.990964][ T44] ? free_module+0x2f0/0x2f0
[ 4793.991190][ T44] ? kfree+0x21d/0x540
[ 4793.991367][ T44] ? load_module+0x20c5/0x2dd0
[ 4793.991599][ T44] load_module+0x1f40/0x2dd0
[ 4793.991832][ T44] ? layout_and_allocate.constprop.0+0x630/0x630
[ 4793.992113][ T44] ? kernel_read_file+0x3ed/0x550
[ 4793.992349][ T44] ? kernel_read_file+0x3c8/0x550
[ 4793.992573][ T44] ? __ia32_sys_fsconfig+0x150/0x150
[ 4793.992800][ T44] ? init_module_from_file+0xe9/0x150
[ 4793.993021][ T44] init_module_from_file+0xe9/0x150
[ 4793.993243][ T44] ? __do_sys_init_module+0x1e0/0x1e0
[ 4793.993509][ T44] ? idempotent_init_module+0x316/0x610
[ 4793.993733][ T44] ? __lock_release+0x5d/0x170
[ 4793.993958][ T44] ? do_raw_spin_unlock+0x58/0x220
[ 4793.994180][ T44] idempotent_init_module+0x331/0x610
[ 4793.994414][ T44] ? init_module_from_file+0x150/0x150
[ 4793.994649][ T44] ? cap_capable+0x94/0x230
[ 4793.994878][ T44] __x64_sys_finit_module+0xca/0x150
[ 4793.995104][ T44] ? do_syscall_64+0x85/0xfd0
[ 4793.995336][ T44] do_syscall_64+0xc1/0xfd0
[ 4793.995562][ T44] entry_SYSCALL_64_after_hwframe+0x4b/0x53
[ 4793.995836][ T44] RIP: 0033:0x7fc863d8b92d
[ 4793.996068][ T44] RSP: 002b:00007ffe23105c28 EFLAGS: 00000246 ORIG_RAX: 0000000000000139
[ 4793.996413][ T44] RAX: ffffffffffffffda RBX: 000055ab78f71ac0 RCX: 00007fc863d8b92d
[ 4793.996752][ T44] RDX: 0000000000000000 RSI: 000055ab485c4a2a RDI: 0000000000000005
[ 4793.997085][ T44] RBP: 0000000000040000 R08: 0000000000000000 R09: 00007ffe23105d60
[ 4793.997440][ T44] R10: 0000000000000005 R11: 0000000000000246 R12: 000055ab485c4a2a
[ 4793.997775][ T44] R13: 000055ab78f71b60 R14: 000055ab78f71da0 R15: 0000000000000000
[ 4793.998131][ T44]
[ 4793.998320][ T44] INFO: task modprobe:412 blocked on an rw-semaphore likely owned by task kworker/u16:1:69
[ 4793.998801][ T44] task:kworker/u16:1 state:R running task stack:24368 pid:69 tgid:69 ppid:2 task_flags:0x4208060 flags:0x00080000
[ 4793.999352][ T44] Workqueue: netns cleanup_net
[ 4793.999582][ T44] Call Trace:
[ 4793.999757][ T44]
[ 4793.999880][ T44] ? __mutex_handoff+0x2b0/0x2b0
[ 4794.000106][ T44] ? rcu_is_watching+0x12/0xb0
[ 4794.000337][ T44] ? rcu_is_watching+0x12/0xb0
[ 4794.000558][ T44] ? trace_contention_end+0xd8/0x140
[ 4794.000781][ T44] ? __mutex_lock+0x19f/0x1190
[ 4794.001004][ T44] ? trace_sched_exit_tp+0xcd/0x130
[ 4794.001229][ T44] ? nf_ct_iterate_cleanup+0x4a/0x340 [nf_conntrack]
[ 4794.001546][ T44] ? ww_mutex_lock+0x160/0x160
[ 4794.001772][ T44] ? io_schedule_timeout+0x150/0x150
[ 4794.002009][ T44] ? nf_ct_iterate_cleanup+0x4a/0x340 [nf_conntrack]
[ 4794.002303][ T44] ? nf_ct_iterate_cleanup+0x4a/0x340 [nf_conntrack]
[ 4794.002586][ T44] ? __nf_conntrack_hash_insert+0x200/0x200 [nf_conntrack]
[ 4794.002927][ T44] ? nf_conntrack_cleanup_net_list+0x19a/0x4b0 [nf_conntrack]
[ 4794.003277][ T44] ? nf_conntrack_cleanup_net_list+0x10e/0x4b0 [nf_conntrack]
[ 4794.003630][ T44] ? nf_conntrack_cleanup_end+0x70/0x70 [nf_conntrack]
[ 4794.003919][ T44] ? nf_conntrack_pernet_exit+0xa5/0x140 [nf_conntrack]
[ 4794.004205][ T44] ? _raw_spin_unlock+0x2d/0x50
[ 4794.004440][ T44] ? ops_undo_list+0x2bf/0x890
[ 4794.004681][ T44] ? rtnl_net_dumpid_one+0x270/0x270
[ 4794.004901][ T44] ? cleanup_net+0x2d6/0x8b0
[ 4794.005129][ T44] ? cleanup_net+0x3b2/0x8b0
[ 4794.005380][ T44] ? net_passive_dec+0x190/0x190
[ 4794.005606][ T44] ? rcu_is_watching+0x12/0xb0
[ 4794.005842][ T44] ? process_one_work+0xe35/0x1650
[ 4794.006075][ T44] ? pwq_dec_nr_in_flight+0x550/0x550
[ 4794.006313][ T44] ? assign_work+0x168/0x240
[ 4794.006547][ T44] ? worker_thread+0x591/0xcf0
[ 4794.006780][ T44] ? rescuer_thread+0xd10/0xd10
[ 4794.007007][ T44] ? kthread+0x37b/0x5f0
[ 4794.007176][ T44] ? kthread_is_per_cpu+0xc0/0xc0
[ 4794.007405][ T44] ? ret_from_fork+0x1b/0x270
[ 4794.007626][ T44] ? __lock_release+0x5d/0x170
[ 4794.007847][ T44] ? rcu_is_watching+0x12/0xb0
[ 4794.008069][ T44] ? kthread_is_per_cpu+0xc0/0xc0
[ 4794.008309][ T44] ? ret_from_fork+0x1db/0x270
[ 4794.008533][ T44] ? kthread_is_per_cpu+0xc0/0xc0
[ 4794.008760][ T44] ? ret_from_fork_asm+0x11/0x20
[ 4794.009008][ T44]
[ 4794.009207][ T44]
[ 4794.009207][ T44] Showing all locks held in the system:
[ 4794.009914][ T44] 1 lock held by khungtaskd/44:
[ 4794.010134][ T44] #0: ffffffffb2788580 (rcu_read_lock){....}-{1:3}, at: debug_show_all_locks+0x36/0x260
[ 4794.010540][ T44] 4 locks held by kworker/u16:1/69:
[ 4794.010764][ T44] 1 lock held by modprobe/412:
[ 4794.010977][ T44] #0: ffffffffb2e800d0 (pernet_ops_rwsem){++++}-{4:4}, at: register_pernet_subsys+0x1a/0x40
[ 4794.011418][ T44]
[ 4794.011525][ T44] =============================================
[ 4794.011525][ T44]
[ 4916.862333][ T44] INFO: task modprobe:412 blocked for more than 614 seconds.
[ 4916.862811][ T44] Not tainted 6.18.0-rc6-virtme #1
[ 4916.863027][ T44] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[ 4916.863443][ T44] task:modprobe state:D stack:25664 pid:412 tgid:412 ppid:1 task_flags:0x400100 flags:0x00080003
[ 4916.863954][ T44] Call Trace:
[ 4916.864126][ T44]
[ 4916.864270][ T44] __schedule+0x880/0x1b10
[ 4916.864516][ T44] ? io_schedule_timeout+0x150/0x150
[ 4916.864753][ T44] ? schedule+0x1ee/0x270
[ 4916.864920][ T44] ? __lock_release+0x5d/0x170
[ 4916.865152][ T44] schedule+0xe2/0x270
[ 4916.865332][ T44] schedule_preempt_disabled+0x18/0x30
[ 4916.865556][ T44] rwsem_down_write_slowpath+0x3c5/0xc60
[ 4916.865780][ T44] ? __lock_acquire+0x449/0x7e0
[ 4916.866021][ T44] ? rwsem_down_read_slowpath+0xd80/0xd80
[ 4916.866264][ T44] ? rcu_is_watching+0x12/0xb0
[ 4916.866491][ T44] ? lock_acquire+0x104/0x160
[ 4916.866713][ T44] ? register_pernet_subsys+0x1a/0x40
[ 4916.866945][ T44] down_write+0x1d6/0x200
[ 4916.867115][ T44] ? rwsem_down_write_slowpath+0xc60/0xc60
[ 4916.867421][ T44] ? find_held_lock+0x2b/0x80
[ 4916.867656][ T44] ? __create_object+0x5e/0xb0
[ 4916.867882][ T44] ? _sub_I_65535_1+0xcf0/0xcf0 [xt_conntrack]
[ 4916.868168][ T44] register_pernet_subsys+0x1a/0x40
[ 4916.868404][ T44] pg_init+0x2c/0x1000 [pktgen]
[ 4916.868632][ T44] ? _sub_I_65535_1+0xcf0/0xcf0 [xt_conntrack]
[ 4916.868906][ T44] do_one_initcall+0x8c/0x1d0
[ 4916.869135][ T44] ? trace_initcall_start+0x130/0x130
[ 4916.869373][ T44] ? rcu_is_watching+0x12/0xb0
[ 4916.869596][ T44] ? trace_kmalloc+0x26/0xd0
[ 4916.869818][ T44] ? __kmalloc_cache_noprof+0x2e4/0x690
[ 4916.870041][ T44] ? debug_objects_oom+0x160/0x160
[ 4916.870275][ T44] ? kasan_unpoison+0x27/0x50
[ 4916.870499][ T44] ? __asan_register_globals+0x5d/0x70
[ 4916.870724][ T44] do_init_module+0x27a/0x860
[ 4916.870953][ T44] ? free_module+0x2f0/0x2f0
[ 4916.871181][ T44] ? kfree+0x21d/0x540
[ 4916.871375][ T44] ? load_module+0x20c5/0x2dd0
[ 4916.871606][ T44] load_module+0x1f40/0x2dd0
[ 4916.871838][ T44] ? layout_and_allocate.constprop.0+0x630/0x630
[ 4916.872115][ T44] ? kernel_read_file+0x3ed/0x550
[ 4916.872351][ T44] ? kernel_read_file+0x3c8/0x550
[ 4916.872590][ T44] ? __ia32_sys_fsconfig+0x150/0x150
[ 4916.872826][ T44] ? init_module_from_file+0xe9/0x150
[ 4916.873048][ T44] init_module_from_file+0xe9/0x150
[ 4916.873279][ T44] ? __do_sys_init_module+0x1e0/0x1e0
[ 4916.873519][ T44] ? idempotent_init_module+0x316/0x610
[ 4916.873872][ T44] ? __lock_release+0x5d/0x170
[ 4916.874268][ T44] ? do_raw_spin_unlock+0x58/0x220
[ 4916.874564][ T44] idempotent_init_module+0x331/0x610
[ 4916.874800][ T44] ? init_module_from_file+0x150/0x150
[ 4916.875034][ T44] ? cap_capable+0x94/0x230
[ 4916.875302][ T44] __x64_sys_finit_module+0xca/0x150
[ 4916.875526][ T44] ? do_syscall_64+0x85/0xfd0
[ 4916.875751][ T44] do_syscall_64+0xc1/0xfd0
[ 4916.875976][ T44] entry_SYSCALL_64_after_hwframe+0x4b/0x53
[ 4916.876264][ T44] RIP: 0033:0x7fc863d8b92d
[ 4916.876503][ T44] RSP: 002b:00007ffe23105c28 EFLAGS: 00000246 ORIG_RAX: 0000000000000139
[ 4916.876842][ T44] RAX: ffffffffffffffda RBX: 000055ab78f71ac0 RCX: 00007fc863d8b92d
[ 4916.877175][ T44] RDX: 0000000000000000 RSI: 000055ab485c4a2a RDI: 0000000000000005
[ 4916.877517][ T44] RBP: 0000000000040000 R08: 0000000000000000 R09: 00007ffe23105d60
[ 4916.877859][ T44] R10: 0000000000000005 R11: 0000000000000246 R12: 000055ab485c4a2a
[ 4916.878202][ T44] R13: 000055ab78f71b60 R14: 000055ab78f71da0 R15: 0000000000000000
[ 4916.878563][ T44]
[ 4916.878747][ T44] INFO: task modprobe:412 blocked on an rw-semaphore likely owned by task kworker/u16:1:69
[ 4916.879219][ T44] task:kworker/u16:1 state:R running task stack:24368 pid:69 tgid:69 ppid:2 task_flags:0x4208060 flags:0x00080000
[ 4916.879797][ T44] Workqueue: netns cleanup_net
[ 4916.880025][ T44] Call Trace:
[ 4916.880194][ T44]
[ 4916.880324][ T44] ? __mutex_trylock_common+0xf9/0x260
[ 4916.880552][ T44] ? __mutex_handoff+0x2b0/0x2b0
[ 4916.880774][ T44] ? rcu_is_watching+0x12/0xb0
[ 4916.881002][ T44] ? rcu_is_watching+0x12/0xb0
[ 4916.881235][ T44] ? __mutex_lock+0x19f/0x1190
[ 4916.881468][ T44] ? trace_sched_exit_tp+0xcd/0x130
[ 4916.881696][ T44] ? nf_ct_iterate_cleanup+0x4a/0x340 [nf_conntrack]
[ 4916.881986][ T44] ? ww_mutex_lock+0x160/0x160
[ 4916.882228][ T44] ? nf_ct_iterate_cleanup+0x4a/0x340 [nf_conntrack]
[ 4916.882520][ T44] ? nf_ct_iterate_cleanup+0x4a/0x340 [nf_conntrack]
[ 4916.882809][ T44] ? __nf_conntrack_hash_insert+0x200/0x200 [nf_conntrack]
[ 4916.883151][ T44] ? nf_conntrack_cleanup_net_list+0x19a/0x4b0 [nf_conntrack]
[ 4916.883518][ T44] ? nf_conntrack_cleanup_net_list+0x10e/0x4b0 [nf_conntrack]
[ 4916.883862][ T44] ? nf_conntrack_cleanup_end+0x70/0x70 [nf_conntrack]
[ 4916.884150][ T44] ? nf_conntrack_pernet_exit+0xa5/0x140 [nf_conntrack]
[ 4916.884450][ T44] ? _raw_spin_unlock+0x2d/0x50
[ 4916.884678][ T44] ? ops_undo_list+0x2bf/0x890
[ 4916.884907][ T44] ? rtnl_net_dumpid_one+0x270/0x270
[ 4916.885127][ T44] ? cleanup_net+0x2d6/0x8b0
[ 4916.885367][ T44] ? cleanup_net+0x3b2/0x8b0
[ 4916.885596][ T44] ? net_passive_dec+0x190/0x190
[ 4916.885820][ T44] ? rcu_is_watching+0x12/0xb0
[ 4916.886044][ T44] ? process_one_work+0xe35/0x1650
[ 4916.886288][ T44] ? pwq_dec_nr_in_flight+0x550/0x550
[ 4916.886519][ T44] ? assign_work+0x168/0x240
[ 4916.886748][ T44] ? worker_thread+0x591/0xcf0
[ 4916.886978][ T44] ? rescuer_thread+0xd10/0xd10
[ 4916.887201][ T44] ? kthread+0x37b/0x5f0
[ 4916.887394][ T44] ? kthread_is_per_cpu+0xc0/0xc0
[ 4916.887617][ T44] ? ret_from_fork+0x1b/0x270
[ 4916.887844][ T44] ? __lock_release+0x5d/0x170
[ 4916.888073][ T44] ? rcu_is_watching+0x12/0xb0
[ 4916.888306][ T44] ? kthread_is_per_cpu+0xc0/0xc0
[ 4916.888530][ T44] ? ret_from_fork+0x1db/0x270
[ 4916.888751][ T44] ? kthread_is_per_cpu+0xc0/0xc0
[ 4916.888970][ T44] ? ret_from_fork_asm+0x11/0x20
[ 4916.889208][ T44]
[ 4916.889414][ T44]
[ 4916.889414][ T44] Showing all locks held in the system:
[ 4916.889743][ T44] 1 lock held by khungtaskd/44:
[ 4916.889958][ T44] #0: ffffffffb2788580 (rcu_read_lock){....}-{1:3}, at: debug_show_all_locks+0x36/0x260
[ 4916.890500][ T44] 4 locks held by kworker/u16:1/69:
[ 4916.890725][ T44] 1 lock held by modprobe/412:
[ 4916.890940][ T44] #0: ffffffffb2e800d0 (pernet_ops_rwsem){++++}-{4:4}, at: register_pernet_subsys+0x1a/0x40
[ 4916.891392][ T44]
[ 4916.891507][ T44] =============================================
[ 4916.891507][ T44]
[ 5039.742473][ T44] INFO: task modprobe:412 blocked for more than 737 seconds.
[ 5039.743139][ T44] Not tainted 6.18.0-rc6-virtme #1
[ 5039.743358][ T44] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[ 5039.743701][ T44] task:modprobe state:D stack:25664 pid:412 tgid:412 ppid:1 task_flags:0x400100 flags:0x00080003
[ 5039.744165][ T44] Call Trace:
[ 5039.744333][ T44]
[ 5039.744449][ T44] __schedule+0x880/0x1b10
[ 5039.744678][ T44] ? io_schedule_timeout+0x150/0x150
[ 5039.744896][ T44] ? schedule+0x1ee/0x270
[ 5039.745050][ T44] ? __lock_release+0x5d/0x170
[ 5039.745276][ T44] schedule+0xe2/0x270
[ 5039.745434][ T44] schedule_preempt_disabled+0x18/0x30
[ 5039.745640][ T44] rwsem_down_write_slowpath+0x3c5/0xc60
[ 5039.745849][ T44] ? __lock_acquire+0x449/0x7e0
[ 5039.746061][ T44] ? rwsem_down_read_slowpath+0xd80/0xd80
[ 5039.746284][ T44] ? rcu_is_watching+0x12/0xb0
[ 5039.746491][ T44] ? lock_acquire+0x104/0x160
[ 5039.746695][ T44] ? register_pernet_subsys+0x1a/0x40
[ 5039.746911][ T44] down_write+0x1d6/0x200
[ 5039.747068][ T44] ? rwsem_down_write_slowpath+0xc60/0xc60
[ 5039.747337][ T44] ? find_held_lock+0x2b/0x80
[ 5039.747551][ T44] ? __create_object+0x5e/0xb0
[ 5039.747766][ T44] ? _sub_I_65535_1+0xcf0/0xcf0 [xt_conntrack]
[ 5039.748023][ T44] register_pernet_subsys+0x1a/0x40
[ 5039.748232][ T44] pg_init+0x2c/0x1000 [pktgen]
[ 5039.748451][ T44] ? _sub_I_65535_1+0xcf0/0xcf0 [xt_conntrack]
[ 5039.748708][ T44] do_one_initcall+0x8c/0x1d0
[ 5039.748917][ T44] ? trace_initcall_start+0x130/0x130
[ 5039.749126][ T44] ? rcu_is_watching+0x12/0xb0
[ 5039.749344][ T44] ? trace_kmalloc+0x26/0xd0
[ 5039.749550][ T44] ? __kmalloc_cache_noprof+0x2e4/0x690
[ 5039.749781][ T44] ? debug_objects_oom+0x160/0x160
[ 5039.750001][ T44] ? kasan_unpoison+0x27/0x50
[ 5039.750219][ T44] ? __asan_register_globals+0x5d/0x70
[ 5039.750448][ T44] do_init_module+0x27a/0x860
[ 5039.750677][ T44] ? free_module+0x2f0/0x2f0
[ 5039.750889][ T44] ? kfree+0x21d/0x540
[ 5039.751046][ T44] ? load_module+0x20c5/0x2dd0
[ 5039.751265][ T44] load_module+0x1f40/0x2dd0
[ 5039.751485][ T44] ? layout_and_allocate.constprop.0+0x630/0x630
[ 5039.751742][ T44] ? kernel_read_file+0x3ed/0x550
[ 5039.751950][ T44] ? kernel_read_file+0x3c8/0x550
[ 5039.752160][ T44] ? __ia32_sys_fsconfig+0x150/0x150
[ 5039.752380][ T44] ? init_module_from_file+0xe9/0x150
[ 5039.752590][ T44] init_module_from_file+0xe9/0x150
[ 5039.752795][ T44] ? __do_sys_init_module+0x1e0/0x1e0
[ 5039.753016][ T44] ? idempotent_init_module+0x316/0x610
[ 5039.753219][ T44] ? __lock_release+0x5d/0x170
[ 5039.753434][ T44] ? do_raw_spin_unlock+0x58/0x220
[ 5039.753642][ T44] idempotent_init_module+0x331/0x610
[ 5039.753876][ T44] ? init_module_from_file+0x150/0x150
[ 5039.754103][ T44] ? cap_capable+0x94/0x230
[ 5039.754335][ T44] __x64_sys_finit_module+0xca/0x150
[ 5039.754553][ T44] ? do_syscall_64+0x85/0xfd0
[ 5039.754777][ T44] do_syscall_64+0xc1/0xfd0
[ 5039.755004][ T44] entry_SYSCALL_64_after_hwframe+0x4b/0x53
[ 5039.755268][ T44] RIP: 0033:0x7fc863d8b92d
[ 5039.755487][ T44] RSP: 002b:00007ffe23105c28 EFLAGS: 00000246 ORIG_RAX: 0000000000000139
[ 5039.755799][ T44] RAX: ffffffffffffffda RBX: 000055ab78f71ac0 RCX: 00007fc863d8b92d
[ 5039.756110][ T44] RDX: 0000000000000000 RSI: 000055ab485c4a2a RDI: 0000000000000005
[ 5039.756427][ T44] RBP: 0000000000040000 R08: 0000000000000000 R09: 00007ffe23105d60
[ 5039.756737][ T44] R10: 0000000000000005 R11: 0000000000000246 R12: 000055ab485c4a2a
[ 5039.757049][ T44] R13: 000055ab78f71b60 R14: 000055ab78f71da0 R15: 0000000000000000
[ 5039.757416][ T44]
[ 5039.757598][ T44] INFO: task modprobe:412 blocked on an rw-semaphore likely owned by task kworker/u16:1:69
[ 5039.758057][ T44] task:kworker/u16:1 state:R running task stack:24368 pid:69 tgid:69 ppid:2 task_flags:0x4208060 flags:0x00080000
[ 5039.758605][ T44] Workqueue: netns cleanup_net
[ 5039.758818][ T44] Call Trace:
[ 5039.758974][ T44]
[ 5039.759079][ T44] ? __mutex_lock+0x19f/0x1190
[ 5039.759298][ T44] ? trace_sched_exit_tp+0xcd/0x130
[ 5039.759509][ T44] ? nf_ct_iterate_cleanup+0x4a/0x340 [nf_conntrack]
[ 5039.759780][ T44] ? ww_mutex_lock+0x160/0x160
[ 5039.760006][ T44] ? nf_ct_iterate_cleanup+0x4a/0x340 [nf_conntrack]
[ 5039.760273][ T44] ? nf_ct_iterate_cleanup+0x4a/0x340 [nf_conntrack]
[ 5039.760535][ T44] ? __nf_conntrack_hash_insert+0x200/0x200 [nf_conntrack]
[ 5039.760850][ T44] ? nf_conntrack_cleanup_net_list+0x19a/0x4b0 [nf_conntrack]
[ 5039.761162][ T44] ? nf_conntrack_cleanup_net_list+0x10e/0x4b0 [nf_conntrack]
[ 5039.761491][ T44] ? nf_conntrack_cleanup_end+0x70/0x70 [nf_conntrack]
[ 5039.761759][ T44] ? nf_conntrack_pernet_exit+0xa5/0x140 [nf_conntrack]
[ 5039.762020][ T44] ? _raw_spin_unlock+0x2d/0x50
[ 5039.762233][ T44] ? ops_undo_list+0x2bf/0x890
[ 5039.762452][ T44] ? rtnl_net_dumpid_one+0x270/0x270
[ 5039.762657][ T44] ? cleanup_net+0x2d6/0x8b0
[ 5039.762888][ T44] ? cleanup_net+0x3b2/0x8b0
[ 5039.763095][ T44] ? net_passive_dec+0x190/0x190
[ 5039.763314][ T44] ? rcu_is_watching+0x12/0xb0
[ 5039.763522][ T44] ? process_one_work+0xe35/0x1650
[ 5039.763740][ T44] ? pwq_dec_nr_in_flight+0x550/0x550
[ 5039.763953][ T44] ? assign_work+0x168/0x240
[ 5039.764162][ T44] ? worker_thread+0x591/0xcf0
[ 5039.764381][ T44] ? rescuer_thread+0xd10/0xd10
[ 5039.764593][ T44] ? kthread+0x37b/0x5f0
[ 5039.764751][ T44] ? kthread_is_per_cpu+0xc0/0xc0
[ 5039.764956][ T44] ? ret_from_fork+0x1b/0x270
[ 5039.765161][ T44] ? __lock_release+0x5d/0x170
[ 5039.765373][ T44] ? rcu_is_watching+0x12/0xb0
[ 5039.765581][ T44] ? kthread_is_per_cpu+0xc0/0xc0
[ 5039.765786][ T44] ? ret_from_fork+0x1db/0x270
[ 5039.765989][ T44] ? kthread_is_per_cpu+0xc0/0xc0
[ 5039.766199][ T44] ? ret_from_fork_asm+0x11/0x20
[ 5039.766427][ T44]
[ 5039.766596][ T44]
[ 5039.766596][ T44] Showing all locks held in the system:
[ 5039.766894][ T44] 1 lock held by khungtaskd/44:
[ 5039.767090][ T44] #0: ffffffffb2788580 (rcu_read_lock){....}-{1:3}, at: debug_show_all_locks+0x36/0x260
[ 5039.767462][ T44] 4 locks held by kworker/u16:1/69:
[ 5039.767674][ T44] 1 lock held by modprobe/412:
[ 5039.767870][ T44] #0: ffffffffb2e800d0 (pernet_ops_rwsem){++++}-{4:4}, at: register_pernet_subsys+0x1a/0x40
[ 5039.768281][ T44]
[ 5039.768381][ T44] =============================================
[ 5039.768381][ T44]
[ 5162.622327][ T44] INFO: task modprobe:412 blocked for more than 860 seconds.
[ 5162.622836][ T44] Not tainted 6.18.0-rc6-virtme #1
[ 5162.623063][ T44] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[ 5162.623567][ T44] task:modprobe state:D stack:25664 pid:412 tgid:412 ppid:1 task_flags:0x400100 flags:0x00080003
[ 5162.624160][ T44] Call Trace:
[ 5162.624411][ T44]
[ 5162.624570][ T44] __schedule+0x880/0x1b10
[ 5162.624885][ T44] ? io_schedule_timeout+0x150/0x150
[ 5162.625203][ T44] ? schedule+0x1ee/0x270
[ 5162.625445][ T44] ? __lock_release+0x5d/0x170
[ 5162.625729][ T44] schedule+0xe2/0x270
[ 5162.625923][ T44] schedule_preempt_disabled+0x18/0x30
[ 5162.626218][ T44] rwsem_down_write_slowpath+0x3c5/0xc60
[ 5162.626480][ T44] ? __lock_acquire+0x449/0x7e0
[ 5162.626724][ T44] ? rwsem_down_read_slowpath+0xd80/0xd80
[ 5162.627002][ T44] ? rcu_is_watching+0x12/0xb0
[ 5162.627311][ T44] ? lock_acquire+0x104/0x160
[ 5162.627613][ T44] ? register_pernet_subsys+0x1a/0x40
[ 5162.627923][ T44] down_write+0x1d6/0x200
[ 5162.628147][ T44] ? rwsem_down_write_slowpath+0xc60/0xc60
[ 5162.628476][ T44] ? find_held_lock+0x2b/0x80
[ 5162.628778][ T44] ? __create_object+0x5e/0xb0
[ 5162.629015][ T44] ? _sub_I_65535_1+0xcf0/0xcf0 [xt_conntrack]
[ 5162.629395][ T44] register_pernet_subsys+0x1a/0x40
[ 5162.629698][ T44] pg_init+0x2c/0x1000 [pktgen]
[ 5162.629997][ T44] ? _sub_I_65535_1+0xcf0/0xcf0 [xt_conntrack]
[ 5162.630320][ T44] do_one_initcall+0x8c/0x1d0
[ 5162.630593][ T44] ? trace_initcall_start+0x130/0x130
[ 5162.630886][ T44] ? rcu_is_watching+0x12/0xb0
[ 5162.631131][ T44] ? trace_kmalloc+0x26/0xd0
[ 5162.631370][ T44] ? __kmalloc_cache_noprof+0x2e4/0x690
[ 5162.631606][ T44] ? debug_objects_oom+0x160/0x160
[ 5162.631840][ T44] ? kasan_unpoison+0x27/0x50
[ 5162.632135][ T44] ? __asan_register_globals+0x5d/0x70
[ 5162.632454][ T44] do_init_module+0x27a/0x860
[ 5162.632760][ T44] ? free_module+0x2f0/0x2f0
[ 5162.633037][ T44] ? kfree+0x21d/0x540
[ 5162.633212][ T44] ? load_module+0x20c5/0x2dd0
[ 5162.633454][ T44] load_module+0x1f40/0x2dd0
[ 5162.633700][ T44] ? layout_and_allocate.constprop.0+0x630/0x630
[ 5162.634033][ T44] ? kernel_read_file+0x3ed/0x550
[ 5162.634323][ T44] ? kernel_read_file+0x3c8/0x550
[ 5162.634560][ T44] ? __ia32_sys_fsconfig+0x150/0x150
[ 5162.634796][ T44] ? init_module_from_file+0xe9/0x150
[ 5162.635074][ T44] init_module_from_file+0xe9/0x150
[ 5162.635378][ T44] ? __do_sys_init_module+0x1e0/0x1e0
[ 5162.635695][ T44] ? idempotent_init_module+0x316/0x610
[ 5162.635985][ T44] ? __lock_release+0x5d/0x170
[ 5162.636220][ T44] ? do_raw_spin_unlock+0x58/0x220
[ 5162.636540][ T44] idempotent_init_module+0x331/0x610
[ 5162.636854][ T44] ? init_module_from_file+0x150/0x150
[ 5162.637162][ T44] ? cap_capable+0x94/0x230
[ 5162.637448][ T44] __x64_sys_finit_module+0xca/0x150
[ 5162.637677][ T44] ? do_syscall_64+0x85/0xfd0
[ 5162.637911][ T44] do_syscall_64+0xc1/0xfd0
[ 5162.638142][ T44] entry_SYSCALL_64_after_hwframe+0x4b/0x53
[ 5162.638499][ T44] RIP: 0033:0x7fc863d8b92d
[ 5162.638807][ T44] RSP: 002b:00007ffe23105c28 EFLAGS: 00000246 ORIG_RAX: 0000000000000139
[ 5162.639261][ T44] RAX: ffffffffffffffda RBX: 000055ab78f71ac0 RCX: 00007fc863d8b92d
[ 5162.639614][ T44] RDX: 0000000000000000 RSI: 000055ab485c4a2a RDI: 0000000000000005
[ 5162.640068][ T44] RBP: 0000000000040000 R08: 0000000000000000 R09: 00007ffe23105d60
[ 5162.640435][ T44] R10: 0000000000000005 R11: 0000000000000246 R12: 000055ab485c4a2a
[ 5162.640867][ T44] R13: 000055ab78f71b60 R14: 000055ab78f71da0 R15: 0000000000000000
[ 5162.641300][ T44]
[ 5162.641494][ T44] INFO: task modprobe:412 blocked on an rw-semaphore likely owned by task kworker/u16:1:69
[ 5162.641985][ T44] task:kworker/u16:1 state:R running task stack:24368 pid:69 tgid:69 ppid:2 task_flags:0x4208060 flags:0x00080000
[ 5162.642689][ T44] Workqueue: netns cleanup_net
[ 5162.642930][ T44] Call Trace:
[ 5162.643104][ T44]
[ 5162.643245][ T44] ? __mutex_trylock_common+0xf9/0x260
[ 5162.643555][ T44] ? __mutex_handoff+0x2b0/0x2b0
[ 5162.643851][ T44] ? rcu_is_watching+0x12/0xb0
[ 5162.644151][ T44] ? rcu_is_watching+0x12/0xb0
[ 5162.644404][ T44] ? __mutex_lock+0x19f/0x1190
[ 5162.644639][ T44] ? trace_sched_exit_tp+0xcd/0x130
[ 5162.644910][ T44] ? nf_ct_iterate_cleanup+0x4a/0x340 [nf_conntrack]
[ 5162.645302][ T44] ? ww_mutex_lock+0x160/0x160
[ 5162.645553][ T44] ? nf_ct_iterate_cleanup+0x4a/0x340 [nf_conntrack]
[ 5162.645854][ T44] ? nf_ct_iterate_cleanup+0x4a/0x340 [nf_conntrack]
[ 5162.646228][ T44] ? __nf_conntrack_hash_insert+0x200/0x200 [nf_conntrack]
[ 5162.646706][ T44] ? nf_conntrack_cleanup_net_list+0x19a/0x4b0 [nf_conntrack]
[ 5162.647128][ T44] ? nf_conntrack_cleanup_net_list+0x10e/0x4b0 [nf_conntrack]
[ 5162.647564][ T44] ? nf_conntrack_cleanup_end+0x70/0x70 [nf_conntrack]
[ 5162.647941][ T44] ? nf_conntrack_pernet_exit+0xa5/0x140 [nf_conntrack]
[ 5162.648338][ T44] ? _raw_spin_unlock+0x2d/0x50
[ 5162.648575][ T44] ? ops_undo_list+0x2bf/0x890
[ 5162.648878][ T44] ? rtnl_net_dumpid_one+0x270/0x270
[ 5162.649160][ T44] ? cleanup_net+0x2d6/0x8b0
[ 5162.649405][ T44] ? cleanup_net+0x3b2/0x8b0
[ 5162.649668][ T44] ? net_passive_dec+0x190/0x190
[ 5162.649965][ T44] ? rcu_is_watching+0x12/0xb0
[ 5162.650197][ T44] ? process_one_work+0xe35/0x1650
[ 5162.650446][ T44] ? pwq_dec_nr_in_flight+0x550/0x550
[ 5162.650683][ T44] ? assign_work+0x168/0x240
[ 5162.650982][ T44] ? worker_thread+0x591/0xcf0
[ 5162.651292][ T44] ? rescuer_thread+0xd10/0xd10
[ 5162.651532][ T44] ? kthread+0x37b/0x5f0
[ 5162.651743][ T44] ? kthread_is_per_cpu+0xc0/0xc0
[ 5162.652035][ T44] ? ret_from_fork+0x1b/0x270
[ 5162.652304][ T44] ? __lock_release+0x5d/0x170
[ 5162.652561][ T44] ? rcu_is_watching+0x12/0xb0
[ 5162.652852][ T44] ? kthread_is_per_cpu+0xc0/0xc0
[ 5162.653138][ T44] ? ret_from_fork+0x1db/0x270
[ 5162.653374][ T44] ? kthread_is_per_cpu+0xc0/0xc0
[ 5162.653657][ T44] ? ret_from_fork_asm+0x11/0x20
[ 5162.653967][ T44]
[ 5162.654179][ T44]
[ 5162.654179][ T44] Showing all locks held in the system:
[ 5162.654749][ T44] 1 lock held by khungtaskd/44:
[ 5162.655040][ T44] #0: ffffffffb2788580 (rcu_read_lock){....}-{1:3}, at: debug_show_all_locks+0x36/0x260
[ 5162.655471][ T44] 4 locks held by kworker/u16:1/69:
[ 5162.655708][ T44] 1 lock held by modprobe/412:
[ 5162.655991][ T44] #0: ffffffffb2e800d0 (pernet_ops_rwsem){++++}-{4:4}, at: register_pernet_subsys+0x1a/0x40
[ 5162.656486][ T44]
[ 5162.656597][ T44] =============================================
[ 5162.656597][ T44]
[ 5285.503331][ T44] INFO: task modprobe:412 blocked for more than 983 seconds.
[ 5285.503869][ T44] Not tainted 6.18.0-rc6-virtme #1
[ 5285.504167][ T44] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[ 5285.504588][ T44] task:modprobe state:D stack:25664 pid:412 tgid:412 ppid:1 task_flags:0x400100 flags:0x00080003
[ 5285.505230][ T44] Call Trace:
[ 5285.505479][ T44]
[ 5285.505629][ T44] __schedule+0x880/0x1b10
[ 5285.505878][ T44] ? io_schedule_timeout+0x150/0x150
[ 5285.506140][ T44] ? schedule+0x1ee/0x270
[ 5285.506325][ T44] ? __lock_release+0x5d/0x170
[ 5285.506587][ T44] schedule+0xe2/0x270
[ 5285.506814][ T44] schedule_preempt_disabled+0x18/0x30
[ 5285.507109][ T44] rwsem_down_write_slowpath+0x3c5/0xc60
[ 5285.507360][ T44] ? __lock_acquire+0x449/0x7e0
[ 5285.507592][ T44] ? rwsem_down_read_slowpath+0xd80/0xd80
[ 5285.507832][ T44] ? rcu_is_watching+0x12/0xb0
[ 5285.508116][ T44] ? lock_acquire+0x104/0x160
[ 5285.508419][ T44] ? register_pernet_subsys+0x1a/0x40
[ 5285.508732][ T44] down_write+0x1d6/0x200
[ 5285.508958][ T44] ? rwsem_down_write_slowpath+0xc60/0xc60
[ 5285.509340][ T44] ? find_held_lock+0x2b/0x80
[ 5285.509587][ T44] ? __create_object+0x5e/0xb0
[ 5285.509892][ T44] ? _sub_I_65535_1+0xcf0/0xcf0 [xt_conntrack]
[ 5285.510180][ T44] register_pernet_subsys+0x1a/0x40
[ 5285.510472][ T44] pg_init+0x2c/0x1000 [pktgen]
[ 5285.510778][ T44] ? _sub_I_65535_1+0xcf0/0xcf0 [xt_conntrack]
[ 5285.511112][ T44] do_one_initcall+0x8c/0x1d0
[ 5285.511355][ T44] ? trace_initcall_start+0x130/0x130
[ 5285.511595][ T44] ? rcu_is_watching+0x12/0xb0
[ 5285.511874][ T44] ? trace_kmalloc+0x26/0xd0
[ 5285.512103][ T44] ? __kmalloc_cache_noprof+0x2e4/0x690
[ 5285.512371][ T44] ? debug_objects_oom+0x160/0x160
[ 5285.512671][ T44] ? kasan_unpoison+0x27/0x50
[ 5285.512967][ T44] ? __asan_register_globals+0x5d/0x70
[ 5285.513276][ T44] do_init_module+0x27a/0x860
[ 5285.513589][ T44] ? free_module+0x2f0/0x2f0
[ 5285.513884][ T44] ? kfree+0x21d/0x540
[ 5285.514108][ T44] ? load_module+0x20c5/0x2dd0
[ 5285.514418][ T44] load_module+0x1f40/0x2dd0
[ 5285.514710][ T44] ? layout_and_allocate.constprop.0+0x630/0x630
[ 5285.515000][ T44] ? kernel_read_file+0x3ed/0x550
[ 5285.515289][ T44] ? kernel_read_file+0x3c8/0x550
[ 5285.515586][ T44] ? __ia32_sys_fsconfig+0x150/0x150
[ 5285.515890][ T44] ? init_module_from_file+0xe9/0x150
[ 5285.516195][ T44] init_module_from_file+0xe9/0x150
[ 5285.516464][ T44] ? __do_sys_init_module+0x1e0/0x1e0
[ 5285.516712][ T44] ? idempotent_init_module+0x316/0x610
[ 5285.517005][ T44] ? __lock_release+0x5d/0x170
[ 5285.517319][ T44] ? do_raw_spin_unlock+0x58/0x220
[ 5285.517560][ T44] idempotent_init_module+0x331/0x610
[ 5285.517795][ T44] ? init_module_from_file+0x150/0x150
[ 5285.518064][ T44] ? cap_capable+0x94/0x230
[ 5285.518373][ T44] __x64_sys_finit_module+0xca/0x150
[ 5285.518666][ T44] ? do_syscall_64+0x85/0xfd0
[ 5285.519033][ T44] do_syscall_64+0xc1/0xfd0
[ 5285.519339][ T44] entry_SYSCALL_64_after_hwframe+0x4b/0x53
[ 5285.519634][ T44] RIP: 0033:0x7fc863d8b92d
[ 5285.519873][ T44] RSP: 002b:00007ffe23105c28 EFLAGS: 00000246 ORIG_RAX: 0000000000000139
[ 5285.520225][ T44] RAX: ffffffffffffffda RBX: 000055ab78f71ac0 RCX: 00007fc863d8b92d
[ 5285.520583][ T44] RDX: 0000000000000000 RSI: 000055ab485c4a2a RDI: 0000000000000005
[ 5285.521017][ T44] RBP: 0000000000040000 R08: 0000000000000000 R09: 00007ffe23105d60
[ 5285.521428][ T44] R10: 0000000000000005 R11: 0000000000000246 R12: 000055ab485c4a2a
[ 5285.521829][ T44] R13: 000055ab78f71b60 R14: 000055ab78f71da0 R15: 0000000000000000
[ 5285.522263][ T44]
[ 5285.522466][ T44] INFO: task modprobe:412 blocked on an rw-semaphore likely owned by task kworker/u16:1:69
[ 5285.523008][ T44] task:kworker/u16:1 state:R running task stack:24368 pid:69 tgid:69 ppid:2 task_flags:0x4208060 flags:0x00080000
[ 5285.523663][ T44] Workqueue: netns cleanup_net
[ 5285.523901][ T44] Call Trace:
[ 5285.524093][ T44]
[ 5285.524246][ T44] ? sched_clock_cpu+0x6c/0x540
[ 5285.524553][ T44] ? __mutex_trylock_common+0xf9/0x260
[ 5285.524792][ T44] ? __mutex_handoff+0x2b0/0x2b0
[ 5285.525028][ T44] ? rcu_is_watching+0x12/0xb0
[ 5285.525320][ T44] ? rcu_is_watching+0x12/0xb0
[ 5285.525618][ T44] ? trace_contention_end+0xd8/0x140
[ 5285.525913][ T44] ? __mutex_lock+0x19f/0x1190
[ 5285.526212][ T44] ? trace_sched_exit_tp+0xcd/0x130
[ 5285.526453][ T44] ? nf_ct_iterate_cleanup+0x4a/0x340 [nf_conntrack]
[ 5285.526758][ T44] ? ww_mutex_lock+0x160/0x160
[ 5285.527077][ T44] ? nf_ct_iterate_cleanup+0x4a/0x340 [nf_conntrack]
[ 5285.527435][ T44] ? nf_ct_iterate_cleanup+0x4a/0x340 [nf_conntrack]
[ 5285.527771][ T44] ? __nf_conntrack_hash_insert+0x200/0x200 [nf_conntrack]
[ 5285.528159][ T44] ? nf_conntrack_cleanup_net_list+0x19a/0x4b0 [nf_conntrack]
[ 5285.528614][ T44] ? nf_conntrack_cleanup_net_list+0x10e/0x4b0 [nf_conntrack]
[ 5285.529068][ T44] ? nf_conntrack_cleanup_end+0x70/0x70 [nf_conntrack]
[ 5285.529428][ T44] ? nf_conntrack_pernet_exit+0xa5/0x140 [nf_conntrack]
[ 5285.529774][ T44] ? _raw_spin_unlock+0x2d/0x50
[ 5285.530073][ T44] ? ops_undo_list+0x2bf/0x890
[ 5285.530317][ T44] ? rtnl_net_dumpid_one+0x270/0x270
[ 5285.530548][ T44] ? cleanup_net+0x2d6/0x8b0
[ 5285.530798][ T44] ? cleanup_net+0x3b2/0x8b0
[ 5285.531100][ T44] ? net_passive_dec+0x190/0x190
[ 5285.531413][ T44] ? rcu_is_watching+0x12/0xb0
[ 5285.531645][ T44] ? process_one_work+0xe35/0x1650
[ 5285.531888][ T44] ? pwq_dec_nr_in_flight+0x550/0x550
[ 5285.532124][ T44] ? assign_work+0x168/0x240
[ 5285.532427][ T44] ? worker_thread+0x591/0xcf0
[ 5285.532730][ T44] ? rescuer_thread+0xd10/0xd10
[ 5285.533031][ T44] ? kthread+0x37b/0x5f0
[ 5285.533239][ T44] ? kthread_is_per_cpu+0xc0/0xc0
[ 5285.533474][ T44] ? ret_from_fork+0x1b/0x270
[ 5285.533706][ T44] ? __lock_release+0x5d/0x170
[ 5285.533936][ T44] ? rcu_is_watching+0x12/0xb0
[ 5285.534230][ T44] ? kthread_is_per_cpu+0xc0/0xc0
[ 5285.534532][ T44] ? ret_from_fork+0x1db/0x270
[ 5285.534836][ T44] ? kthread_is_per_cpu+0xc0/0xc0
[ 5285.535144][ T44] ? ret_from_fork_asm+0x11/0x20
[ 5285.535399][ T44]
[ 5285.535601][ T44]
[ 5285.535601][ T44] Showing all locks held in the system:
[ 5285.535948][ T44] 1 lock held by khungtaskd/44:
[ 5285.536219][ T44] #0: ffffffffb2788580 (rcu_read_lock){....}-{1:3}, at: debug_show_all_locks+0x36/0x260
[ 5285.536746][ T44] 4 locks held by kworker/u16:1/69:
[ 5285.537049][ T44] 1 lock held by modprobe/412:
[ 5285.537309][ T44] #0: ffffffffb2e800d0 (pernet_ops_rwsem){++++}-{4:4}, at: register_pernet_subsys+0x1a/0x40
[ 5285.537893][ T44]
[ 5285.538019][ T44] =============================================
[ 5285.538019][ T44]
[ 5408.382333][ T44] INFO: task modprobe:412 blocked for more than 1105 seconds.
[ 5408.382857][ T44] Not tainted 6.18.0-rc6-virtme #1
[ 5408.383152][ T44] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[ 5408.383605][ T44] task:modprobe state:D stack:25664 pid:412 tgid:412 ppid:1 task_flags:0x400100 flags:0x00080003
[ 5408.384135][ T44] Call Trace:
[ 5408.384366][ T44]
[ 5408.384522][ T44] __schedule+0x880/0x1b10
[ 5408.384832][ T44] ? io_schedule_timeout+0x150/0x150
[ 5408.385143][ T44] ? schedule+0x1ee/0x270
[ 5408.385381][ T44] ? __lock_release+0x5d/0x170
[ 5408.385628][ T44] schedule+0xe2/0x270
[ 5408.385806][ T44] schedule_preempt_disabled+0x18/0x30
[ 5408.386038][ T44] rwsem_down_write_slowpath+0x3c5/0xc60
[ 5408.386335][ T44] ? __lock_acquire+0x449/0x7e0
[ 5408.386630][ T44] ? rwsem_down_read_slowpath+0xd80/0xd80
[ 5408.386935][ T44] ? rcu_is_watching+0x12/0xb0
[ 5408.387231][ T44] ? lock_acquire+0x104/0x160
[ 5408.387490][ T44] ? register_pernet_subsys+0x1a/0x40
[ 5408.387778][ T44] down_write+0x1d6/0x200
[ 5408.387988][ T44] ? rwsem_down_write_slowpath+0xc60/0xc60
[ 5408.388278][ T44] ? find_held_lock+0x2b/0x80
[ 5408.388566][ T44] ? __create_object+0x5e/0xb0
[ 5408.388868][ T44] ? _sub_I_65535_1+0xcf0/0xcf0 [xt_conntrack]
[ 5408.389238][ T44] register_pernet_subsys+0x1a/0x40
[ 5408.389534][ T44] pg_init+0x2c/0x1000 [pktgen]
[ 5408.389774][ T44] ? _sub_I_65535_1+0xcf0/0xcf0 [xt_conntrack]
[ 5408.390059][ T44] do_one_initcall+0x8c/0x1d0
[ 5408.390380][ T44] ? trace_initcall_start+0x130/0x130
[ 5408.390676][ T44] ? rcu_is_watching+0x12/0xb0
[ 5408.390971][ T44] ? trace_kmalloc+0x26/0xd0
[ 5408.391224][ T44] ? __kmalloc_cache_noprof+0x2e4/0x690
[ 5408.391498][ T44] ? debug_objects_oom+0x160/0x160
[ 5408.391801][ T44] ? kasan_unpoison+0x27/0x50
[ 5408.392069][ T44] ? __asan_register_globals+0x5d/0x70
[ 5408.392317][ T44] do_init_module+0x27a/0x860
[ 5408.392562][ T44] ? free_module+0x2f0/0x2f0
[ 5408.392816][ T44] ? kfree+0x21d/0x540
[ 5408.393041][ T44] ? load_module+0x20c5/0x2dd0
[ 5408.393358][ T44] load_module+0x1f40/0x2dd0
[ 5408.393608][ T44] ? layout_and_allocate.constprop.0+0x630/0x630
[ 5408.393970][ T44] ? kernel_read_file+0x3ed/0x550
[ 5408.394207][ T44] ? kernel_read_file+0x3c8/0x550
[ 5408.394495][ T44] ? __ia32_sys_fsconfig+0x150/0x150
[ 5408.394796][ T44] ? init_module_from_file+0xe9/0x150
[ 5408.395093][ T44] init_module_from_file+0xe9/0x150
[ 5408.395403][ T44] ? __do_sys_init_module+0x1e0/0x1e0
[ 5408.395700][ T44] ? idempotent_init_module+0x316/0x610
[ 5408.395932][ T44] ? __lock_release+0x5d/0x170
[ 5408.396164][ T44] ? do_raw_spin_unlock+0x58/0x220
[ 5408.396419][ T44] idempotent_init_module+0x331/0x610
[ 5408.396718][ T44] ? init_module_from_file+0x150/0x150
[ 5408.397024][ T44] ? cap_capable+0x94/0x230
[ 5408.397335][ T44] __x64_sys_finit_module+0xca/0x150
[ 5408.397636][ T44] ? do_syscall_64+0x85/0xfd0
[ 5408.397874][ T44] do_syscall_64+0xc1/0xfd0
[ 5408.398107][ T44] entry_SYSCALL_64_after_hwframe+0x4b/0x53
[ 5408.398406][ T44] RIP: 0033:0x7fc863d8b92d
[ 5408.398711][ T44] RSP: 002b:00007ffe23105c28 EFLAGS: 00000246 ORIG_RAX: 0000000000000139
[ 5408.399158][ T44] RAX: ffffffffffffffda RBX: 000055ab78f71ac0 RCX: 00007fc863d8b92d
[ 5408.399616][ T44] RDX: 0000000000000000 RSI: 000055ab485c4a2a RDI: 0000000000000005
[ 5408.399999][ T44] RBP: 0000000000040000 R08: 0000000000000000 R09: 00007ffe23105d60
[ 5408.400385][ T44] R10: 0000000000000005 R11: 0000000000000246 R12: 000055ab485c4a2a
[ 5408.400828][ T44] R13: 000055ab78f71b60 R14: 000055ab78f71da0 R15: 0000000000000000
[ 5408.401300][ T44]
[ 5408.401502][ T44] INFO: task modprobe:412 blocked on an rw-semaphore likely owned by task kworker/u16:1:69
[ 5408.402073][ T44] task:kworker/u16:1 state:R running task stack:24368 pid:69 tgid:69 ppid:2 task_flags:0x4208060 flags:0x00080000
[ 5408.402736][ T44] Workqueue: netns cleanup_net
[ 5408.403043][ T44] Call Trace:
[ 5408.403274][ T44]
[ 5408.403431][ T44] ? __mutex_trylock_common+0xf9/0x260
[ 5408.403730][ T44] ? __mutex_handoff+0x2b0/0x2b0
[ 5408.404031][ T44] ? rcu_is_watching+0x12/0xb0
[ 5408.404273][ T44] ? rcu_is_watching+0x12/0xb0
[ 5408.404564][ T44] ? trace_contention_end+0xd8/0x140
[ 5408.404859][ T44] ? __mutex_lock+0x19f/0x1190
[ 5408.405153][ T44] ? trace_sched_exit_tp+0xcd/0x130
[ 5408.405475][ T44] ? nf_ct_iterate_cleanup+0x4a/0x340 [nf_conntrack]
[ 5408.405823][ T44] ? ww_mutex_lock+0x160/0x160
[ 5408.406072][ T44] ? nf_ct_iterate_cleanup+0x4a/0x340 [nf_conntrack]
[ 5408.406375][ T44] ? nf_ct_iterate_cleanup+0x4a/0x340 [nf_conntrack]
[ 5408.406759][ T44] ? __nf_conntrack_hash_insert+0x200/0x200 [nf_conntrack]
[ 5408.407167][ T44] ? nf_conntrack_cleanup_net_list+0x19a/0x4b0 [nf_conntrack]
[ 5408.407589][ T44] ? nf_conntrack_cleanup_net_list+0x10e/0x4b0 [nf_conntrack]
[ 5408.408045][ T44] ? nf_conntrack_cleanup_end+0x70/0x70 [nf_conntrack]
[ 5408.408390][ T44] ? nf_conntrack_pernet_exit+0xa5/0x140 [nf_conntrack]
[ 5408.408686][ T44] ? _raw_spin_unlock+0x2d/0x50
[ 5408.408950][ T44] ? ops_undo_list+0x2bf/0x890
[ 5408.409192][ T44] ? rtnl_net_dumpid_one+0x270/0x270
[ 5408.409493][ T44] ? cleanup_net+0x2d6/0x8b0
[ 5408.409796][ T44] ? cleanup_net+0x3b2/0x8b0
[ 5408.410091][ T44] ? net_passive_dec+0x190/0x190
[ 5408.410384][ T44] ? rcu_is_watching+0x12/0xb0
[ 5408.410644][ T44] ? process_one_work+0xe35/0x1650
[ 5408.410950][ T44] ? pwq_dec_nr_in_flight+0x550/0x550
[ 5408.411225][ T44] ? assign_work+0x168/0x240
[ 5408.411464][ T44] ? worker_thread+0x591/0xcf0
[ 5408.411704][ T44] ? rescuer_thread+0xd10/0xd10
[ 5408.411973][ T44] ? kthread+0x37b/0x5f0
[ 5408.412197][ T44] ? kthread_is_per_cpu+0xc0/0xc0
[ 5408.412465][ T44] ? ret_from_fork+0x1b/0x270
[ 5408.412698][ T44] ? __lock_release+0x5d/0x170
[ 5408.412928][ T44] ? rcu_is_watching+0x12/0xb0
[ 5408.413214][ T44] ? kthread_is_per_cpu+0xc0/0xc0
[ 5408.413469][ T44] ? ret_from_fork+0x1db/0x270
[ 5408.413697][ T44] ? kthread_is_per_cpu+0xc0/0xc0
[ 5408.413975][ T44] ? ret_from_fork_asm+0x11/0x20
[ 5408.414224][ T44]
[ 5408.414490][ T44]
[ 5408.414490][ T44] Showing all locks held in the system:
[ 5408.414929][ T44] 1 lock held by khungtaskd/44:
[ 5408.415215][ T44] #0: ffffffffb2788580 (rcu_read_lock){....}-{1:3}, at: debug_show_all_locks+0x36/0x260
[ 5408.415658][ T44] 4 locks held by kworker/u16:1/69:
[ 5408.415955][ T44] 1 lock held by modprobe/412:
[ 5408.416239][ T44] #0: ffffffffb2e800d0 (pernet_ops_rwsem){++++}-{4:4}, at: register_pernet_subsys+0x1a/0x40
[ 5408.416715][ T44]
[ 5408.416828][ T44] =============================================
[ 5408.416828][ T44]
[ 5531.262399][ T44] INFO: task modprobe:412 blocked for more than 1228 seconds.
[ 5531.263156][ T44] Not tainted 6.18.0-rc6-virtme #1
[ 5531.263432][ T44] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[ 5531.263830][ T44] task:modprobe state:D stack:25664 pid:412 tgid:412 ppid:1 task_flags:0x400100 flags:0x00080003
[ 5531.264400][ T44] Call Trace:
[ 5531.264598][ T44]
[ 5531.264733][ T44] __schedule+0x880/0x1b10
[ 5531.264990][ T44] ? io_schedule_timeout+0x150/0x150
[ 5531.265241][ T44] ? schedule+0x1ee/0x270
[ 5531.265447][ T44] ? __lock_release+0x5d/0x170
[ 5531.265713][ T44] schedule+0xe2/0x270
[ 5531.265901][ T44] schedule_preempt_disabled+0x18/0x30
[ 5531.266127][ T44] rwsem_down_write_slowpath+0x3c5/0xc60
[ 5531.266387][ T44] ? __lock_acquire+0x449/0x7e0
[ 5531.266630][ T44] ? rwsem_down_read_slowpath+0xd80/0xd80
[ 5531.266870][ T44] ? rcu_is_watching+0x12/0xb0
[ 5531.267114][ T44] ? lock_acquire+0x104/0x160
[ 5531.267373][ T44] ? register_pernet_subsys+0x1a/0x40
[ 5531.267632][ T44] down_write+0x1d6/0x200
[ 5531.267808][ T44] ? rwsem_down_write_slowpath+0xc60/0xc60
[ 5531.268102][ T44] ? find_held_lock+0x2b/0x80
[ 5531.268339][ T44] ? __create_object+0x5e/0xb0
[ 5531.268600][ T44] ? _sub_I_65535_1+0xcf0/0xcf0 [xt_conntrack]
[ 5531.268895][ T44] register_pernet_subsys+0x1a/0x40
[ 5531.269138][ T44] pg_init+0x2c/0x1000 [pktgen]
[ 5531.269410][ T44] ? _sub_I_65535_1+0xcf0/0xcf0 [xt_conntrack]
[ 5531.269714][ T44] do_one_initcall+0x8c/0x1d0
[ 5531.269957][ T44] ? trace_initcall_start+0x130/0x130
[ 5531.270199][ T44] ? rcu_is_watching+0x12/0xb0
[ 5531.270435][ T44] ? trace_kmalloc+0x26/0xd0
[ 5531.270664][ T44] ? __kmalloc_cache_noprof+0x2e4/0x690
[ 5531.270917][ T44] ? debug_objects_oom+0x160/0x160
[ 5531.271157][ T44] ? kasan_unpoison+0x27/0x50
[ 5531.271407][ T44] ? __asan_register_globals+0x5d/0x70
[ 5531.271637][ T44] do_init_module+0x27a/0x860
[ 5531.271892][ T44] ? free_module+0x2f0/0x2f0
[ 5531.272128][ T44] ? kfree+0x21d/0x540
[ 5531.272311][ T44] ? load_module+0x20c5/0x2dd0
[ 5531.272548][ T44] load_module+0x1f40/0x2dd0
[ 5531.272798][ T44] ? layout_and_allocate.constprop.0+0x630/0x630
[ 5531.273102][ T44] ? kernel_read_file+0x3ed/0x550
[ 5531.273343][ T44] ? kernel_read_file+0x3c8/0x550
[ 5531.273575][ T44] ? __ia32_sys_fsconfig+0x150/0x150
[ 5531.273809][ T44] ? init_module_from_file+0xe9/0x150
[ 5531.274045][ T44] init_module_from_file+0xe9/0x150
[ 5531.274283][ T44] ? __do_sys_init_module+0x1e0/0x1e0
[ 5531.274536][ T44] ? idempotent_init_module+0x316/0x610
[ 5531.274773][ T44] ? __lock_release+0x5d/0x170
[ 5531.275015][ T44] ? do_raw_spin_unlock+0x58/0x220
[ 5531.275258][ T44] idempotent_init_module+0x331/0x610
[ 5531.275492][ T44] ? init_module_from_file+0x150/0x150
[ 5531.275734][ T44] ? cap_capable+0x94/0x230
[ 5531.275971][ T44] __x64_sys_finit_module+0xca/0x150
[ 5531.276218][ T44] ? do_syscall_64+0x85/0xfd0
[ 5531.276461][ T44] do_syscall_64+0xc1/0xfd0
[ 5531.276692][ T44] entry_SYSCALL_64_after_hwframe+0x4b/0x53
[ 5531.276976][ T44] RIP: 0033:0x7fc863d8b92d
[ 5531.277224][ T44] RSP: 002b:00007ffe23105c28 EFLAGS: 00000246 ORIG_RAX: 0000000000000139
[ 5531.277583][ T44] RAX: ffffffffffffffda RBX: 000055ab78f71ac0 RCX: 00007fc863d8b92d
[ 5531.277934][ T44] RDX: 0000000000000000 RSI: 000055ab485c4a2a RDI: 0000000000000005
[ 5531.278295][ T44] RBP: 0000000000040000 R08: 0000000000000000 R09: 00007ffe23105d60
[ 5531.278642][ T44] R10: 0000000000000005 R11: 0000000000000246 R12: 000055ab485c4a2a
[ 5531.278986][ T44] R13: 000055ab78f71b60 R14: 000055ab78f71da0 R15: 0000000000000000
[ 5531.279376][ T44]
[ 5531.279578][ T44] INFO: task modprobe:412 blocked on an rw-semaphore likely owned by task kworker/u16:1:69
[ 5531.280077][ T44] task:kworker/u16:1 state:R running task stack:24368 pid:69 tgid:69 ppid:2 task_flags:0x4208060 flags:0x00080000
[ 5531.280656][ T44] Workqueue: netns cleanup_net
[ 5531.280899][ T44] Call Trace:
[ 5531.281078][ T44]
[ 5531.281206][ T44] ? __mutex_lock+0x19f/0x1190
[ 5531.281449][ T44] ? trace_sched_exit_tp+0xcd/0x130
[ 5531.281685][ T44] ? nf_ct_iterate_cleanup+0x4a/0x340 [nf_conntrack]
[ 5531.281985][ T44] ? ww_mutex_lock+0x160/0x160
[ 5531.282245][ T44] ? nf_ct_iterate_cleanup+0x4a/0x340 [nf_conntrack]
[ 5531.282548][ T44] ? nf_ct_iterate_cleanup+0x4a/0x340 [nf_conntrack]
[ 5531.282848][ T44] ? __nf_conntrack_hash_insert+0x200/0x200 [nf_conntrack]
[ 5531.283212][ T44] ? nf_conntrack_cleanup_net_list+0x19a/0x4b0 [nf_conntrack]
[ 5531.283588][ T44] ? nf_conntrack_cleanup_net_list+0x10e/0x4b0 [nf_conntrack]
[ 5531.283941][ T44] ? nf_conntrack_cleanup_end+0x70/0x70 [nf_conntrack]
[ 5531.284264][ T44] ? nf_conntrack_pernet_exit+0xa5/0x140 [nf_conntrack]
[ 5531.284575][ T44] ? _raw_spin_unlock+0x2d/0x50
[ 5531.284814][ T44] ? ops_undo_list+0x2bf/0x890
[ 5531.285052][ T44] ? rtnl_net_dumpid_one+0x270/0x270
[ 5531.285299][ T44] ? cleanup_net+0x2d6/0x8b0
[ 5531.285542][ T44] ? cleanup_net+0x3b2/0x8b0
[ 5531.285788][ T44] ? net_passive_dec+0x190/0x190
[ 5531.286022][ T44] ? rcu_is_watching+0x12/0xb0
[ 5531.286279][ T44] ? process_one_work+0xe35/0x1650
[ 5531.286524][ T44] ? pwq_dec_nr_in_flight+0x550/0x550
[ 5531.286762][ T44] ? assign_work+0x168/0x240
[ 5531.286996][ T44] ? worker_thread+0x591/0xcf0
[ 5531.287244][ T44] ? rescuer_thread+0xd10/0xd10
[ 5531.287484][ T44] ? kthread+0x37b/0x5f0
[ 5531.287661][ T44] ? kthread_is_per_cpu+0xc0/0xc0
[ 5531.287887][ T44] ? ret_from_fork+0x1b/0x270
[ 5531.288123][ T44] ? __lock_release+0x5d/0x170
[ 5531.288372][ T44] ? rcu_is_watching+0x12/0xb0
[ 5531.288602][ T44] ? kthread_is_per_cpu+0xc0/0xc0
[ 5531.288832][ T44] ? ret_from_fork+0x1db/0x270
[ 5531.289060][ T44] ? kthread_is_per_cpu+0xc0/0xc0
[ 5531.289308][ T44] ? ret_from_fork_asm+0x11/0x20
[ 5531.289564][ T44]
[ 5531.289737][ T44] Future hung task reports are suppressed, see sysctl kernel.hung_task_warnings
[ 5531.290152][ T44]
[ 5531.290152][ T44] Showing all locks held in the system:
[ 5531.290514][ T44] 1 lock held by khungtaskd/44:
[ 5531.290737][ T44] #0: ffffffffb2788580 (rcu_read_lock){....}-{1:3}, at: debug_show_all_locks+0x36/0x260
[ 5531.291144][ T44] 4 locks held by kworker/u16:1/69:
[ 5531.291404][ T44] 1 lock held by modprobe/412:
[ 5531.291624][ T44] #0: ffffffffb2e800d0 (pernet_ops_rwsem){++++}-{4:4}, at: register_pernet_subsys+0x1a/0x40
[ 5531.292091][ T44]
[ 5531.292206][ T44] =============================================
[ 5531.292206][ T44]
WAIT TIMEOUT stderr
Ctrl-C stderr
Ctrl-C stderr