[ 4441.022132][T29700] pktgen: Packet Generator for packet performance testing. Version: 2.75
[ 4671.037251][ T44] INFO: task modprobe:29700 blocked for more than 122 seconds.
[ 4671.037855][ T44] Not tainted 6.17.0-rc6-virtme #1
[ 4671.038086][ T44] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[ 4671.038481][ T44] task:modprobe state:D stack:25712 pid:29700 tgid:29700 ppid:1 task_flags:0x400100 flags:0x00004006
[ 4671.039020][ T44] Call Trace:
[ 4671.039213][ T44]
[ 4671.039340][ T44] __schedule+0x880/0x1b10
[ 4671.039583][ T44] ? io_schedule_timeout+0x150/0x150
[ 4671.039818][ T44] ? schedule+0x1ee/0x270
[ 4671.039983][ T44] ? __lock_release+0x5d/0x170
[ 4671.040255][ T44] schedule+0xe2/0x270
[ 4671.040424][ T44] schedule_preempt_disabled+0x18/0x30
[ 4671.040656][ T44] rwsem_down_write_slowpath+0x3c5/0xcb0
[ 4671.040879][ T44] ? __lock_acquire+0x449/0x7e0
[ 4671.041109][ T44] ? down_read+0x4a0/0x4a0
[ 4671.041348][ T44] ? rcu_is_watching+0x12/0xb0
[ 4671.041583][ T44] ? lock_acquire+0x104/0x160
[ 4671.041800][ T44] ? register_pernet_subsys+0x1a/0x40
[ 4671.042030][ T44] down_write+0x1d6/0x200
[ 4671.042218][ T44] ? rwsem_down_write_slowpath+0xcb0/0xcb0
[ 4671.042574][ T44] ? find_held_lock+0x2b/0x80
[ 4671.042795][ T44] ? __create_object+0x5e/0xb0
[ 4671.043017][ T44] ? _sub_I_65535_1+0x200/0x200 [pktgen]
[ 4671.043281][ T44] register_pernet_subsys+0x1a/0x40
[ 4671.043507][ T44] pg_init+0x2c/0x1000 [pktgen]
[ 4671.043734][ T44] ? _sub_I_65535_1+0x200/0x200 [pktgen]
[ 4671.043953][ T44] do_one_initcall+0x8c/0x1d0
[ 4671.044228][ T44] ? trace_initcall_level+0x130/0x130
[ 4671.044943][ T44] ? rcu_is_watching+0x12/0xb0
[ 4671.045195][ T44] ? trace_kmalloc+0x26/0xd0
[ 4671.045419][ T44] ? __kmalloc_cache_noprof+0x15f/0x360
[ 4671.045650][ T44] ? debug_objects_oom+0x160/0x160
[ 4671.045877][ T44] ? kasan_unpoison+0x27/0x50
[ 4671.046111][ T44] ? __asan_register_globals+0x5d/0x70
[ 4671.046351][ T44] do_init_module+0x27a/0x860
[ 4671.046586][ T44] ? free_module+0x2f0/0x2f0
[ 4671.046817][ T44] ? kfree+0x144/0x320
[ 4671.046982][ T44] ? find_vm_area+0x12/0x50
[ 4671.047227][ T44] load_module+0x1f40/0x2dd0
[ 4671.047461][ T44] ? layout_and_allocate.constprop.0+0x630/0x630
[ 4671.047743][ T44] ? kernel_read_file+0x3ed/0x550
[ 4671.047978][ T44] ? kernel_read_file+0x3c8/0x550
[ 4671.048254][ T44] ? __ia32_sys_fsconfig+0x150/0x150
[ 4671.048493][ T44] ? init_module_from_file+0xe9/0x150
[ 4671.048733][ T44] init_module_from_file+0xe9/0x150
[ 4671.048967][ T44] ? __do_sys_init_module+0x1e0/0x1e0
[ 4671.049241][ T44] ? idempotent_init_module+0x316/0x610
[ 4671.049480][ T44] ? __lock_release+0x5d/0x170
[ 4671.049720][ T44] ? do_raw_spin_unlock+0x58/0x220
[ 4671.049966][ T44] idempotent_init_module+0x331/0x610
[ 4671.050227][ T44] ? init_module_from_file+0x150/0x150
[ 4671.050474][ T44] ? cap_capable+0x94/0x230
[ 4671.050704][ T44] __x64_sys_finit_module+0xca/0x150
[ 4671.050925][ T44] ? do_syscall_64+0x85/0x300
[ 4671.051169][ T44] do_syscall_64+0xc1/0x300
[ 4671.051392][ T44] entry_SYSCALL_64_after_hwframe+0x4b/0x53
[ 4671.051682][ T44] RIP: 0033:0x7fbfb3fafa0d
[ 4671.051917][ T44] RSP: 002b:00007ffde31647a8 EFLAGS: 00000246 ORIG_RAX: 0000000000000139
[ 4671.052288][ T44] RAX: ffffffffffffffda RBX: 0000557a8b230ac0 RCX: 00007fbfb3fafa0d
[ 4671.052633][ T44] RDX: 0000000000000000 RSI: 0000557a53d3fa2a RDI: 0000000000000005
[ 4671.052979][ T44] RBP: 0000000000040000 R08: 0000000000000000 R09: 00007ffde31648e0
[ 4671.053347][ T44] R10: 0000000000000005 R11: 0000000000000246 R12: 0000557a53d3fa2a
[ 4671.053687][ T44] R13: 0000557a8b230b60 R14: 0000557a8b230da0 R15: 0000000000000000
[ 4671.054034][ T44]
[ 4671.054242][ T44] INFO: task modprobe:29700 blocked on an rw-semaphore likely owned by task kworker/u16:0:12
[ 4671.054730][ T44] task:kworker/u16:0 state:R running task stack:24720 pid:12 tgid:12 ppid:2 task_flags:0x4208060 flags:0x00004000
[ 4671.055303][ T44] Workqueue: netns cleanup_net
[ 4671.055545][ T44] Call Trace:
[ 4671.055723][ T44]
[ 4671.055841][ T44] ? __mutex_lock+0x19f/0x11a0
[ 4671.056075][ T44] ? lockdep_hardirqs_on_prepare+0x103/0x1c0
[ 4671.056402][ T44] ? ww_mutex_lock+0x160/0x160
[ 4671.056647][ T44] ? nf_ct_iterate_cleanup+0x4a/0x340 [nf_conntrack]
[ 4671.056937][ T44] ? nf_ct_iterate_cleanup+0x4a/0x340 [nf_conntrack]
[ 4671.057245][ T44] ? __nf_conntrack_hash_insert+0x200/0x200 [nf_conntrack]
[ 4671.057594][ T44] ? nf_conntrack_cleanup_net_list+0x19a/0x4b0 [nf_conntrack]
[ 4671.057953][ T44] ? nf_conntrack_cleanup_net_list+0x10e/0x4b0 [nf_conntrack]
[ 4671.058325][ T44] ? nf_conntrack_cleanup_end+0x70/0x70 [nf_conntrack]
[ 4671.058633][ T44] ? nf_conntrack_pernet_exit+0xa5/0x140 [nf_conntrack]
[ 4671.058945][ T44] ? ops_undo_list+0x2bf/0x890
[ 4671.059207][ T44] ? rtnl_net_dumpid_one+0x270/0x270
[ 4671.059447][ T44] ? cleanup_net+0x2c5/0x890
[ 4671.059690][ T44] ? cleanup_net+0x3a1/0x890
[ 4671.059933][ T44] ? net_passive_dec+0x190/0x190
[ 4671.060207][ T44] ? rcu_is_watching+0x12/0xb0
[ 4671.060452][ T44] ? process_one_work+0xe35/0x1650
[ 4671.060701][ T44] ? pwq_dec_nr_in_flight+0x550/0x550
[ 4671.060945][ T44] ? assign_work+0x168/0x240
[ 4671.061202][ T44] ? worker_thread+0x591/0xcf0
[ 4671.061449][ T44] ? rescuer_thread+0xd10/0xd10
[ 4671.061696][ T44] ? kthread+0x37b/0x5f0
[ 4671.061878][ T44] ? kthread_is_per_cpu+0xc0/0xc0
[ 4671.062126][ T44] ? ret_from_fork+0x1b/0x270
[ 4671.062374][ T44] ? __lock_release+0x5d/0x170
[ 4671.062614][ T44] ? rcu_is_watching+0x12/0xb0
[ 4671.062854][ T44] ? kthread_is_per_cpu+0xc0/0xc0
[ 4671.063115][ T44] ? ret_from_fork+0x1db/0x270
[ 4671.063358][ T44] ? kthread_is_per_cpu+0xc0/0xc0
[ 4671.063596][ T44] ? ret_from_fork_asm+0x11/0x20
[ 4671.063857][ T44]
[ 4671.064079][ T44]
[ 4671.064079][ T44] Showing all locks held in the system:
[ 4671.064793][ T44] 4 locks held by kworker/u16:0/12:
[ 4671.065035][ T44] 1 lock held by khungtaskd/44:
[ 4671.065275][ T44] #0: ffffffff9cb83f80 (rcu_read_lock){....}-{1:3}, at: debug_show_all_locks+0x36/0x260
[ 4671.065700][ T44] 3 locks held by kworker/2:1/15955:
[ 4671.065938][ T44] 1 lock held by modprobe/29700:
[ 4671.066174][ T44] #0: ffffffff9d270630 (pernet_ops_rwsem){++++}-{4:4}, at: register_pernet_subsys+0x1a/0x40
[ 4671.066626][ T44]
[ 4671.066743][ T44] =============================================
[ 4671.066743][ T44]
[ 4793.917250][ T44] INFO: task modprobe:29700 blocked for more than 245 seconds.
[ 4793.917737][ T44] Not tainted 6.17.0-rc6-virtme #1
[ 4793.917974][ T44] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[ 4793.918383][ T44] task:modprobe state:D stack:25712 pid:29700 tgid:29700 ppid:1 task_flags:0x400100 flags:0x00004006
[ 4793.918908][ T44] Call Trace:
[ 4793.919083][ T44]
[ 4793.919222][ T44] __schedule+0x880/0x1b10
[ 4793.919469][ T44] ? io_schedule_timeout+0x150/0x150
[ 4793.919722][ T44] ? schedule+0x1ee/0x270
[ 4793.919895][ T44] ? __lock_release+0x5d/0x170
[ 4793.920141][ T44] schedule+0xe2/0x270
[ 4793.920344][ T44] schedule_preempt_disabled+0x18/0x30
[ 4793.920576][ T44] rwsem_down_write_slowpath+0x3c5/0xcb0
[ 4793.920810][ T44] ? __lock_acquire+0x449/0x7e0
[ 4793.921051][ T44] ? down_read+0x4a0/0x4a0
[ 4793.921302][ T44] ? rcu_is_watching+0x12/0xb0
[ 4793.921550][ T44] ? lock_acquire+0x104/0x160
[ 4793.921780][ T44] ? register_pernet_subsys+0x1a/0x40
[ 4793.922022][ T44] down_write+0x1d6/0x200
[ 4793.922221][ T44] ? rwsem_down_write_slowpath+0xcb0/0xcb0
[ 4793.922512][ T44] ? find_held_lock+0x2b/0x80
[ 4793.922746][ T44] ? __create_object+0x5e/0xb0
[ 4793.922982][ T44] ? _sub_I_65535_1+0x200/0x200 [pktgen]
[ 4793.923235][ T44] register_pernet_subsys+0x1a/0x40
[ 4793.923471][ T44] pg_init+0x2c/0x1000 [pktgen]
[ 4793.923706][ T44] ? _sub_I_65535_1+0x200/0x200 [pktgen]
[ 4793.923940][ T44] do_one_initcall+0x8c/0x1d0
[ 4793.924202][ T44] ? trace_initcall_level+0x130/0x130
[ 4793.924437][ T44] ? rcu_is_watching+0x12/0xb0
[ 4793.924670][ T44] ? trace_kmalloc+0x26/0xd0
[ 4793.924906][ T44] ? __kmalloc_cache_noprof+0x15f/0x360
[ 4793.925140][ T44] ? debug_objects_oom+0x160/0x160
[ 4793.925387][ T44] ? kasan_unpoison+0x27/0x50
[ 4793.925626][ T44] ? __asan_register_globals+0x5d/0x70
[ 4793.925858][ T44] do_init_module+0x27a/0x860
[ 4793.926100][ T44] ? free_module+0x2f0/0x2f0
[ 4793.926342][ T44] ? kfree+0x144/0x320
[ 4793.926519][ T44] ? find_vm_area+0x12/0x50
[ 4793.926765][ T44] load_module+0x1f40/0x2dd0
[ 4793.927009][ T44] ? layout_and_allocate.constprop.0+0x630/0x630
[ 4793.927313][ T44] ? kernel_read_file+0x3ed/0x550
[ 4793.927549][ T44] ? kernel_read_file+0x3c8/0x550
[ 4793.927784][ T44] ? __ia32_sys_fsconfig+0x150/0x150
[ 4793.928023][ T44] ? init_module_from_file+0xe9/0x150
[ 4793.928278][ T44] init_module_from_file+0xe9/0x150
[ 4793.928519][ T44] ? __do_sys_init_module+0x1e0/0x1e0
[ 4793.928770][ T44] ? idempotent_init_module+0x316/0x610
[ 4793.929004][ T44] ? __lock_release+0x5d/0x170
[ 4793.929253][ T44] ? do_raw_spin_unlock+0x58/0x220
[ 4793.929488][ T44] idempotent_init_module+0x331/0x610
[ 4793.929722][ T44] ? init_module_from_file+0x150/0x150
[ 4793.929973][ T44] ? cap_capable+0x94/0x230
[ 4793.930221][ T44] __x64_sys_finit_module+0xca/0x150
[ 4793.930455][ T44] ? do_syscall_64+0x85/0x300
[ 4793.930693][ T44] do_syscall_64+0xc1/0x300
[ 4793.930930][ T44] entry_SYSCALL_64_after_hwframe+0x4b/0x53
[ 4793.931235][ T44] RIP: 0033:0x7fbfb3fafa0d
[ 4793.931478][ T44] RSP: 002b:00007ffde31647a8 EFLAGS: 00000246 ORIG_RAX: 0000000000000139
[ 4793.931839][ T44] RAX: ffffffffffffffda RBX: 0000557a8b230ac0 RCX: 00007fbfb3fafa0d
[ 4793.932219][ T44] RDX: 0000000000000000 RSI: 0000557a53d3fa2a RDI: 0000000000000005
[ 4793.932567][ T44] RBP: 0000000000040000 R08: 0000000000000000 R09: 00007ffde31648e0
[ 4793.932916][ T44] R10: 0000000000000005 R11: 0000000000000246 R12: 0000557a53d3fa2a
[ 4793.933278][ T44] R13: 0000557a8b230b60 R14: 0000557a8b230da0 R15: 0000000000000000
[ 4793.933645][ T44]
[ 4793.933826][ T44] INFO: task modprobe:29700 blocked on an rw-semaphore likely owned by task kworker/u16:0:12
[ 4793.934330][ T44] task:kworker/u16:0 state:R running task stack:24720 pid:12 tgid:12 ppid:2 task_flags:0x4208060 flags:0x00004000
[ 4793.934912][ T44] Workqueue: netns cleanup_net
[ 4793.935147][ T44] Call Trace:
[ 4793.935330][ T44]
[ 4793.935449][ T44] ? __mutex_lock+0x19f/0x11a0
[ 4793.935684][ T44] ? trace_sched_exit_tp+0xcd/0x130
[ 4793.935922][ T44] ? nf_ct_iterate_cleanup+0x4a/0x340 [nf_conntrack]
[ 4793.936247][ T44] ? ww_mutex_lock+0x160/0x160
[ 4793.936499][ T44] ? nf_ct_iterate_cleanup+0x4a/0x340 [nf_conntrack]
[ 4793.936801][ T44] ? nf_ct_iterate_cleanup+0x4a/0x340 [nf_conntrack]
[ 4793.937094][ T44] ? __nf_conntrack_hash_insert+0x200/0x200 [nf_conntrack]
[ 4793.937462][ T44] ? nf_conntrack_cleanup_net_list+0x19a/0x4b0 [nf_conntrack]
[ 4793.937814][ T44] ? nf_conntrack_cleanup_net_list+0x10e/0x4b0 [nf_conntrack]
[ 4793.938176][ T44] ? nf_conntrack_cleanup_end+0x70/0x70 [nf_conntrack]
[ 4793.938481][ T44] ? nf_conntrack_pernet_exit+0xa5/0x140 [nf_conntrack]
[ 4793.938785][ T44] ? ops_undo_list+0x2bf/0x890
[ 4793.939021][ T44] ? rtnl_net_dumpid_one+0x270/0x270
[ 4793.939259][ T44] ? cleanup_net+0x2c5/0x890
[ 4793.939499][ T44] ? cleanup_net+0x3a1/0x890
[ 4793.939733][ T44] ? net_passive_dec+0x190/0x190
[ 4793.939969][ T44] ? rcu_is_watching+0x12/0xb0
[ 4793.940225][ T44] ? process_one_work+0xe35/0x1650
[ 4793.940472][ T44] ? pwq_dec_nr_in_flight+0x550/0x550
[ 4793.940711][ T44] ? assign_work+0x168/0x240
[ 4793.940946][ T44] ? worker_thread+0x591/0xcf0
[ 4793.941192][ T44] ? rescuer_thread+0xd10/0xd10
[ 4793.941433][ T44] ? kthread+0x37b/0x5f0
[ 4793.941612][ T44] ? kthread_is_per_cpu+0xc0/0xc0
[ 4793.941848][ T44] ? ret_from_fork+0x1b/0x270
[ 4793.942080][ T44] ? __lock_release+0x5d/0x170
[ 4793.942344][ T44] ? rcu_is_watching+0x12/0xb0
[ 4793.942589][ T44] ? kthread_is_per_cpu+0xc0/0xc0
[ 4793.942822][ T44] ? ret_from_fork+0x1db/0x270
[ 4793.943052][ T44] ? kthread_is_per_cpu+0xc0/0xc0
[ 4793.943289][ T44] ? ret_from_fork_asm+0x11/0x20
[ 4793.943535][ T44]
[ 4793.943735][ T44]
[ 4793.943735][ T44] Showing all locks held in the system:
[ 4793.944070][ T44] 4 locks held by kworker/u16:0/12:
[ 4793.944698][ T44] 1 lock held by khungtaskd/44:
[ 4793.944921][ T44] #0: ffffffff9cb83f80 (rcu_read_lock){....}-{1:3}, at: debug_show_all_locks+0x36/0x260
[ 4793.945333][ T44] 3 locks held by kworker/2:1/15955:
[ 4793.945567][ T44] 1 lock held by modprobe/29700:
[ 4793.945790][ T44] #0: ffffffff9d270630 (pernet_ops_rwsem){++++}-{4:4}, at: register_pernet_subsys+0x1a/0x40
[ 4793.946245][ T44]
[ 4793.946355][ T44] =============================================
[ 4793.946355][ T44]
[ 4916.798244][ T44] INFO: task modprobe:29700 blocked for more than 368 seconds.
[ 4916.798776][ T44] Not tainted 6.17.0-rc6-virtme #1
[ 4916.798998][ T44] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[ 4916.799433][ T44] task:modprobe state:D stack:25712 pid:29700 tgid:29700 ppid:1 task_flags:0x400100 flags:0x00004006
[ 4916.799959][ T44] Call Trace:
[ 4916.800142][ T44]
[ 4916.800282][ T44] __schedule+0x880/0x1b10
[ 4916.800551][ T44] ? io_schedule_timeout+0x150/0x150
[ 4916.800797][ T44] ? schedule+0x1ee/0x270
[ 4916.800972][ T44] ? __lock_release+0x5d/0x170
[ 4916.801231][ T44] schedule+0xe2/0x270
[ 4916.801409][ T44] schedule_preempt_disabled+0x18/0x30
[ 4916.801642][ T44] rwsem_down_write_slowpath+0x3c5/0xcb0
[ 4916.801877][ T44] ? __lock_acquire+0x449/0x7e0
[ 4916.802118][ T44] ? down_read+0x4a0/0x4a0
[ 4916.802374][ T44] ? rcu_is_watching+0x12/0xb0
[ 4916.802612][ T44] ? lock_acquire+0x104/0x160
[ 4916.802846][ T44] ? register_pernet_subsys+0x1a/0x40
[ 4916.803086][ T44] down_write+0x1d6/0x200
[ 4916.803275][ T44] ? rwsem_down_write_slowpath+0xcb0/0xcb0
[ 4916.803566][ T44] ? find_held_lock+0x2b/0x80
[ 4916.803802][ T44] ? __create_object+0x5e/0xb0
[ 4916.804050][ T44] ? _sub_I_65535_1+0x200/0x200 [pktgen]
[ 4916.804299][ T44] register_pernet_subsys+0x1a/0x40
[ 4916.804533][ T44] pg_init+0x2c/0x1000 [pktgen]
[ 4916.804772][ T44] ? _sub_I_65535_1+0x200/0x200 [pktgen]
[ 4916.805008][ T44] do_one_initcall+0x8c/0x1d0
[ 4916.805259][ T44] ? trace_initcall_level+0x130/0x130
[ 4916.805495][ T44] ? rcu_is_watching+0x12/0xb0
[ 4916.805741][ T44] ? trace_kmalloc+0x26/0xd0
[ 4916.805973][ T44] ? __kmalloc_cache_noprof+0x15f/0x360
[ 4916.806213][ T44] ? debug_objects_oom+0x160/0x160
[ 4916.806447][ T44] ? kasan_unpoison+0x27/0x50
[ 4916.806685][ T44] ? __asan_register_globals+0x5d/0x70
[ 4916.806920][ T44] do_init_module+0x27a/0x860
[ 4916.807173][ T44] ? free_module+0x2f0/0x2f0
[ 4916.807415][ T44] ? kfree+0x144/0x320
[ 4916.807591][ T44] ? find_vm_area+0x12/0x50
[ 4916.807838][ T44] load_module+0x1f40/0x2dd0
[ 4916.808086][ T44] ? layout_and_allocate.constprop.0+0x630/0x630
[ 4916.808384][ T44] ? kernel_read_file+0x3ed/0x550
[ 4916.808621][ T44] ? kernel_read_file+0x3c8/0x550
[ 4916.808860][ T44] ? __ia32_sys_fsconfig+0x150/0x150
[ 4916.809098][ T44] ? init_module_from_file+0xe9/0x150
[ 4916.809344][ T44] init_module_from_file+0xe9/0x150
[ 4916.809577][ T44] ? __do_sys_init_module+0x1e0/0x1e0
[ 4916.809825][ T44] ? idempotent_init_module+0x316/0x610
[ 4916.810054][ T44] ? __lock_release+0x5d/0x170
[ 4916.810295][ T44] ? do_raw_spin_unlock+0x58/0x220
[ 4916.810532][ T44] idempotent_init_module+0x331/0x610
[ 4916.810770][ T44] ? init_module_from_file+0x150/0x150
[ 4916.811015][ T44] ? cap_capable+0x94/0x230
[ 4916.811269][ T44] __x64_sys_finit_module+0xca/0x150
[ 4916.811503][ T44] ? do_syscall_64+0x85/0x300
[ 4916.811739][ T44] do_syscall_64+0xc1/0x300
[ 4916.811976][ T44] entry_SYSCALL_64_after_hwframe+0x4b/0x53
[ 4916.812271][ T44] RIP: 0033:0x7fbfb3fafa0d
[ 4916.812518][ T44] RSP: 002b:00007ffde31647a8 EFLAGS: 00000246 ORIG_RAX: 0000000000000139
[ 4916.812873][ T44] RAX: ffffffffffffffda RBX: 0000557a8b230ac0 RCX: 00007fbfb3fafa0d
[ 4916.813229][ T44] RDX: 0000000000000000 RSI: 0000557a53d3fa2a RDI: 0000000000000005
[ 4916.813580][ T44] RBP: 0000000000040000 R08: 0000000000000000 R09: 00007ffde31648e0
[ 4916.813930][ T44] R10: 0000000000000005 R11: 0000000000000246 R12: 0000557a53d3fa2a
[ 4916.814293][ T44] R13: 0000557a8b230b60 R14: 0000557a8b230da0 R15: 0000000000000000
[ 4916.814657][ T44]
[ 4916.814841][ T44] INFO: task modprobe:29700 blocked on an rw-semaphore likely owned by task kworker/u16:0:12
[ 4916.815343][ T44] task:kworker/u16:0 state:R running task stack:24720 pid:12 tgid:12 ppid:2 task_flags:0x4208060 flags:0x00004000
[ 4916.815927][ T44] Workqueue: netns cleanup_net
[ 4916.816172][ T44] Call Trace:
[ 4916.816354][ T44]
[ 4916.816480][ T44] ? __mutex_trylock_common+0xf9/0x260
[ 4916.816717][ T44] ? __mutex_handoff+0x2b0/0x2b0
[ 4916.816952][ T44] ? rcu_is_watching+0x12/0xb0
[ 4916.817198][ T44] ? rcu_is_watching+0x12/0xb0
[ 4916.817440][ T44] ? trace_contention_end+0xd8/0x140
[ 4916.817676][ T44] ? __mutex_lock+0x19f/0x11a0
[ 4916.817915][ T44] ? trace_sched_exit_tp+0xcd/0x130
[ 4916.818165][ T44] ? nf_ct_iterate_cleanup+0x4a/0x340 [nf_conntrack]
[ 4916.818470][ T44] ? ww_mutex_lock+0x160/0x160
[ 4916.818719][ T44] ? nf_ct_iterate_cleanup+0x4a/0x340 [nf_conntrack]
[ 4916.819019][ T44] ? nf_ct_iterate_cleanup+0x4a/0x340 [nf_conntrack]
[ 4916.819324][ T44] ? __nf_conntrack_hash_insert+0x200/0x200 [nf_conntrack]
[ 4916.819685][ T44] ? nf_conntrack_cleanup_net_list+0x19a/0x4b0 [nf_conntrack]
[ 4916.820040][ T44] ? nf_conntrack_cleanup_net_list+0x10e/0x4b0 [nf_conntrack]
[ 4916.820403][ T44] ? nf_conntrack_cleanup_end+0x70/0x70 [nf_conntrack]
[ 4916.820707][ T44] ? nf_conntrack_pernet_exit+0xa5/0x140 [nf_conntrack]
[ 4916.821009][ T44] ? ops_undo_list+0x2bf/0x890
[ 4916.821279][ T44] ? rtnl_net_dumpid_one+0x270/0x270
[ 4916.821513][ T44] ? cleanup_net+0x2c5/0x890
[ 4916.821754][ T44] ? cleanup_net+0x3a1/0x890
[ 4916.821987][ T44] ? net_passive_dec+0x190/0x190
[ 4916.822231][ T44] ? rcu_is_watching+0x12/0xb0
[ 4916.822475][ T44] ? process_one_work+0xe35/0x1650
[ 4916.822730][ T44] ? pwq_dec_nr_in_flight+0x550/0x550
[ 4916.822973][ T44] ? assign_work+0x168/0x240
[ 4916.823218][ T44] ? worker_thread+0x591/0xcf0
[ 4916.823463][ T44] ? rescuer_thread+0xd10/0xd10
[ 4916.823700][ T44] ? kthread+0x37b/0x5f0
[ 4916.823877][ T44] ? kthread_is_per_cpu+0xc0/0xc0
[ 4916.824109][ T44] ? ret_from_fork+0x1b/0x270
[ 4916.824355][ T44] ? __lock_release+0x5d/0x170
[ 4916.824592][ T44] ? rcu_is_watching+0x12/0xb0
[ 4916.824825][ T44] ? kthread_is_per_cpu+0xc0/0xc0
[ 4916.825064][ T44] ? ret_from_fork+0x1db/0x270
[ 4916.825304][ T44] ? kthread_is_per_cpu+0xc0/0xc0
[ 4916.825538][ T44] ? ret_from_fork_asm+0x11/0x20
[ 4916.825787][ T44]
[ 4916.825978][ T44]
[ 4916.825978][ T44] Showing all locks held in the system:
[ 4916.826322][ T44] 4 locks held by kworker/u16:0/12:
[ 4916.826561][ T44] 1 lock held by khungtaskd/44:
[ 4916.826783][ T44] #0: ffffffff9cb83f80 (rcu_read_lock){....}-{1:3}, at: debug_show_all_locks+0x36/0x260
[ 4916.827201][ T44] 3 locks held by kworker/2:1/15955:
[ 4916.827438][ T44] 1 lock held by modprobe/29700:
[ 4916.827657][ T44] #0: ffffffff9d270630 (pernet_ops_rwsem){++++}-{4:4}, at: register_pernet_subsys+0x1a/0x40
[ 4916.828115][ T44]
[ 4916.828233][ T44] =============================================
[ 4916.828233][ T44]
[ 5039.678326][ T44] INFO: task modprobe:29700 blocked for more than 491 seconds.
[ 5039.678993][ T44] Not tainted 6.17.0-rc6-virtme #1
[ 5039.679293][ T44] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[ 5039.679728][ T44] task:modprobe state:D stack:25712 pid:29700 tgid:29700 ppid:1 task_flags:0x400100 flags:0x00004006
[ 5039.680336][ T44] Call Trace:
[ 5039.680535][ T44]
[ 5039.680680][ T44] __schedule+0x880/0x1b10
[ 5039.680966][ T44] ? io_schedule_timeout+0x150/0x150
[ 5039.681261][ T44] ? schedule+0x1ee/0x270
[ 5039.681479][ T44] ? __lock_release+0x5d/0x170
[ 5039.681750][ T44] schedule+0xe2/0x270
[ 5039.681943][ T44] schedule_preempt_disabled+0x18/0x30
[ 5039.682213][ T44] rwsem_down_write_slowpath+0x3c5/0xcb0
[ 5039.682488][ T44] ? __lock_acquire+0x449/0x7e0
[ 5039.682741][ T44] ? down_read+0x4a0/0x4a0
[ 5039.682999][ T44] ? rcu_is_watching+0x12/0xb0
[ 5039.683265][ T44] ? lock_acquire+0x104/0x160
[ 5039.683525][ T44] ? register_pernet_subsys+0x1a/0x40
[ 5039.683794][ T44] down_write+0x1d6/0x200
[ 5039.683993][ T44] ? rwsem_down_write_slowpath+0xcb0/0xcb0
[ 5039.684321][ T44] ? find_held_lock+0x2b/0x80
[ 5039.684577][ T44] ? __create_object+0x5e/0xb0
[ 5039.684835][ T44] ? _sub_I_65535_1+0x200/0x200 [pktgen]
[ 5039.685106][ T44] register_pernet_subsys+0x1a/0x40
[ 5039.685378][ T44] pg_init+0x2c/0x1000 [pktgen]
[ 5039.685635][ T44] ? _sub_I_65535_1+0x200/0x200 [pktgen]
[ 5039.685891][ T44] do_one_initcall+0x8c/0x1d0
[ 5039.686146][ T44] ? trace_initcall_level+0x130/0x130
[ 5039.686411][ T44] ? rcu_is_watching+0x12/0xb0
[ 5039.686664][ T44] ? trace_kmalloc+0x26/0xd0
[ 5039.686916][ T44] ? __kmalloc_cache_noprof+0x15f/0x360
[ 5039.687187][ T44] ? debug_objects_oom+0x160/0x160
[ 5039.687447][ T44] ? kasan_unpoison+0x27/0x50
[ 5039.687708][ T44] ? __asan_register_globals+0x5d/0x70
[ 5039.687961][ T44] do_init_module+0x27a/0x860
[ 5039.688242][ T44] ? free_module+0x2f0/0x2f0
[ 5039.688495][ T44] ? kfree+0x144/0x320
[ 5039.688685][ T44] ? find_vm_area+0x12/0x50
[ 5039.688943][ T44] load_module+0x1f40/0x2dd0
[ 5039.689238][ T44] ? layout_and_allocate.constprop.0+0x630/0x630
[ 5039.689560][ T44] ? kernel_read_file+0x3ed/0x550
[ 5039.689819][ T44] ? kernel_read_file+0x3c8/0x550
[ 5039.690076][ T44] ? __ia32_sys_fsconfig+0x150/0x150
[ 5039.690351][ T44] ? init_module_from_file+0xe9/0x150
[ 5039.690606][ T44] init_module_from_file+0xe9/0x150
[ 5039.690863][ T44] ? __do_sys_init_module+0x1e0/0x1e0
[ 5039.691139][ T44] ? idempotent_init_module+0x316/0x610
[ 5039.691396][ T44] ? __lock_release+0x5d/0x170
[ 5039.691651][ T44] ? do_raw_spin_unlock+0x58/0x220
[ 5039.691910][ T44] idempotent_init_module+0x331/0x610
[ 5039.692179][ T44] ? init_module_from_file+0x150/0x150
[ 5039.692461][ T44] ? cap_capable+0x94/0x230
[ 5039.692727][ T44] __x64_sys_finit_module+0xca/0x150
[ 5039.693000][ T44] ? do_syscall_64+0x85/0x300
[ 5039.693274][ T44] do_syscall_64+0xc1/0x300
[ 5039.693537][ T44] entry_SYSCALL_64_after_hwframe+0x4b/0x53
[ 5039.693860][ T44] RIP: 0033:0x7fbfb3fafa0d
[ 5039.694134][ T44] RSP: 002b:00007ffde31647a8 EFLAGS: 00000246 ORIG_RAX: 0000000000000139
[ 5039.694537][ T44] RAX: ffffffffffffffda RBX: 0000557a8b230ac0 RCX: 00007fbfb3fafa0d
[ 5039.694933][ T44] RDX: 0000000000000000 RSI: 0000557a53d3fa2a RDI: 0000000000000005
[ 5039.695320][ T44] RBP: 0000000000040000 R08: 0000000000000000 R09: 00007ffde31648e0
[ 5039.695708][ T44] R10: 0000000000000005 R11: 0000000000000246 R12: 0000557a53d3fa2a
[ 5039.696091][ T44] R13: 0000557a8b230b60 R14: 0000557a8b230da0 R15: 0000000000000000
[ 5039.696506][ T44]
[ 5039.696714][ T44] INFO: task modprobe:29700 blocked on an rw-semaphore likely owned by task kworker/u16:0:12
[ 5039.697268][ T44] task:kworker/u16:0 state:R running task stack:24720 pid:12 tgid:12 ppid:2 task_flags:0x4208060 flags:0x00004000
[ 5039.697905][ T44] Workqueue: netns cleanup_net
[ 5039.698187][ T44] Call Trace:
[ 5039.698376][ T44]
[ 5039.698503][ T44] ? __mutex_lock+0x19f/0x11a0
[ 5039.698768][ T44] ? trace_sched_exit_tp+0xcd/0x130
[ 5039.699029][ T44] ? nf_ct_iterate_cleanup+0x4a/0x340 [nf_conntrack]
[ 5039.699375][ T44] ? ww_mutex_lock+0x160/0x160
[ 5039.699653][ T44] ? nf_ct_iterate_cleanup+0x4a/0x340 [nf_conntrack]
[ 5039.699982][ T44] ? nf_ct_iterate_cleanup+0x4a/0x340 [nf_conntrack]
[ 5039.700339][ T44] ? __nf_conntrack_hash_insert+0x200/0x200 [nf_conntrack]
[ 5039.700733][ T44] ? nf_conntrack_cleanup_net_list+0x19a/0x4b0 [nf_conntrack]
[ 5039.701130][ T44] ? nf_conntrack_cleanup_net_list+0x10e/0x4b0 [nf_conntrack]
[ 5039.701528][ T44] ? nf_conntrack_cleanup_end+0x70/0x70 [nf_conntrack]
[ 5039.701855][ T44] ? nf_conntrack_pernet_exit+0xa5/0x140 [nf_conntrack]
[ 5039.702200][ T44] ? ops_undo_list+0x2bf/0x890
[ 5039.702462][ T44] ? rtnl_net_dumpid_one+0x270/0x270
[ 5039.702711][ T44] ? cleanup_net+0x2c5/0x890
[ 5039.702973][ T44] ? cleanup_net+0x3a1/0x890
[ 5039.703234][ T44] ? net_passive_dec+0x190/0x190
[ 5039.703488][ T44] ? rcu_is_watching+0x12/0xb0
[ 5039.703744][ T44] ? process_one_work+0xe35/0x1650
[ 5039.704011][ T44] ? pwq_dec_nr_in_flight+0x550/0x550
[ 5039.704281][ T44] ? assign_work+0x168/0x240
[ 5039.704537][ T44] ? worker_thread+0x591/0xcf0
[ 5039.704802][ T44] ? rescuer_thread+0xd10/0xd10
[ 5039.705058][ T44] ? kthread+0x37b/0x5f0
[ 5039.705270][ T44] ? kthread_is_per_cpu+0xc0/0xc0
[ 5039.705519][ T44] ? ret_from_fork+0x1b/0x270
[ 5039.705776][ T44] ? __lock_release+0x5d/0x170
[ 5039.706035][ T44] ? rcu_is_watching+0x12/0xb0
[ 5039.706295][ T44] ? kthread_is_per_cpu+0xc0/0xc0
[ 5039.706548][ T44] ? ret_from_fork+0x1db/0x270
[ 5039.706799][ T44] ? kthread_is_per_cpu+0xc0/0xc0
[ 5039.707054][ T44] ? ret_from_fork_asm+0x11/0x20
[ 5039.707326][ T44]
[ 5039.707538][ T44]
[ 5039.707538][ T44] Showing all locks held in the system:
[ 5039.707916][ T44] 4 locks held by kworker/u16:0/12:
[ 5039.708215][ T44] 1 lock held by khungtaskd/44:
[ 5039.708464][ T44] #0: ffffffff9cb83f80 (rcu_read_lock){....}-{1:3}, at: debug_show_all_locks+0x36/0x260
[ 5039.708930][ T44] 2 locks held by kworker/2:1/15955:
[ 5039.709207][ T44] 1 lock held by modprobe/29700:
[ 5039.709450][ T44] #0: ffffffff9d270630 (pernet_ops_rwsem){++++}-{4:4}, at: register_pernet_subsys+0x1a/0x40
[ 5039.709949][ T44]
[ 5039.710071][ T44] =============================================
[ 5039.710071][ T44]
[ 5162.558253][ T44] INFO: task modprobe:29700 blocked for more than 614 seconds.
[ 5162.558770][ T44] Not tainted 6.17.0-rc6-virtme #1
[ 5162.559011][ T44] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[ 5162.559458][ T44] task:modprobe state:D stack:25712 pid:29700 tgid:29700 ppid:1 task_flags:0x400100 flags:0x00004006
[ 5162.560000][ T44] Call Trace:
[ 5162.560200][ T44]
[ 5162.560337][ T44] __schedule+0x880/0x1b10
[ 5162.560669][ T44] ? io_schedule_timeout+0x150/0x150
[ 5162.560954][ T44] ? schedule+0x1ee/0x270
[ 5162.561146][ T44] ? __lock_release+0x5d/0x170
[ 5162.564417][ T44] schedule+0xe2/0x270
[ 5162.564611][ T44] schedule_preempt_disabled+0x18/0x30
[ 5162.564860][ T44] rwsem_down_write_slowpath+0x3c5/0xcb0
[ 5162.565105][ T44] ? __lock_acquire+0x449/0x7e0
[ 5162.565359][ T44] ? down_read+0x4a0/0x4a0
[ 5162.565612][ T44] ? rcu_is_watching+0x12/0xb0
[ 5162.565861][ T44] ? lock_acquire+0x104/0x160
[ 5162.566109][ T44] ? register_pernet_subsys+0x1a/0x40
[ 5162.569169][ T44] down_write+0x1d6/0x200
[ 5162.569376][ T44] ? rwsem_down_write_slowpath+0xcb0/0xcb0
[ 5162.569682][ T44] ? find_held_lock+0x2b/0x80
[ 5162.569929][ T44] ? __create_object+0x5e/0xb0
[ 5162.570191][ T44] ? _sub_I_65535_1+0x200/0x200 [pktgen]
[ 5162.570445][ T44] register_pernet_subsys+0x1a/0x40
[ 5162.570691][ T44] pg_init+0x2c/0x1000 [pktgen]
[ 5162.570943][ T44] ? _sub_I_65535_1+0x200/0x200 [pktgen]
[ 5162.571204][ T44] do_one_initcall+0x8c/0x1d0
[ 5162.571455][ T44] ? trace_initcall_level+0x130/0x130
[ 5162.571704][ T44] ? rcu_is_watching+0x12/0xb0
[ 5162.571954][ T44] ? trace_kmalloc+0x26/0xd0
[ 5162.575161][ T44] ? __kmalloc_cache_noprof+0x15f/0x360
[ 5162.575410][ T44] ? debug_objects_oom+0x160/0x160
[ 5162.575659][ T44] ? kasan_unpoison+0x27/0x50
[ 5162.575914][ T44] ? __asan_register_globals+0x5d/0x70
[ 5162.576169][ T44] do_init_module+0x27a/0x860
[ 5162.576426][ T44] ? free_module+0x2f0/0x2f0
[ 5162.576676][ T44] ? kfree+0x144/0x320
[ 5162.576862][ T44] ? find_vm_area+0x12/0x50
[ 5162.577119][ T44] load_module+0x1f40/0x2dd0
[ 5162.580174][ T44] ? layout_and_allocate.constprop.0+0x630/0x630
[ 5162.580482][ T44] ? kernel_read_file+0x3ed/0x550
[ 5162.580741][ T44] ? kernel_read_file+0x3c8/0x550
[ 5162.580988][ T44] ? __ia32_sys_fsconfig+0x150/0x150
[ 5162.581251][ T44] ? init_module_from_file+0xe9/0x150
[ 5162.581497][ T44] init_module_from_file+0xe9/0x150
[ 5162.581742][ T44] ? __do_sys_init_module+0x1e0/0x1e0
[ 5162.582000][ T44] ? idempotent_init_module+0x316/0x610
[ 5162.582253][ T44] ? __lock_release+0x5d/0x170
[ 5162.582506][ T44] ? do_raw_spin_unlock+0x58/0x220
[ 5162.582752][ T44] idempotent_init_module+0x331/0x610
[ 5162.582997][ T44] ? init_module_from_file+0x150/0x150
[ 5162.586173][ T44] ? cap_capable+0x94/0x230
[ 5162.586431][ T44] __x64_sys_finit_module+0xca/0x150
[ 5162.586676][ T44] ? do_syscall_64+0x85/0x300
[ 5162.586923][ T44] do_syscall_64+0xc1/0x300
[ 5162.587183][ T44] entry_SYSCALL_64_after_hwframe+0x4b/0x53
[ 5162.587485][ T44] RIP: 0033:0x7fbfb3fafa0d
[ 5162.587738][ T44] RSP: 002b:00007ffde31647a8 EFLAGS: 00000246 ORIG_RAX: 0000000000000139
[ 5162.588102][ T44] RAX: ffffffffffffffda RBX: 0000557a8b230ac0 RCX: 00007fbfb3fafa0d
[ 5162.591169][ T44] RDX: 0000000000000000 RSI: 0000557a53d3fa2a RDI: 0000000000000005
[ 5162.591790][ T44] RBP: 0000000000040000 R08: 0000000000000000 R09: 00007ffde31648e0
[ 5162.592168][ T44] R10: 0000000000000005 R11: 0000000000000246 R12: 0000557a53d3fa2a
[ 5162.592532][ T44] R13: 0000557a8b230b60 R14: 0000557a8b230da0 R15: 0000000000000000
[ 5162.592908][ T44]
[ 5162.593104][ T44] INFO: task modprobe:29700 blocked on an rw-semaphore likely owned by task kworker/u16:0:12
[ 5162.596161][ T44] task:kworker/u16:0 state:R running task stack:24720 pid:12 tgid:12 ppid:2 task_flags:0x4208060 flags:0x00004000
[ 5162.596751][ T44] Workqueue: netns cleanup_net
[ 5162.597002][ T44] Call Trace:
[ 5162.597200][ T44]
[ 5162.597333][ T44] ? __mutex_lock+0x19f/0x11a0
[ 5162.597589][ T44] ? trace_sched_exit_tp+0xcd/0x130
[ 5162.597845][ T44] ? nf_ct_iterate_cleanup+0x4a/0x340 [nf_conntrack]
[ 5162.598170][ T44] ? irqentry_exit+0x3b/0x80
[ 5162.598436][ T44] ? nf_ct_iterate_cleanup+0x4a/0x340 [nf_conntrack]
[ 5162.598743][ T44] ? nf_ct_iterate_cleanup+0x4a/0x340 [nf_conntrack]
[ 5162.599049][ T44] ? __nf_conntrack_hash_insert+0x200/0x200 [nf_conntrack]
[ 5162.602166][ T44] ? nf_conntrack_cleanup_net_list+0x19a/0x4b0 [nf_conntrack]
[ 5162.602538][ T44] ? nf_conntrack_cleanup_net_list+0x10e/0x4b0 [nf_conntrack]
[ 5162.602906][ T44] ? nf_conntrack_cleanup_end+0x70/0x70 [nf_conntrack]
[ 5162.603227][ T44] ? nf_conntrack_pernet_exit+0xa5/0x140 [nf_conntrack]
[ 5162.603557][ T44] ? ops_undo_list+0x2bf/0x890
[ 5162.603813][ T44] ? rtnl_net_dumpid_one+0x270/0x270
[ 5162.604057][ T44] ? cleanup_net+0x2c5/0x890
[ 5162.606179][ T44] ? cleanup_net+0x3a1/0x890
[ 5162.606446][ T44] ? net_passive_dec+0x190/0x190
[ 5162.606680][ T44] ? rcu_is_watching+0x12/0xb0
[ 5162.606917][ T44] ? process_one_work+0xe35/0x1650
[ 5162.607178][ T44] ? pwq_dec_nr_in_flight+0x550/0x550
[ 5162.607418][ T44] ? assign_work+0x168/0x240
[ 5162.607661][ T44] ? worker_thread+0x591/0xcf0
[ 5162.607898][ T44] ? rescuer_thread+0xd10/0xd10
[ 5162.608130][ T44] ? kthread+0x37b/0x5f0
[ 5162.608327][ T44] ? kthread_is_per_cpu+0xc0/0xc0
[ 5162.608560][ T44] ? ret_from_fork+0x1b/0x270
[ 5162.608792][ T44] ? __lock_release+0x5d/0x170
[ 5162.609024][ T44] ? rcu_is_watching+0x12/0xb0
[ 5162.609282][ T44] ? kthread_is_per_cpu+0xc0/0xc0
[ 5162.609532][ T44] ? ret_from_fork+0x1db/0x270
[ 5162.609764][ T44] ? kthread_is_per_cpu+0xc0/0xc0
[ 5162.609996][ T44] ? ret_from_fork_asm+0x11/0x20
[ 5162.610251][ T44]
[ 5162.610459][ T44]
[ 5162.610459][ T44] Showing all locks held in the system:
[ 5162.610793][ T44] 4 locks held by kworker/u16:0/12:
[ 5162.611034][ T44] 1 lock held by khungtaskd/44:
[ 5162.611266][ T44] #0: ffffffff9cb83f80 (rcu_read_lock){....}-{1:3}, at: debug_show_all_locks+0x36/0x260
[ 5162.611702][ T44] 1 lock held by modprobe/29700:
[ 5162.611921][ T44] #0: ffffffff9d270630 (pernet_ops_rwsem){++++}-{4:4}, at: register_pernet_subsys+0x1a/0x40
[ 5162.612387][ T44]
[ 5162.612500][ T44] =============================================
[ 5162.612500][ T44]
[ 5285.437281][ T44] INFO: task modprobe:29700 blocked for more than 737 seconds.
[ 5285.437877][ T44] Not tainted 6.17.0-rc6-virtme #1
[ 5285.438111][ T44] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[ 5285.438554][ T44] task:modprobe state:D stack:25712 pid:29700 tgid:29700 ppid:1 task_flags:0x400100 flags:0x00004006
[ 5285.439128][ T44] Call Trace:
[ 5285.439380][ T44]
[ 5285.439535][ T44] __schedule+0x880/0x1b10
[ 5285.439812][ T44] ? io_schedule_timeout+0x150/0x150
[ 5285.440083][ T44] ? schedule+0x1ee/0x270
[ 5285.440278][ T44] ? __lock_release+0x5d/0x170
[ 5285.440541][ T44] schedule+0xe2/0x270
[ 5285.440721][ T44] schedule_preempt_disabled+0x18/0x30
[ 5285.440963][ T44] rwsem_down_write_slowpath+0x3c5/0xcb0
[ 5285.441227][ T44] ? __lock_acquire+0x449/0x7e0
[ 5285.441467][ T44] ? down_read+0x4a0/0x4a0
[ 5285.441727][ T44] ? rcu_is_watching+0x12/0xb0
[ 5285.441971][ T44] ? lock_acquire+0x104/0x160
[ 5285.442218][ T44] ? register_pernet_subsys+0x1a/0x40
[ 5285.442479][ T44] down_write+0x1d6/0x200
[ 5285.442671][ T44] ? rwsem_down_write_slowpath+0xcb0/0xcb0
[ 5285.442962][ T44] ? find_held_lock+0x2b/0x80
[ 5285.443232][ T44] ? __create_object+0x5e/0xb0
[ 5285.443481][ T44] ? _sub_I_65535_1+0x200/0x200 [pktgen]
[ 5285.443733][ T44] register_pernet_subsys+0x1a/0x40
[ 5285.443990][ T44] pg_init+0x2c/0x1000 [pktgen]
[ 5285.444251][ T44] ? _sub_I_65535_1+0x200/0x200 [pktgen]
[ 5285.444504][ T44] do_one_initcall+0x8c/0x1d0
[ 5285.444747][ T44] ? trace_initcall_level+0x130/0x130
[ 5285.444987][ T44] ? rcu_is_watching+0x12/0xb0
[ 5285.445232][ T44] ? trace_kmalloc+0x26/0xd0
[ 5285.445469][ T44] ? __kmalloc_cache_noprof+0x15f/0x360
[ 5285.445703][ T44] ? debug_objects_oom+0x160/0x160
[ 5285.445952][ T44] ? kasan_unpoison+0x27/0x50
[ 5285.446223][ T44] ? __asan_register_globals+0x5d/0x70
[ 5285.446475][ T44] do_init_module+0x27a/0x860
[ 5285.446723][ T44] ? free_module+0x2f0/0x2f0
[ 5285.446965][ T44] ? kfree+0x144/0x320
[ 5285.447181][ T44] ? find_vm_area+0x12/0x50
[ 5285.447439][ T44] load_module+0x1f40/0x2dd0
[ 5285.447695][ T44] ? layout_and_allocate.constprop.0+0x630/0x630
[ 5285.447991][ T44] ? kernel_read_file+0x3ed/0x550
[ 5285.448266][ T44] ? kernel_read_file+0x3c8/0x550
[ 5285.448531][ T44] ? __ia32_sys_fsconfig+0x150/0x150
[ 5285.448784][ T44] ? init_module_from_file+0xe9/0x150
[ 5285.449025][ T44] init_module_from_file+0xe9/0x150
[ 5285.449275][ T44] ? __do_sys_init_module+0x1e0/0x1e0
[ 5285.449533][ T44] ? idempotent_init_module+0x316/0x610
[ 5285.449779][ T44] ? __lock_release+0x5d/0x170
[ 5285.450019][ T44] ? do_raw_spin_unlock+0x58/0x220
[ 5285.450283][ T44] idempotent_init_module+0x331/0x610
[ 5285.450522][ T44] ? init_module_from_file+0x150/0x150
[ 5285.450769][ T44] ? cap_capable+0x94/0x230
[ 5285.451012][ T44] __x64_sys_finit_module+0xca/0x150
[ 5285.451274][ T44] ? do_syscall_64+0x85/0x300
[ 5285.451533][ T44] do_syscall_64+0xc1/0x300
[ 5285.451781][ T44] entry_SYSCALL_64_after_hwframe+0x4b/0x53
[ 5285.452081][ T44] RIP: 0033:0x7fbfb3fafa0d
[ 5285.452340][ T44] RSP: 002b:00007ffde31647a8 EFLAGS: 00000246 ORIG_RAX: 0000000000000139
[ 5285.452702][ T44] RAX: ffffffffffffffda RBX: 0000557a8b230ac0 RCX: 00007fbfb3fafa0d
[ 5285.453054][ T44] RDX: 0000000000000000 RSI: 0000557a53d3fa2a RDI: 0000000000000005
[ 5285.453420][ T44] RBP: 0000000000040000 R08: 0000000000000000 R09: 00007ffde31648e0
[ 5285.453771][ T44] R10: 0000000000000005 R11: 0000000000000246 R12: 0000557a53d3fa2a
[ 5285.454139][ T44] R13: 0000557a8b230b60 R14: 0000557a8b230da0 R15: 0000000000000000
[ 5285.454516][ T44]
[ 5285.454697][ T44] INFO: task modprobe:29700 blocked on an rw-semaphore likely owned by task kworker/u16:0:12
[ 5285.455266][ T44] task:kworker/u16:0 state:R running task stack:24720 pid:12 tgid:12 ppid:2 task_flags:0x4208060 flags:0x00004000
[ 5285.455851][ T44] Workqueue: netns cleanup_net
[ 5285.456091][ T44] Call Trace:
[ 5285.456277][ T44]
[ 5285.456404][ T44] ? __mutex_trylock_common+0xf9/0x260
[ 5285.456655][ T44] ? __mutex_handoff+0x2b0/0x2b0
[ 5285.456891][ T44] ? rcu_is_watching+0x12/0xb0
[ 5285.457129][ T44] ? rcu_is_watching+0x12/0xb0
[ 5285.457372][ T44] ? trace_contention_end+0xd8/0x140
[ 5285.457608][ T44] ? __mutex_lock+0x19f/0x11a0
[ 5285.457844][ T44] ? trace_sched_exit_tp+0xcd/0x130
[ 5285.458082][ T44] ? nf_ct_iterate_cleanup+0x4a/0x340 [nf_conntrack]
[ 5285.458414][ T44] ? ww_mutex_lock+0x160/0x160
[ 5285.458667][ T44] ? nf_ct_iterate_cleanup+0x4a/0x340 [nf_conntrack]
[ 5285.458965][ T44] ? nf_ct_iterate_cleanup+0x4a/0x340 [nf_conntrack]
[ 5285.459286][ T44] ? __nf_conntrack_hash_insert+0x200/0x200 [nf_conntrack]
[ 5285.459645][ T44] ? nf_conntrack_cleanup_net_list+0x19a/0x4b0 [nf_conntrack]
[ 5285.460006][ T44] ? nf_conntrack_cleanup_net_list+0x10e/0x4b0 [nf_conntrack]
[ 5285.460371][ T44] ? nf_conntrack_cleanup_end+0x70/0x70 [nf_conntrack]
[ 5285.460676][ T44] ? nf_conntrack_pernet_exit+0xa5/0x140 [nf_conntrack]
[ 5285.460993][ T44] ? ops_undo_list+0x2bf/0x890
[ 5285.461244][ T44] ? rtnl_net_dumpid_one+0x270/0x270
[ 5285.461492][ T44] ? cleanup_net+0x2c5/0x890
[ 5285.461744][ T44] ? cleanup_net+0x3a1/0x890
[ 5285.461980][ T44] ? net_passive_dec+0x190/0x190
[ 5285.462226][ T44] ? rcu_is_watching+0x12/0xb0
[ 5285.462470][ T44] ? process_one_work+0xe35/0x1650
[ 5285.462718][ T44] ? pwq_dec_nr_in_flight+0x550/0x550
[ 5285.462969][ T44] ? assign_work+0x168/0x240
[ 5285.463231][ T44] ? worker_thread+0x591/0xcf0
[ 5285.463475][ T44] ? rescuer_thread+0xd10/0xd10
[ 5285.463724][ T44] ? kthread+0x37b/0x5f0
[ 5285.463912][ T44] ? kthread_is_per_cpu+0xc0/0xc0
[ 5285.464148][ T44] ? ret_from_fork+0x1b/0x270
[ 5285.464396][ T44] ? __lock_release+0x5d/0x170
[ 5285.464637][ T44] ? rcu_is_watching+0x12/0xb0
[ 5285.464870][ T44] ? kthread_is_per_cpu+0xc0/0xc0
[ 5285.465107][ T44] ? ret_from_fork+0x1db/0x270
[ 5285.465345][ T44] ? kthread_is_per_cpu+0xc0/0xc0
[ 5285.465599][ T44] ? ret_from_fork_asm+0x11/0x20
[ 5285.465854][ T44]
[ 5285.466078][ T44]
[ 5285.466078][ T44] Showing all locks held in the system:
[ 5285.466659][ T44] 4 locks held by kworker/u16:0/12:
[ 5285.466903][ T44] 1 lock held by khungtaskd/44:
[ 5285.467128][ T44] #0: ffffffff9cb83f80 (rcu_read_lock){....}-{1:3}, at: debug_show_all_locks+0x36/0x260
[ 5285.467557][ T44] 3 locks held by kworker/2:1/15955:
[ 5285.467803][ T44] 1 lock held by modprobe/29700:
[ 5285.468028][ T44] #0: ffffffff9d270630 (pernet_ops_rwsem){++++}-{4:4}, at: register_pernet_subsys+0x1a/0x40
[ 5285.468507][ T44]
[ 5285.468621][ T44] =============================================
[ 5285.468621][ T44]
[ 5408.318272][ T44] INFO: task modprobe:29700 blocked for more than 860 seconds.
[ 5408.318792][ T44] Not tainted 6.17.0-rc6-virtme #1
[ 5408.319021][ T44] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[ 5408.319448][ T44] task:modprobe state:D stack:25712 pid:29700 tgid:29700 ppid:1 task_flags:0x400100 flags:0x00004006
[ 5408.319989][ T44] Call Trace:
[ 5408.320177][ T44]
[ 5408.320305][ T44] __schedule+0x880/0x1b10
[ 5408.320566][ T44] ? io_schedule_timeout+0x150/0x150
[ 5408.320826][ T44] ? schedule+0x1ee/0x270
[ 5408.321002][ T44] ? __lock_release+0x5d/0x170
[ 5408.321272][ T44] schedule+0xe2/0x270
[ 5408.321460][ T44] schedule_preempt_disabled+0x18/0x30
[ 5408.321690][ T44] rwsem_down_write_slowpath+0x3c5/0xcb0
[ 5408.321926][ T44] ? __lock_acquire+0x449/0x7e0
[ 5408.322171][ T44] ? down_read+0x4a0/0x4a0
[ 5408.322419][ T44] ? rcu_is_watching+0x12/0xb0
[ 5408.322660][ T44] ? lock_acquire+0x104/0x160
[ 5408.322895][ T44] ? register_pernet_subsys+0x1a/0x40
[ 5408.323138][ T44] down_write+0x1d6/0x200
[ 5408.323324][ T44] ? rwsem_down_write_slowpath+0xcb0/0xcb0
[ 5408.323614][ T44] ? find_held_lock+0x2b/0x80
[ 5408.323845][ T44] ? __create_object+0x5e/0xb0
[ 5408.324081][ T44] ? _sub_I_65535_1+0x200/0x200 [pktgen]
[ 5408.324341][ T44] register_pernet_subsys+0x1a/0x40
[ 5408.324576][ T44] pg_init+0x2c/0x1000 [pktgen]
[ 5408.324812][ T44] ? _sub_I_65535_1+0x200/0x200 [pktgen]
[ 5408.325045][ T44] do_one_initcall+0x8c/0x1d0
[ 5408.325307][ T44] ? trace_initcall_level+0x130/0x130
[ 5408.325544][ T44] ? rcu_is_watching+0x12/0xb0
[ 5408.325777][ T44] ? trace_kmalloc+0x26/0xd0
[ 5408.326033][ T44] ? __kmalloc_cache_noprof+0x15f/0x360
[ 5408.326273][ T44] ? debug_objects_oom+0x160/0x160
[ 5408.326509][ T44] ? kasan_unpoison+0x27/0x50
[ 5408.326746][ T44] ? __asan_register_globals+0x5d/0x70
[ 5408.326984][ T44] do_init_module+0x27a/0x860
[ 5408.327231][ T44] ? free_module+0x2f0/0x2f0
[ 5408.327473][ T44] ? kfree+0x144/0x320
[ 5408.327651][ T44] ? find_vm_area+0x12/0x50
[ 5408.327892][ T44] load_module+0x1f40/0x2dd0
[ 5408.328140][ T44] ? layout_and_allocate.constprop.0+0x630/0x630
[ 5408.328443][ T44] ? kernel_read_file+0x3ed/0x550
[ 5408.328682][ T44] ? kernel_read_file+0x3c8/0x550
[ 5408.328921][ T44] ? __ia32_sys_fsconfig+0x150/0x150
[ 5408.329184][ T44] ? init_module_from_file+0xe9/0x150
[ 5408.329418][ T44] init_module_from_file+0xe9/0x150
[ 5408.329655][ T44] ? __do_sys_init_module+0x1e0/0x1e0
[ 5408.329903][ T44] ? idempotent_init_module+0x316/0x610
[ 5408.330135][ T44] ? __lock_release+0x5d/0x170
[ 5408.330378][ T44] ? do_raw_spin_unlock+0x58/0x220
[ 5408.330613][ T44] idempotent_init_module+0x331/0x610
[ 5408.330848][ T44] ? init_module_from_file+0x150/0x150
[ 5408.331097][ T44] ? cap_capable+0x94/0x230
[ 5408.331346][ T44] __x64_sys_finit_module+0xca/0x150
[ 5408.331580][ T44] ? do_syscall_64+0x85/0x300
[ 5408.331815][ T44] do_syscall_64+0xc1/0x300
[ 5408.332049][ T44] entry_SYSCALL_64_after_hwframe+0x4b/0x53
[ 5408.332343][ T44] RIP: 0033:0x7fbfb3fafa0d
[ 5408.332585][ T44] RSP: 002b:00007ffde31647a8 EFLAGS: 00000246 ORIG_RAX: 0000000000000139
[ 5408.332946][ T44] RAX: ffffffffffffffda RBX: 0000557a8b230ac0 RCX: 00007fbfb3fafa0d
[ 5408.333327][ T44] RDX: 0000000000000000 RSI: 0000557a53d3fa2a RDI: 0000000000000005
[ 5408.333677][ T44] RBP: 0000000000040000 R08: 0000000000000000 R09: 00007ffde31648e0
[ 5408.334025][ T44] R10: 0000000000000005 R11: 0000000000000246 R12: 0000557a53d3fa2a
[ 5408.334387][ T44] R13: 0000557a8b230b60 R14: 0000557a8b230da0 R15: 0000000000000000
[ 5408.334753][ T44]
[ 5408.334931][ T44] INFO: task modprobe:29700 blocked on an rw-semaphore likely owned by task kworker/u16:0:12
[ 5408.335436][ T44] task:kworker/u16:0 state:R running task stack:24720 pid:12 tgid:12 ppid:2 task_flags:0x4208060 flags:0x00004000
[ 5408.336013][ T44] Workqueue: netns cleanup_net
[ 5408.336259][ T44] Call Trace:
[ 5408.336434][ T44]
[ 5408.336554][ T44] ? __hrtimer_start_range_ns+0x34b/0xbf0
[ 5408.336793][ T44] ? find_held_lock+0x2b/0x80
[ 5408.337025][ T44] ? start_dl_timer+0x348/0x510
[ 5408.337280][ T44] ? __lock_release+0x5d/0x170
[ 5408.337521][ T44] ? rcu_read_lock_any_held+0x3f/0xa0
[ 5408.337758][ T44] ? validate_chain+0x15e/0x4d0
[ 5408.337996][ T44] ? __lock_acquire+0x449/0x7e0
[ 5408.338243][ T44] ? __mutex_trylock_common+0xf9/0x260
[ 5408.338481][ T44] ? __mutex_handoff+0x2b0/0x2b0
[ 5408.338714][ T44] ? rcu_is_watching+0x12/0xb0
[ 5408.338951][ T44] ? rcu_is_watching+0x12/0xb0
[ 5408.339192][ T44] ? trace_contention_end+0xd8/0x140
[ 5408.339427][ T44] ? __mutex_lock+0x19f/0x11a0
[ 5408.339666][ T44] ? trace_sched_exit_tp+0xcd/0x130
[ 5408.339903][ T44] ? nf_ct_iterate_cleanup+0x4a/0x340 [nf_conntrack]
[ 5408.340213][ T44] ? ww_mutex_lock+0x160/0x160
[ 5408.340466][ T44] ? nf_ct_iterate_cleanup+0x4a/0x340 [nf_conntrack]
[ 5408.340759][ T44] ? nf_ct_iterate_cleanup+0x4a/0x340 [nf_conntrack]
[ 5408.341053][ T44] ? __nf_conntrack_hash_insert+0x200/0x200 [nf_conntrack]
[ 5408.341435][ T44] ? nf_conntrack_cleanup_net_list+0x19a/0x4b0 [nf_conntrack]
[ 5408.341790][ T44] ? nf_conntrack_cleanup_net_list+0x10e/0x4b0 [nf_conntrack]
[ 5408.342143][ T44] ? nf_conntrack_cleanup_end+0x70/0x70 [nf_conntrack]
[ 5408.342450][ T44] ? nf_conntrack_pernet_exit+0xa5/0x140 [nf_conntrack]
[ 5408.342754][ T44] ? ops_undo_list+0x2bf/0x890
[ 5408.342996][ T44] ? rtnl_net_dumpid_one+0x270/0x270
[ 5408.343232][ T44] ? cleanup_net+0x2c5/0x890
[ 5408.343474][ T44] ? cleanup_net+0x3a1/0x890
[ 5408.343705][ T44] ? net_passive_dec+0x190/0x190
[ 5408.343941][ T44] ? rcu_is_watching+0x12/0xb0
[ 5408.344180][ T44] ? process_one_work+0xe35/0x1650
[ 5408.344425][ T44] ? pwq_dec_nr_in_flight+0x550/0x550
[ 5408.344670][ T44] ? assign_work+0x168/0x240
[ 5408.344905][ T44] ? worker_thread+0x591/0xcf0
[ 5408.345147][ T44] ? rescuer_thread+0xd10/0xd10
[ 5408.345392][ T44] ? kthread+0x37b/0x5f0
[ 5408.345572][ T44] ? kthread_is_per_cpu+0xc0/0xc0
[ 5408.345804][ T44] ? ret_from_fork+0x1b/0x270
[ 5408.346038][ T44] ? __lock_release+0x5d/0x170
[ 5408.346294][ T44] ? rcu_is_watching+0x12/0xb0
[ 5408.346544][ T44] ? kthread_is_per_cpu+0xc0/0xc0
[ 5408.346779][ T44] ? ret_from_fork+0x1db/0x270
[ 5408.347013][ T44] ? kthread_is_per_cpu+0xc0/0xc0
[ 5408.347257][ T44] ? ret_from_fork_asm+0x11/0x20
[ 5408.347514][ T44]
[ 5408.347731][ T44]
[ 5408.347731][ T44] Showing all locks held in the system:
[ 5408.348068][ T44] 4 locks held by kworker/u16:0/12:
[ 5408.348674][ T44] 1 lock held by khungtaskd/44:
[ 5408.348905][ T44] #0: ffffffff9cb83f80 (rcu_read_lock){....}-{1:3}, at: debug_show_all_locks+0x36/0x260
[ 5408.349321][ T44] 2 locks held by kworker/2:1/15955:
[ 5408.349551][ T44] 1 lock held by modprobe/29700:
[ 5408.349773][ T44] #0: ffffffff9d270630 (pernet_ops_rwsem){++++}-{4:4}, at: register_pernet_subsys+0x1a/0x40
[ 5408.350235][ T44]
[ 5408.350347][ T44] =============================================
[ 5408.350347][ T44]
[ 5531.198294][ T44] INFO: task modprobe:29700 blocked for more than 983 seconds.
[ 5531.198856][ T44] Not tainted 6.17.0-rc6-virtme #1
[ 5531.199082][ T44] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[ 5531.199495][ T44] task:modprobe state:D stack:25712 pid:29700 tgid:29700 ppid:1 task_flags:0x400100 flags:0x00004006
[ 5531.200036][ T44] Call Trace:
[ 5531.200231][ T44]
[ 5531.200364][ T44] __schedule+0x880/0x1b10
[ 5531.200620][ T44] ? io_schedule_timeout+0x150/0x150
[ 5531.200869][ T44] ? schedule+0x1ee/0x270
[ 5531.201057][ T44] ? __lock_release+0x5d/0x170
[ 5531.201317][ T44] schedule+0xe2/0x270
[ 5531.201498][ T44] schedule_preempt_disabled+0x18/0x30
[ 5531.201745][ T44] rwsem_down_write_slowpath+0x3c5/0xcb0
[ 5531.201982][ T44] ? __lock_acquire+0x449/0x7e0
[ 5531.202228][ T44] ? down_read+0x4a0/0x4a0
[ 5531.202486][ T44] ? rcu_is_watching+0x12/0xb0
[ 5531.202745][ T44] ? lock_acquire+0x104/0x160
[ 5531.202981][ T44] ? register_pernet_subsys+0x1a/0x40
[ 5531.203242][ T44] down_write+0x1d6/0x200
[ 5531.203424][ T44] ? rwsem_down_write_slowpath+0xcb0/0xcb0
[ 5531.203721][ T44] ? find_held_lock+0x2b/0x80
[ 5531.203958][ T44] ? __create_object+0x5e/0xb0
[ 5531.204209][ T44] ? _sub_I_65535_1+0x200/0x200 [pktgen]
[ 5531.204462][ T44] register_pernet_subsys+0x1a/0x40
[ 5531.204700][ T44] pg_init+0x2c/0x1000 [pktgen]
[ 5531.204938][ T44] ? _sub_I_65535_1+0x200/0x200 [pktgen]
[ 5531.205184][ T44] do_one_initcall+0x8c/0x1d0
[ 5531.205425][ T44] ? trace_initcall_level+0x130/0x130
[ 5531.205664][ T44] ? rcu_is_watching+0x12/0xb0
[ 5531.205900][ T44] ? trace_kmalloc+0x26/0xd0
[ 5531.206161][ T44] ? __kmalloc_cache_noprof+0x15f/0x360
[ 5531.206397][ T44] ? debug_objects_oom+0x160/0x160
[ 5531.206645][ T44] ? kasan_unpoison+0x27/0x50
[ 5531.206884][ T44] ? __asan_register_globals+0x5d/0x70
[ 5531.207117][ T44] do_init_module+0x27a/0x860
[ 5531.207375][ T44] ? free_module+0x2f0/0x2f0
[ 5531.207610][ T44] ? kfree+0x144/0x320
[ 5531.207787][ T44] ? find_vm_area+0x12/0x50
[ 5531.208039][ T44] load_module+0x1f40/0x2dd0
[ 5531.208297][ T44] ? layout_and_allocate.constprop.0+0x630/0x630
[ 5531.208592][ T44] ? kernel_read_file+0x3ed/0x550
[ 5531.208832][ T44] ? kernel_read_file+0x3c8/0x550
[ 5531.209069][ T44] ? __ia32_sys_fsconfig+0x150/0x150
[ 5531.209322][ T44] ? init_module_from_file+0xe9/0x150
[ 5531.209556][ T44] init_module_from_file+0xe9/0x150
[ 5531.209803][ T44] ? __do_sys_init_module+0x1e0/0x1e0
[ 5531.210054][ T44] ? idempotent_init_module+0x316/0x610
[ 5531.210294][ T44] ? __lock_release+0x5d/0x170
[ 5531.210533][ T44] ? do_raw_spin_unlock+0x58/0x220
[ 5531.210773][ T44] idempotent_init_module+0x331/0x610
[ 5531.211010][ T44] ? init_module_from_file+0x150/0x150
[ 5531.211272][ T44] ? cap_capable+0x94/0x230
[ 5531.211516][ T44] __x64_sys_finit_module+0xca/0x150
[ 5531.211755][ T44] ? do_syscall_64+0x85/0x300
[ 5531.211993][ T44] do_syscall_64+0xc1/0x300
[ 5531.212237][ T44] entry_SYSCALL_64_after_hwframe+0x4b/0x53
[ 5531.212528][ T44] RIP: 0033:0x7fbfb3fafa0d
[ 5531.212773][ T44] RSP: 002b:00007ffde31647a8 EFLAGS: 00000246 ORIG_RAX: 0000000000000139
[ 5531.213137][ T44] RAX: ffffffffffffffda RBX: 0000557a8b230ac0 RCX: 00007fbfb3fafa0d
[ 5531.213500][ T44] RDX: 0000000000000000 RSI: 0000557a53d3fa2a RDI: 0000000000000005
[ 5531.213854][ T44] RBP: 0000000000040000 R08: 0000000000000000 R09: 00007ffde31648e0
[ 5531.214217][ T44] R10: 0000000000000005 R11: 0000000000000246 R12: 0000557a53d3fa2a
[ 5531.214578][ T44] R13: 0000557a8b230b60 R14: 0000557a8b230da0 R15: 0000000000000000
[ 5531.214949][ T44]
[ 5531.215134][ T44] INFO: task modprobe:29700 blocked on an rw-semaphore likely owned by task kworker/u16:0:12
[ 5531.215648][ T44] task:kworker/u16:0 state:R running task stack:24720 pid:12 tgid:12 ppid:2 task_flags:0x4208060 flags:0x00004000
[ 5531.216237][ T44] Workqueue: netns cleanup_net
[ 5531.216482][ T44] Call Trace:
[ 5531.216658][ T44]
[ 5531.216783][ T44] ? __mutex_trylock_common+0xf9/0x260
[ 5531.217024][ T44] ? __mutex_handoff+0x2b0/0x2b0
[ 5531.217269][ T44] ? rcu_is_watching+0x12/0xb0
[ 5531.217512][ T44] ? rcu_is_watching+0x12/0xb0
[ 5531.217747][ T44] ? trace_contention_end+0xd8/0x140
[ 5531.217993][ T44] ? __mutex_lock+0x19f/0x11a0
[ 5531.218236][ T44] ? trace_sched_exit_tp+0xcd/0x130
[ 5531.218476][ T44] ? __schedule+0x13c2/0x1b10
[ 5531.218713][ T44] ? nf_ct_iterate_cleanup+0x4a/0x340 [nf_conntrack]
[ 5531.219023][ T44] ? ww_mutex_lock+0x160/0x160
[ 5531.219285][ T44] ? nf_ct_iterate_cleanup+0x4a/0x340 [nf_conntrack]
[ 5531.219584][ T44] ? nf_ct_iterate_cleanup+0x4a/0x340 [nf_conntrack]
[ 5531.219889][ T44] ? __nf_conntrack_hash_insert+0x200/0x200 [nf_conntrack]
[ 5531.220259][ T44] ? nf_conntrack_cleanup_net_list+0x19a/0x4b0 [nf_conntrack]
[ 5531.220619][ T44] ? nf_conntrack_cleanup_net_list+0x10e/0x4b0 [nf_conntrack]
[ 5531.220978][ T44] ? nf_conntrack_cleanup_end+0x70/0x70 [nf_conntrack]
[ 5531.221287][ T44] ? nf_conntrack_pernet_exit+0xa5/0x140 [nf_conntrack]
[ 5531.221603][ T44] ? ops_undo_list+0x2bf/0x890
[ 5531.221843][ T44] ? rtnl_net_dumpid_one+0x270/0x270
[ 5531.222082][ T44] ? cleanup_net+0x2c5/0x890
[ 5531.222344][ T44] ? cleanup_net+0x3a1/0x890
[ 5531.222582][ T44] ? net_passive_dec+0x190/0x190
[ 5531.222823][ T44] ? rcu_is_watching+0x12/0xb0
[ 5531.223061][ T44] ? process_one_work+0xe35/0x1650
[ 5531.223324][ T44] ? pwq_dec_nr_in_flight+0x550/0x550
[ 5531.223568][ T44] ? assign_work+0x168/0x240
[ 5531.223805][ T44] ? worker_thread+0x591/0xcf0
[ 5531.224049][ T44] ? rescuer_thread+0xd10/0xd10
[ 5531.224291][ T44] ? kthread+0x37b/0x5f0
[ 5531.224472][ T44] ? kthread_is_per_cpu+0xc0/0xc0
[ 5531.224703][ T44] ? ret_from_fork+0x1b/0x270
[ 5531.224937][ T44] ? __lock_release+0x5d/0x170
[ 5531.225185][ T44] ? rcu_is_watching+0x12/0xb0
[ 5531.225422][ T44] ? kthread_is_per_cpu+0xc0/0xc0
[ 5531.225658][ T44] ? ret_from_fork+0x1db/0x270
[ 5531.225890][ T44] ? kthread_is_per_cpu+0xc0/0xc0
[ 5531.226124][ T44] ? ret_from_fork_asm+0x11/0x20
[ 5531.226381][ T44]
[ 5531.226592][ T44]
[ 5531.226592][ T44] Showing all locks held in the system:
[ 5531.226924][ T44] 4 locks held by kworker/u16:0/12:
[ 5531.227184][ T44] 1 lock held by khungtaskd/44:
[ 5531.227410][ T44] #0: ffffffff9cb83f80 (rcu_read_lock){....}-{1:3}, at: debug_show_all_locks+0x36/0x260
[ 5531.227826][ T44] 3 locks held by kworker/2:1/15955:
[ 5531.228061][ T44] 1 lock held by modprobe/29700:
[ 5531.228292][ T44] #0: ffffffff9d270630 (pernet_ops_rwsem){++++}-{4:4}, at: register_pernet_subsys+0x1a/0x40
[ 5531.228768][ T44]
[ 5531.228880][ T44] =============================================
[ 5531.228880][ T44]
[ 5654.078259][ T44] INFO: task modprobe:29700 blocked for more than 1105 seconds.
[ 5654.078853][ T44] Not tainted 6.17.0-rc6-virtme #1
[ 5654.079144][ T44] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[ 5654.079639][ T44] task:modprobe state:D stack:25712 pid:29700 tgid:29700 ppid:1 task_flags:0x400100 flags:0x00004006
[ 5654.080182][ T44] Call Trace:
[ 5654.080406][ T44]
[ 5654.080563][ T44] __schedule+0x880/0x1b10
[ 5654.080837][ T44] ? io_schedule_timeout+0x150/0x150
[ 5654.081107][ T44] ? schedule+0x1ee/0x270
[ 5654.081352][ T44] ? __lock_release+0x5d/0x170
[ 5654.081647][ T44] schedule+0xe2/0x270
[ 5654.081822][ T44] schedule_preempt_disabled+0x18/0x30
[ 5654.082051][ T44] rwsem_down_write_slowpath+0x3c5/0xcb0
[ 5654.082349][ T44] ? __lock_acquire+0x449/0x7e0
[ 5654.082648][ T44] ? down_read+0x4a0/0x4a0
[ 5654.082906][ T44] ? rcu_is_watching+0x12/0xb0
[ 5654.083191][ T44] ? lock_acquire+0x104/0x160
[ 5654.083486][ T44] ? register_pernet_subsys+0x1a/0x40
[ 5654.083779][ T44] down_write+0x1d6/0x200
[ 5654.083954][ T44] ? rwsem_down_write_slowpath+0xcb0/0xcb0
[ 5654.084251][ T44] ? find_held_lock+0x2b/0x80
[ 5654.084489][ T44] ? __create_object+0x5e/0xb0
[ 5654.084799][ T44] ? _sub_I_65535_1+0x200/0x200 [pktgen]
[ 5654.085108][ T44] register_pernet_subsys+0x1a/0x40
[ 5654.085430][ T44] pg_init+0x2c/0x1000 [pktgen]
[ 5654.085714][ T44] ? _sub_I_65535_1+0x200/0x200 [pktgen]
[ 5654.085955][ T44] do_one_initcall+0x8c/0x1d0
[ 5654.086260][ T44] ? trace_initcall_level+0x130/0x130
[ 5654.086562][ T44] ? rcu_is_watching+0x12/0xb0
[ 5654.086801][ T44] ? trace_kmalloc+0x26/0xd0
[ 5654.087038][ T44] ? __kmalloc_cache_noprof+0x15f/0x360
[ 5654.087280][ T44] ? debug_objects_oom+0x160/0x160
[ 5654.087594][ T44] ? kasan_unpoison+0x27/0x50
[ 5654.087896][ T44] ? __asan_register_globals+0x5d/0x70
[ 5654.088204][ T44] do_init_module+0x27a/0x860
[ 5654.088481][ T44] ? free_module+0x2f0/0x2f0
[ 5654.088731][ T44] ? kfree+0x144/0x320
[ 5654.088961][ T44] ? find_vm_area+0x12/0x50
[ 5654.089288][ T44] load_module+0x1f40/0x2dd0
[ 5654.089600][ T44] ? layout_and_allocate.constprop.0+0x630/0x630
[ 5654.089922][ T44] ? kernel_read_file+0x3ed/0x550
[ 5654.090196][ T44] ? kernel_read_file+0x3c8/0x550
[ 5654.090464][ T44] ? __ia32_sys_fsconfig+0x150/0x150
[ 5654.090700][ T44] ? init_module_from_file+0xe9/0x150
[ 5654.090996][ T44] init_module_from_file+0xe9/0x150
[ 5654.091308][ T44] ? __do_sys_init_module+0x1e0/0x1e0
[ 5654.091606][ T44] ? idempotent_init_module+0x316/0x610
[ 5654.091836][ T44] ? __lock_release+0x5d/0x170
[ 5654.092068][ T44] ? do_raw_spin_unlock+0x58/0x220
[ 5654.092387][ T44] idempotent_init_module+0x331/0x610
[ 5654.092678][ T44] ? init_module_from_file+0x150/0x150
[ 5654.092925][ T44] ? cap_capable+0x94/0x230
[ 5654.093228][ T44] __x64_sys_finit_module+0xca/0x150
[ 5654.093526][ T44] ? do_syscall_64+0x85/0x300
[ 5654.093779][ T44] do_syscall_64+0xc1/0x300
[ 5654.094010][ T44] entry_SYSCALL_64_after_hwframe+0x4b/0x53
[ 5654.094304][ T44] RIP: 0033:0x7fbfb3fafa0d
[ 5654.094603][ T44] RSP: 002b:00007ffde31647a8 EFLAGS: 00000246 ORIG_RAX: 0000000000000139
[ 5654.095053][ T44] RAX: ffffffffffffffda RBX: 0000557a8b230ac0 RCX: 00007fbfb3fafa0d
[ 5654.095510][ T44] RDX: 0000000000000000 RSI: 0000557a53d3fa2a RDI: 0000000000000005
[ 5654.095881][ T44] RBP: 0000000000040000 R08: 0000000000000000 R09: 00007ffde31648e0
[ 5654.096335][ T44] R10: 0000000000000005 R11: 0000000000000246 R12: 0000557a53d3fa2a
[ 5654.096693][ T44] R13: 0000557a8b230b60 R14: 0000557a8b230da0 R15: 0000000000000000
[ 5654.097149][ T44]
[ 5654.097384][ T44] INFO: task modprobe:29700 blocked on an rw-semaphore likely owned by task kworker/u16:0:12
[ 5654.097881][ T44] task:kworker/u16:0 state:R running task stack:24720 pid:12 tgid:12 ppid:2 task_flags:0x4208060 flags:0x00004000
[ 5654.098455][ T44] Workqueue: netns cleanup_net
[ 5654.098732][ T44] Call Trace:
[ 5654.098959][ T44]
[ 5654.099111][ T44] ? __lock_acquire+0x449/0x7e0
[ 5654.099421][ T44] ? __mutex_trylock_common+0xf9/0x260
[ 5654.099725][ T44] ? __mutex_handoff+0x2b0/0x2b0
[ 5654.099974][ T44] ? rcu_is_watching+0x12/0xb0
[ 5654.100250][ T44] ? rcu_is_watching+0x12/0xb0
[ 5654.100503][ T44] ? trace_contention_end+0xd8/0x140
[ 5654.100735][ T44] ? __mutex_lock+0x19f/0x11a0
[ 5654.101015][ T44] ? trace_sched_exit_tp+0xcd/0x130
[ 5654.101335][ T44] ? nf_ct_iterate_cleanup+0x4a/0x340 [nf_conntrack]
[ 5654.101636][ T44] ? ww_mutex_lock+0x160/0x160
[ 5654.101920][ T44] ? nf_conntrack_cleanup_net_list+0x19a/0x4b0 [nf_conntrack]
[ 5654.102388][ T44] ? nf_ct_iterate_cleanup+0x4a/0x340 [nf_conntrack]
[ 5654.102768][ T44] ? nf_ct_iterate_cleanup+0x4a/0x340 [nf_conntrack]
[ 5654.103143][ T44] ? __nf_conntrack_hash_insert+0x200/0x200 [nf_conntrack]
[ 5654.103603][ T44] ? nf_conntrack_cleanup_net_list+0x19a/0x4b0 [nf_conntrack]
[ 5654.103982][ T44] ? nf_conntrack_cleanup_net_list+0x10e/0x4b0 [nf_conntrack]
[ 5654.104340][ T44] ? nf_conntrack_cleanup_end+0x70/0x70 [nf_conntrack]
[ 5654.104724][ T44] ? nf_conntrack_pernet_exit+0xa5/0x140 [nf_conntrack]
[ 5654.105063][ T44] ? ops_undo_list+0x2bf/0x890
[ 5654.105316][ T44] ? rtnl_net_dumpid_one+0x270/0x270
[ 5654.105579][ T44] ? cleanup_net+0x2c5/0x890
[ 5654.105882][ T44] ? cleanup_net+0x3a1/0x890
[ 5654.106184][ T44] ? net_passive_dec+0x190/0x190
[ 5654.106488][ T44] ? rcu_is_watching+0x12/0xb0
[ 5654.106731][ T44] ? process_one_work+0xe35/0x1650
[ 5654.107024][ T44] ? pwq_dec_nr_in_flight+0x550/0x550
[ 5654.107338][ T44] ? assign_work+0x168/0x240
[ 5654.107573][ T44] ? worker_thread+0x591/0xcf0
[ 5654.107827][ T44] ? rescuer_thread+0xd10/0xd10
[ 5654.108130][ T44] ? kthread+0x37b/0x5f0
[ 5654.108365][ T44] ? kthread_is_per_cpu+0xc0/0xc0
[ 5654.108667][ T44] ? ret_from_fork+0x1b/0x270
[ 5654.108916][ T44] ? __lock_release+0x5d/0x170
[ 5654.109148][ T44] ? rcu_is_watching+0x12/0xb0
[ 5654.109417][ T44] ? kthread_is_per_cpu+0xc0/0xc0
[ 5654.109715][ T44] ? ret_from_fork+0x1db/0x270
[ 5654.110012][ T44] ? kthread_is_per_cpu+0xc0/0xc0
[ 5654.110316][ T44] ? ret_from_fork_asm+0x11/0x20
[ 5654.110602][ T44]
[ 5654.110813][ T44]
[ 5654.110813][ T44] Showing all locks held in the system:
[ 5654.111565][ T44] 4 locks held by kworker/u16:0/12:
[ 5654.111801][ T44] 1 lock held by khungtaskd/44:
[ 5654.112021][ T44] #0: ffffffff9cb83f80 (rcu_read_lock){....}-{1:3}, at: debug_show_all_locks+0x36/0x260
[ 5654.112434][ T44] 3 locks held by kworker/2:1/15955:
[ 5654.112718][ T44] 1 lock held by modprobe/29700:
[ 5654.113006][ T44] #0: ffffffff9d270630 (pernet_ops_rwsem){++++}-{4:4}, at: register_pernet_subsys+0x1a/0x40
[ 5654.113600][ T44]
[ 5654.113741][ T44] =============================================
[ 5654.113741][ T44]
[ 5776.958257][ T44] INFO: task modprobe:29700 blocked for more than 1228 seconds.
[ 5776.958749][ T44] Not tainted 6.17.0-rc6-virtme #1
[ 5776.958969][ T44] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[ 5776.959372][ T44] task:modprobe state:D stack:25712 pid:29700 tgid:29700 ppid:1 task_flags:0x400100 flags:0x00004006
[ 5776.960041][ T44] Call Trace:
[ 5776.960282][ T44]
[ 5776.960444][ T44] __schedule+0x880/0x1b10
[ 5776.960740][ T44] ? io_schedule_timeout+0x150/0x150
[ 5776.961027][ T44] ? schedule+0x1ee/0x270
[ 5776.961264][ T44] ? __lock_release+0x5d/0x170
[ 5776.961541][ T44] schedule+0xe2/0x270
[ 5776.961717][ T44] schedule_preempt_disabled+0x18/0x30
[ 5776.961984][ T44] rwsem_down_write_slowpath+0x3c5/0xcb0
[ 5776.962306][ T44] ? __lock_acquire+0x449/0x7e0
[ 5776.962608][ T44] ? down_read+0x4a0/0x4a0
[ 5776.962900][ T44] ? rcu_is_watching+0x12/0xb0
[ 5776.963134][ T44] ? lock_acquire+0x104/0x160
[ 5776.963427][ T44] ? register_pernet_subsys+0x1a/0x40
[ 5776.963740][ T44] down_write+0x1d6/0x200
[ 5776.963919][ T44] ? rwsem_down_write_slowpath+0xcb0/0xcb0
[ 5776.964265][ T44] ? find_held_lock+0x2b/0x80
[ 5776.964541][ T44] ? __create_object+0x5e/0xb0
[ 5776.964790][ T44] ? _sub_I_65535_1+0x200/0x200 [pktgen]
[ 5776.965060][ T44] register_pernet_subsys+0x1a/0x40
[ 5776.965386][ T44] pg_init+0x2c/0x1000 [pktgen]
[ 5776.965689][ T44] ? _sub_I_65535_1+0x200/0x200 [pktgen]
[ 5776.965993][ T44] do_one_initcall+0x8c/0x1d0
[ 5776.966257][ T44] ? trace_initcall_level+0x130/0x130
[ 5776.966493][ T44] ? rcu_is_watching+0x12/0xb0
[ 5776.966724][ T44] ? trace_kmalloc+0x26/0xd0
[ 5776.966973][ T44] ? __kmalloc_cache_noprof+0x15f/0x360
[ 5776.967283][ T44] ? debug_objects_oom+0x160/0x160
[ 5776.967584][ T44] ? kasan_unpoison+0x27/0x50
[ 5776.967890][ T44] ? __asan_register_globals+0x5d/0x70
[ 5776.968127][ T44] do_init_module+0x27a/0x860
[ 5776.968377][ T44] ? free_module+0x2f0/0x2f0
[ 5776.968667][ T44] ? kfree+0x144/0x320
[ 5776.968901][ T44] ? find_vm_area+0x12/0x50
[ 5776.969228][ T44] load_module+0x1f40/0x2dd0
[ 5776.969541][ T44] ? layout_and_allocate.constprop.0+0x630/0x630
[ 5776.969828][ T44] ? kernel_read_file+0x3ed/0x550
[ 5776.970063][ T44] ? kernel_read_file+0x3c8/0x550
[ 5776.970333][ T44] ? __ia32_sys_fsconfig+0x150/0x150
[ 5776.970605][ T44] ? init_module_from_file+0xe9/0x150
[ 5776.970841][ T44] init_module_from_file+0xe9/0x150
[ 5776.971139][ T44] ? __do_sys_init_module+0x1e0/0x1e0
[ 5776.971463][ T44] ? idempotent_init_module+0x316/0x610
[ 5776.971694][ T44] ? __lock_release+0x5d/0x170
[ 5776.971942][ T44] ? do_raw_spin_unlock+0x58/0x220
[ 5776.972249][ T44] idempotent_init_module+0x331/0x610
[ 5776.972549][ T44] ? init_module_from_file+0x150/0x150
[ 5776.972867][ T44] ? cap_capable+0x94/0x230
[ 5776.973105][ T44] __x64_sys_finit_module+0xca/0x150
[ 5776.973352][ T44] ? do_syscall_64+0x85/0x300
[ 5776.973609][ T44] do_syscall_64+0xc1/0x300
[ 5776.973909][ T44] entry_SYSCALL_64_after_hwframe+0x4b/0x53
[ 5776.974288][ T44] RIP: 0033:0x7fbfb3fafa0d
[ 5776.974603][ T44] RSP: 002b:00007ffde31647a8 EFLAGS: 00000246 ORIG_RAX: 0000000000000139
[ 5776.974958][ T44] RAX: ffffffffffffffda RBX: 0000557a8b230ac0 RCX: 00007fbfb3fafa0d
[ 5776.975323][ T44] RDX: 0000000000000000 RSI: 0000557a53d3fa2a RDI: 0000000000000005
[ 5776.975769][ T44] RBP: 0000000000040000 R08: 0000000000000000 R09: 00007ffde31648e0
[ 5776.976135][ T44] R10: 0000000000000005 R11: 0000000000000246 R12: 0000557a53d3fa2a
[ 5776.976561][ T44] R13: 0000557a8b230b60 R14: 0000557a8b230da0 R15: 0000000000000000
[ 5776.976922][ T44]
[ 5776.977159][ T44] INFO: task modprobe:29700 blocked on an rw-semaphore likely owned by task kworker/u16:0:12
[ 5776.977791][ T44] task:kworker/u16:0 state:R running task stack:24720 pid:12 tgid:12 ppid:2 task_flags:0x4208060 flags:0x00004000
[ 5776.978440][ T44] Workqueue: netns cleanup_net
[ 5776.978675][ T44] Call Trace:
[ 5776.978886][ T44]
[ 5776.979043][ T44] ? __mutex_trylock_common+0xf9/0x260
[ 5776.979355][ T44] ? __mutex_handoff+0x2b0/0x2b0
[ 5776.979654][ T44] ? rcu_is_watching+0x12/0xb0
[ 5776.979905][ T44] ? rcu_is_watching+0x12/0xb0
[ 5776.980137][ T44] ? trace_contention_end+0xd8/0x140
[ 5776.980418][ T44] ? __mutex_lock+0x19f/0x11a0
[ 5776.980719][ T44] ? trace_sched_exit_tp+0xcd/0x130
[ 5776.980992][ T44] ? nf_ct_iterate_cleanup+0x4a/0x340 [nf_conntrack]
[ 5776.981300][ T44] ? ww_mutex_lock+0x160/0x160
[ 5776.981592][ T44] ? nf_ct_iterate_cleanup+0x4a/0x340 [nf_conntrack]
[ 5776.981969][ T44] ? nf_ct_iterate_cleanup+0x4a/0x340 [nf_conntrack]
[ 5776.982368][ T44] ? __nf_conntrack_hash_insert+0x200/0x200 [nf_conntrack]
[ 5776.982808][ T44] ? nf_conntrack_cleanup_net_list+0x19a/0x4b0 [nf_conntrack]
[ 5776.983162][ T44] ? nf_conntrack_cleanup_net_list+0x10e/0x4b0 [nf_conntrack]
[ 5776.983564][ T44] ? nf_conntrack_cleanup_end+0x70/0x70 [nf_conntrack]
[ 5776.983936][ T44] ? nf_conntrack_pernet_exit+0xa5/0x140 [nf_conntrack]
[ 5776.984251][ T44] ? ops_undo_list+0x2bf/0x890
[ 5776.984486][ T44] ? rtnl_net_dumpid_one+0x270/0x270
[ 5776.984735][ T44] ? cleanup_net+0x2c5/0x890
[ 5776.984974][ T44] ? cleanup_net+0x3a1/0x890
[ 5776.985215][ T44] ? net_passive_dec+0x190/0x190
[ 5776.985510][ T44] ? rcu_is_watching+0x12/0xb0
[ 5776.985810][ T44] ? process_one_work+0xe35/0x1650
[ 5776.986069][ T44] ? pwq_dec_nr_in_flight+0x550/0x550
[ 5776.986328][ T44] ? assign_work+0x168/0x240
[ 5776.986606][ T44] ? worker_thread+0x591/0xcf0
[ 5776.986911][ T44] ? rescuer_thread+0xd10/0xd10
[ 5776.987221][ T44] ? kthread+0x37b/0x5f0
[ 5776.987452][ T44] ? kthread_is_per_cpu+0xc0/0xc0
[ 5776.987708][ T44] ? ret_from_fork+0x1b/0x270
[ 5776.987960][ T44] ? __lock_release+0x5d/0x170
[ 5776.988269][ T44] ? rcu_is_watching+0x12/0xb0
[ 5776.988572][ T44] ? kthread_is_per_cpu+0xc0/0xc0
[ 5776.988869][ T44] ? ret_from_fork+0x1db/0x270
[ 5776.989097][ T44] ? kthread_is_per_cpu+0xc0/0xc0
[ 5776.989349][ T44] ? ret_from_fork_asm+0x11/0x20
[ 5776.989660][ T44]
[ 5776.989867][ T44] Future hung task reports are suppressed, see sysctl kernel.hung_task_warnings
[ 5776.990297][ T44]
[ 5776.990297][ T44] Showing all locks held in the system:
[ 5776.990713][ T44] 4 locks held by kworker/u16:0/12:
[ 5776.991014][ T44] 1 lock held by khungtaskd/44:
[ 5776.991638][ T44] #0: ffffffff9cb83f80 (rcu_read_lock){....}-{1:3}, at: debug_show_all_locks+0x36/0x260
[ 5776.992047][ T44] 3 locks held by kworker/2:1/15955:
[ 5776.992281][ T44] 1 lock held by modprobe/29700:
[ 5776.992577][ T44] #0: ffffffff9d270630 (pernet_ops_rwsem){++++}-{4:4}, at: register_pernet_subsys+0x1a/0x40
[ 5776.993102][ T44]
[ 5776.993218][ T44] =============================================
[ 5776.993218][ T44]
WAIT TIMEOUT stderr
Ctrl-C stderr
Ctrl-C stderr