[ 4008.317863][ T70] ------------[ cut here ]------------ [ 4008.318470][ T70] conntrack cleanup blocked for 60s [ 4008.318473][ T70] WARNING: net/netfilter/nf_conntrack_core.c:2511 at 0x0, CPU#2: kworker/u16:1/70 [ 4008.319268][ T70] Modules linked in: mpls_router(+) xt_conntrack xt_length nft_compat act_ct nf_flow_table nf_tables act_gact sctp_diag sctp ip6_gre ip_gre gre cls_bpf sch_ingress netdevsim xfrm_user openvswitch psample nf_nat nf_conntrack nf_defrag_ipv6 nf_defrag_ipv4 nsh geneve vxlan act_csum act_pedit cls_flower sch_prio [ 4008.320757][ T70] CPU: 2 UID: 0 PID: 70 Comm: kworker/u16:1 Not tainted 6.18.0-virtme #1 PREEMPT(full) [ 4008.321237][ T70] Hardware name: Bochs Bochs, BIOS Bochs 01/01/2011 [ 4008.321584][ T70] Workqueue: netns cleanup_net [ 4008.321890][ T70] RIP: 0010:nf_conntrack_cleanup_net_list+0x4a0/0x540 [nf_conntrack] [ 4008.322312][ T70] Code: 00 00 00 65 48 2b 05 ff 11 06 e0 0f 85 a0 00 00 00 48 81 c4 c0 00 00 00 5b 5d 41 5c 41 5d 41 5e 41 5f c3 48 8d 3d 00 d2 ea ff <67> 48 0f b9 3a e9 0f ff ff ff 48 c7 c7 bc 0f 36 c0 e8 0a 4f c4 db [ 4008.323242][ T70] RSP: 0018:ffffc900004b7988 EFLAGS: 00010297 [ 4008.323590][ T70] RAX: 0000000100389315 RBX: ffffc900004b7ae8 RCX: ffffffffc04a6b0a [ 4008.323998][ T70] RDX: 0000000100389314 RSI: 0000000000000004 RDI: ffffffffc0354030 [ 4008.324393][ T70] RBP: ffff88800bbc7260 R08: 0000000000000000 R09: ffffed1001778e4c [ 4008.324799][ T70] R10: ffff88800bbc7263 R11: ffff888008868a90 R12: 0000000000000001 [ 4008.325205][ T70] R13: ffff88800bb7a600 R14: dffffc0000000000 R15: ffffc900004b79e8 [ 4008.325603][ T70] FS: 0000000000000000(0000) GS:ffff8880ccc18000(0000) knlGS:0000000000000000 [ 4008.326073][ T70] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 4008.326814][ T70] CR2: 00007fe1ce0e74d8 CR3: 0000000011545002 CR4: 0000000000772ef0 [ 4008.327219][ T70] PKRU: 55555554 [ 4008.327442][ T70] Call Trace: [ 4008.327654][ T70] [ 4008.327819][ T70] ? nf_conntrack_cleanup_net_list+0x137/0x540 [nf_conntrack] [ 4008.328232][ T70] ? nf_conntrack_cleanup_end+0x70/0x70 [nf_conntrack] [ 4008.328573][ T70] ? nf_conntrack_pernet_exit+0xa5/0x140 [nf_conntrack] [ 4008.328928][ T70] ? nf_conntrack_pernet_exit+0xa5/0x140 [nf_conntrack] [ 4008.329275][ T70] ? _raw_spin_unlock+0x2d/0x50 [ 4008.329557][ T70] ops_undo_list+0x2bf/0x890 [ 4008.329854][ T70] ? netns_install+0x280/0x280 [ 4008.330134][ T70] ? cleanup_net+0x2d6/0x8e0 [ 4008.330415][ T70] cleanup_net+0x3b2/0x8e0 [ 4008.330688][ T70] ? net_passive_dec+0x190/0x190 [ 4008.330977][ T70] ? rcu_is_watching+0x12/0xb0 [ 4008.331252][ T70] ? lock_acquire+0x104/0x140 [ 4008.331528][ T70] process_one_work+0x880/0x1810 [ 4008.331835][ T70] ? pwq_dec_nr_in_flight+0x550/0x550 [ 4008.332115][ T70] ? assign_work+0x168/0x240 [ 4008.332389][ T70] worker_thread+0x591/0xcf0 [ 4008.332668][ T70] ? bh_worker+0x6f0/0x6f0 [ 4008.332952][ T70] kthread+0x37b/0x5f0 [ 4008.333168][ T70] ? kthread_is_per_cpu+0xc0/0xc0 [ 4008.333438][ T70] ? ret_from_fork+0x71/0x540 [ 4008.333721][ T70] ? __lock_release+0x5d/0x160 [ 4008.334010][ T70] ? lock_acquire+0x104/0x140 [ 4008.334282][ T70] ? rcu_is_watching+0x12/0xb0 [ 4008.334552][ T70] ? kthread_is_per_cpu+0xc0/0xc0 [ 4008.334833][ T70] ret_from_fork+0x42f/0x540 [ 4008.335106][ T70] ? arch_exit_to_user_mode_prepare.constprop.0+0x140/0x140 [ 4008.335498][ T70] ? kthread_is_per_cpu+0xc0/0xc0 [ 4008.335770][ T70] ? __switch_to+0x5c8/0xd50 [ 4008.336060][ T70] ? kthread_is_per_cpu+0xc0/0xc0 [ 4008.336331][ T70] ret_from_fork_asm+0x11/0x20 [ 4008.336619][ T70] [ 4008.336839][ T70] irq event stamp: 33959443 [ 4008.337112][ T70] hardirqs last enabled at (33959453): [] __up_console_sem+0x67/0x70 [ 4008.337570][ T70] hardirqs last disabled at (33959462): [] __up_console_sem+0x4c/0x70 [ 4008.338040][ T70] softirqs last enabled at (33958914): [] handle_softirqs+0x524/0x820 [ 4008.338497][ T70] softirqs last disabled at (33958907): [] __irq_exit_rcu+0x6c/0xe0 [ 4008.338956][ T70] ---[ end trace 0000000000000000 ]--- [ 4179.560879][ T44] INFO: task modprobe:1447 blocked for more than 122 seconds. [ 4179.561421][ T44] Tainted: G W 6.18.0-virtme #1 [ 4179.561767][ T44] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. [ 4179.562174][ T44] task:modprobe state:D stack:25208 pid:1447 tgid:1447 ppid:63 task_flags:0x400100 flags:0x00080000 [ 4179.562762][ T44] Call Trace: [ 4179.563020][ T44] [ 4179.563148][ T44] __schedule+0x90a/0x1930 [ 4179.563395][ T44] ? io_schedule_timeout+0x150/0x150 [ 4179.563643][ T44] ? __lock_release+0x5d/0x160 [ 4179.563890][ T44] schedule+0xe2/0x270 [ 4179.564069][ T44] schedule_preempt_disabled+0x18/0x30 [ 4179.564290][ T44] rwsem_down_write_slowpath+0x3c5/0xc60 [ 4179.564528][ T44] ? rwsem_down_read_slowpath+0xd80/0xd80 [ 4179.564759][ T44] ? rcu_is_watching+0x12/0xb0 [ 4179.565002][ T44] ? lock_acquire+0x104/0x140 [ 4179.565233][ T44] down_write+0x1d6/0x200 [ 4179.565403][ T44] ? rwsem_down_write_slowpath+0xc60/0xc60 [ 4179.565694][ T44] ? __create_object+0x5e/0xb0 [ 4179.566019][ T44] ? __lock_release+0x5d/0x160 [ 4179.566410][ T44] register_pernet_subsys+0x1a/0x40 [ 4179.566831][ T44] ? conntrack_mt_exit+0xcf0/0xcf0 [xt_conntrack] [ 4179.567255][ T44] mpls_init+0x17/0x1000 [mpls_router] [ 4179.567650][ T44] ? conntrack_mt_exit+0xcf0/0xcf0 [xt_conntrack] [ 4179.568124][ T44] do_one_initcall+0xc5/0x3d0 [ 4179.568483][ T44] ? trace_event_raw_event_initcall_level+0x1f0/0x1f0 [ 4179.568993][ T44] ? rcu_is_watching+0x12/0xb0 [ 4179.569376][ T44] ? trace_kmalloc+0x24/0xb0 [ 4179.570288][ T44] ? __kmalloc_cache_noprof+0x2e4/0x710 [ 4179.570681][ T44] ? debug_objects_fill_pool+0x3e0/0x3e0 [ 4179.571073][ T44] ? kasan_unpoison+0x27/0x50 [ 4179.571440][ T44] do_init_module+0x27a/0x860 [ 4179.571823][ T44] ? free_module+0x3f0/0x3f0 [ 4179.572183][ T44] ? kfree+0x2bb/0x5f0 [ 4179.572451][ T44] ? nft_chain_route_fini+0x680/0x680 [nf_tables] [ 4179.572943][ T44] ? load_module+0x20c5/0x2dd0 [ 4179.573322][ T44] load_module+0x1f40/0x2dd0 [ 4179.573719][ T44] ? layout_and_allocate.constprop.0+0x630/0x630 [ 4179.574217][ T44] ? kernel_read_file+0x3ed/0x550 [ 4179.574582][ T44] ? kernel_read_file+0x3c8/0x550 [ 4179.574970][ T44] ? __ia32_sys_fsconfig+0x150/0x150 [ 4179.575347][ T44] ? init_module_from_file+0x159/0x180 [ 4179.575718][ T44] init_module_from_file+0x159/0x180 [ 4179.576098][ T44] ? __do_sys_init_module+0x1e0/0x1e0 [ 4179.576467][ T44] ? __lock_acquire+0x664/0xca0 [ 4179.576888][ T44] ? idempotent_init_module+0x316/0x610 [ 4179.577252][ T44] ? __lock_release+0x5d/0x160 [ 4179.577623][ T44] ? do_raw_spin_unlock+0x58/0x220 [ 4179.578019][ T44] idempotent_init_module+0x331/0x610 [ 4179.578378][ T44] ? init_module_from_file+0x180/0x180 [ 4179.578749][ T44] ? rcu_is_watching+0x12/0xb0 [ 4179.579124][ T44] ? cap_capable+0x17b/0x340 [ 4179.579514][ T44] __x64_sys_finit_module+0xca/0x150 [ 4179.579895][ T44] ? do_syscall_64+0x85/0xfc0 [ 4179.580267][ T44] do_syscall_64+0xc1/0xfc0 [ 4179.580633][ T44] entry_SYSCALL_64_after_hwframe+0x4b/0x53 [ 4179.581102][ T44] RIP: 0033:0x7f7e31c1c92d [ 4179.581492][ T44] RSP: 002b:00007fff4d76cb68 EFLAGS: 00000246 ORIG_RAX: 0000000000000139 [ 4179.582130][ T44] RAX: ffffffffffffffda RBX: 0000565009fefdf0 RCX: 00007f7e31c1c92d [ 4179.582740][ T44] RDX: 0000000000000000 RSI: 00005650052c9a2a RDI: 0000000000000000 [ 4179.583365][ T44] RBP: 0000000000040000 R08: 0000000000000000 R09: 0000000000000000 [ 4179.583987][ T44] R10: 0000000000000000 R11: 0000000000000246 R12: 00005650052c9a2a [ 4179.584578][ T44] R13: 0000565009ff0000 R14: 0000565009ff02a0 R15: 0000000000000000 [ 4179.585210][ T44] [ 4179.585509][ T44] INFO: task modprobe:1447 is blocked on an rw-semaphore, but the owner is not found. [ 4179.586157][ T44] [ 4179.586157][ T44] Showing all locks held in the system: [ 4179.586673][ T44] 1 lock held by khungtaskd/44: [ 4179.586942][ T44] #0: ffffffff9f17fe20 (rcu_read_lock){....}-{1:3}, at: debug_show_all_locks+0x36/0x260 [ 4179.587324][ T44] 2 locks held by kworker/u20:1/63: [ 4179.587545][ T44] #0: ffff888001099d48 ((wq_completion)events_unbound#2){+.+.}-{0:0}, at: process_one_work+0x1034/0x1810 [ 4179.588005][ T44] #1: ffffc90000447cb8 ((work_completion)(&sub_info->work)){+.+.}-{0:0}, at: process_one_work+0x802/0x1810 [ 4179.588496][ T44] 5 locks held by kworker/u16:1/70: [ 4179.588721][ T44] 1 lock held by modprobe/1447: [ 4179.589092][ T44] #0: ffffffff9f868cd0 (pernet_ops_rwsem){++++}-{4:4}, at: register_pernet_subsys+0x1a/0x40 [ 4179.589588][ T44] [ 4179.589738][ T44] ============================================= [ 4179.589738][ T44] [ 4302.440874][ T44] INFO: task modprobe:1447 blocked for more than 245 seconds. [ 4302.441281][ T44] Tainted: G W 6.18.0-virtme #1 [ 4302.441538][ T44] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. [ 4302.441925][ T44] task:modprobe state:D stack:25208 pid:1447 tgid:1447 ppid:63 task_flags:0x400100 flags:0x00080000 [ 4302.442402][ T44] Call Trace: [ 4302.442581][ T44] [ 4302.442700][ T44] __schedule+0x90a/0x1930 [ 4302.442945][ T44] ? io_schedule_timeout+0x150/0x150 [ 4302.443171][ T44] ? __lock_release+0x5d/0x160 [ 4302.443394][ T44] schedule+0xe2/0x270 [ 4302.443570][ T44] schedule_preempt_disabled+0x18/0x30 [ 4302.443794][ T44] rwsem_down_write_slowpath+0x3c5/0xc60 [ 4302.444020][ T44] ? rwsem_down_read_slowpath+0xd80/0xd80 [ 4302.444259][ T44] ? rcu_is_watching+0x12/0xb0 [ 4302.444477][ T44] ? lock_acquire+0x104/0x140 [ 4302.444706][ T44] down_write+0x1d6/0x200 [ 4302.444887][ T44] ? rwsem_down_write_slowpath+0xc60/0xc60 [ 4302.445162][ T44] ? __create_object+0x5e/0xb0 [ 4302.445380][ T44] ? __lock_release+0x5d/0x160 [ 4302.445603][ T44] register_pernet_subsys+0x1a/0x40 [ 4302.445862][ T44] ? conntrack_mt_exit+0xcf0/0xcf0 [xt_conntrack] [ 4302.446131][ T44] mpls_init+0x17/0x1000 [mpls_router] [ 4302.446352][ T44] ? conntrack_mt_exit+0xcf0/0xcf0 [xt_conntrack] [ 4302.446623][ T44] do_one_initcall+0xc5/0x3d0 [ 4302.446855][ T44] ? trace_event_raw_event_initcall_level+0x1f0/0x1f0 [ 4302.447125][ T44] ? rcu_is_watching+0x12/0xb0 [ 4302.447342][ T44] ? trace_kmalloc+0x24/0xb0 [ 4302.447579][ T44] ? __kmalloc_cache_noprof+0x2e4/0x710 [ 4302.447808][ T44] ? debug_objects_fill_pool+0x3e0/0x3e0 [ 4302.448029][ T44] ? kasan_unpoison+0x27/0x50 [ 4302.448254][ T44] do_init_module+0x27a/0x860 [ 4302.448484][ T44] ? free_module+0x3f0/0x3f0 [ 4302.448710][ T44] ? kfree+0x2bb/0x5f0 [ 4302.448886][ T44] ? nft_chain_route_fini+0x680/0x680 [nf_tables] [ 4302.449185][ T44] ? load_module+0x20c5/0x2dd0 [ 4302.449409][ T44] load_module+0x1f40/0x2dd0 [ 4302.449636][ T44] ? layout_and_allocate.constprop.0+0x630/0x630 [ 4302.449933][ T44] ? kernel_read_file+0x3ed/0x550 [ 4302.450157][ T44] ? kernel_read_file+0x3c8/0x550 [ 4302.450376][ T44] ? __ia32_sys_fsconfig+0x150/0x150 [ 4302.450600][ T44] ? init_module_from_file+0x159/0x180 [ 4302.450828][ T44] init_module_from_file+0x159/0x180 [ 4302.451058][ T44] ? __do_sys_init_module+0x1e0/0x1e0 [ 4302.451279][ T44] ? __lock_acquire+0x664/0xca0 [ 4302.451510][ T44] ? idempotent_init_module+0x316/0x610 [ 4302.451734][ T44] ? __lock_release+0x5d/0x160 [ 4302.451969][ T44] ? do_raw_spin_unlock+0x58/0x220 [ 4302.452192][ T44] idempotent_init_module+0x331/0x610 [ 4302.452431][ T44] ? init_module_from_file+0x180/0x180 [ 4302.452660][ T44] ? rcu_is_watching+0x12/0xb0 [ 4302.452890][ T44] ? cap_capable+0x17b/0x340 [ 4302.453117][ T44] __x64_sys_finit_module+0xca/0x150 [ 4302.453332][ T44] ? do_syscall_64+0x85/0xfc0 [ 4302.453555][ T44] do_syscall_64+0xc1/0xfc0 [ 4302.453775][ T44] entry_SYSCALL_64_after_hwframe+0x4b/0x53 [ 4302.454068][ T44] RIP: 0033:0x7f7e31c1c92d [ 4302.454310][ T44] RSP: 002b:00007fff4d76cb68 EFLAGS: 00000246 ORIG_RAX: 0000000000000139 [ 4302.454640][ T44] RAX: ffffffffffffffda RBX: 0000565009fefdf0 RCX: 00007f7e31c1c92d [ 4302.454982][ T44] RDX: 0000000000000000 RSI: 00005650052c9a2a RDI: 0000000000000000 [ 4302.455307][ T44] RBP: 0000000000040000 R08: 0000000000000000 R09: 0000000000000000 [ 4302.455654][ T44] R10: 0000000000000000 R11: 0000000000000246 R12: 00005650052c9a2a [ 4302.456000][ T44] R13: 0000565009ff0000 R14: 0000565009ff02a0 R15: 0000000000000000 [ 4302.456339][ T44] [ 4302.456504][ T44] INFO: task modprobe:1447 is blocked on an rw-semaphore, but the owner is not found. [ 4302.456878][ T44] [ 4302.456878][ T44] Showing all locks held in the system: [ 4302.457195][ T44] 1 lock held by khungtaskd/44: [ 4302.457398][ T44] #0: ffffffff9f17fe20 (rcu_read_lock){....}-{1:3}, at: debug_show_all_locks+0x36/0x260 [ 4302.457766][ T44] 2 locks held by kworker/u20:1/63: [ 4302.458125][ T44] #0: ffff888001099d48 ((wq_completion)events_unbound#2){+.+.}-{0:0}, at: process_one_work+0x1034/0x1810 [ 4302.458548][ T44] #1: ffffc90000447cb8 ((work_completion)(&sub_info->work)){+.+.}-{0:0}, at: process_one_work+0x802/0x1810 [ 4302.459030][ T44] 4 locks held by kworker/u16:1/70: [ 4302.459243][ T44] 1 lock held by modprobe/1447: [ 4302.459443][ T44] #0: ffffffff9f868cd0 (pernet_ops_rwsem){++++}-{4:4}, at: register_pernet_subsys+0x1a/0x40 [ 4302.459866][ T44] [ 4302.459970][ T44] ============================================= [ 4302.459970][ T44] [ 4425.321868][ T44] INFO: task modprobe:1447 blocked for more than 368 seconds. [ 4425.322335][ T44] Tainted: G W 6.18.0-virtme #1 [ 4425.322585][ T44] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. [ 4425.322973][ T44] task:modprobe state:D stack:25208 pid:1447 tgid:1447 ppid:63 task_flags:0x400100 flags:0x00080000 [ 4425.323462][ T44] Call Trace: [ 4425.323629][ T44] [ 4425.323747][ T44] __schedule+0x90a/0x1930 [ 4425.324007][ T44] ? io_schedule_timeout+0x150/0x150 [ 4425.324232][ T44] ? __lock_release+0x5d/0x160 [ 4425.324460][ T44] schedule+0xe2/0x270 [ 4425.324626][ T44] schedule_preempt_disabled+0x18/0x30 [ 4425.324864][ T44] rwsem_down_write_slowpath+0x3c5/0xc60 [ 4425.325095][ T44] ? rwsem_down_read_slowpath+0xd80/0xd80 [ 4425.325320][ T44] ? rcu_is_watching+0x12/0xb0 [ 4425.325540][ T44] ? lock_acquire+0x104/0x140 [ 4425.325761][ T44] down_write+0x1d6/0x200 [ 4425.325940][ T44] ? rwsem_down_write_slowpath+0xc60/0xc60 [ 4425.326214][ T44] ? __create_object+0x5e/0xb0 [ 4425.326430][ T44] ? __lock_release+0x5d/0x160 [ 4425.326654][ T44] register_pernet_subsys+0x1a/0x40 [ 4425.326906][ T44] ? conntrack_mt_exit+0xcf0/0xcf0 [xt_conntrack] [ 4425.327179][ T44] mpls_init+0x17/0x1000 [mpls_router] [ 4425.327398][ T44] ? conntrack_mt_exit+0xcf0/0xcf0 [xt_conntrack] [ 4425.327665][ T44] do_one_initcall+0xc5/0x3d0 [ 4425.327910][ T44] ? trace_event_raw_event_initcall_level+0x1f0/0x1f0 [ 4425.328193][ T44] ? rcu_is_watching+0x12/0xb0 [ 4425.328408][ T44] ? trace_kmalloc+0x24/0xb0 [ 4425.328625][ T44] ? __kmalloc_cache_noprof+0x2e4/0x710 [ 4425.328852][ T44] ? debug_objects_fill_pool+0x3e0/0x3e0 [ 4425.329073][ T44] ? kasan_unpoison+0x27/0x50 [ 4425.329292][ T44] do_init_module+0x27a/0x860 [ 4425.329518][ T44] ? free_module+0x3f0/0x3f0 [ 4425.329738][ T44] ? kfree+0x2bb/0x5f0 [ 4425.329922][ T44] ? nft_chain_route_fini+0x680/0x680 [nf_tables] [ 4425.330209][ T44] ? load_module+0x20c5/0x2dd0 [ 4425.330433][ T44] load_module+0x1f40/0x2dd0 [ 4425.330662][ T44] ? layout_and_allocate.constprop.0+0x630/0x630 [ 4425.330957][ T44] ? kernel_read_file+0x3ed/0x550 [ 4425.331182][ T44] ? kernel_read_file+0x3c8/0x550 [ 4425.331401][ T44] ? __ia32_sys_fsconfig+0x150/0x150 [ 4425.331630][ T44] ? init_module_from_file+0x159/0x180 [ 4425.331862][ T44] init_module_from_file+0x159/0x180 [ 4425.332080][ T44] ? __do_sys_init_module+0x1e0/0x1e0 [ 4425.332308][ T44] ? __lock_acquire+0x664/0xca0 [ 4425.332537][ T44] ? idempotent_init_module+0x316/0x610 [ 4425.332758][ T44] ? __lock_release+0x5d/0x160 [ 4425.333003][ T44] ? do_raw_spin_unlock+0x58/0x220 [ 4425.333238][ T44] idempotent_init_module+0x331/0x610 [ 4425.333458][ T44] ? init_module_from_file+0x180/0x180 [ 4425.333689][ T44] ? rcu_is_watching+0x12/0xb0 [ 4425.333915][ T44] ? cap_capable+0x17b/0x340 [ 4425.334143][ T44] __x64_sys_finit_module+0xca/0x150 [ 4425.334361][ T44] ? do_syscall_64+0x85/0xfc0 [ 4425.334587][ T44] do_syscall_64+0xc1/0xfc0 [ 4425.334848][ T44] entry_SYSCALL_64_after_hwframe+0x4b/0x53 [ 4425.335121][ T44] RIP: 0033:0x7f7e31c1c92d [ 4425.335353][ T44] RSP: 002b:00007fff4d76cb68 EFLAGS: 00000246 ORIG_RAX: 0000000000000139 [ 4425.335693][ T44] RAX: ffffffffffffffda RBX: 0000565009fefdf0 RCX: 00007f7e31c1c92d [ 4425.336036][ T44] RDX: 0000000000000000 RSI: 00005650052c9a2a RDI: 0000000000000000 [ 4425.336395][ T44] RBP: 0000000000040000 R08: 0000000000000000 R09: 0000000000000000 [ 4425.336723][ T44] R10: 0000000000000000 R11: 0000000000000246 R12: 00005650052c9a2a [ 4425.337058][ T44] R13: 0000565009ff0000 R14: 0000565009ff02a0 R15: 0000000000000000 [ 4425.337402][ T44] [ 4425.337571][ T44] INFO: task modprobe:1447 is blocked on an rw-semaphore, but the owner is not found. [ 4425.337968][ T44] [ 4425.337968][ T44] Showing all locks held in the system: [ 4425.338287][ T44] 1 lock held by khungtaskd/44: [ 4425.338497][ T44] #0: ffffffff9f17fe20 (rcu_read_lock){....}-{1:3}, at: debug_show_all_locks+0x36/0x260 [ 4425.339059][ T44] 2 locks held by kworker/u20:1/63: [ 4425.339266][ T44] #0: ffff888001099d48 ((wq_completion)events_unbound#2){+.+.}-{0:0}, at: process_one_work+0x1034/0x1810 [ 4425.339695][ T44] #1: ffffc90000447cb8 ((work_completion)(&sub_info->work)){+.+.}-{0:0}, at: process_one_work+0x802/0x1810 [ 4425.340167][ T44] 4 locks held by kworker/u16:1/70: [ 4425.340383][ T44] 1 lock held by modprobe/1447: [ 4425.340582][ T44] #0: ffffffff9f868cd0 (pernet_ops_rwsem){++++}-{4:4}, at: register_pernet_subsys+0x1a/0x40 [ 4425.341009][ T44] [ 4425.341113][ T44] ============================================= [ 4425.341113][ T44] [ 4548.201860][ T44] INFO: task modprobe:1447 blocked for more than 491 seconds. [ 4548.202267][ T44] Tainted: G W 6.18.0-virtme #1 [ 4548.202519][ T44] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. [ 4548.202920][ T44] task:modprobe state:D stack:25208 pid:1447 tgid:1447 ppid:63 task_flags:0x400100 flags:0x00080000 [ 4548.203442][ T44] Call Trace: [ 4548.203612][ T44] [ 4548.203732][ T44] __schedule+0x90a/0x1930 [ 4548.203990][ T44] ? io_schedule_timeout+0x150/0x150 [ 4548.204251][ T44] ? __lock_release+0x5d/0x160 [ 4548.204476][ T44] schedule+0xe2/0x270 [ 4548.204643][ T44] schedule_preempt_disabled+0x18/0x30 [ 4548.204868][ T44] rwsem_down_write_slowpath+0x3c5/0xc60 [ 4548.205091][ T44] ? rwsem_down_read_slowpath+0xd80/0xd80 [ 4548.205317][ T44] ? rcu_is_watching+0x12/0xb0 [ 4548.205545][ T44] ? lock_acquire+0x104/0x140 [ 4548.205770][ T44] down_write+0x1d6/0x200 [ 4548.205957][ T44] ? rwsem_down_write_slowpath+0xc60/0xc60 [ 4548.206230][ T44] ? __create_object+0x5e/0xb0 [ 4548.206447][ T44] ? __lock_release+0x5d/0x160 [ 4548.206669][ T44] register_pernet_subsys+0x1a/0x40 [ 4548.206926][ T44] ? conntrack_mt_exit+0xcf0/0xcf0 [xt_conntrack] [ 4548.207217][ T44] mpls_init+0x17/0x1000 [mpls_router] [ 4548.207441][ T44] ? conntrack_mt_exit+0xcf0/0xcf0 [xt_conntrack] [ 4548.207706][ T44] do_one_initcall+0xc5/0x3d0 [ 4548.207943][ T44] ? trace_event_raw_event_initcall_level+0x1f0/0x1f0 [ 4548.208217][ T44] ? rcu_is_watching+0x12/0xb0 [ 4548.208443][ T44] ? trace_kmalloc+0x24/0xb0 [ 4548.208666][ T44] ? __kmalloc_cache_noprof+0x2e4/0x710 [ 4548.208900][ T44] ? debug_objects_fill_pool+0x3e0/0x3e0 [ 4548.209119][ T44] ? kasan_unpoison+0x27/0x50 [ 4548.209340][ T44] do_init_module+0x27a/0x860 [ 4548.209563][ T44] ? free_module+0x3f0/0x3f0 [ 4548.209778][ T44] ? kfree+0x2bb/0x5f0 [ 4548.209958][ T44] ? nft_chain_route_fini+0x680/0x680 [nf_tables] [ 4548.210242][ T44] ? load_module+0x20c5/0x2dd0 [ 4548.210474][ T44] load_module+0x1f40/0x2dd0 [ 4548.210705][ T44] ? layout_and_allocate.constprop.0+0x630/0x630 [ 4548.211001][ T44] ? kernel_read_file+0x3ed/0x550 [ 4548.211229][ T44] ? kernel_read_file+0x3c8/0x550 [ 4548.211453][ T44] ? __ia32_sys_fsconfig+0x150/0x150 [ 4548.211683][ T44] ? init_module_from_file+0x159/0x180 [ 4548.211915][ T44] init_module_from_file+0x159/0x180 [ 4548.212144][ T44] ? __do_sys_init_module+0x1e0/0x1e0 [ 4548.212368][ T44] ? __lock_acquire+0x664/0xca0 [ 4548.212601][ T44] ? idempotent_init_module+0x316/0x610 [ 4548.212828][ T44] ? __lock_release+0x5d/0x160 [ 4548.213055][ T44] ? do_raw_spin_unlock+0x58/0x220 [ 4548.213275][ T44] idempotent_init_module+0x331/0x610 [ 4548.213495][ T44] ? init_module_from_file+0x180/0x180 [ 4548.213739][ T44] ? rcu_is_watching+0x12/0xb0 [ 4548.213971][ T44] ? cap_capable+0x17b/0x340 [ 4548.214198][ T44] __x64_sys_finit_module+0xca/0x150 [ 4548.214413][ T44] ? do_syscall_64+0x85/0xfc0 [ 4548.214632][ T44] do_syscall_64+0xc1/0xfc0 [ 4548.214857][ T44] entry_SYSCALL_64_after_hwframe+0x4b/0x53 [ 4548.215129][ T44] RIP: 0033:0x7f7e31c1c92d [ 4548.215355][ T44] RSP: 002b:00007fff4d76cb68 EFLAGS: 00000246 ORIG_RAX: 0000000000000139 [ 4548.215688][ T44] RAX: ffffffffffffffda RBX: 0000565009fefdf0 RCX: 00007f7e31c1c92d [ 4548.216042][ T44] RDX: 0000000000000000 RSI: 00005650052c9a2a RDI: 0000000000000000 [ 4548.216369][ T44] RBP: 0000000000040000 R08: 0000000000000000 R09: 0000000000000000 [ 4548.216692][ T44] R10: 0000000000000000 R11: 0000000000000246 R12: 00005650052c9a2a [ 4548.217039][ T44] R13: 0000565009ff0000 R14: 0000565009ff02a0 R15: 0000000000000000 [ 4548.217389][ T44] [ 4548.217553][ T44] INFO: task modprobe:1447 is blocked on an rw-semaphore, but the owner is not found. [ 4548.217929][ T44] [ 4548.217929][ T44] Showing all locks held in the system: [ 4548.218250][ T44] 1 lock held by khungtaskd/44: [ 4548.218465][ T44] #0: ffffffff9f17fe20 (rcu_read_lock){....}-{1:3}, at: debug_show_all_locks+0x36/0x260 [ 4548.218985][ T44] 2 locks held by kworker/u20:1/63: [ 4548.219192][ T44] #0: ffff888001099d48 ((wq_completion)events_unbound#2){+.+.}-{0:0}, at: process_one_work+0x1034/0x1810 [ 4548.219605][ T44] #1: ffffc90000447cb8 ((work_completion)(&sub_info->work)){+.+.}-{0:0}, at: process_one_work+0x802/0x1810 [ 4548.220086][ T44] 4 locks held by kworker/u16:1/70: [ 4548.220301][ T44] 1 lock held by modprobe/1447: [ 4548.220511][ T44] #0: ffffffff9f868cd0 (pernet_ops_rwsem){++++}-{4:4}, at: register_pernet_subsys+0x1a/0x40 [ 4548.220935][ T44] [ 4548.221039][ T44] ============================================= [ 4548.221039][ T44] [ 4671.081865][ T44] INFO: task modprobe:1447 blocked for more than 614 seconds. [ 4671.082319][ T44] Tainted: G W 6.18.0-virtme #1 [ 4671.082573][ T44] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. [ 4671.082964][ T44] task:modprobe state:D stack:25208 pid:1447 tgid:1447 ppid:63 task_flags:0x400100 flags:0x00080000 [ 4671.083475][ T44] Call Trace: [ 4671.083641][ T44] [ 4671.083758][ T44] __schedule+0x90a/0x1930 [ 4671.084018][ T44] ? io_schedule_timeout+0x150/0x150 [ 4671.084242][ T44] ? __lock_release+0x5d/0x160 [ 4671.084473][ T44] schedule+0xe2/0x270 [ 4671.084639][ T44] schedule_preempt_disabled+0x18/0x30 [ 4671.084862][ T44] rwsem_down_write_slowpath+0x3c5/0xc60 [ 4671.085096][ T44] ? rwsem_down_read_slowpath+0xd80/0xd80 [ 4671.085326][ T44] ? rcu_is_watching+0x12/0xb0 [ 4671.085550][ T44] ? lock_acquire+0x104/0x140 [ 4671.085776][ T44] down_write+0x1d6/0x200 [ 4671.085959][ T44] ? rwsem_down_write_slowpath+0xc60/0xc60 [ 4671.086234][ T44] ? __create_object+0x5e/0xb0 [ 4671.086452][ T44] ? __lock_release+0x5d/0x160 [ 4671.086685][ T44] register_pernet_subsys+0x1a/0x40 [ 4671.086941][ T44] ? conntrack_mt_exit+0xcf0/0xcf0 [xt_conntrack] [ 4671.087215][ T44] mpls_init+0x17/0x1000 [mpls_router] [ 4671.087436][ T44] ? conntrack_mt_exit+0xcf0/0xcf0 [xt_conntrack] [ 4671.087718][ T44] do_one_initcall+0xc5/0x3d0 [ 4671.087950][ T44] ? trace_event_raw_event_initcall_level+0x1f0/0x1f0 [ 4671.088221][ T44] ? rcu_is_watching+0x12/0xb0 [ 4671.088451][ T44] ? trace_kmalloc+0x24/0xb0 [ 4671.088665][ T44] ? __kmalloc_cache_noprof+0x2e4/0x710 [ 4671.088893][ T44] ? debug_objects_fill_pool+0x3e0/0x3e0 [ 4671.089115][ T44] ? kasan_unpoison+0x27/0x50 [ 4671.089334][ T44] do_init_module+0x27a/0x860 [ 4671.089558][ T44] ? free_module+0x3f0/0x3f0 [ 4671.089775][ T44] ? kfree+0x2bb/0x5f0 [ 4671.089953][ T44] ? nft_chain_route_fini+0x680/0x680 [nf_tables] [ 4671.090245][ T44] ? load_module+0x20c5/0x2dd0 [ 4671.090468][ T44] load_module+0x1f40/0x2dd0 [ 4671.090695][ T44] ? layout_and_allocate.constprop.0+0x630/0x630 [ 4671.090989][ T44] ? kernel_read_file+0x3ed/0x550 [ 4671.091213][ T44] ? kernel_read_file+0x3c8/0x550 [ 4671.091434][ T44] ? __ia32_sys_fsconfig+0x150/0x150 [ 4671.091659][ T44] ? init_module_from_file+0x159/0x180 [ 4671.091898][ T44] init_module_from_file+0x159/0x180 [ 4671.092114][ T44] ? __do_sys_init_module+0x1e0/0x1e0 [ 4671.092331][ T44] ? __lock_acquire+0x664/0xca0 [ 4671.092557][ T44] ? idempotent_init_module+0x316/0x610 [ 4671.092770][ T44] ? __lock_release+0x5d/0x160 [ 4671.092999][ T44] ? do_raw_spin_unlock+0x58/0x220 [ 4671.093218][ T44] idempotent_init_module+0x331/0x610 [ 4671.093448][ T44] ? init_module_from_file+0x180/0x180 [ 4671.093671][ T44] ? rcu_is_watching+0x12/0xb0 [ 4671.093895][ T44] ? cap_capable+0x17b/0x340 [ 4671.094117][ T44] __x64_sys_finit_module+0xca/0x150 [ 4671.094333][ T44] ? do_syscall_64+0x85/0xfc0 [ 4671.094551][ T44] do_syscall_64+0xc1/0xfc0 [ 4671.094767][ T44] entry_SYSCALL_64_after_hwframe+0x4b/0x53 [ 4671.095067][ T44] RIP: 0033:0x7f7e31c1c92d [ 4671.095295][ T44] RSP: 002b:00007fff4d76cb68 EFLAGS: 00000246 ORIG_RAX: 0000000000000139 [ 4671.095621][ T44] RAX: ffffffffffffffda RBX: 0000565009fefdf0 RCX: 00007f7e31c1c92d [ 4671.095956][ T44] RDX: 0000000000000000 RSI: 00005650052c9a2a RDI: 0000000000000000 [ 4671.096280][ T44] RBP: 0000000000040000 R08: 0000000000000000 R09: 0000000000000000 [ 4671.096618][ T44] R10: 0000000000000000 R11: 0000000000000246 R12: 00005650052c9a2a [ 4671.096952][ T44] R13: 0000565009ff0000 R14: 0000565009ff02a0 R15: 0000000000000000 [ 4671.097290][ T44] [ 4671.097453][ T44] INFO: task modprobe:1447 is blocked on an rw-semaphore, but the owner is not found. [ 4671.097819][ T44] [ 4671.097819][ T44] Showing all locks held in the system: [ 4671.098143][ T44] 1 lock held by khungtaskd/44: [ 4671.098346][ T44] #0: ffffffff9f17fe20 (rcu_read_lock){....}-{1:3}, at: debug_show_all_locks+0x36/0x260 [ 4671.098716][ T44] 2 locks held by kworker/u20:1/63: [ 4671.099070][ T44] #0: ffff888001099d48 ((wq_completion)events_unbound#2){+.+.}-{0:0}, at: process_one_work+0x1034/0x1810 [ 4671.099494][ T44] #1: ffffc90000447cb8 ((work_completion)(&sub_info->work)){+.+.}-{0:0}, at: process_one_work+0x802/0x1810 [ 4671.099961][ T44] 4 locks held by kworker/u16:1/70: [ 4671.100174][ T44] 1 lock held by modprobe/1447: [ 4671.100374][ T44] #0: ffffffff9f868cd0 (pernet_ops_rwsem){++++}-{4:4}, at: register_pernet_subsys+0x1a/0x40 [ 4671.100793][ T44] [ 4671.100911][ T44] ============================================= [ 4671.100911][ T44] [ 4793.961861][ T44] INFO: task modprobe:1447 blocked for more than 737 seconds. [ 4793.962310][ T44] Tainted: G W 6.18.0-virtme #1 [ 4793.962560][ T44] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. [ 4793.962955][ T44] task:modprobe state:D stack:25208 pid:1447 tgid:1447 ppid:63 task_flags:0x400100 flags:0x00080000 [ 4793.963445][ T44] Call Trace: [ 4793.963619][ T44] [ 4793.963736][ T44] __schedule+0x90a/0x1930 [ 4793.963981][ T44] ? io_schedule_timeout+0x150/0x150 [ 4793.964205][ T44] ? __lock_release+0x5d/0x160 [ 4793.964428][ T44] schedule+0xe2/0x270 [ 4793.964595][ T44] schedule_preempt_disabled+0x18/0x30 [ 4793.964837][ T44] rwsem_down_write_slowpath+0x3c5/0xc60 [ 4793.965058][ T44] ? rwsem_down_read_slowpath+0xd80/0xd80 [ 4793.965285][ T44] ? rcu_is_watching+0x12/0xb0 [ 4793.965504][ T44] ? lock_acquire+0x104/0x140 [ 4793.965730][ T44] down_write+0x1d6/0x200 [ 4793.965917][ T44] ? rwsem_down_write_slowpath+0xc60/0xc60 [ 4793.966190][ T44] ? __create_object+0x5e/0xb0 [ 4793.966418][ T44] ? __lock_release+0x5d/0x160 [ 4793.966635][ T44] register_pernet_subsys+0x1a/0x40 [ 4793.966883][ T44] ? conntrack_mt_exit+0xcf0/0xcf0 [xt_conntrack] [ 4793.967152][ T44] mpls_init+0x17/0x1000 [mpls_router] [ 4793.967371][ T44] ? conntrack_mt_exit+0xcf0/0xcf0 [xt_conntrack] [ 4793.967640][ T44] do_one_initcall+0xc5/0x3d0 [ 4793.967866][ T44] ? trace_event_raw_event_initcall_level+0x1f0/0x1f0 [ 4793.968155][ T44] ? rcu_is_watching+0x12/0xb0 [ 4793.968371][ T44] ? trace_kmalloc+0x24/0xb0 [ 4793.968586][ T44] ? __kmalloc_cache_noprof+0x2e4/0x710 [ 4793.968810][ T44] ? debug_objects_fill_pool+0x3e0/0x3e0 [ 4793.969031][ T44] ? kasan_unpoison+0x27/0x50 [ 4793.969250][ T44] do_init_module+0x27a/0x860 [ 4793.969470][ T44] ? free_module+0x3f0/0x3f0 [ 4793.969691][ T44] ? kfree+0x2bb/0x5f0 [ 4793.969865][ T44] ? nft_chain_route_fini+0x680/0x680 [nf_tables] [ 4793.970153][ T44] ? load_module+0x20c5/0x2dd0 [ 4793.970374][ T44] load_module+0x1f40/0x2dd0 [ 4793.970599][ T44] ? layout_and_allocate.constprop.0+0x630/0x630 [ 4793.970876][ T44] ? kernel_read_file+0x3ed/0x550 [ 4793.971096][ T44] ? kernel_read_file+0x3c8/0x550 [ 4793.971315][ T44] ? __ia32_sys_fsconfig+0x150/0x150 [ 4793.971544][ T44] ? init_module_from_file+0x159/0x180 [ 4793.971765][ T44] init_module_from_file+0x159/0x180 [ 4793.972007][ T44] ? __do_sys_init_module+0x1e0/0x1e0 [ 4793.972229][ T44] ? __lock_acquire+0x664/0xca0 [ 4793.972457][ T44] ? idempotent_init_module+0x316/0x610 [ 4793.972671][ T44] ? __lock_release+0x5d/0x160 [ 4793.972899][ T44] ? do_raw_spin_unlock+0x58/0x220 [ 4793.973128][ T44] idempotent_init_module+0x331/0x610 [ 4793.973348][ T44] ? init_module_from_file+0x180/0x180 [ 4793.973572][ T44] ? rcu_is_watching+0x12/0xb0 [ 4793.973795][ T44] ? cap_capable+0x17b/0x340 [ 4793.974020][ T44] __x64_sys_finit_module+0xca/0x150 [ 4793.974233][ T44] ? do_syscall_64+0x85/0xfc0 [ 4793.974451][ T44] do_syscall_64+0xc1/0xfc0 [ 4793.974672][ T44] entry_SYSCALL_64_after_hwframe+0x4b/0x53 [ 4793.974959][ T44] RIP: 0033:0x7f7e31c1c92d [ 4793.975187][ T44] RSP: 002b:00007fff4d76cb68 EFLAGS: 00000246 ORIG_RAX: 0000000000000139 [ 4793.975513][ T44] RAX: ffffffffffffffda RBX: 0000565009fefdf0 RCX: 00007f7e31c1c92d [ 4793.975850][ T44] RDX: 0000000000000000 RSI: 00005650052c9a2a RDI: 0000000000000000 [ 4793.976189][ T44] RBP: 0000000000040000 R08: 0000000000000000 R09: 0000000000000000 [ 4793.976516][ T44] R10: 0000000000000000 R11: 0000000000000246 R12: 00005650052c9a2a [ 4793.976863][ T44] R13: 0000565009ff0000 R14: 0000565009ff02a0 R15: 0000000000000000 [ 4793.977206][ T44] [ 4793.977369][ T44] INFO: task modprobe:1447 is blocked on an rw-semaphore, but the owner is not found. [ 4793.977735][ T44] [ 4793.977735][ T44] Showing all locks held in the system: [ 4793.978202][ T44] 1 lock held by khungtaskd/44: [ 4793.978410][ T44] #0: ffffffff9f17fe20 (rcu_read_lock){....}-{1:3}, at: debug_show_all_locks+0x36/0x260 [ 4793.978775][ T44] 2 locks held by kworker/u20:1/63: [ 4793.978987][ T44] #0: ffff888001099d48 ((wq_completion)events_unbound#2){+.+.}-{0:0}, at: process_one_work+0x1034/0x1810 [ 4793.979404][ T44] #1: ffffc90000447cb8 ((work_completion)(&sub_info->work)){+.+.}-{0:0}, at: process_one_work+0x802/0x1810 [ 4793.979875][ T44] 4 locks held by kworker/u16:1/70: [ 4793.980089][ T44] 1 lock held by modprobe/1447: [ 4793.980291][ T44] #0: ffffffff9f868cd0 (pernet_ops_rwsem){++++}-{4:4}, at: register_pernet_subsys+0x1a/0x40 [ 4793.980708][ T44] [ 4793.980839][ T44] ============================================= [ 4793.980839][ T44] [ 4916.840864][ T44] INFO: task modprobe:1447 blocked for more than 860 seconds. [ 4916.841313][ T44] Tainted: G W 6.18.0-virtme #1 [ 4916.841561][ T44] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. [ 4916.841964][ T44] task:modprobe state:D stack:25208 pid:1447 tgid:1447 ppid:63 task_flags:0x400100 flags:0x00080000 [ 4916.842436][ T44] Call Trace: [ 4916.842601][ T44] [ 4916.842720][ T44] __schedule+0x90a/0x1930 [ 4916.842967][ T44] ? io_schedule_timeout+0x150/0x150 [ 4916.843190][ T44] ? __lock_release+0x5d/0x160 [ 4916.843438][ T44] schedule+0xe2/0x270 [ 4916.843605][ T44] schedule_preempt_disabled+0x18/0x30 [ 4916.843832][ T44] rwsem_down_write_slowpath+0x3c5/0xc60 [ 4916.844054][ T44] ? rwsem_down_read_slowpath+0xd80/0xd80 [ 4916.844281][ T44] ? rcu_is_watching+0x12/0xb0 [ 4916.844498][ T44] ? lock_acquire+0x104/0x140 [ 4916.844723][ T44] down_write+0x1d6/0x200 [ 4916.844898][ T44] ? rwsem_down_write_slowpath+0xc60/0xc60 [ 4916.845182][ T44] ? __create_object+0x5e/0xb0 [ 4916.845402][ T44] ? __lock_release+0x5d/0x160 [ 4916.845618][ T44] register_pernet_subsys+0x1a/0x40 [ 4916.845869][ T44] ? conntrack_mt_exit+0xcf0/0xcf0 [xt_conntrack] [ 4916.846140][ T44] mpls_init+0x17/0x1000 [mpls_router] [ 4916.846361][ T44] ? conntrack_mt_exit+0xcf0/0xcf0 [xt_conntrack] [ 4916.846638][ T44] do_one_initcall+0xc5/0x3d0 [ 4916.846867][ T44] ? trace_event_raw_event_initcall_level+0x1f0/0x1f0 [ 4916.847137][ T44] ? rcu_is_watching+0x12/0xb0 [ 4916.847356][ T44] ? trace_kmalloc+0x24/0xb0 [ 4916.847573][ T44] ? __kmalloc_cache_noprof+0x2e4/0x710 [ 4916.847799][ T44] ? debug_objects_fill_pool+0x3e0/0x3e0 [ 4916.848018][ T44] ? kasan_unpoison+0x27/0x50 [ 4916.848246][ T44] do_init_module+0x27a/0x860 [ 4916.848469][ T44] ? free_module+0x3f0/0x3f0 [ 4916.848686][ T44] ? kfree+0x2bb/0x5f0 [ 4916.848863][ T44] ? nft_chain_route_fini+0x680/0x680 [nf_tables] [ 4916.849152][ T44] ? load_module+0x20c5/0x2dd0 [ 4916.849385][ T44] load_module+0x1f40/0x2dd0 [ 4916.849614][ T44] ? layout_and_allocate.constprop.0+0x630/0x630 [ 4916.849910][ T44] ? kernel_read_file+0x3ed/0x550 [ 4916.850139][ T44] ? kernel_read_file+0x3c8/0x550 [ 4916.850359][ T44] ? __ia32_sys_fsconfig+0x150/0x150 [ 4916.850583][ T44] ? init_module_from_file+0x159/0x180 [ 4916.850807][ T44] init_module_from_file+0x159/0x180 [ 4916.851026][ T44] ? __do_sys_init_module+0x1e0/0x1e0 [ 4916.851244][ T44] ? __lock_acquire+0x664/0xca0 [ 4916.851472][ T44] ? idempotent_init_module+0x316/0x610 [ 4916.851698][ T44] ? __lock_release+0x5d/0x160 [ 4916.851925][ T44] ? do_raw_spin_unlock+0x58/0x220 [ 4916.852146][ T44] idempotent_init_module+0x331/0x610 [ 4916.852364][ T44] ? init_module_from_file+0x180/0x180 [ 4916.852591][ T44] ? rcu_is_watching+0x12/0xb0 [ 4916.852814][ T44] ? cap_capable+0x17b/0x340 [ 4916.853039][ T44] __x64_sys_finit_module+0xca/0x150 [ 4916.853265][ T44] ? do_syscall_64+0x85/0xfc0 [ 4916.853482][ T44] do_syscall_64+0xc1/0xfc0 [ 4916.853700][ T44] entry_SYSCALL_64_after_hwframe+0x4b/0x53 [ 4916.853991][ T44] RIP: 0033:0x7f7e31c1c92d [ 4916.854219][ T44] RSP: 002b:00007fff4d76cb68 EFLAGS: 00000246 ORIG_RAX: 0000000000000139 [ 4916.854547][ T44] RAX: ffffffffffffffda RBX: 0000565009fefdf0 RCX: 00007f7e31c1c92d [ 4916.854889][ T44] RDX: 0000000000000000 RSI: 00005650052c9a2a RDI: 0000000000000000 [ 4916.855226][ T44] RBP: 0000000000040000 R08: 0000000000000000 R09: 0000000000000000 [ 4916.855551][ T44] R10: 0000000000000000 R11: 0000000000000246 R12: 00005650052c9a2a [ 4916.855886][ T44] R13: 0000565009ff0000 R14: 0000565009ff02a0 R15: 0000000000000000 [ 4916.856237][ T44] [ 4916.856403][ T44] INFO: task modprobe:1447 is blocked on an rw-semaphore, but the owner is not found. [ 4916.856765][ T44] [ 4916.856765][ T44] Showing all locks held in the system: [ 4916.857226][ T44] 1 lock held by khungtaskd/44: [ 4916.857434][ T44] #0: ffffffff9f17fe20 (rcu_read_lock){....}-{1:3}, at: debug_show_all_locks+0x36/0x260 [ 4916.857817][ T44] 2 locks held by kworker/u20:1/63: [ 4916.858025][ T44] #0: ffff888001099d48 ((wq_completion)events_unbound#2){+.+.}-{0:0}, at: process_one_work+0x1034/0x1810 [ 4916.858439][ T44] #1: ffffc90000447cb8 ((work_completion)(&sub_info->work)){+.+.}-{0:0}, at: process_one_work+0x802/0x1810 [ 4916.858911][ T44] 3 locks held by kworker/u16:1/70: [ 4916.859124][ T44] 1 lock held by modprobe/1447: [ 4916.859334][ T44] #0: ffffffff9f868cd0 (pernet_ops_rwsem){++++}-{4:4}, at: register_pernet_subsys+0x1a/0x40 [ 4916.859748][ T44] [ 4916.859877][ T44] ============================================= [ 4916.859877][ T44] [ 5039.721894][ T44] INFO: task modprobe:1447 blocked for more than 983 seconds. [ 5039.722288][ T44] Tainted: G W 6.18.0-virtme #1 [ 5039.722546][ T44] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. [ 5039.722946][ T44] task:modprobe state:D stack:25208 pid:1447 tgid:1447 ppid:63 task_flags:0x400100 flags:0x00080000 [ 5039.723413][ T44] Call Trace: [ 5039.723581][ T44] [ 5039.723696][ T44] __schedule+0x90a/0x1930 [ 5039.723939][ T44] ? io_schedule_timeout+0x150/0x150 [ 5039.724175][ T44] ? __lock_release+0x5d/0x160 [ 5039.724396][ T44] schedule+0xe2/0x270 [ 5039.724571][ T44] schedule_preempt_disabled+0x18/0x30 [ 5039.724799][ T44] rwsem_down_write_slowpath+0x3c5/0xc60 [ 5039.725021][ T44] ? rwsem_down_read_slowpath+0xd80/0xd80 [ 5039.725248][ T44] ? rcu_is_watching+0x12/0xb0 [ 5039.725464][ T44] ? lock_acquire+0x104/0x140 [ 5039.725695][ T44] down_write+0x1d6/0x200 [ 5039.725867][ T44] ? rwsem_down_write_slowpath+0xc60/0xc60 [ 5039.726138][ T44] ? __create_object+0x5e/0xb0 [ 5039.726362][ T44] ? __lock_release+0x5d/0x160 [ 5039.726588][ T44] register_pernet_subsys+0x1a/0x40 [ 5039.726834][ T44] ? conntrack_mt_exit+0xcf0/0xcf0 [xt_conntrack] [ 5039.727104][ T44] mpls_init+0x17/0x1000 [mpls_router] [ 5039.727336][ T44] ? conntrack_mt_exit+0xcf0/0xcf0 [xt_conntrack] [ 5039.727602][ T44] do_one_initcall+0xc5/0x3d0 [ 5039.727833][ T44] ? trace_event_raw_event_initcall_level+0x1f0/0x1f0 [ 5039.728107][ T44] ? rcu_is_watching+0x12/0xb0 [ 5039.728322][ T44] ? trace_kmalloc+0x24/0xb0 [ 5039.728542][ T44] ? __kmalloc_cache_noprof+0x2e4/0x710 [ 5039.728757][ T44] ? debug_objects_fill_pool+0x3e0/0x3e0 [ 5039.728991][ T44] ? kasan_unpoison+0x27/0x50 [ 5039.729212][ T44] do_init_module+0x27a/0x860 [ 5039.729438][ T44] ? free_module+0x3f0/0x3f0 [ 5039.729657][ T44] ? kfree+0x2bb/0x5f0 [ 5039.729828][ T44] ? nft_chain_route_fini+0x680/0x680 [nf_tables] [ 5039.730112][ T44] ? load_module+0x20c5/0x2dd0 [ 5039.730332][ T44] load_module+0x1f40/0x2dd0 [ 5039.730559][ T44] ? layout_and_allocate.constprop.0+0x630/0x630 [ 5039.730856][ T44] ? kernel_read_file+0x3ed/0x550 [ 5039.731081][ T44] ? kernel_read_file+0x3c8/0x550 [ 5039.731300][ T44] ? __ia32_sys_fsconfig+0x150/0x150 [ 5039.731521][ T44] ? init_module_from_file+0x159/0x180 [ 5039.731741][ T44] init_module_from_file+0x159/0x180 [ 5039.731964][ T44] ? __do_sys_init_module+0x1e0/0x1e0 [ 5039.732192][ T44] ? __lock_acquire+0x664/0xca0 [ 5039.732421][ T44] ? idempotent_init_module+0x316/0x610 [ 5039.732640][ T44] ? __lock_release+0x5d/0x160 [ 5039.732866][ T44] ? do_raw_spin_unlock+0x58/0x220 [ 5039.733084][ T44] idempotent_init_module+0x331/0x610 [ 5039.733302][ T44] ? init_module_from_file+0x180/0x180 [ 5039.733526][ T44] ? rcu_is_watching+0x12/0xb0 [ 5039.733756][ T44] ? cap_capable+0x17b/0x340 [ 5039.734005][ T44] __x64_sys_finit_module+0xca/0x150 [ 5039.734220][ T44] ? do_syscall_64+0x85/0xfc0 [ 5039.734439][ T44] do_syscall_64+0xc1/0xfc0 [ 5039.734666][ T44] entry_SYSCALL_64_after_hwframe+0x4b/0x53 [ 5039.734954][ T44] RIP: 0033:0x7f7e31c1c92d [ 5039.735180][ T44] RSP: 002b:00007fff4d76cb68 EFLAGS: 00000246 ORIG_RAX: 0000000000000139 [ 5039.735516][ T44] RAX: ffffffffffffffda RBX: 0000565009fefdf0 RCX: 00007f7e31c1c92d [ 5039.735849][ T44] RDX: 0000000000000000 RSI: 00005650052c9a2a RDI: 0000000000000000 [ 5039.736171][ T44] RBP: 0000000000040000 R08: 0000000000000000 R09: 0000000000000000 [ 5039.736494][ T44] R10: 0000000000000000 R11: 0000000000000246 R12: 00005650052c9a2a [ 5039.736841][ T44] R13: 0000565009ff0000 R14: 0000565009ff02a0 R15: 0000000000000000 [ 5039.737181][ T44] [ 5039.737343][ T44] INFO: task modprobe:1447 is blocked on an rw-semaphore, but the owner is not found. [ 5039.737706][ T44] [ 5039.737706][ T44] Showing all locks held in the system: [ 5039.738166][ T44] 1 lock held by khungtaskd/44: [ 5039.738374][ T44] #0: ffffffff9f17fe20 (rcu_read_lock){....}-{1:3}, at: debug_show_all_locks+0x36/0x260 [ 5039.738750][ T44] 2 locks held by kworker/u20:1/63: [ 5039.738961][ T44] #0: ffff888001099d48 ((wq_completion)events_unbound#2){+.+.}-{0:0}, at: process_one_work+0x1034/0x1810 [ 5039.739372][ T44] #1: ffffc90000447cb8 ((work_completion)(&sub_info->work)){+.+.}-{0:0}, at: process_one_work+0x802/0x1810 [ 5039.739852][ T44] 4 locks held by kworker/u16:1/70: [ 5039.740064][ T44] 1 lock held by modprobe/1447: [ 5039.740270][ T44] #0: ffffffff9f868cd0 (pernet_ops_rwsem){++++}-{4:4}, at: register_pernet_subsys+0x1a/0x40 [ 5039.740678][ T44] [ 5039.740785][ T44] ============================================= [ 5039.740785][ T44] [ 5162.601868][ T44] INFO: task modprobe:1447 blocked for more than 1105 seconds. [ 5162.602327][ T44] Tainted: G W 6.18.0-virtme #1 [ 5162.602575][ T44] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. [ 5162.602948][ T44] task:modprobe state:D stack:25208 pid:1447 tgid:1447 ppid:63 task_flags:0x400100 flags:0x00080000 [ 5162.603417][ T44] Call Trace: [ 5162.603578][ T44] [ 5162.603704][ T44] __schedule+0x90a/0x1930 [ 5162.603958][ T44] ? io_schedule_timeout+0x150/0x150 [ 5162.604186][ T44] ? __lock_release+0x5d/0x160 [ 5162.604408][ T44] schedule+0xe2/0x270 [ 5162.604574][ T44] schedule_preempt_disabled+0x18/0x30 [ 5162.604798][ T44] rwsem_down_write_slowpath+0x3c5/0xc60 [ 5162.605021][ T44] ? rwsem_down_read_slowpath+0xd80/0xd80 [ 5162.605253][ T44] ? rcu_is_watching+0x12/0xb0 [ 5162.605469][ T44] ? lock_acquire+0x104/0x140 [ 5162.605705][ T44] down_write+0x1d6/0x200 [ 5162.605881][ T44] ? rwsem_down_write_slowpath+0xc60/0xc60 [ 5162.606153][ T44] ? __create_object+0x5e/0xb0 [ 5162.606368][ T44] ? __lock_release+0x5d/0x160 [ 5162.606584][ T44] register_pernet_subsys+0x1a/0x40 [ 5162.606812][ T44] ? conntrack_mt_exit+0xcf0/0xcf0 [xt_conntrack] [ 5162.607090][ T44] mpls_init+0x17/0x1000 [mpls_router] [ 5162.607311][ T44] ? conntrack_mt_exit+0xcf0/0xcf0 [xt_conntrack] [ 5162.607576][ T44] do_one_initcall+0xc5/0x3d0 [ 5162.607801][ T44] ? trace_event_raw_event_initcall_level+0x1f0/0x1f0 [ 5162.608072][ T44] ? rcu_is_watching+0x12/0xb0 [ 5162.608287][ T44] ? trace_kmalloc+0x24/0xb0 [ 5162.608506][ T44] ? __kmalloc_cache_noprof+0x2e4/0x710 [ 5162.608727][ T44] ? debug_objects_fill_pool+0x3e0/0x3e0 [ 5162.608955][ T44] ? kasan_unpoison+0x27/0x50 [ 5162.609174][ T44] do_init_module+0x27a/0x860 [ 5162.609395][ T44] ? free_module+0x3f0/0x3f0 [ 5162.609610][ T44] ? kfree+0x2bb/0x5f0 [ 5162.609775][ T44] ? nft_chain_route_fini+0x680/0x680 [nf_tables] [ 5162.610073][ T44] ? load_module+0x20c5/0x2dd0 [ 5162.610299][ T44] load_module+0x1f40/0x2dd0 [ 5162.610528][ T44] ? layout_and_allocate.constprop.0+0x630/0x630 [ 5162.610803][ T44] ? kernel_read_file+0x3ed/0x550 [ 5162.611022][ T44] ? kernel_read_file+0x3c8/0x550 [ 5162.611241][ T44] ? __ia32_sys_fsconfig+0x150/0x150 [ 5162.611466][ T44] ? init_module_from_file+0x159/0x180 [ 5162.611681][ T44] init_module_from_file+0x159/0x180 [ 5162.611914][ T44] ? __do_sys_init_module+0x1e0/0x1e0 [ 5162.612137][ T44] ? __lock_acquire+0x664/0xca0 [ 5162.612363][ T44] ? idempotent_init_module+0x316/0x610 [ 5162.612575][ T44] ? __lock_release+0x5d/0x160 [ 5162.612809][ T44] ? do_raw_spin_unlock+0x58/0x220 [ 5162.613030][ T44] idempotent_init_module+0x331/0x610 [ 5162.613248][ T44] ? init_module_from_file+0x180/0x180 [ 5162.613493][ T44] ? rcu_is_watching+0x12/0xb0 [ 5162.613708][ T44] ? cap_capable+0x17b/0x340 [ 5162.613943][ T44] __x64_sys_finit_module+0xca/0x150 [ 5162.614159][ T44] ? do_syscall_64+0x85/0xfc0 [ 5162.614379][ T44] do_syscall_64+0xc1/0xfc0 [ 5162.614595][ T44] entry_SYSCALL_64_after_hwframe+0x4b/0x53 [ 5162.614868][ T44] RIP: 0033:0x7f7e31c1c92d [ 5162.615126][ T44] RSP: 002b:00007fff4d76cb68 EFLAGS: 00000246 ORIG_RAX: 0000000000000139 [ 5162.615454][ T44] RAX: ffffffffffffffda RBX: 0000565009fefdf0 RCX: 00007f7e31c1c92d [ 5162.615782][ T44] RDX: 0000000000000000 RSI: 00005650052c9a2a RDI: 0000000000000000 [ 5162.616112][ T44] RBP: 0000000000040000 R08: 0000000000000000 R09: 0000000000000000 [ 5162.616434][ T44] R10: 0000000000000000 R11: 0000000000000246 R12: 00005650052c9a2a [ 5162.616765][ T44] R13: 0000565009ff0000 R14: 0000565009ff02a0 R15: 0000000000000000 [ 5162.617110][ T44] [ 5162.617277][ T44] INFO: task modprobe:1447 is blocked on an rw-semaphore, but the owner is not found. [ 5162.617617][ T44] [ 5162.617617][ T44] Showing all locks held in the system: [ 5162.617939][ T44] 1 lock held by khungtaskd/44: [ 5162.618141][ T44] #0: ffffffff9f17fe20 (rcu_read_lock){....}-{1:3}, at: debug_show_all_locks+0x36/0x260 [ 5162.618505][ T44] 2 locks held by kworker/u20:1/63: [ 5162.618711][ T44] #0: ffff888001099d48 ((wq_completion)events_unbound#2){+.+.}-{0:0}, at: process_one_work+0x1034/0x1810 [ 5162.619129][ T44] #1: ffffc90000447cb8 ((work_completion)(&sub_info->work)){+.+.}-{0:0}, at: process_one_work+0x802/0x1810 [ 5162.619595][ T44] 5 locks held by kworker/u16:1/70: [ 5162.619815][ T44] 1 lock held by modprobe/1447: [ 5162.620018][ T44] #0: ffffffff9f868cd0 (pernet_ops_rwsem){++++}-{4:4}, at: register_pernet_subsys+0x1a/0x40 [ 5162.620424][ T44] [ 5162.620527][ T44] ============================================= [ 5162.620527][ T44] [ 5285.480870][ T44] INFO: task modprobe:1447 blocked for more than 1228 seconds. [ 5285.481321][ T44] Tainted: G W 6.18.0-virtme #1 [ 5285.481572][ T44] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. [ 5285.481942][ T44] task:modprobe state:D stack:25208 pid:1447 tgid:1447 ppid:63 task_flags:0x400100 flags:0x00080000 [ 5285.482425][ T44] Call Trace: [ 5285.482593][ T44] [ 5285.482717][ T44] __schedule+0x90a/0x1930 [ 5285.482970][ T44] ? io_schedule_timeout+0x150/0x150 [ 5285.483219][ T44] ? __lock_release+0x5d/0x160 [ 5285.483449][ T44] schedule+0xe2/0x270 [ 5285.483616][ T44] schedule_preempt_disabled+0x18/0x30 [ 5285.483842][ T44] rwsem_down_write_slowpath+0x3c5/0xc60 [ 5285.484064][ T44] ? rwsem_down_read_slowpath+0xd80/0xd80 [ 5285.484292][ T44] ? rcu_is_watching+0x12/0xb0 [ 5285.484510][ T44] ? lock_acquire+0x104/0x140 [ 5285.484748][ T44] down_write+0x1d6/0x200 [ 5285.484920][ T44] ? rwsem_down_write_slowpath+0xc60/0xc60 [ 5285.485193][ T44] ? __create_object+0x5e/0xb0 [ 5285.485413][ T44] ? __lock_release+0x5d/0x160 [ 5285.485632][ T44] register_pernet_subsys+0x1a/0x40 [ 5285.485862][ T44] ? conntrack_mt_exit+0xcf0/0xcf0 [xt_conntrack] [ 5285.486131][ T44] mpls_init+0x17/0x1000 [mpls_router] [ 5285.486361][ T44] ? conntrack_mt_exit+0xcf0/0xcf0 [xt_conntrack] [ 5285.486627][ T44] do_one_initcall+0xc5/0x3d0 [ 5285.486855][ T44] ? trace_event_raw_event_initcall_level+0x1f0/0x1f0 [ 5285.487126][ T44] ? rcu_is_watching+0x12/0xb0 [ 5285.487343][ T44] ? trace_kmalloc+0x24/0xb0 [ 5285.487563][ T44] ? __kmalloc_cache_noprof+0x2e4/0x710 [ 5285.487782][ T44] ? debug_objects_fill_pool+0x3e0/0x3e0 [ 5285.488019][ T44] ? kasan_unpoison+0x27/0x50 [ 5285.488240][ T44] do_init_module+0x27a/0x860 [ 5285.488462][ T44] ? free_module+0x3f0/0x3f0 [ 5285.488678][ T44] ? kfree+0x2bb/0x5f0 [ 5285.488851][ T44] ? nft_chain_route_fini+0x680/0x680 [nf_tables] [ 5285.489135][ T44] ? load_module+0x20c5/0x2dd0 [ 5285.489358][ T44] load_module+0x1f40/0x2dd0 [ 5285.489585][ T44] ? layout_and_allocate.constprop.0+0x630/0x630 [ 5285.489872][ T44] ? kernel_read_file+0x3ed/0x550 [ 5285.490093][ T44] ? kernel_read_file+0x3c8/0x550 [ 5285.490313][ T44] ? __ia32_sys_fsconfig+0x150/0x150 [ 5285.490537][ T44] ? init_module_from_file+0x159/0x180 [ 5285.490753][ T44] init_module_from_file+0x159/0x180 [ 5285.490978][ T44] ? __do_sys_init_module+0x1e0/0x1e0 [ 5285.491204][ T44] ? __lock_acquire+0x664/0xca0 [ 5285.491433][ T44] ? idempotent_init_module+0x316/0x610 [ 5285.491649][ T44] ? __lock_release+0x5d/0x160 [ 5285.491876][ T44] ? do_raw_spin_unlock+0x58/0x220 [ 5285.492097][ T44] idempotent_init_module+0x331/0x610 [ 5285.492316][ T44] ? init_module_from_file+0x180/0x180 [ 5285.492542][ T44] ? rcu_is_watching+0x12/0xb0 [ 5285.492763][ T44] ? cap_capable+0x17b/0x340 [ 5285.492994][ T44] __x64_sys_finit_module+0xca/0x150 [ 5285.493217][ T44] ? do_syscall_64+0x85/0xfc0 [ 5285.493439][ T44] do_syscall_64+0xc1/0xfc0 [ 5285.493657][ T44] entry_SYSCALL_64_after_hwframe+0x4b/0x53 [ 5285.493933][ T44] RIP: 0033:0x7f7e31c1c92d [ 5285.494160][ T44] RSP: 002b:00007fff4d76cb68 EFLAGS: 00000246 ORIG_RAX: 0000000000000139 [ 5285.494498][ T44] RAX: ffffffffffffffda RBX: 0000565009fefdf0 RCX: 00007f7e31c1c92d [ 5285.494833][ T44] RDX: 0000000000000000 RSI: 00005650052c9a2a RDI: 0000000000000000 [ 5285.495158][ T44] RBP: 0000000000040000 R08: 0000000000000000 R09: 0000000000000000 [ 5285.495483][ T44] R10: 0000000000000000 R11: 0000000000000246 R12: 00005650052c9a2a [ 5285.495825][ T44] R13: 0000565009ff0000 R14: 0000565009ff02a0 R15: 0000000000000000 [ 5285.496168][ T44] [ 5285.496332][ T44] INFO: task modprobe:1447 is blocked on an rw-semaphore, but the owner is not found. [ 5285.496675][ T44] Future hung task reports are suppressed, see sysctl kernel.hung_task_warnings [ 5285.497031][ T44] [ 5285.497031][ T44] Showing all locks held in the system: [ 5285.497351][ T44] 1 lock held by khungtaskd/44: [ 5285.497558][ T44] #0: ffffffff9f17fe20 (rcu_read_lock){....}-{1:3}, at: debug_show_all_locks+0x36/0x260 [ 5285.497935][ T44] 2 locks held by kworker/u20:1/63: [ 5285.498139][ T44] #0: ffff888001099d48 ((wq_completion)events_unbound#2){+.+.}-{0:0}, at: process_one_work+0x1034/0x1810 [ 5285.498554][ T44] #1: ffffc90000447cb8 ((work_completion)(&sub_info->work)){+.+.}-{0:0}, at: process_one_work+0x802/0x1810 [ 5285.499036][ T44] 4 locks held by kworker/u16:1/70: [ 5285.499250][ T44] 1 lock held by modprobe/1447: [ 5285.499451][ T44] #0: ffffffff9f868cd0 (pernet_ops_rwsem){++++}-{4:4}, at: register_pernet_subsys+0x1a/0x40 [ 5285.499868][ T44] [ 5285.499972][ T44] ============================================= [ 5285.499972][ T44] WAIT TIMEOUT stderr Ctrl-C stderr Ctrl-C stderr