[ 4179.590335][ T44] INFO: task modprobe:18131 blocked for more than 122 seconds. [ 4179.590945][ T44] Not tainted 6.18.0-rc2-virtme #1 [ 4179.591187][ T44] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. [ 4179.591600][ T44] task:modprobe state:D stack:25664 pid:18131 tgid:18131 ppid:9353 task_flags:0x400100 flags:0x00080001 [ 4179.592165][ T44] Call Trace: [ 4179.592366][ T44] [ 4179.592506][ T44] __schedule+0x880/0x1b10 [ 4179.592760][ T44] ? io_schedule_timeout+0x150/0x150 [ 4179.593031][ T44] ? schedule+0x1ee/0x270 [ 4179.593208][ T44] ? __lock_release+0x5d/0x170 [ 4179.593489][ T44] schedule+0xe2/0x270 [ 4179.593693][ T44] schedule_preempt_disabled+0x18/0x30 [ 4179.593932][ T44] rwsem_down_write_slowpath+0x3c5/0xc60 [ 4179.594176][ T44] ? __lock_acquire+0x449/0x7e0 [ 4179.594411][ T44] ? rwsem_down_read_slowpath+0xd80/0xd80 [ 4179.594644][ T44] ? rcu_is_watching+0x12/0xb0 [ 4179.594868][ T44] ? lock_acquire+0x104/0x160 [ 4179.595103][ T44] ? register_pernet_subsys+0x1a/0x40 [ 4179.595356][ T44] down_write+0x1d6/0x200 [ 4179.595603][ T44] ? rwsem_down_write_slowpath+0xc60/0xc60 [ 4179.595882][ T44] ? find_held_lock+0x2b/0x80 [ 4179.596116][ T44] ? __create_object+0x5e/0xb0 [ 4179.596351][ T44] ? __lock_release+0x5d/0x170 [ 4179.596576][ T44] ? _sub_I_65535_1+0xcf0/0xcf0 [xt_conntrack] [ 4179.596853][ T44] register_pernet_subsys+0x1a/0x40 [ 4179.597093][ T44] vxlan_init_module+0x17/0x1000 [vxlan] [ 4179.597349][ T44] do_one_initcall+0x8c/0x1d0 [ 4179.597576][ T44] ? trace_initcall_start+0x130/0x130 [ 4179.597799][ T44] ? rcu_is_watching+0x12/0xb0 [ 4179.598036][ T44] ? trace_kmalloc+0x26/0xd0 [ 4179.598642][ T44] ? __kmalloc_cache_noprof+0x2e4/0x690 [ 4179.598872][ T44] ? debug_objects_oom+0x160/0x160 [ 4179.599111][ T44] ? kasan_unpoison+0x27/0x50 [ 4179.599357][ T44] ? __asan_register_globals+0x5d/0x70 [ 4179.599587][ T44] do_init_module+0x27a/0x860 [ 4179.599818][ T44] ? free_module+0x2f0/0x2f0 [ 4179.600052][ T44] ? kfree+0x21d/0x540 [ 4179.600221][ T44] ? load_module+0x20c5/0x2dd0 [ 4179.600461][ T44] load_module+0x1f40/0x2dd0 [ 4179.600699][ T44] ? layout_and_allocate.constprop.0+0x630/0x630 [ 4179.600983][ T44] ? kernel_read_file+0x3ed/0x550 [ 4179.601219][ T44] ? kernel_read_file+0x3c8/0x550 [ 4179.601456][ T44] ? __ia32_sys_fsconfig+0x150/0x150 [ 4179.601686][ T44] ? init_module_from_file+0xe9/0x150 [ 4179.601910][ T44] init_module_from_file+0xe9/0x150 [ 4179.602149][ T44] ? __do_sys_init_module+0x1e0/0x1e0 [ 4179.602394][ T44] ? idempotent_init_module+0x316/0x610 [ 4179.602623][ T44] ? __lock_release+0x5d/0x170 [ 4179.602850][ T44] ? do_raw_spin_unlock+0x58/0x220 [ 4179.603100][ T44] idempotent_init_module+0x331/0x610 [ 4179.603335][ T44] ? init_module_from_file+0x150/0x150 [ 4179.603580][ T44] ? cap_capable+0x94/0x230 [ 4179.603813][ T44] __x64_sys_finit_module+0xca/0x150 [ 4179.604046][ T44] ? do_syscall_64+0x85/0xfd0 [ 4179.604290][ T44] do_syscall_64+0xc1/0xfd0 [ 4179.604516][ T44] entry_SYSCALL_64_after_hwframe+0x4b/0x53 [ 4179.604791][ T44] RIP: 0033:0x7fa5db66f92d [ 4179.605043][ T44] RSP: 002b:00007fff6f7d3d98 EFLAGS: 00000246 ORIG_RAX: 0000000000000139 [ 4179.605409][ T44] RAX: ffffffffffffffda RBX: 0000555763c52c30 RCX: 00007fa5db66f92d [ 4179.605746][ T44] RDX: 0000000000000000 RSI: 000055573b805a2a RDI: 0000000000000000 [ 4179.606106][ T44] RBP: 0000000000040000 R08: 0000000000000000 R09: 00007fff6f7d3ed0 [ 4179.606461][ T44] R10: 0000000000000000 R11: 0000000000000246 R12: 000055573b805a2a [ 4179.606801][ T44] R13: 0000555763c52f50 R14: 0000555763c52fa0 R15: 0000555763c52c9e [ 4179.607169][ T44] [ 4179.607356][ T44] INFO: task modprobe:18131 blocked on an rw-semaphore likely owned by task kworker/u16:0:12 [ 4179.607839][ T44] task:kworker/u16:0 state:R running task stack:23392 pid:12 tgid:12 ppid:2 task_flags:0x4208060 flags:0x00080000 [ 4179.608412][ T44] Workqueue: netns cleanup_net [ 4179.608645][ T44] Call Trace: [ 4179.608815][ T44] [ 4179.608939][ T44] ? __mutex_trylock_common+0xf9/0x260 [ 4179.609189][ T44] ? __mutex_handoff+0x2b0/0x2b0 [ 4179.609419][ T44] ? rcu_is_watching+0x12/0xb0 [ 4179.609645][ T44] ? rcu_is_watching+0x12/0xb0 [ 4179.609869][ T44] ? trace_contention_end+0xd8/0x140 [ 4179.610102][ T44] ? __mutex_lock+0x19f/0x1190 [ 4179.610343][ T44] ? trace_sched_exit_tp+0xcd/0x130 [ 4179.610576][ T44] ? irqentry_exit+0x3b/0x80 [ 4179.610822][ T44] ? nf_ct_iterate_cleanup+0x4a/0x340 [nf_conntrack] [ 4179.611127][ T44] ? __nf_conntrack_hash_insert+0x200/0x200 [nf_conntrack] [ 4179.611481][ T44] ? nf_conntrack_cleanup_net_list+0x19a/0x4b0 [nf_conntrack] [ 4179.611822][ T44] ? nf_conntrack_cleanup_net_list+0x10e/0x4b0 [nf_conntrack] [ 4179.612171][ T44] ? nf_conntrack_cleanup_end+0x70/0x70 [nf_conntrack] [ 4179.612476][ T44] ? nf_conntrack_pernet_exit+0xa5/0x140 [nf_conntrack] [ 4179.612769][ T44] ? _raw_spin_unlock+0x2d/0x50 [ 4179.613001][ T44] ? ops_undo_list+0x2bf/0x890 [ 4179.613235][ T44] ? rtnl_net_dumpid_one+0x270/0x270 [ 4179.613476][ T44] ? cleanup_net+0x2d6/0x8b0 [ 4179.613707][ T44] ? cleanup_net+0x3b2/0x8b0 [ 4179.613933][ T44] ? net_passive_dec+0x190/0x190 [ 4179.614175][ T44] ? rcu_is_watching+0x12/0xb0 [ 4179.614420][ T44] ? process_one_work+0xe35/0x1650 [ 4179.614654][ T44] ? pwq_dec_nr_in_flight+0x550/0x550 [ 4179.614890][ T44] ? assign_work+0x168/0x240 [ 4179.615126][ T44] ? worker_thread+0x591/0xcf0 [ 4179.615372][ T44] ? rescuer_thread+0xd10/0xd10 [ 4179.615599][ T44] ? kthread+0x37b/0x5f0 [ 4179.615807][ T44] ? kthread_is_per_cpu+0xc0/0xc0 [ 4179.616056][ T44] ? ret_from_fork+0x1b/0x270 [ 4179.616310][ T44] ? __lock_release+0x5d/0x170 [ 4179.616548][ T44] ? rcu_is_watching+0x12/0xb0 [ 4179.616792][ T44] ? kthread_is_per_cpu+0xc0/0xc0 [ 4179.617049][ T44] ? ret_from_fork+0x1db/0x270 [ 4179.617296][ T44] ? kthread_is_per_cpu+0xc0/0xc0 [ 4179.617536][ T44] ? ret_from_fork_asm+0x11/0x20 [ 4179.617792][ T44] [ 4179.617973][ T44] [ 4179.617973][ T44] Showing all locks held in the system: [ 4179.618460][ T44] 4 locks held by kworker/u16:0/12: [ 4179.618702][ T44] 1 lock held by khungtaskd/44: [ 4179.618927][ T44] #0: ffffffff9b9881c0 (rcu_read_lock){....}-{1:3}, at: debug_show_all_locks+0x36/0x260 [ 4179.619365][ T44] 2 locks held by kworker/u20:1/9353: [ 4179.619595][ T44] #0: ffff888001099d48 ((wq_completion)events_unbound){+.+.}-{0:0}, at: process_one_work+0x7e5/0x1650 [ 4179.620072][ T44] #1: ffffc9000352fd30 ((work_completion)(&sub_info->work)){+.+.}-{0:0}, at: process_one_work+0xded/0x1650 [ 4179.620610][ T44] 1 lock held by modprobe/18131: [ 4179.620839][ T44] #0: ffffffff9c07fb50 (pernet_ops_rwsem){++++}-{4:4}, at: register_pernet_subsys+0x1a/0x40 [ 4179.621318][ T44] [ 4179.621438][ T44] ============================================= [ 4179.621438][ T44] [ 4302.469364][ T44] INFO: task modprobe:18131 blocked for more than 245 seconds. [ 4302.469964][ T44] Not tainted 6.18.0-rc2-virtme #1 [ 4302.470339][ T44] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. [ 4302.470926][ T44] task:modprobe state:D stack:25664 pid:18131 tgid:18131 ppid:9353 task_flags:0x400100 flags:0x00080001 [ 4302.471782][ T44] Call Trace: [ 4302.472065][ T44] [ 4302.472275][ T44] __schedule+0x880/0x1b10 [ 4302.472666][ T44] ? io_schedule_timeout+0x150/0x150 [ 4302.473052][ T44] ? schedule+0x1ee/0x270 [ 4302.473346][ T44] ? __lock_release+0x5d/0x170 [ 4302.473727][ T44] schedule+0xe2/0x270 [ 4302.474008][ T44] schedule_preempt_disabled+0x18/0x30 [ 4302.474409][ T44] rwsem_down_write_slowpath+0x3c5/0xc60 [ 4302.474780][ T44] ? __lock_acquire+0x449/0x7e0 [ 4302.475151][ T44] ? rwsem_down_read_slowpath+0xd80/0xd80 [ 4302.475558][ T44] ? rcu_is_watching+0x12/0xb0 [ 4302.475940][ T44] ? lock_acquire+0x104/0x160 [ 4302.476335][ T44] ? register_pernet_subsys+0x1a/0x40 [ 4302.476733][ T44] down_write+0x1d6/0x200 [ 4302.477023][ T44] ? rwsem_down_write_slowpath+0xc60/0xc60 [ 4302.477495][ T44] ? find_held_lock+0x2b/0x80 [ 4302.477865][ T44] ? __create_object+0x5e/0xb0 [ 4302.478219][ T44] ? __lock_release+0x5d/0x170 [ 4302.478613][ T44] ? _sub_I_65535_1+0xcf0/0xcf0 [xt_conntrack] [ 4302.479073][ T44] register_pernet_subsys+0x1a/0x40 [ 4302.479455][ T44] vxlan_init_module+0x17/0x1000 [vxlan] [ 4302.479870][ T44] do_one_initcall+0x8c/0x1d0 [ 4302.480258][ T44] ? trace_initcall_start+0x130/0x130 [ 4302.480632][ T44] ? rcu_is_watching+0x12/0xb0 [ 4302.480998][ T44] ? trace_kmalloc+0x26/0xd0 [ 4302.481381][ T44] ? __kmalloc_cache_noprof+0x2e4/0x690 [ 4302.481746][ T44] ? debug_objects_oom+0x160/0x160 [ 4302.482115][ T44] ? kasan_unpoison+0x27/0x50 [ 4302.482513][ T44] ? __asan_register_globals+0x5d/0x70 [ 4302.482896][ T44] do_init_module+0x27a/0x860 [ 4302.483295][ T44] ? free_module+0x2f0/0x2f0 [ 4302.483675][ T44] ? kfree+0x21d/0x540 [ 4302.483953][ T44] ? load_module+0x20c5/0x2dd0 [ 4302.484347][ T44] load_module+0x1f40/0x2dd0 [ 4302.484737][ T44] ? layout_and_allocate.constprop.0+0x630/0x630 [ 4302.485198][ T44] ? kernel_read_file+0x3ed/0x550 [ 4302.485598][ T44] ? kernel_read_file+0x3c8/0x550 [ 4302.485964][ T44] ? __ia32_sys_fsconfig+0x150/0x150 [ 4302.486359][ T44] ? init_module_from_file+0xe9/0x150 [ 4302.486726][ T44] init_module_from_file+0xe9/0x150 [ 4302.487085][ T44] ? __do_sys_init_module+0x1e0/0x1e0 [ 4302.487506][ T44] ? idempotent_init_module+0x316/0x610 [ 4302.487876][ T44] ? __lock_release+0x5d/0x170 [ 4302.488267][ T44] ? do_raw_spin_unlock+0x58/0x220 [ 4302.488652][ T44] idempotent_init_module+0x331/0x610 [ 4302.489022][ T44] ? init_module_from_file+0x150/0x150 [ 4302.489432][ T44] ? cap_capable+0x94/0x230 [ 4302.489794][ T44] __x64_sys_finit_module+0xca/0x150 [ 4302.490154][ T44] ? do_syscall_64+0x85/0xfd0 [ 4302.490555][ T44] do_syscall_64+0xc1/0xfd0 [ 4302.490906][ T44] entry_SYSCALL_64_after_hwframe+0x4b/0x53 [ 4302.491369][ T44] RIP: 0033:0x7fa5db66f92d [ 4302.491764][ T44] RSP: 002b:00007fff6f7d3d98 EFLAGS: 00000246 ORIG_RAX: 0000000000000139 [ 4302.492331][ T44] RAX: ffffffffffffffda RBX: 0000555763c52c30 RCX: 00007fa5db66f92d [ 4302.492855][ T44] RDX: 0000000000000000 RSI: 000055573b805a2a RDI: 0000000000000000 [ 4302.493205][ T44] RBP: 0000000000040000 R08: 0000000000000000 R09: 00007fff6f7d3ed0 [ 4302.493558][ T44] R10: 0000000000000000 R11: 0000000000000246 R12: 000055573b805a2a [ 4302.493896][ T44] R13: 0000555763c52f50 R14: 0000555763c52fa0 R15: 0000555763c52c9e [ 4302.494254][ T44] [ 4302.494431][ T44] INFO: task modprobe:18131 blocked on an rw-semaphore likely owned by task kworker/u16:0:12 [ 4302.494908][ T44] task:kworker/u16:0 state:R running task stack:23392 pid:12 tgid:12 ppid:2 task_flags:0x4208060 flags:0x00080000 [ 4302.495454][ T44] Workqueue: netns cleanup_net [ 4302.495683][ T44] Call Trace: [ 4302.495849][ T44] [ 4302.495961][ T44] ? update_se+0x3bf/0x560 [ 4302.496193][ T44] ? __mutex_trylock_common+0xf9/0x260 [ 4302.496421][ T44] ? __mutex_handoff+0x2b0/0x2b0 [ 4302.496647][ T44] ? rcu_is_watching+0x12/0xb0 [ 4302.496868][ T44] ? trace_contention_end+0xd8/0x140 [ 4302.497090][ T44] ? __mutex_lock+0x19f/0x1190 [ 4302.497320][ T44] ? trace_sched_exit_tp+0xcd/0x130 [ 4302.497550][ T44] ? irqentry_exit+0x3b/0x80 [ 4302.497774][ T44] ? lockdep_hardirqs_on+0x7c/0x110 [ 4302.498008][ T44] ? nf_ct_iterate_cleanup+0x4a/0x340 [nf_conntrack] [ 4302.498303][ T44] ? __nf_conntrack_hash_insert+0x200/0x200 [nf_conntrack] [ 4302.498644][ T44] ? nf_conntrack_cleanup_net_list+0x19a/0x4b0 [nf_conntrack] [ 4302.498980][ T44] ? nf_conntrack_cleanup_net_list+0x10e/0x4b0 [nf_conntrack] [ 4302.499329][ T44] ? nf_conntrack_cleanup_end+0x70/0x70 [nf_conntrack] [ 4302.499619][ T44] ? nf_conntrack_pernet_exit+0xa5/0x140 [nf_conntrack] [ 4302.499902][ T44] ? _raw_spin_unlock+0x2d/0x50 [ 4302.500127][ T44] ? ops_undo_list+0x2bf/0x890 [ 4302.500369][ T44] ? rtnl_net_dumpid_one+0x270/0x270 [ 4302.500606][ T44] ? cleanup_net+0x2d6/0x8b0 [ 4302.500835][ T44] ? cleanup_net+0x3b2/0x8b0 [ 4302.501066][ T44] ? net_passive_dec+0x190/0x190 [ 4302.501298][ T44] ? rcu_is_watching+0x12/0xb0 [ 4302.501524][ T44] ? process_one_work+0xe35/0x1650 [ 4302.501753][ T44] ? pwq_dec_nr_in_flight+0x550/0x550 [ 4302.501979][ T44] ? assign_work+0x168/0x240 [ 4302.502199][ T44] ? worker_thread+0x591/0xcf0 [ 4302.502439][ T44] ? rescuer_thread+0xd10/0xd10 [ 4302.502662][ T44] ? kthread+0x37b/0x5f0 [ 4302.502855][ T44] ? kthread_is_per_cpu+0xc0/0xc0 [ 4302.503074][ T44] ? ret_from_fork+0x1b/0x270 [ 4302.503301][ T44] ? __lock_release+0x5d/0x170 [ 4302.503523][ T44] ? rcu_is_watching+0x12/0xb0 [ 4302.503740][ T44] ? kthread_is_per_cpu+0xc0/0xc0 [ 4302.503963][ T44] ? ret_from_fork+0x1db/0x270 [ 4302.504180][ T44] ? kthread_is_per_cpu+0xc0/0xc0 [ 4302.504409][ T44] ? ret_from_fork_asm+0x11/0x20 [ 4302.504655][ T44] [ 4302.504844][ T44] [ 4302.504844][ T44] Showing all locks held in the system: [ 4302.505183][ T44] 4 locks held by kworker/u16:0/12: [ 4302.505517][ T44] 1 lock held by khungtaskd/44: [ 4302.505749][ T44] #0: ffffffff9b9881c0 (rcu_read_lock){....}-{1:3}, at: debug_show_all_locks+0x36/0x260 [ 4302.506147][ T44] 2 locks held by kworker/u20:1/9353: [ 4302.506382][ T44] #0: ffff888001099d48 ((wq_completion)events_unbound){+.+.}-{0:0}, at: process_one_work+0x7e5/0x1650 [ 4302.506840][ T44] #1: ffffc9000352fd30 ((work_completion)(&sub_info->work)){+.+.}-{0:0}, at: process_one_work+0xded/0x1650 [ 4302.507368][ T44] 1 lock held by modprobe/18131: [ 4302.507596][ T44] #0: ffffffff9c07fb50 (pernet_ops_rwsem){++++}-{4:4}, at: register_pernet_subsys+0x1a/0x40 [ 4302.508061][ T44] [ 4302.508173][ T44] ============================================= [ 4302.508173][ T44] [ 4425.349333][ T44] INFO: task modprobe:18131 blocked for more than 368 seconds. [ 4425.349815][ T44] Not tainted 6.18.0-rc2-virtme #1 [ 4425.350025][ T44] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. [ 4425.350412][ T44] task:modprobe state:D stack:25664 pid:18131 tgid:18131 ppid:9353 task_flags:0x400100 flags:0x00080001 [ 4425.350909][ T44] Call Trace: [ 4425.351076][ T44] [ 4425.351194][ T44] __schedule+0x880/0x1b10 [ 4425.351455][ T44] ? io_schedule_timeout+0x150/0x150 [ 4425.351691][ T44] ? schedule+0x1ee/0x270 [ 4425.351857][ T44] ? __lock_release+0x5d/0x170 [ 4425.352099][ T44] schedule+0xe2/0x270 [ 4425.352284][ T44] schedule_preempt_disabled+0x18/0x30 [ 4425.352504][ T44] rwsem_down_write_slowpath+0x3c5/0xc60 [ 4425.352724][ T44] ? __lock_acquire+0x449/0x7e0 [ 4425.352943][ T44] ? rwsem_down_read_slowpath+0xd80/0xd80 [ 4425.353172][ T44] ? rcu_is_watching+0x12/0xb0 [ 4425.353407][ T44] ? lock_acquire+0x104/0x160 [ 4425.353627][ T44] ? register_pernet_subsys+0x1a/0x40 [ 4425.353863][ T44] down_write+0x1d6/0x200 [ 4425.354030][ T44] ? rwsem_down_write_slowpath+0xc60/0xc60 [ 4425.354309][ T44] ? find_held_lock+0x2b/0x80 [ 4425.354534][ T44] ? __create_object+0x5e/0xb0 [ 4425.354758][ T44] ? __lock_release+0x5d/0x170 [ 4425.354980][ T44] ? _sub_I_65535_1+0xcf0/0xcf0 [xt_conntrack] [ 4425.355260][ T44] register_pernet_subsys+0x1a/0x40 [ 4425.355487][ T44] vxlan_init_module+0x17/0x1000 [vxlan] [ 4425.355723][ T44] do_one_initcall+0x8c/0x1d0 [ 4425.355944][ T44] ? trace_initcall_start+0x130/0x130 [ 4425.356162][ T44] ? rcu_is_watching+0x12/0xb0 [ 4425.356390][ T44] ? trace_kmalloc+0x26/0xd0 [ 4425.356612][ T44] ? __kmalloc_cache_noprof+0x2e4/0x690 [ 4425.356831][ T44] ? debug_objects_oom+0x160/0x160 [ 4425.357060][ T44] ? kasan_unpoison+0x27/0x50 [ 4425.357293][ T44] ? __asan_register_globals+0x5d/0x70 [ 4425.357514][ T44] do_init_module+0x27a/0x860 [ 4425.357742][ T44] ? free_module+0x2f0/0x2f0 [ 4425.357963][ T44] ? kfree+0x21d/0x540 [ 4425.358127][ T44] ? load_module+0x20c5/0x2dd0 [ 4425.358361][ T44] load_module+0x1f40/0x2dd0 [ 4425.358591][ T44] ? layout_and_allocate.constprop.0+0x630/0x630 [ 4425.358871][ T44] ? kernel_read_file+0x3ed/0x550 [ 4425.359093][ T44] ? kernel_read_file+0x3c8/0x550 [ 4425.359321][ T44] ? __ia32_sys_fsconfig+0x150/0x150 [ 4425.359549][ T44] ? init_module_from_file+0xe9/0x150 [ 4425.359768][ T44] init_module_from_file+0xe9/0x150 [ 4425.359986][ T44] ? __do_sys_init_module+0x1e0/0x1e0 [ 4425.360222][ T44] ? idempotent_init_module+0x316/0x610 [ 4425.360452][ T44] ? __lock_release+0x5d/0x170 [ 4425.360673][ T44] ? do_raw_spin_unlock+0x58/0x220 [ 4425.360892][ T44] idempotent_init_module+0x331/0x610 [ 4425.361114][ T44] ? init_module_from_file+0x150/0x150 [ 4425.361354][ T44] ? cap_capable+0x94/0x230 [ 4425.361578][ T44] __x64_sys_finit_module+0xca/0x150 [ 4425.361805][ T44] ? do_syscall_64+0x85/0xfd0 [ 4425.362031][ T44] do_syscall_64+0xc1/0xfd0 [ 4425.362261][ T44] entry_SYSCALL_64_after_hwframe+0x4b/0x53 [ 4425.362533][ T44] RIP: 0033:0x7fa5db66f92d [ 4425.362762][ T44] RSP: 002b:00007fff6f7d3d98 EFLAGS: 00000246 ORIG_RAX: 0000000000000139 [ 4425.363098][ T44] RAX: ffffffffffffffda RBX: 0000555763c52c30 RCX: 00007fa5db66f92d [ 4425.363439][ T44] RDX: 0000000000000000 RSI: 000055573b805a2a RDI: 0000000000000000 [ 4425.363768][ T44] RBP: 0000000000040000 R08: 0000000000000000 R09: 00007fff6f7d3ed0 [ 4425.364097][ T44] R10: 0000000000000000 R11: 0000000000000246 R12: 000055573b805a2a [ 4425.364453][ T44] R13: 0000555763c52f50 R14: 0000555763c52fa0 R15: 0000555763c52c9e [ 4425.364823][ T44] [ 4425.365009][ T44] INFO: task modprobe:18131 blocked on an rw-semaphore likely owned by task kworker/u16:0:12 [ 4425.365480][ T44] task:kworker/u16:0 state:R running task stack:23392 pid:12 tgid:12 ppid:2 task_flags:0x4208060 flags:0x00080000 [ 4425.366026][ T44] Workqueue: netns cleanup_net [ 4425.366261][ T44] Call Trace: [ 4425.366428][ T44] [ 4425.366554][ T44] ? __mutex_trylock_common+0xf9/0x260 [ 4425.366779][ T44] ? __mutex_handoff+0x2b0/0x2b0 [ 4425.366999][ T44] ? rcu_is_watching+0x12/0xb0 [ 4425.367224][ T44] ? rcu_is_watching+0x12/0xb0 [ 4425.367452][ T44] ? trace_contention_end+0xd8/0x140 [ 4425.367672][ T44] ? __mutex_lock+0x19f/0x1190 [ 4425.367892][ T44] ? trace_sched_exit_tp+0xcd/0x130 [ 4425.368119][ T44] ? __schedule+0x3d9/0x1b10 [ 4425.368349][ T44] ? irqentry_exit+0x3b/0x80 [ 4425.368570][ T44] ? lockdep_hardirqs_on+0x7c/0x110 [ 4425.368802][ T44] ? nf_ct_iterate_cleanup+0x4a/0x340 [nf_conntrack] [ 4425.369088][ T44] ? __nf_conntrack_hash_insert+0x200/0x200 [nf_conntrack] [ 4425.369437][ T44] ? nf_conntrack_cleanup_net_list+0x19a/0x4b0 [nf_conntrack] [ 4425.369779][ T44] ? nf_conntrack_cleanup_net_list+0x10e/0x4b0 [nf_conntrack] [ 4425.370118][ T44] ? nf_conntrack_cleanup_end+0x70/0x70 [nf_conntrack] [ 4425.370414][ T44] ? nf_conntrack_pernet_exit+0xa5/0x140 [nf_conntrack] [ 4425.370697][ T44] ? _raw_spin_unlock+0x2d/0x50 [ 4425.370924][ T44] ? ops_undo_list+0x2bf/0x890 [ 4425.371149][ T44] ? rtnl_net_dumpid_one+0x270/0x270 [ 4425.371380][ T44] ? cleanup_net+0x2d6/0x8b0 [ 4425.371609][ T44] ? cleanup_net+0x3b2/0x8b0 [ 4425.371830][ T44] ? net_passive_dec+0x190/0x190 [ 4425.372059][ T44] ? rcu_is_watching+0x12/0xb0 [ 4425.372293][ T44] ? process_one_work+0xe35/0x1650 [ 4425.372522][ T44] ? pwq_dec_nr_in_flight+0x550/0x550 [ 4425.372751][ T44] ? assign_work+0x168/0x240 [ 4425.372974][ T44] ? worker_thread+0x591/0xcf0 [ 4425.373206][ T44] ? rescuer_thread+0xd10/0xd10 [ 4425.373433][ T44] ? kthread+0x37b/0x5f0 [ 4425.373602][ T44] ? kthread_is_per_cpu+0xc0/0xc0 [ 4425.373818][ T44] ? ret_from_fork+0x1b/0x270 [ 4425.374037][ T44] ? __lock_release+0x5d/0x170 [ 4425.374262][ T44] ? rcu_is_watching+0x12/0xb0 [ 4425.374481][ T44] ? kthread_is_per_cpu+0xc0/0xc0 [ 4425.374701][ T44] ? ret_from_fork+0x1db/0x270 [ 4425.374925][ T44] ? kthread_is_per_cpu+0xc0/0xc0 [ 4425.375148][ T44] ? ret_from_fork_asm+0x11/0x20 [ 4425.375393][ T44] [ 4425.375567][ T44] [ 4425.375567][ T44] Showing all locks held in the system: [ 4425.375881][ T44] 4 locks held by kworker/u16:0/12: [ 4425.376100][ T44] 1 lock held by khungtaskd/44: [ 4425.376454][ T44] #0: ffffffff9b9881c0 (rcu_read_lock){....}-{1:3}, at: debug_show_all_locks+0x36/0x260 [ 4425.376845][ T44] 2 locks held by kworker/u20:1/9353: [ 4425.377056][ T44] #0: ffff888001099d48 ((wq_completion)events_unbound){+.+.}-{0:0}, at: process_one_work+0x7e5/0x1650 [ 4425.377485][ T44] #1: ffffc9000352fd30 ((work_completion)(&sub_info->work)){+.+.}-{0:0}, at: process_one_work+0xded/0x1650 [ 4425.377973][ T44] 1 lock held by modprobe/18131: [ 4425.378187][ T44] #0: ffffffff9c07fb50 (pernet_ops_rwsem){++++}-{4:4}, at: register_pernet_subsys+0x1a/0x40 [ 4425.378619][ T44] [ 4425.378728][ T44] ============================================= [ 4425.378728][ T44] [ 4548.230323][ T44] INFO: task modprobe:18131 blocked for more than 491 seconds. [ 4548.230820][ T44] Not tainted 6.18.0-rc2-virtme #1 [ 4548.231027][ T44] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. [ 4548.231430][ T44] task:modprobe state:D stack:25664 pid:18131 tgid:18131 ppid:9353 task_flags:0x400100 flags:0x00080001 [ 4548.231923][ T44] Call Trace: [ 4548.232090][ T44] [ 4548.232209][ T44] __schedule+0x880/0x1b10 [ 4548.232476][ T44] ? io_schedule_timeout+0x150/0x150 [ 4548.232713][ T44] ? schedule+0x1ee/0x270 [ 4548.232886][ T44] ? __lock_release+0x5d/0x170 [ 4548.233114][ T44] schedule+0xe2/0x270 [ 4548.233295][ T44] schedule_preempt_disabled+0x18/0x30 [ 4548.233514][ T44] rwsem_down_write_slowpath+0x3c5/0xc60 [ 4548.233736][ T44] ? __lock_acquire+0x449/0x7e0 [ 4548.233956][ T44] ? rwsem_down_read_slowpath+0xd80/0xd80 [ 4548.234184][ T44] ? rcu_is_watching+0x12/0xb0 [ 4548.234424][ T44] ? lock_acquire+0x104/0x160 [ 4548.234640][ T44] ? register_pernet_subsys+0x1a/0x40 [ 4548.234870][ T44] down_write+0x1d6/0x200 [ 4548.235036][ T44] ? rwsem_down_write_slowpath+0xc60/0xc60 [ 4548.235315][ T44] ? find_held_lock+0x2b/0x80 [ 4548.235542][ T44] ? __create_object+0x5e/0xb0 [ 4548.235762][ T44] ? __lock_release+0x5d/0x170 [ 4548.235984][ T44] ? _sub_I_65535_1+0xcf0/0xcf0 [xt_conntrack] [ 4548.236276][ T44] register_pernet_subsys+0x1a/0x40 [ 4548.236499][ T44] vxlan_init_module+0x17/0x1000 [vxlan] [ 4548.236736][ T44] do_one_initcall+0x8c/0x1d0 [ 4548.236958][ T44] ? trace_initcall_start+0x130/0x130 [ 4548.237175][ T44] ? rcu_is_watching+0x12/0xb0 [ 4548.237405][ T44] ? trace_kmalloc+0x26/0xd0 [ 4548.237627][ T44] ? __kmalloc_cache_noprof+0x2e4/0x690 [ 4548.237852][ T44] ? debug_objects_oom+0x160/0x160 [ 4548.238074][ T44] ? kasan_unpoison+0x27/0x50 [ 4548.238302][ T44] ? __asan_register_globals+0x5d/0x70 [ 4548.238528][ T44] do_init_module+0x27a/0x860 [ 4548.238759][ T44] ? free_module+0x2f0/0x2f0 [ 4548.238981][ T44] ? kfree+0x21d/0x540 [ 4548.239147][ T44] ? load_module+0x20c5/0x2dd0 [ 4548.239387][ T44] load_module+0x1f40/0x2dd0 [ 4548.239618][ T44] ? layout_and_allocate.constprop.0+0x630/0x630 [ 4548.239892][ T44] ? kernel_read_file+0x3ed/0x550 [ 4548.240115][ T44] ? kernel_read_file+0x3c8/0x550 [ 4548.240344][ T44] ? __ia32_sys_fsconfig+0x150/0x150 [ 4548.240569][ T44] ? init_module_from_file+0xe9/0x150 [ 4548.240787][ T44] init_module_from_file+0xe9/0x150 [ 4548.241019][ T44] ? __do_sys_init_module+0x1e0/0x1e0 [ 4548.241261][ T44] ? idempotent_init_module+0x316/0x610 [ 4548.241489][ T44] ? __lock_release+0x5d/0x170 [ 4548.241713][ T44] ? do_raw_spin_unlock+0x58/0x220 [ 4548.241935][ T44] idempotent_init_module+0x331/0x610 [ 4548.242157][ T44] ? init_module_from_file+0x150/0x150 [ 4548.242399][ T44] ? cap_capable+0x94/0x230 [ 4548.242631][ T44] __x64_sys_finit_module+0xca/0x150 [ 4548.242850][ T44] ? do_syscall_64+0x85/0xfd0 [ 4548.243071][ T44] do_syscall_64+0xc1/0xfd0 [ 4548.243300][ T44] entry_SYSCALL_64_after_hwframe+0x4b/0x53 [ 4548.243573][ T44] RIP: 0033:0x7fa5db66f92d [ 4548.243807][ T44] RSP: 002b:00007fff6f7d3d98 EFLAGS: 00000246 ORIG_RAX: 0000000000000139 [ 4548.244148][ T44] RAX: ffffffffffffffda RBX: 0000555763c52c30 RCX: 00007fa5db66f92d [ 4548.244486][ T44] RDX: 0000000000000000 RSI: 000055573b805a2a RDI: 0000000000000000 [ 4548.244815][ T44] RBP: 0000000000040000 R08: 0000000000000000 R09: 00007fff6f7d3ed0 [ 4548.245145][ T44] R10: 0000000000000000 R11: 0000000000000246 R12: 000055573b805a2a [ 4548.245479][ T44] R13: 0000555763c52f50 R14: 0000555763c52fa0 R15: 0000555763c52c9e [ 4548.245835][ T44] [ 4548.246006][ T44] INFO: task modprobe:18131 blocked on an rw-semaphore likely owned by task kworker/u16:0:12 [ 4548.246477][ T44] task:kworker/u16:0 state:R running task stack:23392 pid:12 tgid:12 ppid:2 task_flags:0x4208060 flags:0x00080000 [ 4548.247024][ T44] Workqueue: netns cleanup_net [ 4548.247254][ T44] Call Trace: [ 4548.247420][ T44] [ 4548.247538][ T44] ? __mutex_trylock_common+0xf9/0x260 [ 4548.247761][ T44] ? __mutex_handoff+0x2b0/0x2b0 [ 4548.247983][ T44] ? rcu_is_watching+0x12/0xb0 [ 4548.248207][ T44] ? rcu_is_watching+0x12/0xb0 [ 4548.248433][ T44] ? trace_contention_end+0xd8/0x140 [ 4548.248656][ T44] ? __mutex_lock+0x19f/0x1190 [ 4548.248882][ T44] ? trace_sched_exit_tp+0xcd/0x130 [ 4548.249112][ T44] ? irqentry_exit+0x3b/0x80 [ 4548.249338][ T44] ? lockdep_hardirqs_on+0x7c/0x110 [ 4548.249562][ T44] ? __mutex_unlock_slowpath+0x149/0x4f0 [ 4548.249787][ T44] ? nf_ct_iterate_cleanup+0x89/0x340 [nf_conntrack] [ 4548.250074][ T44] ? __nf_conntrack_hash_insert+0x200/0x200 [nf_conntrack] [ 4548.250431][ T44] ? nf_conntrack_cleanup_net_list+0x19a/0x4b0 [nf_conntrack] [ 4548.250774][ T44] ? nf_conntrack_cleanup_net_list+0x10e/0x4b0 [nf_conntrack] [ 4548.251112][ T44] ? nf_conntrack_cleanup_end+0x70/0x70 [nf_conntrack] [ 4548.251414][ T44] ? nf_conntrack_pernet_exit+0xa5/0x140 [nf_conntrack] [ 4548.251698][ T44] ? _raw_spin_unlock+0x2d/0x50 [ 4548.251926][ T44] ? ops_undo_list+0x2bf/0x890 [ 4548.252152][ T44] ? rtnl_net_dumpid_one+0x270/0x270 [ 4548.252376][ T44] ? cleanup_net+0x2d6/0x8b0 [ 4548.252604][ T44] ? cleanup_net+0x3b2/0x8b0 [ 4548.252825][ T44] ? net_passive_dec+0x190/0x190 [ 4548.253049][ T44] ? rcu_is_watching+0x12/0xb0 [ 4548.253279][ T44] ? process_one_work+0xe35/0x1650 [ 4548.253511][ T44] ? pwq_dec_nr_in_flight+0x550/0x550 [ 4548.253746][ T44] ? assign_work+0x168/0x240 [ 4548.253971][ T44] ? worker_thread+0x591/0xcf0 [ 4548.254203][ T44] ? rescuer_thread+0xd10/0xd10 [ 4548.254434][ T44] ? kthread+0x37b/0x5f0 [ 4548.254605][ T44] ? kthread_is_per_cpu+0xc0/0xc0 [ 4548.254823][ T44] ? ret_from_fork+0x1b/0x270 [ 4548.255041][ T44] ? __lock_release+0x5d/0x170 [ 4548.255264][ T44] ? rcu_is_watching+0x12/0xb0 [ 4548.255489][ T44] ? kthread_is_per_cpu+0xc0/0xc0 [ 4548.255711][ T44] ? ret_from_fork+0x1db/0x270 [ 4548.255930][ T44] ? kthread_is_per_cpu+0xc0/0xc0 [ 4548.256148][ T44] ? ret_from_fork_asm+0x11/0x20 [ 4548.256389][ T44] [ 4548.256565][ T44] [ 4548.256565][ T44] Showing all locks held in the system: [ 4548.256879][ T44] 4 locks held by kworker/u16:0/12: [ 4548.257104][ T44] 1 lock held by khungtaskd/44: [ 4548.257481][ T44] #0: ffffffff9b9881c0 (rcu_read_lock){....}-{1:3}, at: debug_show_all_locks+0x36/0x260 [ 4548.257869][ T44] 2 locks held by kworker/u20:1/9353: [ 4548.258075][ T44] #0: ffff888001099d48 ((wq_completion)events_unbound){+.+.}-{0:0}, at: process_one_work+0x7e5/0x1650 [ 4548.258514][ T44] #1: ffffc9000352fd30 ((work_completion)(&sub_info->work)){+.+.}-{0:0}, at: process_one_work+0xded/0x1650 [ 4548.259015][ T44] 1 lock held by modprobe/18131: [ 4548.259224][ T44] #0: ffffffff9c07fb50 (pernet_ops_rwsem){++++}-{4:4}, at: register_pernet_subsys+0x1a/0x40 [ 4548.259656][ T44] [ 4548.259761][ T44] ============================================= [ 4548.259761][ T44] [ 4671.110347][ T44] INFO: task modprobe:18131 blocked for more than 614 seconds. [ 4671.110972][ T44] Not tainted 6.18.0-rc2-virtme #1 [ 4671.111356][ T44] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. [ 4671.111982][ T44] task:modprobe state:D stack:25664 pid:18131 tgid:18131 ppid:9353 task_flags:0x400100 flags:0x00080001 [ 4671.112846][ T44] Call Trace: [ 4671.113129][ T44] [ 4671.113342][ T44] __schedule+0x880/0x1b10 [ 4671.113743][ T44] ? io_schedule_timeout+0x150/0x150 [ 4671.114138][ T44] ? schedule+0x1ee/0x270 [ 4671.114435][ T44] ? __lock_release+0x5d/0x170 [ 4671.114837][ T44] schedule+0xe2/0x270 [ 4671.115122][ T44] schedule_preempt_disabled+0x18/0x30 [ 4671.115504][ T44] rwsem_down_write_slowpath+0x3c5/0xc60 [ 4671.115979][ T44] ? __lock_acquire+0x449/0x7e0 [ 4671.116363][ T44] ? rwsem_down_read_slowpath+0xd80/0xd80 [ 4671.116754][ T44] ? rcu_is_watching+0x12/0xb0 [ 4671.117128][ T44] ? lock_acquire+0x104/0x160 [ 4671.117520][ T44] ? register_pernet_subsys+0x1a/0x40 [ 4671.117910][ T44] down_write+0x1d6/0x200 [ 4671.118195][ T44] ? rwsem_down_write_slowpath+0xc60/0xc60 [ 4671.118665][ T44] ? find_held_lock+0x2b/0x80 [ 4671.119045][ T44] ? __create_object+0x5e/0xb0 [ 4671.119430][ T44] ? __lock_release+0x5d/0x170 [ 4671.119795][ T44] ? _sub_I_65535_1+0xcf0/0xcf0 [xt_conntrack] [ 4671.120286][ T44] register_pernet_subsys+0x1a/0x40 [ 4671.120673][ T44] vxlan_init_module+0x17/0x1000 [vxlan] [ 4671.121072][ T44] do_one_initcall+0x8c/0x1d0 [ 4671.121464][ T44] ? trace_initcall_start+0x130/0x130 [ 4671.121805][ T44] ? rcu_is_watching+0x12/0xb0 [ 4671.122089][ T44] ? trace_kmalloc+0x26/0xd0 [ 4671.122451][ T44] ? __kmalloc_cache_noprof+0x2e4/0x690 [ 4671.122834][ T44] ? debug_objects_oom+0x160/0x160 [ 4671.123211][ T44] ? kasan_unpoison+0x27/0x50 [ 4671.123599][ T44] ? __asan_register_globals+0x5d/0x70 [ 4671.123979][ T44] do_init_module+0x27a/0x860 [ 4671.124378][ T44] ? free_module+0x2f0/0x2f0 [ 4671.124750][ T44] ? kfree+0x21d/0x540 [ 4671.125033][ T44] ? load_module+0x20c5/0x2dd0 [ 4671.125434][ T44] load_module+0x1f40/0x2dd0 [ 4671.125839][ T44] ? layout_and_allocate.constprop.0+0x630/0x630 [ 4671.126315][ T44] ? kernel_read_file+0x3ed/0x550 [ 4671.126696][ T44] ? kernel_read_file+0x3c8/0x550 [ 4671.127076][ T44] ? __ia32_sys_fsconfig+0x150/0x150 [ 4671.127473][ T44] ? init_module_from_file+0xe9/0x150 [ 4671.127846][ T44] init_module_from_file+0xe9/0x150 [ 4671.128217][ T44] ? __do_sys_init_module+0x1e0/0x1e0 [ 4671.128660][ T44] ? idempotent_init_module+0x316/0x610 [ 4671.129032][ T44] ? __lock_release+0x5d/0x170 [ 4671.129421][ T44] ? do_raw_spin_unlock+0x58/0x220 [ 4671.129799][ T44] idempotent_init_module+0x331/0x610 [ 4671.130178][ T44] ? init_module_from_file+0x150/0x150 [ 4671.130591][ T44] ? cap_capable+0x94/0x230 [ 4671.130985][ T44] __x64_sys_finit_module+0xca/0x150 [ 4671.131377][ T44] ? do_syscall_64+0x85/0xfd0 [ 4671.131753][ T44] do_syscall_64+0xc1/0xfd0 [ 4671.132130][ T44] entry_SYSCALL_64_after_hwframe+0x4b/0x53 [ 4671.132602][ T44] RIP: 0033:0x7fa5db66f92d [ 4671.132993][ T44] RSP: 002b:00007fff6f7d3d98 EFLAGS: 00000246 ORIG_RAX: 0000000000000139 [ 4671.133571][ T44] RAX: ffffffffffffffda RBX: 0000555763c52c30 RCX: 00007fa5db66f92d [ 4671.134140][ T44] RDX: 0000000000000000 RSI: 000055573b805a2a RDI: 0000000000000000 [ 4671.134709][ T44] RBP: 0000000000040000 R08: 0000000000000000 R09: 00007fff6f7d3ed0 [ 4671.135279][ T44] R10: 0000000000000000 R11: 0000000000000246 R12: 000055573b805a2a [ 4671.135838][ T44] R13: 0000555763c52f50 R14: 0000555763c52fa0 R15: 0000555763c52c9e [ 4671.136451][ T44] [ 4671.136743][ T44] INFO: task modprobe:18131 blocked on an rw-semaphore likely owned by task kworker/u16:0:12 [ 4671.137539][ T44] task:kworker/u16:0 state:R running task stack:23392 pid:12 tgid:12 ppid:2 task_flags:0x4208060 flags:0x00080000 [ 4671.138456][ T44] Workqueue: netns cleanup_net [ 4671.138845][ T44] Call Trace: [ 4671.139125][ T44] [ 4671.139337][ T44] ? __mutex_trylock_common+0xf9/0x260 [ 4671.139721][ T44] ? __mutex_handoff+0x2b0/0x2b0 [ 4671.140098][ T44] ? rcu_is_watching+0x12/0xb0 [ 4671.140489][ T44] ? rcu_is_watching+0x12/0xb0 [ 4671.140804][ T44] ? trace_contention_end+0xd8/0x140 [ 4671.141109][ T44] ? __mutex_lock+0x19f/0x1190 [ 4671.141510][ T44] ? trace_sched_exit_tp+0xcd/0x130 [ 4671.141912][ T44] ? irqentry_exit+0x3b/0x80 [ 4671.142294][ T44] ? lockdep_hardirqs_on+0x7c/0x110 [ 4671.142696][ T44] ? nf_ct_iterate_cleanup+0x4a/0x340 [nf_conntrack] [ 4671.143181][ T44] ? nf_ct_iterate_cleanup+0x4a/0x340 [nf_conntrack] [ 4671.143673][ T44] ? __nf_conntrack_hash_insert+0x200/0x200 [nf_conntrack] [ 4671.144270][ T44] ? nf_conntrack_cleanup_net_list+0x19a/0x4b0 [nf_conntrack] [ 4671.144843][ T44] ? nf_conntrack_cleanup_net_list+0x10e/0x4b0 [nf_conntrack] [ 4671.145433][ T44] ? nf_conntrack_cleanup_end+0x70/0x70 [nf_conntrack] [ 4671.145925][ T44] ? nf_conntrack_pernet_exit+0xa5/0x140 [nf_conntrack] [ 4671.146415][ T44] ? _raw_spin_unlock+0x2d/0x50 [ 4671.146799][ T44] ? ops_undo_list+0x2bf/0x890 [ 4671.147188][ T44] ? rtnl_net_dumpid_one+0x270/0x270 [ 4671.147569][ T44] ? cleanup_net+0x2d6/0x8b0 [ 4671.147958][ T44] ? cleanup_net+0x3b2/0x8b0 [ 4671.148344][ T44] ? net_passive_dec+0x190/0x190 [ 4671.148726][ T44] ? rcu_is_watching+0x12/0xb0 [ 4671.149108][ T44] ? process_one_work+0xe35/0x1650 [ 4671.149513][ T44] ? pwq_dec_nr_in_flight+0x550/0x550 [ 4671.149909][ T44] ? assign_work+0x168/0x240 [ 4671.150303][ T44] ? worker_thread+0x591/0xcf0 [ 4671.150700][ T44] ? rescuer_thread+0xd10/0xd10 [ 4671.151077][ T44] ? kthread+0x37b/0x5f0 [ 4671.151372][ T44] ? kthread_is_per_cpu+0xc0/0xc0 [ 4671.151741][ T44] ? ret_from_fork+0x1b/0x270 [ 4671.152116][ T44] ? __lock_release+0x5d/0x170 [ 4671.152501][ T44] ? rcu_is_watching+0x12/0xb0 [ 4671.152876][ T44] ? kthread_is_per_cpu+0xc0/0xc0 [ 4671.153272][ T44] ? ret_from_fork+0x1db/0x270 [ 4671.153643][ T44] ? kthread_is_per_cpu+0xc0/0xc0 [ 4671.154017][ T44] ? ret_from_fork_asm+0x11/0x20 [ 4671.154428][ T44] [ 4671.154724][ T44] [ 4671.154724][ T44] Showing all locks held in the system: [ 4671.155411][ T44] 4 locks held by kworker/u16:0/12: [ 4671.155779][ T44] 1 lock held by khungtaskd/44: [ 4671.156142][ T44] #0: ffffffff9b9881c0 (rcu_read_lock){....}-{1:3}, at: debug_show_all_locks+0x36/0x260 [ 4671.156802][ T44] 2 locks held by kworker/u20:1/9353: [ 4671.157150][ T44] #0: ffff888001099d48 ((wq_completion)events_unbound){+.+.}-{0:0}, at: process_one_work+0x7e5/0x1650 [ 4671.157882][ T44] #1: ffffc9000352fd30 ((work_completion)(&sub_info->work)){+.+.}-{0:0}, at: process_one_work+0xded/0x1650 [ 4671.158718][ T44] 1 lock held by modprobe/18131: [ 4671.159074][ T44] #0: ffffffff9c07fb50 (pernet_ops_rwsem){++++}-{4:4}, at: register_pernet_subsys+0x1a/0x40 [ 4671.159668][ T44] [ 4671.159839][ T44] ============================================= [ 4671.159839][ T44] [ 4793.990350][ T44] INFO: task modprobe:18131 blocked for more than 737 seconds. [ 4793.990912][ T44] Not tainted 6.18.0-rc2-virtme #1 [ 4793.991191][ T44] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. [ 4793.991706][ T44] task:modprobe state:D stack:25664 pid:18131 tgid:18131 ppid:9353 task_flags:0x400100 flags:0x00080001 [ 4793.992382][ T44] Call Trace: [ 4793.992604][ T44] [ 4793.992766][ T44] __schedule+0x880/0x1b10 [ 4793.993105][ T44] ? io_schedule_timeout+0x150/0x150 [ 4793.993460][ T44] ? schedule+0x1ee/0x270 [ 4793.993682][ T44] ? __lock_release+0x5d/0x170 [ 4793.994007][ T44] schedule+0xe2/0x270 [ 4793.994234][ T44] schedule_preempt_disabled+0x18/0x30 [ 4793.994584][ T44] rwsem_down_write_slowpath+0x3c5/0xc60 [ 4793.994908][ T44] ? __lock_acquire+0x449/0x7e0 [ 4793.995209][ T44] ? rwsem_down_read_slowpath+0xd80/0xd80 [ 4793.995547][ T44] ? rcu_is_watching+0x12/0xb0 [ 4793.995882][ T44] ? lock_acquire+0x104/0x160 [ 4793.996250][ T44] ? register_pernet_subsys+0x1a/0x40 [ 4793.996593][ T44] down_write+0x1d6/0x200 [ 4793.996861][ T44] ? rwsem_down_write_slowpath+0xc60/0xc60 [ 4793.997322][ T44] ? find_held_lock+0x2b/0x80 [ 4793.997678][ T44] ? __create_object+0x5e/0xb0 [ 4793.998035][ T44] ? __lock_release+0x5d/0x170 [ 4793.998407][ T44] ? _sub_I_65535_1+0xcf0/0xcf0 [xt_conntrack] [ 4793.998857][ T44] register_pernet_subsys+0x1a/0x40 [ 4793.999185][ T44] vxlan_init_module+0x17/0x1000 [vxlan] [ 4793.999517][ T44] do_one_initcall+0x8c/0x1d0 [ 4793.999802][ T44] ? trace_initcall_start+0x130/0x130 [ 4794.000089][ T44] ? rcu_is_watching+0x12/0xb0 [ 4794.000400][ T44] ? trace_kmalloc+0x26/0xd0 [ 4794.000692][ T44] ? __kmalloc_cache_noprof+0x2e4/0x690 [ 4794.000959][ T44] ? debug_objects_oom+0x160/0x160 [ 4794.001249][ T44] ? kasan_unpoison+0x27/0x50 [ 4794.001530][ T44] ? __asan_register_globals+0x5d/0x70 [ 4794.001807][ T44] do_init_module+0x27a/0x860 [ 4794.002096][ T44] ? free_module+0x2f0/0x2f0 [ 4794.002395][ T44] ? kfree+0x21d/0x540 [ 4794.002613][ T44] ? load_module+0x20c5/0x2dd0 [ 4794.002905][ T44] load_module+0x1f40/0x2dd0 [ 4794.003202][ T44] ? layout_and_allocate.constprop.0+0x630/0x630 [ 4794.003552][ T44] ? kernel_read_file+0x3ed/0x550 [ 4794.003842][ T44] ? kernel_read_file+0x3c8/0x550 [ 4794.004126][ T44] ? __ia32_sys_fsconfig+0x150/0x150 [ 4794.004424][ T44] ? init_module_from_file+0xe9/0x150 [ 4794.004715][ T44] init_module_from_file+0xe9/0x150 [ 4794.004993][ T44] ? __do_sys_init_module+0x1e0/0x1e0 [ 4794.005301][ T44] ? idempotent_init_module+0x316/0x610 [ 4794.005579][ T44] ? __lock_release+0x5d/0x170 [ 4794.005860][ T44] ? do_raw_spin_unlock+0x58/0x220 [ 4794.006137][ T44] idempotent_init_module+0x331/0x610 [ 4794.006438][ T44] ? init_module_from_file+0x150/0x150 [ 4794.006742][ T44] ? cap_capable+0x94/0x230 [ 4794.007028][ T44] __x64_sys_finit_module+0xca/0x150 [ 4794.007318][ T44] ? do_syscall_64+0x85/0xfd0 [ 4794.007604][ T44] do_syscall_64+0xc1/0xfd0 [ 4794.007898][ T44] entry_SYSCALL_64_after_hwframe+0x4b/0x53 [ 4794.008268][ T44] RIP: 0033:0x7fa5db66f92d [ 4794.008573][ T44] RSP: 002b:00007fff6f7d3d98 EFLAGS: 00000246 ORIG_RAX: 0000000000000139 [ 4794.009017][ T44] RAX: ffffffffffffffda RBX: 0000555763c52c30 RCX: 00007fa5db66f92d [ 4794.009475][ T44] RDX: 0000000000000000 RSI: 000055573b805a2a RDI: 0000000000000000 [ 4794.009904][ T44] RBP: 0000000000040000 R08: 0000000000000000 R09: 00007fff6f7d3ed0 [ 4794.010355][ T44] R10: 0000000000000000 R11: 0000000000000246 R12: 000055573b805a2a [ 4794.010779][ T44] R13: 0000555763c52f50 R14: 0000555763c52fa0 R15: 0000555763c52c9e [ 4794.011221][ T44] [ 4794.011452][ T44] INFO: task modprobe:18131 blocked on an rw-semaphore likely owned by task kworker/u16:0:12 [ 4794.012042][ T44] task:kworker/u16:0 state:R running task stack:23392 pid:12 tgid:12 ppid:2 task_flags:0x4208060 flags:0x00080000 [ 4794.012754][ T44] Workqueue: netns cleanup_net [ 4794.013050][ T44] Call Trace: [ 4794.013275][ T44] [ 4794.013433][ T44] ? __mutex_trylock_common+0xf9/0x260 [ 4794.013728][ T44] ? __mutex_handoff+0x2b0/0x2b0 [ 4794.014019][ T44] ? rcu_is_watching+0x12/0xb0 [ 4794.014325][ T44] ? rcu_is_watching+0x12/0xb0 [ 4794.014626][ T44] ? trace_contention_end+0xd8/0x140 [ 4794.014910][ T44] ? __mutex_lock+0x19f/0x1190 [ 4794.015191][ T44] ? trace_sched_exit_tp+0xcd/0x130 [ 4794.015492][ T44] ? irqentry_exit+0x3b/0x80 [ 4794.015767][ T44] ? lockdep_hardirqs_on+0x7c/0x110 [ 4794.016040][ T44] ? asm_common_interrupt+0x26/0x40 [ 4794.016344][ T44] ? nf_ct_iterate_cleanup+0x9b/0x340 [nf_conntrack] [ 4794.016720][ T44] ? __nf_conntrack_hash_insert+0x200/0x200 [nf_conntrack] [ 4794.017146][ T44] ? nf_conntrack_cleanup_net_list+0x19a/0x4b0 [nf_conntrack] [ 4794.017592][ T44] ? nf_conntrack_cleanup_net_list+0x10e/0x4b0 [nf_conntrack] [ 4794.018025][ T44] ? nf_conntrack_cleanup_end+0x70/0x70 [nf_conntrack] [ 4794.018404][ T44] ? nf_conntrack_pernet_exit+0xa5/0x140 [nf_conntrack] [ 4794.018775][ T44] ? _raw_spin_unlock+0x2d/0x50 [ 4794.019068][ T44] ? ops_undo_list+0x2bf/0x890 [ 4794.019383][ T44] ? rtnl_net_dumpid_one+0x270/0x270 [ 4794.019672][ T44] ? cleanup_net+0x2d6/0x8b0 [ 4794.019987][ T44] ? cleanup_net+0x3b2/0x8b0 [ 4794.020280][ T44] ? net_passive_dec+0x190/0x190 [ 4794.020571][ T44] ? rcu_is_watching+0x12/0xb0 [ 4794.020860][ T44] ? process_one_work+0xe35/0x1650 [ 4794.021170][ T44] ? pwq_dec_nr_in_flight+0x550/0x550 [ 4794.021479][ T44] ? assign_work+0x168/0x240 [ 4794.021770][ T44] ? worker_thread+0x591/0xcf0 [ 4794.022067][ T44] ? rescuer_thread+0xd10/0xd10 [ 4794.022357][ T44] ? kthread+0x37b/0x5f0 [ 4794.022579][ T44] ? kthread_is_per_cpu+0xc0/0xc0 [ 4794.022865][ T44] ? ret_from_fork+0x1b/0x270 [ 4794.023169][ T44] ? __lock_release+0x5d/0x170 [ 4794.023474][ T44] ? rcu_is_watching+0x12/0xb0 [ 4794.023763][ T44] ? kthread_is_per_cpu+0xc0/0xc0 [ 4794.024051][ T44] ? ret_from_fork+0x1db/0x270 [ 4794.024344][ T44] ? kthread_is_per_cpu+0xc0/0xc0 [ 4794.024629][ T44] ? ret_from_fork_asm+0x11/0x20 [ 4794.025012][ T44] [ 4794.025309][ T44] [ 4794.025309][ T44] Showing all locks held in the system: [ 4794.025826][ T44] 4 locks held by kworker/u16:0/12: [ 4794.026171][ T44] 1 lock held by khungtaskd/44: [ 4794.026756][ T44] #0: ffffffff9b9881c0 (rcu_read_lock){....}-{1:3}, at: debug_show_all_locks+0x36/0x260 [ 4794.027395][ T44] 2 locks held by kworker/u20:1/9353: [ 4794.027761][ T44] #0: ffff888001099d48 ((wq_completion)events_unbound){+.+.}-{0:0}, at: process_one_work+0x7e5/0x1650 [ 4794.028441][ T44] #1: ffffc9000352fd30 ((work_completion)(&sub_info->work)){+.+.}-{0:0}, at: process_one_work+0xded/0x1650 [ 4794.029078][ T44] 1 lock held by modprobe/18131: [ 4794.029364][ T44] #0: ffffffff9c07fb50 (pernet_ops_rwsem){++++}-{4:4}, at: register_pernet_subsys+0x1a/0x40 [ 4794.029914][ T44] [ 4794.030068][ T44] ============================================= [ 4794.030068][ T44] [ 4916.870310][ T44] INFO: task modprobe:18131 blocked for more than 860 seconds. [ 4916.870709][ T44] Not tainted 6.18.0-rc2-virtme #1 [ 4916.870923][ T44] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. [ 4916.871332][ T44] task:modprobe state:D stack:25664 pid:18131 tgid:18131 ppid:9353 task_flags:0x400100 flags:0x00080001 [ 4916.871841][ T44] Call Trace: [ 4916.872012][ T44] [ 4916.872134][ T44] __schedule+0x880/0x1b10 [ 4916.872381][ T44] ? io_schedule_timeout+0x150/0x150 [ 4916.872617][ T44] ? schedule+0x1ee/0x270 [ 4916.872785][ T44] ? __lock_release+0x5d/0x170 [ 4916.873015][ T44] schedule+0xe2/0x270 [ 4916.873183][ T44] schedule_preempt_disabled+0x18/0x30 [ 4916.873421][ T44] rwsem_down_write_slowpath+0x3c5/0xc60 [ 4916.873645][ T44] ? __lock_acquire+0x449/0x7e0 [ 4916.873867][ T44] ? rwsem_down_read_slowpath+0xd80/0xd80 [ 4916.874099][ T44] ? rcu_is_watching+0x12/0xb0 [ 4916.874334][ T44] ? lock_acquire+0x104/0x160 [ 4916.874557][ T44] ? register_pernet_subsys+0x1a/0x40 [ 4916.874790][ T44] down_write+0x1d6/0x200 [ 4916.874966][ T44] ? rwsem_down_write_slowpath+0xc60/0xc60 [ 4916.875249][ T44] ? find_held_lock+0x2b/0x80 [ 4916.875479][ T44] ? __create_object+0x5e/0xb0 [ 4916.875702][ T44] ? __lock_release+0x5d/0x170 [ 4916.875925][ T44] ? _sub_I_65535_1+0xcf0/0xcf0 [xt_conntrack] [ 4916.876203][ T44] register_pernet_subsys+0x1a/0x40 [ 4916.876437][ T44] vxlan_init_module+0x17/0x1000 [vxlan] [ 4916.876683][ T44] do_one_initcall+0x8c/0x1d0 [ 4916.876907][ T44] ? trace_initcall_start+0x130/0x130 [ 4916.877132][ T44] ? rcu_is_watching+0x12/0xb0 [ 4916.877363][ T44] ? trace_kmalloc+0x26/0xd0 [ 4916.877585][ T44] ? __kmalloc_cache_noprof+0x2e4/0x690 [ 4916.877805][ T44] ? debug_objects_oom+0x160/0x160 [ 4916.878028][ T44] ? kasan_unpoison+0x27/0x50 [ 4916.878258][ T44] ? __asan_register_globals+0x5d/0x70 [ 4916.878487][ T44] do_init_module+0x27a/0x860 [ 4916.878717][ T44] ? free_module+0x2f0/0x2f0 [ 4916.878940][ T44] ? kfree+0x21d/0x540 [ 4916.879109][ T44] ? load_module+0x20c5/0x2dd0 [ 4916.879350][ T44] load_module+0x1f40/0x2dd0 [ 4916.879588][ T44] ? layout_and_allocate.constprop.0+0x630/0x630 [ 4916.879866][ T44] ? kernel_read_file+0x3ed/0x550 [ 4916.880101][ T44] ? kernel_read_file+0x3c8/0x550 [ 4916.880334][ T44] ? __ia32_sys_fsconfig+0x150/0x150 [ 4916.880567][ T44] ? init_module_from_file+0xe9/0x150 [ 4916.880791][ T44] init_module_from_file+0xe9/0x150 [ 4916.881012][ T44] ? __do_sys_init_module+0x1e0/0x1e0 [ 4916.881258][ T44] ? idempotent_init_module+0x316/0x610 [ 4916.881479][ T44] ? __lock_release+0x5d/0x170 [ 4916.881709][ T44] ? do_raw_spin_unlock+0x58/0x220 [ 4916.881934][ T44] idempotent_init_module+0x331/0x610 [ 4916.882157][ T44] ? init_module_from_file+0x150/0x150 [ 4916.882399][ T44] ? cap_capable+0x94/0x230 [ 4916.882627][ T44] __x64_sys_finit_module+0xca/0x150 [ 4916.882850][ T44] ? do_syscall_64+0x85/0xfd0 [ 4916.883073][ T44] do_syscall_64+0xc1/0xfd0 [ 4916.883313][ T44] entry_SYSCALL_64_after_hwframe+0x4b/0x53 [ 4916.883590][ T44] RIP: 0033:0x7fa5db66f92d [ 4916.883831][ T44] RSP: 002b:00007fff6f7d3d98 EFLAGS: 00000246 ORIG_RAX: 0000000000000139 [ 4916.884171][ T44] RAX: ffffffffffffffda RBX: 0000555763c52c30 RCX: 00007fa5db66f92d [ 4916.884511][ T44] RDX: 0000000000000000 RSI: 000055573b805a2a RDI: 0000000000000000 [ 4916.884848][ T44] RBP: 0000000000040000 R08: 0000000000000000 R09: 00007fff6f7d3ed0 [ 4916.885179][ T44] R10: 0000000000000000 R11: 0000000000000246 R12: 000055573b805a2a [ 4916.885524][ T44] R13: 0000555763c52f50 R14: 0000555763c52fa0 R15: 0000555763c52c9e [ 4916.885876][ T44] [ 4916.886047][ T44] INFO: task modprobe:18131 blocked on an rw-semaphore likely owned by task kworker/u16:0:12 [ 4916.886535][ T44] task:kworker/u16:0 state:R running task stack:23392 pid:12 tgid:12 ppid:2 task_flags:0x4208060 flags:0x00080000 [ 4916.887079][ T44] Workqueue: netns cleanup_net [ 4916.887313][ T44] Call Trace: [ 4916.887481][ T44] [ 4916.887608][ T44] ? __mutex_handoff+0x2b0/0x2b0 [ 4916.887870][ T44] ? rcu_is_watching+0x12/0xb0 [ 4916.888109][ T44] ? rcu_is_watching+0x12/0xb0 [ 4916.888347][ T44] ? trace_contention_end+0xd8/0x140 [ 4916.888571][ T44] ? __mutex_lock+0x19f/0x1190 [ 4916.888795][ T44] ? trace_sched_exit_tp+0xcd/0x130 [ 4916.889027][ T44] ? irqentry_exit+0x3b/0x80 [ 4916.889263][ T44] ? lockdep_hardirqs_on+0x7c/0x110 [ 4916.889501][ T44] ? lock_is_held_type+0xaa/0x140 [ 4916.889735][ T44] ? nf_ct_iterate_cleanup+0x79/0x340 [nf_conntrack] [ 4916.890031][ T44] ? __nf_conntrack_hash_insert+0x200/0x200 [nf_conntrack] [ 4916.890388][ T44] ? nf_conntrack_cleanup_net_list+0x19a/0x4b0 [nf_conntrack] [ 4916.890732][ T44] ? nf_conntrack_cleanup_net_list+0x10e/0x4b0 [nf_conntrack] [ 4916.891080][ T44] ? nf_conntrack_cleanup_end+0x70/0x70 [nf_conntrack] [ 4916.891382][ T44] ? nf_conntrack_pernet_exit+0xa5/0x140 [nf_conntrack] [ 4916.891666][ T44] ? _raw_spin_unlock+0x2d/0x50 [ 4916.891895][ T44] ? ops_undo_list+0x2bf/0x890 [ 4916.892132][ T44] ? rtnl_net_dumpid_one+0x270/0x270 [ 4916.892361][ T44] ? cleanup_net+0x2d6/0x8b0 [ 4916.892597][ T44] ? cleanup_net+0x3b2/0x8b0 [ 4916.892823][ T44] ? net_passive_dec+0x190/0x190 [ 4916.893056][ T44] ? rcu_is_watching+0x12/0xb0 [ 4916.893296][ T44] ? process_one_work+0xe35/0x1650 [ 4916.893533][ T44] ? pwq_dec_nr_in_flight+0x550/0x550 [ 4916.893765][ T44] ? assign_work+0x168/0x240 [ 4916.893992][ T44] ? worker_thread+0x591/0xcf0 [ 4916.894225][ T44] ? rescuer_thread+0xd10/0xd10 [ 4916.894458][ T44] ? kthread+0x37b/0x5f0 [ 4916.894629][ T44] ? kthread_is_per_cpu+0xc0/0xc0 [ 4916.894858][ T44] ? ret_from_fork+0x1b/0x270 [ 4916.895084][ T44] ? __lock_release+0x5d/0x170 [ 4916.895314][ T44] ? rcu_is_watching+0x12/0xb0 [ 4916.895537][ T44] ? kthread_is_per_cpu+0xc0/0xc0 [ 4916.895758][ T44] ? ret_from_fork+0x1db/0x270 [ 4916.895980][ T44] ? kthread_is_per_cpu+0xc0/0xc0 [ 4916.896203][ T44] ? ret_from_fork_asm+0x11/0x20 [ 4916.896450][ T44] [ 4916.896620][ T44] [ 4916.896620][ T44] Showing all locks held in the system: [ 4916.896942][ T44] 4 locks held by kworker/u16:0/12: [ 4916.897164][ T44] 1 lock held by khungtaskd/44: [ 4916.897385][ T44] #0: ffffffff9b9881c0 (rcu_read_lock){....}-{1:3}, at: debug_show_all_locks+0x36/0x260 [ 4916.897775][ T44] 2 locks held by kworker/u20:1/9353: [ 4916.897986][ T44] #0: ffff888001099d48 ((wq_completion)events_unbound){+.+.}-{0:0}, at: process_one_work+0x7e5/0x1650 [ 4916.898426][ T44] #1: ffffc9000352fd30 ((work_completion)(&sub_info->work)){+.+.}-{0:0}, at: process_one_work+0xded/0x1650 [ 4916.898916][ T44] 1 lock held by modprobe/18131: [ 4916.899129][ T44] #0: ffffffff9c07fb50 (pernet_ops_rwsem){++++}-{4:4}, at: register_pernet_subsys+0x1a/0x40 [ 4916.899581][ T44] [ 4916.899694][ T44] ============================================= [ 4916.899694][ T44] [ 5039.750335][ T44] INFO: task modprobe:18131 blocked for more than 983 seconds. [ 5039.750948][ T44] Not tainted 6.18.0-rc2-virtme #1 [ 5039.751232][ T44] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. [ 5039.751662][ T44] task:modprobe state:D stack:25664 pid:18131 tgid:18131 ppid:9353 task_flags:0x400100 flags:0x00080001 [ 5039.752214][ T44] Call Trace: [ 5039.752452][ T44] [ 5039.752611][ T44] __schedule+0x880/0x1b10 [ 5039.752896][ T44] ? io_schedule_timeout+0x150/0x150 [ 5039.753143][ T44] ? schedule+0x1ee/0x270 [ 5039.753350][ T44] ? __lock_release+0x5d/0x170 [ 5039.753657][ T44] schedule+0xe2/0x270 [ 5039.753868][ T44] schedule_preempt_disabled+0x18/0x30 [ 5039.754112][ T44] rwsem_down_write_slowpath+0x3c5/0xc60 [ 5039.754358][ T44] ? __lock_acquire+0x449/0x7e0 [ 5039.754591][ T44] ? rwsem_down_read_slowpath+0xd80/0xd80 [ 5039.754838][ T44] ? rcu_is_watching+0x12/0xb0 [ 5039.755115][ T44] ? lock_acquire+0x104/0x160 [ 5039.755420][ T44] ? register_pernet_subsys+0x1a/0x40 [ 5039.755730][ T44] down_write+0x1d6/0x200 [ 5039.755963][ T44] ? rwsem_down_write_slowpath+0xc60/0xc60 [ 5039.756290][ T44] ? find_held_lock+0x2b/0x80 [ 5039.756533][ T44] ? __create_object+0x5e/0xb0 [ 5039.756773][ T44] ? __lock_release+0x5d/0x170 [ 5039.757008][ T44] ? _sub_I_65535_1+0xcf0/0xcf0 [xt_conntrack] [ 5039.757324][ T44] register_pernet_subsys+0x1a/0x40 [ 5039.757622][ T44] vxlan_init_module+0x17/0x1000 [vxlan] [ 5039.757946][ T44] do_one_initcall+0x8c/0x1d0 [ 5039.758231][ T44] ? trace_initcall_start+0x130/0x130 [ 5039.758534][ T44] ? rcu_is_watching+0x12/0xb0 [ 5039.758845][ T44] ? trace_kmalloc+0x26/0xd0 [ 5039.759109][ T44] ? __kmalloc_cache_noprof+0x2e4/0x690 [ 5039.759350][ T44] ? debug_objects_oom+0x160/0x160 [ 5039.759589][ T44] ? kasan_unpoison+0x27/0x50 [ 5039.759897][ T44] ? __asan_register_globals+0x5d/0x70 [ 5039.760197][ T44] do_init_module+0x27a/0x860 [ 5039.760510][ T44] ? free_module+0x2f0/0x2f0 [ 5039.760808][ T44] ? kfree+0x21d/0x540 [ 5039.761031][ T44] ? load_module+0x20c5/0x2dd0 [ 5039.761345][ T44] load_module+0x1f40/0x2dd0 [ 5039.761590][ T44] ? layout_and_allocate.constprop.0+0x630/0x630 [ 5039.761880][ T44] ? kernel_read_file+0x3ed/0x550 [ 5039.762161][ T44] ? kernel_read_file+0x3c8/0x550 [ 5039.762471][ T44] ? __ia32_sys_fsconfig+0x150/0x150 [ 5039.762773][ T44] ? init_module_from_file+0xe9/0x150 [ 5039.763045][ T44] init_module_from_file+0xe9/0x150 [ 5039.763287][ T44] ? __do_sys_init_module+0x1e0/0x1e0 [ 5039.763539][ T44] ? idempotent_init_module+0x316/0x610 [ 5039.763779][ T44] ? __lock_release+0x5d/0x170 [ 5039.764017][ T44] ? do_raw_spin_unlock+0x58/0x220 [ 5039.764257][ T44] idempotent_init_module+0x331/0x610 [ 5039.764496][ T44] ? init_module_from_file+0x150/0x150 [ 5039.764741][ T44] ? cap_capable+0x94/0x230 [ 5039.765045][ T44] __x64_sys_finit_module+0xca/0x150 [ 5039.765346][ T44] ? do_syscall_64+0x85/0xfd0 [ 5039.765650][ T44] do_syscall_64+0xc1/0xfd0 [ 5039.765934][ T44] entry_SYSCALL_64_after_hwframe+0x4b/0x53 [ 5039.766224][ T44] RIP: 0033:0x7fa5db66f92d [ 5039.766501][ T44] RSP: 002b:00007fff6f7d3d98 EFLAGS: 00000246 ORIG_RAX: 0000000000000139 [ 5039.766952][ T44] RAX: ffffffffffffffda RBX: 0000555763c52c30 RCX: 00007fa5db66f92d [ 5039.767333][ T44] RDX: 0000000000000000 RSI: 000055573b805a2a RDI: 0000000000000000 [ 5039.767784][ T44] RBP: 0000000000040000 R08: 0000000000000000 R09: 00007fff6f7d3ed0 [ 5039.768180][ T44] R10: 0000000000000000 R11: 0000000000000246 R12: 000055573b805a2a [ 5039.768537][ T44] R13: 0000555763c52f50 R14: 0000555763c52fa0 R15: 0000555763c52c9e [ 5039.768907][ T44] [ 5039.769088][ T44] INFO: task modprobe:18131 blocked on an rw-semaphore likely owned by task kworker/u16:0:12 [ 5039.769608][ T44] task:kworker/u16:0 state:R running task stack:23392 pid:12 tgid:12 ppid:2 task_flags:0x4208060 flags:0x00080000 [ 5039.770328][ T44] Workqueue: netns cleanup_net [ 5039.770635][ T44] Call Trace: [ 5039.770858][ T44] [ 5039.770977][ T44] ? lockdep_hardirqs_on+0x7c/0x110 [ 5039.771292][ T44] ? __mutex_trylock_common+0xf9/0x260 [ 5039.771593][ T44] ? __mutex_handoff+0x2b0/0x2b0 [ 5039.771893][ T44] ? rcu_is_watching+0x12/0xb0 [ 5039.772191][ T44] ? rcu_is_watching+0x12/0xb0 [ 5039.772464][ T44] ? trace_contention_end+0xd8/0x140 [ 5039.772698][ T44] ? __mutex_lock+0x19f/0x1190 [ 5039.772934][ T44] ? trace_sched_exit_tp+0xcd/0x130 [ 5039.773213][ T44] ? nf_ct_iterate_cleanup+0x4a/0x340 [nf_conntrack] [ 5039.773631][ T44] ? nf_ct_iterate_cleanup+0x4a/0x340 [nf_conntrack] [ 5039.773942][ T44] ? __nf_conntrack_hash_insert+0x200/0x200 [nf_conntrack] [ 5039.774402][ T44] ? nf_conntrack_cleanup_net_list+0x19a/0x4b0 [nf_conntrack] [ 5039.774855][ T44] ? nf_conntrack_cleanup_net_list+0x10e/0x4b0 [nf_conntrack] [ 5039.775275][ T44] ? nf_conntrack_cleanup_end+0x70/0x70 [nf_conntrack] [ 5039.775581][ T44] ? nf_conntrack_pernet_exit+0xa5/0x140 [nf_conntrack] [ 5039.775883][ T44] ? _raw_spin_unlock+0x2d/0x50 [ 5039.776122][ T44] ? ops_undo_list+0x2bf/0x890 [ 5039.776371][ T44] ? rtnl_net_dumpid_one+0x270/0x270 [ 5039.776604][ T44] ? cleanup_net+0x2d6/0x8b0 [ 5039.776846][ T44] ? cleanup_net+0x3b2/0x8b0 [ 5039.777090][ T44] ? net_passive_dec+0x190/0x190 [ 5039.777333][ T44] ? rcu_is_watching+0x12/0xb0 [ 5039.777570][ T44] ? process_one_work+0xe35/0x1650 [ 5039.777815][ T44] ? pwq_dec_nr_in_flight+0x550/0x550 [ 5039.778109][ T44] ? assign_work+0x168/0x240 [ 5039.778418][ T44] ? worker_thread+0x591/0xcf0 [ 5039.778734][ T44] ? rescuer_thread+0xd10/0xd10 [ 5039.779013][ T44] ? kthread+0x37b/0x5f0 [ 5039.779195][ T44] ? kthread_is_per_cpu+0xc0/0xc0 [ 5039.779435][ T44] ? ret_from_fork+0x1b/0x270 [ 5039.779689][ T44] ? __lock_release+0x5d/0x170 [ 5039.779980][ T44] ? rcu_is_watching+0x12/0xb0 [ 5039.780215][ T44] ? kthread_is_per_cpu+0xc0/0xc0 [ 5039.780461][ T44] ? ret_from_fork+0x1db/0x270 [ 5039.780695][ T44] ? kthread_is_per_cpu+0xc0/0xc0 [ 5039.780977][ T44] ? ret_from_fork_asm+0x11/0x20 [ 5039.781304][ T44] [ 5039.781534][ T44] [ 5039.781534][ T44] Showing all locks held in the system: [ 5039.781963][ T44] 4 locks held by kworker/u16:0/12: [ 5039.782269][ T44] 1 lock held by khungtaskd/44: [ 5039.782509][ T44] #0: ffffffff9b9881c0 (rcu_read_lock){....}-{1:3}, at: debug_show_all_locks+0x36/0x260 [ 5039.782936][ T44] 2 locks held by kworker/u20:1/9353: [ 5039.783228][ T44] #0: ffff888001099d48 ((wq_completion)events_unbound){+.+.}-{0:0}, at: process_one_work+0x7e5/0x1650 [ 5039.783820][ T44] #1: ffffc9000352fd30 ((work_completion)(&sub_info->work)){+.+.}-{0:0}, at: process_one_work+0xded/0x1650 [ 5039.784374][ T44] 1 lock held by modprobe/18131: [ 5039.784657][ T44] #0: ffffffff9c07fb50 (pernet_ops_rwsem){++++}-{4:4}, at: register_pernet_subsys+0x1a/0x40 [ 5039.785189][ T44] [ 5039.785311][ T44] ============================================= [ 5039.785311][ T44] [ 5162.629345][ T44] INFO: task modprobe:18131 blocked for more than 1105 seconds. [ 5162.629855][ T44] Not tainted 6.18.0-rc2-virtme #1 [ 5162.630084][ T44] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. [ 5162.630507][ T44] task:modprobe state:D stack:25664 pid:18131 tgid:18131 ppid:9353 task_flags:0x400100 flags:0x00080001 [ 5162.631033][ T44] Call Trace: [ 5162.631229][ T44] [ 5162.631414][ T44] __schedule+0x880/0x1b10 [ 5162.631733][ T44] ? io_schedule_timeout+0x150/0x150 [ 5162.631980][ T44] ? schedule+0x1ee/0x270 [ 5162.632158][ T44] ? __lock_release+0x5d/0x170 [ 5162.632424][ T44] schedule+0xe2/0x270 [ 5162.632624][ T44] schedule_preempt_disabled+0x18/0x30 [ 5162.632857][ T44] rwsem_down_write_slowpath+0x3c5/0xc60 [ 5162.633096][ T44] ? __lock_acquire+0x449/0x7e0 [ 5162.633401][ T44] ? rwsem_down_read_slowpath+0xd80/0xd80 [ 5162.633706][ T44] ? rcu_is_watching+0x12/0xb0 [ 5162.634002][ T44] ? lock_acquire+0x104/0x160 [ 5162.634267][ T44] ? register_pernet_subsys+0x1a/0x40 [ 5162.634533][ T44] down_write+0x1d6/0x200 [ 5162.634758][ T44] ? rwsem_down_write_slowpath+0xc60/0xc60 [ 5162.635105][ T44] ? find_held_lock+0x2b/0x80 [ 5162.635349][ T44] ? __create_object+0x5e/0xb0 [ 5162.635590][ T44] ? __lock_release+0x5d/0x170 [ 5162.635892][ T44] ? _sub_I_65535_1+0xcf0/0xcf0 [xt_conntrack] [ 5162.636260][ T44] register_pernet_subsys+0x1a/0x40 [ 5162.636504][ T44] vxlan_init_module+0x17/0x1000 [vxlan] [ 5162.636753][ T44] do_one_initcall+0x8c/0x1d0 [ 5162.636993][ T44] ? trace_initcall_start+0x130/0x130 [ 5162.637231][ T44] ? rcu_is_watching+0x12/0xb0 [ 5162.637485][ T44] ? trace_kmalloc+0x26/0xd0 [ 5162.637780][ T44] ? __kmalloc_cache_noprof+0x2e4/0x690 [ 5162.638086][ T44] ? debug_objects_oom+0x160/0x160 [ 5162.638364][ T44] ? kasan_unpoison+0x27/0x50 [ 5162.638665][ T44] ? __asan_register_globals+0x5d/0x70 [ 5162.638915][ T44] do_init_module+0x27a/0x860 [ 5162.639155][ T44] ? free_module+0x2f0/0x2f0 [ 5162.639400][ T44] ? kfree+0x21d/0x540 [ 5162.639631][ T44] ? load_module+0x20c5/0x2dd0 [ 5162.639935][ T44] load_module+0x1f40/0x2dd0 [ 5162.640206][ T44] ? layout_and_allocate.constprop.0+0x630/0x630 [ 5162.640508][ T44] ? kernel_read_file+0x3ed/0x550 [ 5162.640748][ T44] ? kernel_read_file+0x3c8/0x550 [ 5162.640984][ T44] ? __ia32_sys_fsconfig+0x150/0x150 [ 5162.641226][ T44] ? init_module_from_file+0xe9/0x150 [ 5162.641466][ T44] init_module_from_file+0xe9/0x150 [ 5162.641701][ T44] ? __do_sys_init_module+0x1e0/0x1e0 [ 5162.641959][ T44] ? idempotent_init_module+0x316/0x610 [ 5162.642192][ T44] ? __lock_release+0x5d/0x170 [ 5162.642451][ T44] ? do_raw_spin_unlock+0x58/0x220 [ 5162.642750][ T44] idempotent_init_module+0x331/0x610 [ 5162.643007][ T44] ? init_module_from_file+0x150/0x150 [ 5162.643258][ T44] ? cap_capable+0x94/0x230 [ 5162.643522][ T44] __x64_sys_finit_module+0xca/0x150 [ 5162.643820][ T44] ? do_syscall_64+0x85/0xfd0 [ 5162.644110][ T44] do_syscall_64+0xc1/0xfd0 [ 5162.644355][ T44] entry_SYSCALL_64_after_hwframe+0x4b/0x53 [ 5162.644650][ T44] RIP: 0033:0x7fa5db66f92d [ 5162.644894][ T44] RSP: 002b:00007fff6f7d3d98 EFLAGS: 00000246 ORIG_RAX: 0000000000000139 [ 5162.645263][ T44] RAX: ffffffffffffffda RBX: 0000555763c52c30 RCX: 00007fa5db66f92d [ 5162.645627][ T44] RDX: 0000000000000000 RSI: 000055573b805a2a RDI: 0000000000000000 [ 5162.645979][ T44] RBP: 0000000000040000 R08: 0000000000000000 R09: 00007fff6f7d3ed0 [ 5162.646411][ T44] R10: 0000000000000000 R11: 0000000000000246 R12: 000055573b805a2a [ 5162.646857][ T44] R13: 0000555763c52f50 R14: 0000555763c52fa0 R15: 0000555763c52c9e [ 5162.647226][ T44] [ 5162.647415][ T44] INFO: task modprobe:18131 blocked on an rw-semaphore likely owned by task kworker/u16:0:12 [ 5162.647921][ T44] task:kworker/u16:0 state:R running task stack:23392 pid:12 tgid:12 ppid:2 task_flags:0x4208060 flags:0x00080000 [ 5162.648499][ T44] Workqueue: netns cleanup_net [ 5162.648806][ T44] Call Trace: [ 5162.649043][ T44] [ 5162.649191][ T44] ? __mutex_trylock_common+0xf9/0x260 [ 5162.649436][ T44] ? __mutex_handoff+0x2b0/0x2b0 [ 5162.649671][ T44] ? rcu_is_watching+0x12/0xb0 [ 5162.649981][ T44] ? rcu_is_watching+0x12/0xb0 [ 5162.650281][ T44] ? trace_contention_end+0xd8/0x140 [ 5162.650578][ T44] ? __mutex_lock+0x19f/0x1190 [ 5162.650846][ T44] ? trace_sched_exit_tp+0xcd/0x130 [ 5162.651100][ T44] ? irqentry_exit+0x3b/0x80 [ 5162.651337][ T44] ? lockdep_hardirqs_on+0x7c/0x110 [ 5162.651592][ T44] ? asm_common_interrupt+0x26/0x40 [ 5162.651897][ T44] ? nf_ct_iterate_cleanup+0x4a/0x340 [nf_conntrack] [ 5162.652247][ T44] ? __nf_conntrack_hash_insert+0x200/0x200 [nf_conntrack] [ 5162.652680][ T44] ? nf_conntrack_cleanup_net_list+0x19a/0x4b0 [nf_conntrack] [ 5162.653116][ T44] ? nf_conntrack_cleanup_net_list+0x10e/0x4b0 [nf_conntrack] [ 5162.653485][ T44] ? nf_conntrack_cleanup_end+0x70/0x70 [nf_conntrack] [ 5162.653811][ T44] ? nf_conntrack_pernet_exit+0xa5/0x140 [nf_conntrack] [ 5162.654187][ T44] ? _raw_spin_unlock+0x2d/0x50 [ 5162.654492][ T44] ? ops_undo_list+0x2bf/0x890 [ 5162.654796][ T44] ? rtnl_net_dumpid_one+0x270/0x270 [ 5162.655096][ T44] ? cleanup_net+0x2d6/0x8b0 [ 5162.655410][ T44] ? cleanup_net+0x3b2/0x8b0 [ 5162.655707][ T44] ? net_passive_dec+0x190/0x190 [ 5162.656003][ T44] ? rcu_is_watching+0x12/0xb0 [ 5162.656279][ T44] ? process_one_work+0xe35/0x1650 [ 5162.656527][ T44] ? pwq_dec_nr_in_flight+0x550/0x550 [ 5162.656768][ T44] ? assign_work+0x168/0x240 [ 5162.657059][ T44] ? worker_thread+0x591/0xcf0 [ 5162.657370][ T44] ? rescuer_thread+0xd10/0xd10 [ 5162.657626][ T44] ? kthread+0x37b/0x5f0 [ 5162.657804][ T44] ? kthread_is_per_cpu+0xc0/0xc0 [ 5162.658059][ T44] ? ret_from_fork+0x1b/0x270 [ 5162.658364][ T44] ? __lock_release+0x5d/0x170 [ 5162.658631][ T44] ? rcu_is_watching+0x12/0xb0 [ 5162.658863][ T44] ? kthread_is_per_cpu+0xc0/0xc0 [ 5162.659104][ T44] ? ret_from_fork+0x1db/0x270 [ 5162.659344][ T44] ? kthread_is_per_cpu+0xc0/0xc0 [ 5162.659639][ T44] ? ret_from_fork_asm+0x11/0x20 [ 5162.659945][ T44] [ 5162.660131][ T44] [ 5162.660131][ T44] Showing all locks held in the system: [ 5162.660611][ T44] 4 locks held by kworker/u16:0/12: [ 5162.660884][ T44] 1 lock held by khungtaskd/44: [ 5162.661114][ T44] #0: ffffffff9b9881c0 (rcu_read_lock){....}-{1:3}, at: debug_show_all_locks+0x36/0x260 [ 5162.661652][ T44] 2 locks held by kworker/u20:1/9353: [ 5162.661940][ T44] #0: ffff888001099d48 ((wq_completion)events_unbound){+.+.}-{0:0}, at: process_one_work+0x7e5/0x1650 [ 5162.662413][ T44] #1: ffffc9000352fd30 ((work_completion)(&sub_info->work)){+.+.}-{0:0}, at: process_one_work+0xded/0x1650 [ 5162.663030][ T44] 1 lock held by modprobe/18131: [ 5162.663297][ T44] #0: ffffffff9c07fb50 (pernet_ops_rwsem){++++}-{4:4}, at: register_pernet_subsys+0x1a/0x40 [ 5162.663750][ T44] [ 5162.663864][ T44] ============================================= [ 5162.663864][ T44] [ 5285.509345][ T44] INFO: task modprobe:18131 blocked for more than 1228 seconds. [ 5285.509989][ T44] Not tainted 6.18.0-rc2-virtme #1 [ 5285.510323][ T44] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. [ 5285.510719][ T44] task:modprobe state:D stack:25664 pid:18131 tgid:18131 ppid:9353 task_flags:0x400100 flags:0x00080001 [ 5285.511254][ T44] Call Trace: [ 5285.511446][ T44] [ 5285.511577][ T44] __schedule+0x880/0x1b10 [ 5285.511829][ T44] ? io_schedule_timeout+0x150/0x150 [ 5285.512085][ T44] ? schedule+0x1ee/0x270 [ 5285.512292][ T44] ? __lock_release+0x5d/0x170 [ 5285.512548][ T44] schedule+0xe2/0x270 [ 5285.512728][ T44] schedule_preempt_disabled+0x18/0x30 [ 5285.512962][ T44] rwsem_down_write_slowpath+0x3c5/0xc60 [ 5285.513201][ T44] ? __lock_acquire+0x449/0x7e0 [ 5285.513446][ T44] ? rwsem_down_read_slowpath+0xd80/0xd80 [ 5285.513762][ T44] ? rcu_is_watching+0x12/0xb0 [ 5285.514058][ T44] ? lock_acquire+0x104/0x160 [ 5285.514309][ T44] ? register_pernet_subsys+0x1a/0x40 [ 5285.514553][ T44] down_write+0x1d6/0x200 [ 5285.514732][ T44] ? rwsem_down_write_slowpath+0xc60/0xc60 [ 5285.515067][ T44] ? find_held_lock+0x2b/0x80 [ 5285.515376][ T44] ? __create_object+0x5e/0xb0 [ 5285.515675][ T44] ? __lock_release+0x5d/0x170 [ 5285.515981][ T44] ? _sub_I_65535_1+0xcf0/0xcf0 [xt_conntrack] [ 5285.516358][ T44] register_pernet_subsys+0x1a/0x40 [ 5285.516594][ T44] vxlan_init_module+0x17/0x1000 [vxlan] [ 5285.516845][ T44] do_one_initcall+0x8c/0x1d0 [ 5285.517100][ T44] ? trace_initcall_start+0x130/0x130 [ 5285.517418][ T44] ? rcu_is_watching+0x12/0xb0 [ 5285.517720][ T44] ? trace_kmalloc+0x26/0xd0 [ 5285.518011][ T44] ? __kmalloc_cache_noprof+0x2e4/0x690 [ 5285.518255][ T44] ? debug_objects_oom+0x160/0x160 [ 5285.518496][ T44] ? kasan_unpoison+0x27/0x50 [ 5285.518754][ T44] ? __asan_register_globals+0x5d/0x70 [ 5285.519036][ T44] do_init_module+0x27a/0x860 [ 5285.519337][ T44] ? free_module+0x2f0/0x2f0 [ 5285.519632][ T44] ? kfree+0x21d/0x540 [ 5285.519865][ T44] ? load_module+0x20c5/0x2dd0 [ 5285.520171][ T44] load_module+0x1f40/0x2dd0 [ 5285.520427][ T44] ? layout_and_allocate.constprop.0+0x630/0x630 [ 5285.520801][ T44] ? kernel_read_file+0x3ed/0x550 [ 5285.521112][ T44] ? kernel_read_file+0x3c8/0x550 [ 5285.521363][ T44] ? __ia32_sys_fsconfig+0x150/0x150 [ 5285.521602][ T44] ? init_module_from_file+0xe9/0x150 [ 5285.521862][ T44] init_module_from_file+0xe9/0x150 [ 5285.522148][ T44] ? __do_sys_init_module+0x1e0/0x1e0 [ 5285.522406][ T44] ? idempotent_init_module+0x316/0x610 [ 5285.522641][ T44] ? __lock_release+0x5d/0x170 [ 5285.522893][ T44] ? do_raw_spin_unlock+0x58/0x220 [ 5285.523193][ T44] idempotent_init_module+0x331/0x610 [ 5285.523501][ T44] ? init_module_from_file+0x150/0x150 [ 5285.523820][ T44] ? cap_capable+0x94/0x230 [ 5285.524092][ T44] __x64_sys_finit_module+0xca/0x150 [ 5285.524353][ T44] ? do_syscall_64+0x85/0xfd0 [ 5285.524656][ T44] do_syscall_64+0xc1/0xfd0 [ 5285.524955][ T44] entry_SYSCALL_64_after_hwframe+0x4b/0x53 [ 5285.525333][ T44] RIP: 0033:0x7fa5db66f92d [ 5285.525611][ T44] RSP: 002b:00007fff6f7d3d98 EFLAGS: 00000246 ORIG_RAX: 0000000000000139 [ 5285.525974][ T44] RAX: ffffffffffffffda RBX: 0000555763c52c30 RCX: 00007fa5db66f92d [ 5285.526335][ T44] RDX: 0000000000000000 RSI: 000055573b805a2a RDI: 0000000000000000 [ 5285.526736][ T44] RBP: 0000000000040000 R08: 0000000000000000 R09: 00007fff6f7d3ed0 [ 5285.527090][ T44] R10: 0000000000000000 R11: 0000000000000246 R12: 000055573b805a2a [ 5285.527462][ T44] R13: 0000555763c52f50 R14: 0000555763c52fa0 R15: 0000555763c52c9e [ 5285.527834][ T44] [ 5285.528065][ T44] INFO: task modprobe:18131 blocked on an rw-semaphore likely owned by task kworker/u16:0:12 [ 5285.528709][ T44] task:kworker/u16:0 state:R running task stack:23392 pid:12 tgid:12 ppid:2 task_flags:0x4208060 flags:0x00080000 [ 5285.529377][ T44] Workqueue: netns cleanup_net [ 5285.529688][ T44] Call Trace: [ 5285.529910][ T44] [ 5285.530058][ T44] ? __mutex_trylock_common+0xf9/0x260 [ 5285.530302][ T44] ? __mutex_handoff+0x2b0/0x2b0 [ 5285.530535][ T44] ? rcu_is_watching+0x12/0xb0 [ 5285.530847][ T44] ? rcu_is_watching+0x12/0xb0 [ 5285.531148][ T44] ? trace_contention_end+0xd8/0x140 [ 5285.531393][ T44] ? __mutex_lock+0x19f/0x1190 [ 5285.531628][ T44] ? trace_sched_exit_tp+0xcd/0x130 [ 5285.531925][ T44] ? rcu_is_watching+0x12/0xb0 [ 5285.532219][ T44] ? trace_irq_disable.constprop.0+0xd0/0x130 [ 5285.532594][ T44] ? irqentry_exit+0x3b/0x80 [ 5285.532889][ T44] ? common_interrupt+0x9b/0xe0 [ 5285.533120][ T44] ? lockdep_hardirqs_on+0x7c/0x110 [ 5285.533380][ T44] ? nf_ct_iterate_cleanup+0x89/0x340 [nf_conntrack] [ 5285.533691][ T44] ? __nf_conntrack_hash_insert+0x200/0x200 [nf_conntrack] [ 5285.534155][ T44] ? nf_conntrack_cleanup_net_list+0x19a/0x4b0 [nf_conntrack] [ 5285.534600][ T44] ? nf_conntrack_cleanup_net_list+0x10e/0x4b0 [nf_conntrack] [ 5285.534960][ T44] ? nf_conntrack_cleanup_end+0x70/0x70 [nf_conntrack] [ 5285.535328][ T44] ? nf_conntrack_pernet_exit+0xa5/0x140 [nf_conntrack] [ 5285.535711][ T44] ? _raw_spin_unlock+0x2d/0x50 [ 5285.536013][ T44] ? ops_undo_list+0x2bf/0x890 [ 5285.536275][ T44] ? rtnl_net_dumpid_one+0x270/0x270 [ 5285.536526][ T44] ? cleanup_net+0x2d6/0x8b0 [ 5285.536816][ T44] ? cleanup_net+0x3b2/0x8b0 [ 5285.537048][ T44] ? net_passive_dec+0x190/0x190 [ 5285.537351][ T44] ? rcu_is_watching+0x12/0xb0 [ 5285.537651][ T44] ? process_one_work+0xe35/0x1650 [ 5285.537930][ T44] ? pwq_dec_nr_in_flight+0x550/0x550 [ 5285.538171][ T44] ? assign_work+0x168/0x240 [ 5285.538489][ T44] ? worker_thread+0x591/0xcf0 [ 5285.538796][ T44] ? rescuer_thread+0xd10/0xd10 [ 5285.539096][ T44] ? kthread+0x37b/0x5f0 [ 5285.539325][ T44] ? kthread_is_per_cpu+0xc0/0xc0 [ 5285.539568][ T44] ? ret_from_fork+0x1b/0x270 [ 5285.539805][ T44] ? __lock_release+0x5d/0x170 [ 5285.540113][ T44] ? rcu_is_watching+0x12/0xb0 [ 5285.540418][ T44] ? kthread_is_per_cpu+0xc0/0xc0 [ 5285.540719][ T44] ? ret_from_fork+0x1db/0x270 [ 5285.540981][ T44] ? kthread_is_per_cpu+0xc0/0xc0 [ 5285.541215][ T44] ? ret_from_fork_asm+0x11/0x20 [ 5285.541471][ T44] [ 5285.541662][ T44] Future hung task reports are suppressed, see sysctl kernel.hung_task_warnings [ 5285.542100][ T44] [ 5285.542100][ T44] Showing all locks held in the system: [ 5285.542530][ T44] 4 locks held by kworker/u16:0/12: [ 5285.542767][ T44] 1 lock held by khungtaskd/44: [ 5285.543040][ T44] #0: ffffffff9b9881c0 (rcu_read_lock){....}-{1:3}, at: debug_show_all_locks+0x36/0x260 [ 5285.543575][ T44] 2 locks held by kworker/u20:1/9353: [ 5285.543870][ T44] #0: ffff888001099d48 ((wq_completion)events_unbound){+.+.}-{0:0}, at: process_one_work+0x7e5/0x1650 [ 5285.544383][ T44] #1: ffffc9000352fd30 ((work_completion)(&sub_info->work)){+.+.}-{0:0}, at: process_one_work+0xded/0x1650 [ 5285.544901][ T44] 1 lock held by modprobe/18131: [ 5285.545145][ T44] #0: ffffffff9c07fb50 (pernet_ops_rwsem){++++}-{4:4}, at: register_pernet_subsys+0x1a/0x40 [ 5285.545741][ T44] [ 5285.545889][ T44] ============================================= [ 5285.545889][ T44] WAIT TIMEOUT stderr Ctrl-C stderr Ctrl-C stderr