[ 24.627074][ T10] [ 24.627175][ T10] ===================================================== [ 24.627345][ T10] WARNING: SOFTIRQ-safe -> SOFTIRQ-unsafe lock order detected [ 24.627553][ T10] 6.9.0-rc1-virtme #1 Not tainted [ 24.627692][ T10] ----------------------------------------------------- [ 24.627866][ T10] kworker/u16:0/10 [HC0[0]:SC0[2]:HE1:SE0] is trying to acquire: [ 24.628065][ T10] ffff888001a70968 (&p->alloc_lock){+.+.}-{2:2}, at: __get_task_comm+0x27/0x70 [ 24.628311][ T10] [ 24.628311][ T10] and this task is already holding: [ 24.628525][ T10] ffff88803613f218 (&ul->lock){+.-.}-{2:2}, at: rt6_uncached_list_flush_dev+0x138/0x840 [ 24.628780][ T10] which would create a new lock dependency: [ 24.628951][ T10] (&ul->lock){+.-.}-{2:2} -> (&p->alloc_lock){+.+.}-{2:2} [ 24.629164][ T10] [ 24.629164][ T10] but this new dependency connects a SOFTIRQ-irq-safe lock: [ 24.629395][ T10] (&ul->lock){+.-.}-{2:2} [ 24.629398][ T10] [ 24.629398][ T10] ... which became SOFTIRQ-irq-safe at: [ 24.629731][ T10] __lock_acquire+0x797/0x1570 [ 24.629876][ T10] lock_acquire.part.0+0xeb/0x330 [ 24.630013][ T10] _raw_spin_lock_bh+0x38/0x50 [ 24.630154][ T10] rt6_uncached_list_del+0x77/0x200 [ 24.630292][ T10] ip6_dst_destroy+0x74/0x1e0 [ 24.630429][ T10] dst_destroy+0xbc/0x370 [ 24.630567][ T10] rcu_do_batch+0x3c0/0xfb0 [ 24.630703][ T10] rcu_core+0x2be/0x500 [ 24.630805][ T10] __do_softirq+0x1f8/0x5df [ 24.630945][ T10] irq_exit_rcu+0x97/0xc0 [ 24.631085][ T10] sysvec_apic_timer_interrupt+0x75/0x80 [ 24.631224][ T10] asm_sysvec_apic_timer_interrupt+0x1a/0x20 [ 24.631402][ T10] default_idle+0xf/0x20 [ 24.631504][ T10] default_idle_call+0x6d/0xb0 [ 24.631637][ T10] cpuidle_idle_call+0x1f4/0x280 [ 24.631782][ T10] do_idle+0xf9/0x160 [ 24.631880][ T10] cpu_startup_entry+0x54/0x60 [ 24.632008][ T10] start_secondary+0x21c/0x2b0 [ 24.632142][ T10] common_startup_64+0x12c/0x138 [ 24.632275][ T10] [ 24.632275][ T10] to a SOFTIRQ-irq-unsafe lock: [ 24.632436][ T10] (&p->alloc_lock){+.+.}-{2:2} [ 24.632439][ T10] [ 24.632439][ T10] ... which became SOFTIRQ-irq-unsafe at: [ 24.632776][ T10] ... [ 24.632777][ T10] __lock_acquire+0x797/0x1570 [ 24.632981][ T10] lock_acquire.part.0+0xeb/0x330 [ 24.633117][ T10] _raw_spin_lock+0x30/0x40 [ 24.633251][ T10] set_mems_allowed+0x1d/0x210 [ 24.633389][ T10] kernel_init_freeable+0x72/0x310 [ 24.633526][ T10] kernel_init+0x20/0x200 [ 24.633660][ T10] ret_from_fork+0x31/0x70 [ 24.633813][ T10] ret_from_fork_asm+0x1a/0x30 [ 24.633949][ T10] [ 24.633949][ T10] other info that might help us debug this: [ 24.633949][ T10] [ 24.634220][ T10] Possible interrupt unsafe locking scenario: [ 24.634220][ T10] [ 24.634420][ T10] CPU0 CPU1 [ 24.634557][ T10] ---- ---- [ 24.634707][ T10] lock(&p->alloc_lock); [ 24.634814][ T10] local_irq_disable(); [ 24.634989][ T10] lock(&ul->lock); [ 24.635161][ T10] lock(&p->alloc_lock); [ 24.635329][ T10] [ 24.635427][ T10] lock(&ul->lock); [ 24.635534][ T10] [ 24.635534][ T10] *** DEADLOCK *** [ 24.635534][ T10] [ 24.635739][ T10] 5 locks held by kworker/u16:0/10: [ 24.635876][ T10] #0: ffff88800190a948 ((wq_completion)netns){+.+.}-{0:0}, at: process_one_work+0xda6/0x1730 [ 24.636174][ T10] #1: ffffc900000afda0 (net_cleanup_work){+.+.}-{0:0}, at: process_one_work+0xde2/0x1730 [ 24.636421][ T10] #2: ffffffff9b59bc50 (pernet_ops_rwsem){++++}-{3:3}, at: cleanup_net+0xbc/0xb60 [ 24.636681][ T10] #3: ffffffff9b5b7208 (rtnl_mutex){+.+.}-{3:3}, at: default_device_exit_batch+0x81/0x2d0 [ 24.636959][ T10] #4: ffff88803613f218 (&ul->lock){+.-.}-{2:2}, at: rt6_uncached_list_flush_dev+0x138/0x840 [ 24.637237][ T10] [ 24.637237][ T10] the dependencies between SOFTIRQ-irq-safe lock and the holding lock: [ 24.637506][ T10] -> (&ul->lock){+.-.}-{2:2} { [ 24.637649][ T10] HARDIRQ-ON-W at: [ 24.637761][ T10] __lock_acquire+0x797/0x1570 [ 24.637950][ T10] lock_acquire.part.0+0xeb/0x330 [ 24.638121][ T10] _raw_spin_lock_bh+0x38/0x50 [ 24.638288][ T10] rt6_uncached_list_add+0x55/0x1b0 [ 24.638458][ T10] icmp6_dst_alloc+0x2c0/0x450 [ 24.638632][ T10] mld_sendpack+0x529/0xbc0 [ 24.638836][ T10] mld_ifc_work+0x32/0x1f0 [ 24.639008][ T10] process_one_work+0xe2c/0x1730 [ 24.639185][ T10] worker_thread+0x587/0xd30 [ 24.639361][ T10] kthread+0x28a/0x350 [ 24.639499][ T10] ret_from_fork+0x31/0x70 [ 24.639676][ T10] ret_from_fork_asm+0x1a/0x30 [ 24.639854][ T10] IN-SOFTIRQ-W at: [ 24.639964][ T10] __lock_acquire+0x797/0x1570 [ 24.640138][ T10] lock_acquire.part.0+0xeb/0x330 [ 24.640331][ T10] _raw_spin_lock_bh+0x38/0x50 [ 24.640504][ T10] rt6_uncached_list_del+0x77/0x200 [ 24.640692][ T10] ip6_dst_destroy+0x74/0x1e0 [ 24.640870][ T10] dst_destroy+0xbc/0x370 [ 24.641039][ T10] rcu_do_batch+0x3c0/0xfb0 [ 24.641211][ T10] rcu_core+0x2be/0x500 [ 24.641388][ T10] __do_softirq+0x1f8/0x5df [ 24.641554][ T10] irq_exit_rcu+0x97/0xc0 [ 24.641717][ T10] sysvec_apic_timer_interrupt+0x75/0x80 [ 24.641910][ T10] asm_sysvec_apic_timer_interrupt+0x1a/0x20 [ 24.642109][ T10] default_idle+0xf/0x20 [ 24.642274][ T10] default_idle_call+0x6d/0xb0 [ 24.642442][ T10] cpuidle_idle_call+0x1f4/0x280 [ 24.642604][ T10] do_idle+0xf9/0x160 [ 24.642735][ T10] cpu_startup_entry+0x54/0x60 [ 24.642917][ T10] start_secondary+0x21c/0x2b0 [ 24.643083][ T10] common_startup_64+0x12c/0x138 [ 24.643249][ T10] INITIAL USE at: [ 24.643431][ T10] __lock_acquire+0x797/0x1570 [ 24.643603][ T10] lock_acquire.part.0+0xeb/0x330 [ 24.643772][ T10] _raw_spin_lock_bh+0x38/0x50 [ 24.644032][ T10] rt6_uncached_list_add+0x55/0x1b0 [ 24.644199][ T10] icmp6_dst_alloc+0x2c0/0x450 [ 24.644365][ T10] mld_sendpack+0x529/0xbc0 [ 24.644530][ T10] mld_ifc_work+0x32/0x1f0 [ 24.644696][ T10] process_one_work+0xe2c/0x1730 [ 24.644861][ T10] worker_thread+0x587/0xd30 [ 24.645028][ T10] kthread+0x28a/0x350 [ 24.645171][ T10] ret_from_fork+0x31/0x70 [ 24.645348][ T10] ret_from_fork_asm+0x1a/0x30 [ 24.645517][ T10] } [ 24.645592][ T10] ... key at: [] __key.15+0x0/0x40 [ 24.645866][ T10] [ 24.645866][ T10] the dependencies between the lock to be acquired [ 24.645867][ T10] and SOFTIRQ-irq-unsafe lock: [ 24.646228][ T10] -> (&p->alloc_lock){+.+.}-{2:2} { [ 24.646371][ T10] HARDIRQ-ON-W at: [ 24.646567][ T10] __lock_acquire+0x797/0x1570 [ 24.646735][ T10] lock_acquire.part.0+0xeb/0x330 [ 24.646908][ T10] _raw_spin_lock+0x30/0x40 [ 24.647081][ T10] set_mems_allowed+0x1d/0x210 [ 24.647249][ T10] kernel_init_freeable+0x72/0x310 [ 24.647422][ T10] kernel_init+0x20/0x200 [ 24.647685][ T10] ret_from_fork+0x31/0x70 [ 24.647866][ T10] ret_from_fork_asm+0x1a/0x30 [ 24.648046][ T10] SOFTIRQ-ON-W at: [ 24.648150][ T10] __lock_acquire+0x797/0x1570 [ 24.648336][ T10] lock_acquire.part.0+0xeb/0x330 [ 24.648595][ T10] _raw_spin_lock+0x30/0x40 [ 24.648770][ T10] set_mems_allowed+0x1d/0x210 [ 24.648949][ T10] kernel_init_freeable+0x72/0x310 [ 24.649300][ T10] kernel_init+0x20/0x200 [ 24.649474][ T10] ret_from_fork+0x31/0x70 [ 24.649644][ T10] ret_from_fork_asm+0x1a/0x30 [ 24.649919][ T10] INITIAL USE at: [ 24.650019][ T10] __lock_acquire+0x797/0x1570 [ 24.650186][ T10] lock_acquire.part.0+0xeb/0x330 [ 24.650359][ T10] _raw_spin_lock+0x30/0x40 [ 24.650609][ T10] set_mems_allowed+0x1d/0x210 [ 24.650782][ T10] kernel_init_freeable+0x72/0x310 [ 24.650947][ T10] kernel_init+0x20/0x200 [ 24.651197][ T10] ret_from_fork+0x31/0x70 [ 24.651470][ T10] ret_from_fork_asm+0x1a/0x30 [ 24.651642][ T10] } [ 24.651735][ T10] ... key at: [] __key.388+0x0/0x40 [ 24.651942][ T10] ... acquired at: [ 24.652131][ T10] check_prev_add+0x194/0xce0 [ 24.652280][ T10] validate_chain+0x71d/0x9b0 [ 24.652428][ T10] __lock_acquire+0xaf0/0x1570 [ 24.652565][ T10] lock_acquire.part.0+0xeb/0x330 [ 24.652703][ T10] _raw_spin_lock+0x30/0x40 [ 24.652840][ T10] __get_task_comm+0x27/0x70 [ 24.652978][ T10] ref_tracker_alloc+0x2ee/0x490 [ 24.653200][ T10] rt6_uncached_list_flush_dev+0x494/0x840 [ 24.653371][ T10] rt6_disable_ip+0xd9/0x140 [ 24.653505][ T10] addrconf_ifdown.isra.0+0x103/0x1560 [ 24.653647][ T10] addrconf_notify+0xd1/0xd60 [ 24.653783][ T10] notifier_call_chain+0xcd/0x150 [ 24.654001][ T10] dev_close_many+0x2d8/0x650 [ 24.654141][ T10] unregister_netdevice_many_notify+0x3d5/0x1190 [ 24.654315][ T10] default_device_exit_batch+0x234/0x2d0 [ 24.654480][ T10] cleanup_net+0x4f3/0xb60 [ 24.654801][ T10] process_one_work+0xe2c/0x1730 [ 24.654941][ T10] worker_thread+0x587/0xd30 [ 24.655161][ T10] kthread+0x28a/0x350 [ 24.655266][ T10] ret_from_fork+0x31/0x70 [ 24.655408][ T10] ret_from_fork_asm+0x1a/0x30 [ 24.655545][ T10] [ 24.655616][ T10] [ 24.655616][ T10] stack backtrace: [ 24.655802][ T10] CPU: 1 PID: 10 Comm: kworker/u16:0 Not tainted 6.9.0-rc1-virtme #1 [ 24.656106][ T10] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS rel-1.16.3-0-ga6ed6b701f0a-prebuilt.qemu.org 04/01/2014 [ 24.656407][ T10] Workqueue: netns cleanup_net [ 24.656549][ T10] Call Trace: [ 24.656670][ T10] [ 24.656745][ T10] dump_stack_lvl+0x82/0xd0 [ 24.656889][ T10] print_bad_irq_dependency+0x957/0xd40 [ 24.657026][ T10] ? __pfx_print_bad_irq_dependency+0x10/0x10 [ 24.657199][ T10] ? __pfx_usage_skip+0x10/0x10 [ 24.657344][ T10] ? __bfs+0x246/0x650 [ 24.657450][ T10] ? __pfx_usage_match+0x10/0x10 [ 24.657587][ T10] check_irq_usage+0x596/0x850 [ 24.657728][ T10] ? arch_stack_walk+0x8c/0xf0 [ 24.657879][ T10] ? __pfx_check_irq_usage+0x10/0x10 [ 24.658096][ T10] ? __bfs+0x246/0x650 [ 24.658196][ T10] ? __pfx_hlock_conflict+0x10/0x10 [ 24.658343][ T10] ? check_path.constprop.0+0x24/0x50 [ 24.658482][ T10] ? check_noncircular+0x14c/0x3e0 [ 24.658618][ T10] ? __pfx_check_noncircular+0x10/0x10 [ 24.658766][ T10] ? check_prev_add+0x194/0xce0 [ 24.658903][ T10] check_prev_add+0x194/0xce0 [ 24.659042][ T10] ? validate_chain+0x130/0x9b0 [ 24.659258][ T10] validate_chain+0x71d/0x9b0 [ 24.659398][ T10] ? mark_lock+0x38/0x3e0 [ 24.659502][ T10] ? __pfx_validate_chain+0x10/0x10 [ 24.659641][ T10] ? hlock_class+0x4e/0x130 [ 24.659776][ T10] ? mark_lock+0x38/0x3e0 [ 24.659982][ T10] __lock_acquire+0xaf0/0x1570 [ 24.660120][ T10] lock_acquire.part.0+0xeb/0x330 [ 24.660257][ T10] ? __get_task_comm+0x27/0x70 [ 24.660390][ T10] ? mark_lock+0x38/0x3e0 [ 24.660490][ T10] ? __pfx_lock_acquire.part.0+0x10/0x10 [ 24.660705][ T10] ? trace_lock_acquire+0x135/0x1c0 [ 24.660837][ T10] ? __get_task_comm+0x27/0x70 [ 24.660966][ T10] ? lock_acquire+0x32/0xc0 [ 24.661094][ T10] ? __get_task_comm+0x27/0x70 [ 24.661375][ T10] _raw_spin_lock+0x30/0x40 [ 24.661507][ T10] ? __get_task_comm+0x27/0x70 [ 24.661639][ T10] __get_task_comm+0x27/0x70 [ 24.661772][ T10] ref_tracker_alloc+0x2ee/0x490 [ 24.661979][ T10] ? __pfx_ref_tracker_alloc+0x10/0x10 [ 24.662107][ T10] ? __lock_release+0x103/0x460 [ 24.662235][ T10] ? rt6_uncached_list_flush_dev+0x494/0x840 [ 24.662394][ T10] ? rt6_disable_ip+0xd9/0x140 [ 24.662595][ T10] ? addrconf_ifdown.isra.0+0x103/0x1560 [ 24.662724][ T10] ? addrconf_notify+0xd1/0xd60 [ 24.662851][ T10] ? notifier_call_chain+0xcd/0x150 [ 24.662980][ T10] ? dev_close_many+0x2d8/0x650 [ 24.663198][ T10] ? unregister_netdevice_many_notify+0x3d5/0x1190 [ 24.663364][ T10] ? default_device_exit_batch+0x234/0x2d0 [ 24.663528][ T10] ? cleanup_net+0x4f3/0xb60 [ 24.663659][ T10] ? process_one_work+0xe2c/0x1730 [ 24.663865][ T10] ? worker_thread+0x587/0xd30 [ 24.663996][ T10] ? kthread+0x28a/0x350 [ 24.664097][ T10] ? ret_from_fork+0x31/0x70 [ 24.664225][ T10] ? ret_from_fork_asm+0x1a/0x30 [ 24.664357][ T10] ? in6_dev_get+0xcc/0x210 [ 24.664562][ T10] rt6_uncached_list_flush_dev+0x494/0x840 [ 24.664723][ T10] rt6_disable_ip+0xd9/0x140 [ 24.664851][ T10] ? __pfx_rt6_disable_ip+0x10/0x10 [ 24.664980][ T10] ? mark_lock+0x38/0x3e0 [ 24.665176][ T10] addrconf_ifdown.isra.0+0x103/0x1560 [ 24.665312][ T10] ? __try_to_del_timer_sync+0xe7/0x140 [ 24.665452][ T10] ? __pfx_addrconf_ifdown.isra.0+0x10/0x10 [ 24.665624][ T10] addrconf_notify+0xd1/0xd60 [ 24.665757][ T10] notifier_call_chain+0xcd/0x150 [ 24.665904][ T10] dev_close_many+0x2d8/0x650 [ 24.666041][ T10] ? __pfx_dev_close_many+0x10/0x10 [ 24.666176][ T10] ? default_device_exit_batch+0x81/0x2d0 [ 24.666389][ T10] ? trace_contention_end+0xdc/0x140 [ 24.666538][ T10] unregister_netdevice_many_notify+0x3d5/0x1190 [ 24.666708][ T10] ? mutex_is_locked+0x17/0x50 [ 24.666845][ T10] ? __pfx_unregister_netdevice_many_notify+0x10/0x10 [ 24.667124][ T10] ? __pfx_unregister_netdevice_queue+0x10/0x10 [ 24.667293][ T10] default_device_exit_batch+0x234/0x2d0 [ 24.667429][ T10] ? __pfx_default_device_exit_batch+0x10/0x10 [ 24.667603][ T10] ? ops_exit_list+0xb4/0x170 [ 24.667739][ T10] cleanup_net+0x4f3/0xb60 [ 24.667879][ T10] ? __pfx_lock_acquire.part.0+0x10/0x10 [ 24.668011][ T10] ? __pfx_cleanup_net+0x10/0x10 [ 24.668144][ T10] ? trace_lock_acquire+0x135/0x1c0 [ 24.668355][ T10] ? process_one_work+0xde2/0x1730 [ 24.668489][ T10] ? lock_acquire+0x32/0xc0 [ 24.668620][ T10] ? process_one_work+0xde2/0x1730 [ 24.668782][ T10] process_one_work+0xe2c/0x1730 [ 24.669181][ T10] ? __pfx___lock_release+0x10/0x10 [ 24.669325][ T10] ? __pfx_process_one_work+0x10/0x10 [ 24.669466][ T10] ? assign_work+0x16c/0x240 [ 24.669602][ T10] worker_thread+0x587/0xd30 [ 24.669744][ T10] ? __pfx_worker_thread+0x10/0x10 [ 24.669882][ T10] kthread+0x28a/0x350 [ 24.669984][ T10] ? __pfx_kthread+0x10/0x10 [ 24.670122][ T10] ret_from_fork+0x31/0x70 [ 24.670251][ T10] ? __pfx_kthread+0x10/0x10 [ 24.670470][ T10] ret_from_fork_asm+0x1a/0x30 [ 24.670620][ T10] [ 27.237692][ T282] veth0: renamed from ioam-veth-alpha [ 27.340582][ T283] veth0: renamed from ioam-veth-betaL [ 27.438579][ T284] veth1: renamed from ioam-veth-betaR [ 27.558366][ T285] veth0: renamed from ioam-veth-gamma