[ 963.988107][T11628] netdevsim netdevsim1337 eni1337np1: renamed from eth0 [ 964.085086][T11632] netdevsim netdevsim1057 eni1057np1: renamed from eth1 [ 964.217976][T11646] netdevsim netdevsim5417 eni5417np1: renamed from eth0 [ 986.953720][T11612] [ 986.953882][T11612] ====================================================== [ 986.954128][T11612] WARNING: possible circular locking dependency detected [ 986.954364][T11612] 6.13.0-rc2-virtme #1 Not tainted [ 986.954558][T11612] ------------------------------------------------------ [ 986.954794][T11612] hw_stats_l3.sh/11612 is trying to acquire lock: [ 986.955027][T11612] ffff8880091b9690 (kn->active#25){++++}-{0:0}, at: __kernfs_remove+0x1f9/0x590 [ 986.955387][T11612] [ 986.955387][T11612] but task is already holding lock: [ 986.955665][T11612] ffffffff93fe1248 (rtnl_mutex){+.+.}-{4:4}, at: nsim_destroy+0x6b/0x5f0 [netdevsim] [ 986.956016][T11612] [ 986.956016][T11612] which lock already depends on the new lock. [ 986.956016][T11612] [ 986.956384][T11612] [ 986.956384][T11612] the existing dependency chain (in reverse order) is: [ 986.956726][T11612] [ 986.956726][T11612] -> #1 (rtnl_mutex){+.+.}-{4:4}: [ 986.957012][T11612] __lock_acquire+0xb9a/0x1680 [ 986.957212][T11612] lock_acquire.part.0+0xeb/0x330 [ 986.957406][T11612] __mutex_lock+0x169/0xbc0 [ 986.957598][T11612] phys_port_name_show+0xbe/0x200 [ 986.957791][T11612] dev_attr_show+0x43/0xc0 [ 986.957982][T11612] sysfs_kf_seq_show+0x1fb/0x3d0 [ 986.958170][T11612] seq_read_iter+0x40e/0x1090 [ 986.958358][T11612] vfs_read+0x755/0xce0 [ 986.958547][T11612] ksys_read+0xf8/0x1d0 [ 986.958739][T11612] do_syscall_64+0xc1/0x1d0 [ 986.958922][T11612] entry_SYSCALL_64_after_hwframe+0x77/0x7f [ 986.959158][T11612] [ 986.959158][T11612] -> #0 (kn->active#25){++++}-{0:0}: [ 986.959450][T11612] check_prev_add+0x182/0xce0 [ 986.959647][T11612] validate_chain+0x7bd/0xae0 [ 986.959841][T11612] __lock_acquire+0xb9a/0x1680 [ 986.960031][T11612] lock_acquire.part.0+0xeb/0x330 [ 986.960219][T11612] kernfs_drain+0x384/0x460 [ 986.960410][T11612] __kernfs_remove+0x1f9/0x590 [ 986.960596][T11612] kernfs_remove_by_name_ns+0xa5/0x100 [ 986.960834][T11612] remove_files+0x8c/0x1a0 [ 986.961023][T11612] sysfs_remove_group+0x7b/0x160 [ 986.961210][T11612] sysfs_remove_groups+0x53/0xa0 [ 986.961402][T11612] device_remove_attrs+0x1db/0x260 [ 986.961590][T11612] device_del+0x337/0x8c0 [ 986.961775][T11612] unregister_netdevice_many_notify+0x43d/0x1580 [ 986.962013][T11612] unregister_netdevice_queue+0x2a4/0x410 [ 986.962245][T11612] nsim_destroy+0xe8/0x5f0 [netdevsim] [ 986.962499][T11612] __nsim_dev_port_del+0x17e/0x250 [netdevsim] [ 986.962741][T11612] nsim_dev_reload_destroy+0xe0/0x470 [netdevsim] [ 986.962987][T11612] nsim_drv_remove+0x51/0x1d0 [netdevsim] [ 986.963228][T11612] device_release_driver_internal+0x3bf/0x590 [ 986.963461][T11612] bus_remove_device+0x1f1/0x3f0 [ 986.963643][T11612] device_del+0x33f/0x8c0 [ 986.963831][T11612] device_unregister+0x17/0xb0 [ 986.964020][T11612] del_device_store+0x2f3/0x4f0 [netdevsim] [ 986.964262][T11612] kernfs_fop_write_iter+0x2ba/0x460 [ 986.964502][T11612] vfs_write+0xa81/0x11e0 [ 986.964690][T11612] ksys_write+0xf8/0x1d0 [ 986.964873][T11612] do_syscall_64+0xc1/0x1d0 [ 986.965059][T11612] entry_SYSCALL_64_after_hwframe+0x77/0x7f [ 986.965290][T11612] [ 986.965290][T11612] other info that might help us debug this: [ 986.965290][T11612] [ 986.965655][T11612] Possible unsafe locking scenario: [ 986.965655][T11612] [ 986.965932][T11612] CPU0 CPU1 [ 986.966121][T11612] ---- ---- [ 986.966306][T11612] lock(rtnl_mutex); [ 986.966444][T11612] lock(kn->active#25); [ 986.966675][T11612] lock(rtnl_mutex); [ 986.966930][T11612] lock(kn->active#25); [ 986.967076][T11612] [ 986.967076][T11612] *** DEADLOCK *** [ 986.967076][T11612] [ 986.967354][T11612] 7 locks held by hw_stats_l3.sh/11612: [ 986.967541][T11612] #0: ffff88800a9093f0 (sb_writers#8){.+.+}-{0:0}, at: ksys_write+0xf8/0x1d0 [ 986.967874][T11612] #1: ffff88800bebd088 (&of->mutex){+.+.}-{4:4}, at: kernfs_fop_write_iter+0x20b/0x460 [ 986.968204][T11612] #2: ffff88800d795210 (kn->active#38){.+.+}-{0:0}, at: kernfs_fop_write_iter+0x22e/0x460 [ 986.968581][T11612] #3: ffffffffc03035e8 (nsim_bus_dev_list_lock){+.+.}-{4:4}, at: del_device_store+0x9f/0x4f0 [netdevsim] [ 986.968964][T11612] #4: ffff88800d5068e8 (&dev->mutex){....}-{4:4}, at: device_release_driver_internal+0xa7/0x590 [ 986.969338][T11612] #5: ffff88800d500a50 (&devlink->lock_key#2){+.+.}-{4:4}, at: nsim_drv_remove+0x49/0x1d0 [netdevsim] [ 986.969720][T11612] #6: ffffffff93fe1248 (rtnl_mutex){+.+.}-{4:4}, at: nsim_destroy+0x6b/0x5f0 [netdevsim] [ 986.970053][T11612] [ 986.970053][T11612] stack backtrace: [ 986.970284][T11612] CPU: 0 UID: 0 PID: 11612 Comm: hw_stats_l3.sh Not tainted 6.13.0-rc2-virtme #1 [ 986.970612][T11612] Hardware name: Bochs Bochs, BIOS Bochs 01/01/2011 [ 986.970845][T11612] Call Trace: [ 986.970990][T11612] [ 986.971086][T11612] dump_stack_lvl+0x82/0xd0 [ 986.971278][T11612] print_circular_bug+0x2e2/0x4b0 [ 986.971467][T11612] check_noncircular+0x2f9/0x3e0 [ 986.971654][T11612] ? __pfx_check_noncircular+0x10/0x10 [ 986.971844][T11612] ? __kernel_text_address+0x12/0x40 [ 986.972034][T11612] ? unwind_get_return_address+0x5e/0xa0 [ 986.972237][T11612] ? __pfx_stack_trace_consume_entry+0x10/0x10 [ 986.972476][T11612] ? arch_stack_walk+0xa2/0xf0 [ 986.972666][T11612] check_prev_add+0x182/0xce0 [ 986.972854][T11612] ? __pfx_stack_trace_save+0x10/0x10 [ 986.973042][T11612] validate_chain+0x7bd/0xae0 [ 986.973230][T11612] ? __pfx_validate_chain+0x10/0x10 [ 986.973418][T11612] ? mark_lock+0x335/0x3e0 [ 986.973609][T11612] __lock_acquire+0xb9a/0x1680 [ 986.973804][T11612] ? kasan_addr_to_slab+0x40/0x80 [ 986.973997][T11612] lock_acquire.part.0+0xeb/0x330 [ 986.974183][T11612] ? __kernfs_remove+0x1f9/0x590 [ 986.974371][T11612] ? __pfx___lock_release+0x10/0x10 [ 986.974554][T11612] ? __pfx_lock_acquire.part.0+0x10/0x10 [ 986.974738][T11612] ? trace_lock_acquire+0x14c/0x1f0 [ 986.974922][T11612] ? __virt_addr_valid+0x22b/0x430 [ 986.975114][T11612] ? lock_acquire+0x32/0xc0 [ 986.975298][T11612] ? __kernfs_remove+0x1f9/0x590 [ 986.975593][T11612] kernfs_drain+0x384/0x460 [ 986.975885][T11612] ? __kernfs_remove+0x1f9/0x590 [ 986.976179][T11612] ? __pfx_kernfs_drain+0x10/0x10 [ 986.976366][T11612] ? lock_acquire+0x32/0xc0 [ 986.976549][T11612] __kernfs_remove+0x1f9/0x590 [ 986.976747][T11612] kernfs_remove_by_name_ns+0xa5/0x100 [ 986.976938][T11612] remove_files+0x8c/0x1a0 [ 986.977229][T11612] sysfs_remove_group+0x7b/0x160 [ 986.977415][T11612] sysfs_remove_groups+0x53/0xa0 [ 986.977811][T11612] device_remove_attrs+0x1db/0x260 [ 986.977997][T11612] ? __pfx_device_remove_attrs+0x10/0x10 [ 986.978185][T11612] device_del+0x337/0x8c0 [ 986.978324][T11612] ? __pfx_device_del+0x10/0x10 [ 986.978513][T11612] unregister_netdevice_many_notify+0x43d/0x1580 [ 986.978746][T11612] ? __mutex_trylock_common+0xfa/0x260 [ 986.978934][T11612] ? __pfx___mutex_trylock_common+0x10/0x10 [ 986.979161][T11612] ? __pfx_unregister_netdevice_many_notify+0x10/0x10 [ 986.979391][T11612] ? __mutex_lock+0x190/0xbc0 [ 986.979582][T11612] ? nsim_destroy+0x6b/0x5f0 [netdevsim] [ 986.979781][T11612] ? __pfx___mutex_lock+0x10/0x10 [ 986.979970][T11612] unregister_netdevice_queue+0x2a4/0x410 [ 986.980156][T11612] ? __pfx_do_raw_spin_lock+0x10/0x10 [ 986.980452][T11612] ? __pfx_unregister_netdevice_queue+0x10/0x10 [ 986.980686][T11612] nsim_destroy+0xe8/0x5f0 [netdevsim] [ 986.980879][T11612] __nsim_dev_port_del+0x17e/0x250 [netdevsim] [ 986.981119][T11612] nsim_dev_reload_destroy+0xe0/0x470 [netdevsim] [ 986.981579][T11612] nsim_drv_remove+0x51/0x1d0 [netdevsim] [ 986.981779][T11612] device_release_driver_internal+0x3bf/0x590 [ 986.982012][T11612] ? klist_put+0xb1/0x170 [ 986.982157][T11612] bus_remove_device+0x1f1/0x3f0 [ 986.982342][T11612] device_del+0x33f/0x8c0 [ 986.982481][T11612] ? __pfx_device_del+0x10/0x10 [ 986.982665][T11612] ? lock_acquire.part.0+0xeb/0x330 [ 986.982852][T11612] ? kernfs_fop_write_iter+0x22e/0x460 [ 986.983037][T11612] device_unregister+0x17/0xb0 [ 986.983327][T11612] del_device_store+0x2f3/0x4f0 [netdevsim] [ 986.983568][T11612] ? __pfx_del_device_store+0x10/0x10 [netdevsim] [ 986.983808][T11612] ? __pfx_sysfs_kf_write+0x10/0x10 [ 986.984097][T11612] ? sysfs_file_ops+0x11e/0x170 [ 986.984283][T11612] ? __pfx_sysfs_kf_write+0x10/0x10 [ 986.984469][T11612] kernfs_fop_write_iter+0x2ba/0x460 [ 986.984655][T11612] vfs_write+0xa81/0x11e0 [ 986.984797][T11612] ? __pfx_vfs_write+0x10/0x10 [ 986.984980][T11612] ? __lock_release+0x103/0x460 [ 986.985165][T11612] ? do_user_addr_fault+0x972/0xe30 [ 986.985359][T11612] ? __pfx___up_read+0x10/0x10 [ 986.985757][T11612] ksys_write+0xf8/0x1d0 [ 986.985908][T11612] ? __pfx_ksys_write+0x10/0x10 [ 986.986098][T11612] ? do_user_addr_fault+0x97c/0xe30 [ 986.986286][T11612] do_syscall_64+0xc1/0x1d0 [ 986.986474][T11612] entry_SYSCALL_64_after_hwframe+0x77/0x7f [ 986.986811][T11612] RIP: 0033:0x7f1407b0eb77 [ 986.987009][T11612] Code: Unable to access opcode bytes at 0x7f1407b0eb4d. [ 986.987252][T11612] RSP: 002b:00007ffc05944428 EFLAGS: 00000246 ORIG_RAX: 0000000000000001 [ 986.987647][T11612] RAX: ffffffffffffffda RBX: 0000000000000005 RCX: 00007f1407b0eb77 [ 986.987920][T11612] RDX: 0000000000000005 RSI: 000055ad382adc60 RDI: 0000000000000001 [ 986.988197][T11612] RBP: 000055ad382adc60 R08: 0000000000000000 R09: 00007f1407b814e0 [ 986.988580][T11612] R10: 00007f1407b813e0 R11: 0000000000000246 R12: 0000000000000005 [ 986.988860][T11612] R13: 00007f1407bca760 R14: 0000000000000005 R15: 00007f1407bc59c0 [ 986.989245][T11612]