[ 41.797457][ T406] netdevsim netdevsim1337 eni1337np1: renamed from eth0 [ 41.809325][ T409] netdevsim netdevsim1057 eni1057np1: renamed from eth1 [ 41.934017][ T422] netdevsim netdevsim5417 eni5417np1: renamed from eth2 [ 65.651514][ T834] ip (834) used greatest stack depth: 23888 bytes left [ 65.887535][ T388] [ 65.887727][ T388] ====================================================== [ 65.888060][ T388] WARNING: possible circular locking dependency detected [ 65.888398][ T388] 6.13.0-rc2-virtme #1 Not tainted [ 65.888669][ T388] ------------------------------------------------------ [ 65.889011][ T388] hw_stats_l3.sh/388 is trying to acquire lock: [ 65.889353][ T388] ffff888008d9a5b0 (kn->active#25){++++}-{0:0}, at: __kernfs_remove+0x1f9/0x590 [ 65.889842][ T388] [ 65.889842][ T388] but task is already holding lock: [ 65.890237][ T388] ffffffffa59e1208 (rtnl_mutex){+.+.}-{4:4}, at: nsim_destroy+0x6b/0x5f0 [netdevsim] [ 65.890731][ T388] [ 65.890731][ T388] which lock already depends on the new lock. [ 65.890731][ T388] [ 65.891287][ T388] [ 65.891287][ T388] the existing dependency chain (in reverse order) is: [ 65.891761][ T388] [ 65.891761][ T388] -> #1 (rtnl_mutex){+.+.}-{4:4}: [ 65.892182][ T388] __lock_acquire+0xb9a/0x1680 [ 65.892460][ T388] lock_acquire.part.0+0xeb/0x330 [ 65.892731][ T388] __mutex_lock+0x169/0xbc0 [ 65.893001][ T388] phys_port_name_show+0xbe/0x200 [ 65.893276][ T388] dev_attr_show+0x43/0xc0 [ 65.893548][ T388] sysfs_kf_seq_show+0x1fb/0x3d0 [ 65.893845][ T388] seq_read_iter+0x40e/0x1090 [ 65.894108][ T388] vfs_read+0x755/0xce0 [ 65.894379][ T388] ksys_read+0xf8/0x1d0 [ 65.894647][ T388] do_syscall_64+0xc1/0x1d0 [ 65.894904][ T388] entry_SYSCALL_64_after_hwframe+0x77/0x7f [ 65.895225][ T388] [ 65.895225][ T388] -> #0 (kn->active#25){++++}-{0:0}: [ 65.895633][ T388] check_prev_add+0x182/0xce0 [ 65.895920][ T388] validate_chain+0x7bd/0xae0 [ 65.896183][ T388] __lock_acquire+0xb9a/0x1680 [ 65.896442][ T388] lock_acquire.part.0+0xeb/0x330 [ 65.896709][ T388] kernfs_drain+0x384/0x460 [ 65.896983][ T388] __kernfs_remove+0x1f9/0x590 [ 65.897249][ T388] kernfs_remove_by_name_ns+0xa5/0x100 [ 65.897568][ T388] remove_files+0x8c/0x1a0 [ 65.897842][ T388] sysfs_remove_group+0x7b/0x160 [ 65.898106][ T388] sysfs_remove_groups+0x53/0xa0 [ 65.898380][ T388] device_remove_attrs+0x1db/0x260 [ 65.898646][ T388] device_del+0x337/0x8c0 [ 65.898928][ T388] unregister_netdevice_many_notify+0x43d/0x1580 [ 65.899270][ T388] unregister_netdevice_queue+0x2a4/0x410 [ 65.899609][ T388] nsim_destroy+0xe8/0x5f0 [netdevsim] [ 65.899975][ T388] __nsim_dev_port_del+0x17e/0x250 [netdevsim] [ 65.900327][ T388] nsim_dev_reload_destroy+0xe0/0x470 [netdevsim] [ 65.900682][ T388] nsim_drv_remove+0x51/0x1d0 [netdevsim] [ 65.901043][ T388] device_release_driver_internal+0x3bf/0x590 [ 65.901383][ T388] bus_remove_device+0x1f1/0x3f0 [ 65.901654][ T388] device_del+0x33f/0x8c0 [ 65.901917][ T388] device_unregister+0x17/0xb0 [ 65.902185][ T388] del_device_store+0x2f3/0x4f0 [netdevsim] [ 65.902545][ T388] kernfs_fop_write_iter+0x2ba/0x460 [ 65.902878][ T388] vfs_write+0xa81/0x11e0 [ 65.903183][ T388] ksys_write+0xf8/0x1d0 [ 65.903459][ T388] do_syscall_64+0xc1/0x1d0 [ 65.903732][ T388] entry_SYSCALL_64_after_hwframe+0x77/0x7f [ 65.904071][ T388] [ 65.904071][ T388] other info that might help us debug this: [ 65.904071][ T388] [ 65.904571][ T388] Possible unsafe locking scenario: [ 65.904571][ T388] [ 65.904960][ T388] CPU0 CPU1 [ 65.905212][ T388] ---- ---- [ 65.905460][ T388] lock(rtnl_mutex); [ 65.905661][ T388] lock(kn->active#25); [ 65.905994][ T388] lock(rtnl_mutex); [ 65.906333][ T388] lock(kn->active#25); [ 65.906541][ T388] [ 65.906541][ T388] *** DEADLOCK *** [ 65.906541][ T388] [ 65.906944][ T388] 7 locks held by hw_stats_l3.sh/388: [ 65.907217][ T388] #0: ffff888009e893f0 (sb_writers#8){.+.+}-{0:0}, at: ksys_write+0xf8/0x1d0 [ 65.907704][ T388] #1: ffff88800b851a88 (&of->mutex){+.+.}-{4:4}, at: kernfs_fop_write_iter+0x20b/0x460 [ 65.908180][ T388] #2: ffff88800940bb10 (kn->active#35){.+.+}-{0:0}, at: kernfs_fop_write_iter+0x22e/0x460 [ 65.908755][ T388] #3: ffffffffc04b25e8 (nsim_bus_dev_list_lock){+.+.}-{4:4}, at: del_device_store+0x9f/0x4f0 [netdevsim] [ 65.909326][ T388] #4: ffff888005a3e8e8 (&dev->mutex){....}-{4:4}, at: device_release_driver_internal+0xa7/0x590 [ 65.909837][ T388] #5: ffff888005a3d250 (&devlink->lock_key#2){+.+.}-{4:4}, at: nsim_drv_remove+0x49/0x1d0 [netdevsim] [ 65.910372][ T388] #6: ffffffffa59e1208 (rtnl_mutex){+.+.}-{4:4}, at: nsim_destroy+0x6b/0x5f0 [netdevsim] [ 65.910824][ T388] [ 65.910824][ T388] stack backtrace: [ 65.911149][ T388] CPU: 1 UID: 0 PID: 388 Comm: hw_stats_l3.sh Not tainted 6.13.0-rc2-virtme #1 [ 65.911602][ T388] Hardware name: Bochs Bochs, BIOS Bochs 01/01/2011 [ 65.911941][ T388] Call Trace: [ 65.912145][ T388] [ 65.912296][ T388] dump_stack_lvl+0x82/0xd0 [ 65.912575][ T388] print_circular_bug+0x2e2/0x4b0 [ 65.912851][ T388] check_noncircular+0x2f9/0x3e0 [ 65.913118][ T388] ? __pfx_check_noncircular+0x10/0x10 [ 65.913371][ T388] ? __kernel_text_address+0x12/0x40 [ 65.913624][ T388] ? unwind_get_return_address+0x5e/0xa0 [ 65.913891][ T388] ? __pfx_stack_trace_consume_entry+0x10/0x10 [ 65.914219][ T388] ? arch_stack_walk+0xa2/0xf0 [ 65.914500][ T388] check_prev_add+0x182/0xce0 [ 65.914768][ T388] ? __pfx_stack_trace_save+0x10/0x10 [ 65.915038][ T388] validate_chain+0x7bd/0xae0 [ 65.915311][ T388] ? __pfx_validate_chain+0x10/0x10 [ 65.915582][ T388] ? mark_lock+0x335/0x3e0 [ 65.915830][ T388] __lock_acquire+0xb9a/0x1680 [ 65.916090][ T388] ? kasan_addr_to_slab+0x40/0x80 [ 65.916355][ T388] lock_acquire.part.0+0xeb/0x330 [ 65.916620][ T388] ? __kernfs_remove+0x1f9/0x590 [ 65.916890][ T388] ? __pfx___lock_release+0x10/0x10 [ 65.917153][ T388] ? __pfx_lock_acquire.part.0+0x10/0x10 [ 65.917418][ T388] ? trace_lock_acquire+0x14c/0x1f0 [ 65.917680][ T388] ? __virt_addr_valid+0x22b/0x430 [ 65.917959][ T388] ? lock_acquire+0x32/0xc0 [ 65.918225][ T388] ? __kernfs_remove+0x1f9/0x590 [ 65.918496][ T388] kernfs_drain+0x384/0x460 [ 65.918770][ T388] ? __kernfs_remove+0x1f9/0x590 [ 65.919051][ T388] ? __pfx_kernfs_drain+0x10/0x10 [ 65.919301][ T388] ? lock_acquire+0x32/0xc0 [ 65.919566][ T388] __kernfs_remove+0x1f9/0x590 [ 65.919836][ T388] kernfs_remove_by_name_ns+0xa5/0x100 [ 65.920110][ T388] remove_files+0x8c/0x1a0 [ 65.920382][ T388] sysfs_remove_group+0x7b/0x160 [ 65.920652][ T388] sysfs_remove_groups+0x53/0xa0 [ 65.920922][ T388] device_remove_attrs+0x1db/0x260 [ 65.921384][ T388] ? __pfx_device_remove_attrs+0x10/0x10 [ 65.921672][ T388] device_del+0x337/0x8c0 [ 65.921884][ T388] ? __pfx_device_del+0x10/0x10 [ 65.922166][ T388] unregister_netdevice_many_notify+0x43d/0x1580 [ 65.922702][ T388] ? __pfx_unregister_netdevice_many_notify+0x10/0x10 [ 65.923025][ T388] ? lockdep_hardirqs_on_prepare+0x275/0x410 [ 65.923304][ T388] ? __mutex_lock+0x34d/0xbc0 [ 65.923594][ T388] ? nsim_destroy+0x6b/0x5f0 [netdevsim] [ 65.923777][ T388] ? __pfx___mutex_lock+0x10/0x10 [ 65.923946][ T388] unregister_netdevice_queue+0x2a4/0x410 [ 65.924160][ T388] ? __pfx_unregister_netdevice_queue+0x10/0x10 [ 65.924475][ T388] nsim_destroy+0xe8/0x5f0 [netdevsim] [ 65.924651][ T388] __nsim_dev_port_del+0x17e/0x250 [netdevsim] [ 65.924867][ T388] nsim_dev_reload_destroy+0xe0/0x470 [netdevsim] [ 65.925144][ T388] nsim_drv_remove+0x51/0x1d0 [netdevsim] [ 65.925628][ T388] device_release_driver_internal+0x3bf/0x590 [ 65.925955][ T388] ? klist_put+0xb1/0x170 [ 65.926099][ T388] bus_remove_device+0x1f1/0x3f0 [ 65.926268][ T388] device_del+0x33f/0x8c0 [ 65.926503][ T388] ? __pfx_device_del+0x10/0x10 [ 65.926669][ T388] ? lock_acquire.part.0+0xeb/0x330 [ 65.926835][ T388] ? kernfs_fop_write_iter+0x22e/0x460 [ 65.927014][ T388] device_unregister+0x17/0xb0 [ 65.927293][ T388] del_device_store+0x2f3/0x4f0 [netdevsim] [ 65.927514][ T388] ? __pfx_del_device_store+0x10/0x10 [netdevsim] [ 65.927733][ T388] ? __pfx_sysfs_kf_write+0x10/0x10 [ 65.928016][ T388] ? sysfs_file_ops+0x11e/0x170 [ 65.928227][ T388] ? __pfx_sysfs_kf_write+0x10/0x10 [ 65.928392][ T388] kernfs_fop_write_iter+0x2ba/0x460 [ 65.928558][ T388] vfs_write+0xa81/0x11e0 [ 65.928689][ T388] ? __pfx_vfs_write+0x10/0x10 [ 65.928976][ T388] ? __lock_release+0x103/0x460 [ 65.929239][ T388] ? do_user_addr_fault+0x972/0xe30 [ 65.929517][ T388] ? __pfx___up_read+0x10/0x10 [ 65.929792][ T388] ksys_write+0xf8/0x1d0 [ 65.930192][ T388] ? __pfx_ksys_write+0x10/0x10 [ 65.930465][ T388] ? do_user_addr_fault+0x97c/0xe30 [ 65.930737][ T388] do_syscall_64+0xc1/0x1d0 [ 65.931019][ T388] entry_SYSCALL_64_after_hwframe+0x77/0x7f [ 65.931552][ T388] RIP: 0033:0x7f4d16c0eb77 [ 65.931833][ T388] Code: 0b 00 f7 d8 64 89 02 48 c7 c0 ff ff ff ff eb b7 0f 1f 00 f3 0f 1e fa 64 8b 04 25 18 00 00 00 85 c0 75 10 b8 01 00 00 00 0f 05 <48> 3d 00 f0 ff ff 77 51 c3 48 83 ec 28 48 89 54 24 18 48 89 74 24 [ 65.932974][ T388] RSP: 002b:00007ffdb7ed3428 EFLAGS: 00000246 ORIG_RAX: 0000000000000001 [ 65.933398][ T388] RAX: ffffffffffffffda RBX: 0000000000000005 RCX: 00007f4d16c0eb77 [ 65.933905][ T388] RDX: 0000000000000005 RSI: 000055a480ce4c60 RDI: 0000000000000001 [ 65.934249][ T388] RBP: 000055a480ce4c60 R08: 0000000000000000 R09: 00007f4d16c814e0 [ 65.934499][ T388] R10: 00007f4d16c813e0 R11: 0000000000000246 R12: 0000000000000005 [ 65.934868][ T388] R13: 00007f4d16cca760 R14: 0000000000000005 R15: 00007f4d16cc59c0 [ 65.935133][ T388]