[ 31.157999][ T263] ip (263) used greatest stack depth: 24360 bytes left [ 33.399417][ T280] ip (280) used greatest stack depth: 23984 bytes left [ 35.056002][ T291] ip (291) used greatest stack depth: 22848 bytes left [ 37.859810][ C3] [ 37.860015][ C3] ======================================================== [ 37.860355][ C3] WARNING: possible irq lock inversion dependency detected [ 37.860716][ C3] 6.9.0-rc1-virtme #1 Not tainted [ 37.860954][ C3] -------------------------------------------------------- [ 37.861289][ C3] swapper/3/0 just changed the state of lock: [ 37.861564][ C3] ffff888033575580 (&ndev->lock){++-.}-{2:2}, at: addrconf_rs_timer+0xa9/0x730 [ 37.861963][ C3] but this lock took another, SOFTIRQ-unsafe lock in the past: [ 37.862290][ C3] (&p->alloc_lock){+.+.}-{2:2} [ 37.862297][ C3] [ 37.862297][ C3] [ 37.862297][ C3] and interrupts could create inverse lock ordering between them. [ 37.862297][ C3] [ 37.863105][ C3] [ 37.863105][ C3] other info that might help us debug this: [ 37.863435][ C3] Possible interrupt unsafe locking scenario: [ 37.863435][ C3] [ 37.863765][ C3] CPU0 CPU1 [ 37.863981][ C3] ---- ---- [ 37.864204][ C3] lock(&p->alloc_lock); [ 37.864374][ C3] local_irq_disable(); [ 37.864648][ C3] lock(&ndev->lock); [ 37.864928][ C3] lock(&p->alloc_lock); [ 37.865208][ C3] [ 37.865372][ C3] lock(&ndev->lock); [ 37.865543][ C3] [ 37.865543][ C3] *** DEADLOCK *** [ 37.865543][ C3] [ 37.865861][ C3] 1 lock held by swapper/3/0: [ 37.866087][ C3] #0: ffffc90000288b68 ((&ndev->rs_timer)){+.-.}-{0:0}, at: call_timer_fn+0xe8/0x230 [ 37.866500][ C3] [ 37.866500][ C3] the shortest dependencies between 2nd lock and 1st lock: [ 37.866883][ C3] -> (&p->alloc_lock){+.+.}-{2:2} { [ 37.867116][ C3] HARDIRQ-ON-W at: [ 37.867291][ C3] __lock_acquire+0x797/0x1570 [ 37.867575][ C3] lock_acquire.part.0+0xeb/0x330 [ 37.867860][ C3] _raw_spin_lock+0x30/0x40 [ 37.868145][ C3] set_mems_allowed+0x1d/0x210 [ 37.868430][ C3] kernel_init_freeable+0x72/0x310 [ 37.868717][ C3] kernel_init+0x20/0x200 [ 37.869000][ C3] ret_from_fork+0x31/0x70 [ 37.869281][ C3] ret_from_fork_asm+0x1a/0x30 [ 37.869563][ C3] SOFTIRQ-ON-W at: [ 37.869740][ C3] __lock_acquire+0x797/0x1570 [ 37.870030][ C3] lock_acquire.part.0+0xeb/0x330 [ 37.870323][ C3] _raw_spin_lock+0x30/0x40 [ 37.870607][ C3] set_mems_allowed+0x1d/0x210 [ 37.870890][ C3] kernel_init_freeable+0x72/0x310 [ 37.871166][ C3] kernel_init+0x20/0x200 [ 37.871450][ C3] ret_from_fork+0x31/0x70 [ 37.871736][ C3] ret_from_fork_asm+0x1a/0x30 [ 37.872014][ C3] INITIAL USE at: [ 37.872182][ C3] __lock_acquire+0x797/0x1570 [ 37.872462][ C3] lock_acquire.part.0+0xeb/0x330 [ 37.872742][ C3] _raw_spin_lock+0x30/0x40 [ 37.873021][ C3] set_mems_allowed+0x1d/0x210 [ 37.873309][ C3] kernel_init_freeable+0x72/0x310 [ 37.873591][ C3] kernel_init+0x20/0x200 [ 37.873868][ C3] ret_from_fork+0x31/0x70 [ 37.874150][ C3] ret_from_fork_asm+0x1a/0x30 [ 37.874426][ C3] } [ 37.874538][ C3] ... key at: [] __key.387+0x0/0x40 [ 37.874880][ C3] ... acquired at: [ 37.875047][ C3] __lock_acquire+0xaf0/0x1570 [ 37.875271][ C3] lock_acquire.part.0+0xeb/0x330 [ 37.875493][ C3] _raw_spin_lock+0x30/0x40 [ 37.875715][ C3] __get_task_comm+0x27/0x70 [ 37.875944][ C3] ref_tracker_alloc+0x2ee/0x490 [ 37.876171][ C3] netdev_get_by_index+0x5e/0x80 [ 37.876398][ C3] fib6_nh_init+0x3d8/0x15b0 [ 37.876625][ C3] ip6_route_info_create+0xf18/0x17d0 [ 37.876851][ C3] ip6_route_add+0x1c/0x140 [ 37.877075][ C3] addrconf_prefix_route+0x1cc/0x2b0 [ 37.877298][ C3] addrconf_permanent_addr+0x32f/0x670 [ 37.877527][ C3] addrconf_notify+0x749/0xd60 [ 37.877754][ C3] notifier_call_chain+0xcd/0x150 [ 37.877976][ C3] __dev_notify_flags+0xe6/0x250 [ 37.878197][ C3] dev_change_flags+0xec/0x160 [ 37.878418][ C3] do_setlink+0x7d6/0x21e0 [ 37.878634][ C3] __rtnl_newlink+0xaa7/0xd80 [ 37.878852][ C3] rtnl_newlink+0x63/0xa0 [ 37.879070][ C3] rtnetlink_rcv_msg+0x2fb/0xbf0 [ 37.879288][ C3] netlink_rcv_skb+0x130/0x360 [ 37.879506][ C3] netlink_unicast+0x449/0x710 [ 37.879719][ C3] netlink_sendmsg+0x723/0xbe0 [ 37.879929][ C3] ____sys_sendmsg+0x7b2/0xa10 [ 37.880140][ C3] ___sys_sendmsg+0xee/0x170 [ 37.880372][ C3] __sys_sendmsg+0xcd/0x170 [ 37.880580][ C3] do_syscall_64+0xc6/0x1e0 [ 37.880796][ C3] entry_SYSCALL_64_after_hwframe+0x6d/0x75 [ 37.881065][ C3] [ 37.881181][ C3] -> (&ndev->lock){++-.}-{2:2} { [ 37.881403][ C3] HARDIRQ-ON-W at: [ 37.881562][ C3] __lock_acquire+0x797/0x1570 [ 37.881831][ C3] lock_acquire.part.0+0xeb/0x330 [ 37.882109][ C3] _raw_write_lock_bh+0x38/0x50 [ 37.882394][ C3] addrconf_permanent_addr+0x6e/0x670 [ 37.882661][ C3] addrconf_notify+0x749/0xd60 [ 37.882924][ C3] notifier_call_chain+0xcd/0x150 [ 37.883192][ C3] __dev_notify_flags+0xe6/0x250 [ 37.883457][ C3] dev_change_flags+0xec/0x160 [ 37.883725][ C3] do_setlink+0x7d6/0x21e0 [ 37.883996][ C3] __rtnl_newlink+0xaa7/0xd80 [ 37.884261][ C3] rtnl_newlink+0x63/0xa0 [ 37.884525][ C3] rtnetlink_rcv_msg+0x2fb/0xbf0 [ 37.884794][ C3] netlink_rcv_skb+0x130/0x360 [ 37.885068][ C3] netlink_unicast+0x449/0x710 [ 37.885337][ C3] netlink_sendmsg+0x723/0xbe0 [ 37.885611][ C3] ____sys_sendmsg+0x7b2/0xa10 [ 37.885880][ C3] ___sys_sendmsg+0xee/0x170 [ 37.886146][ C3] __sys_sendmsg+0xcd/0x170 [ 37.886415][ C3] do_syscall_64+0xc6/0x1e0 [ 37.886687][ C3] entry_SYSCALL_64_after_hwframe+0x6d/0x75 [ 37.887014][ C3] HARDIRQ-ON-R at: [ 37.887182][ C3] __lock_acquire+0x797/0x1570 [ 37.887452][ C3] lock_acquire.part.0+0xeb/0x330 [ 37.887722][ C3] _raw_read_lock_bh+0x44/0x80 [ 37.888034][ C3] inet6_fill_ifla6_attrs+0x516/0x740 [ 37.888320][ C3] inet6_fill_link_af+0x5e/0xe0 [ 37.888607][ C3] rtnl_fill_ifinfo+0x1612/0x29e0 [ 37.888890][ C3] rtmsg_ifinfo_build_skb+0x13c/0x240 [ 37.889176][ C3] rtmsg_ifinfo_event.part.0+0x2d/0x120 [ 37.889518][ C3] rtmsg_ifinfo+0x5b/0xa0 [ 37.889807][ C3] register_netdevice+0x1177/0x1690 [ 37.890093][ C3] register_netdev+0x20/0x40 [ 37.890395][ C3] sit_init_net+0x250/0x4f0 [ 37.890695][ C3] ops_init+0x9b/0x560 [ 37.890910][ C3] register_pernet_operations+0x2db/0x710 [ 37.891244][ C3] register_pernet_device+0x2a/0x60 [ 37.891510][ C3] sit_init+0x23/0xe0 [ 37.891729][ C3] do_one_initcall+0x8d/0x1e0 [ 37.892031][ C3] do_initcalls+0x1b2/0x3e0 [ 37.892321][ C3] kernel_init_freeable+0x232/0x310 [ 37.892608][ C3] kernel_init+0x20/0x200 [ 37.892896][ C3] ret_from_fork+0x31/0x70 [ 37.893184][ C3] ret_from_fork_asm+0x1a/0x30 [ 37.893469][ C3] IN-SOFTIRQ-W at: [ 37.893641][ C3] __lock_acquire+0x797/0x1570 [ 37.893928][ C3] lock_acquire.part.0+0xeb/0x330 [ 37.894217][ C3] _raw_write_lock+0x30/0x40 [ 37.894509][ C3] addrconf_rs_timer+0xa9/0x730 [ 37.894804][ C3] call_timer_fn+0x13b/0x230 [ 37.895091][ C3] __run_timers+0x545/0x810 [ 37.895378][ C3] timer_expire_remote+0x9b/0xe0 [ 37.895672][ C3] tmigr_handle_remote_cpu+0x270/0x430 [ 37.895963][ C3] tmigr_handle_remote_up+0x19f/0x2e0 [ 37.896248][ C3] tmigr_handle_remote+0x21d/0x3a0 [ 37.896553][ C3] __do_softirq+0x1f8/0x5df [ 37.896832][ C3] irq_exit_rcu+0x97/0xc0 [ 37.897135][ C3] sysvec_apic_timer_interrupt+0x75/0x80 [ 37.897486][ C3] asm_sysvec_apic_timer_interrupt+0x1a/0x20 [ 37.897831][ C3] default_idle+0xf/0x20 [ 37.898123][ C3] default_idle_call+0x6d/0xb0 [ 37.898412][ C3] cpuidle_idle_call+0x1f4/0x280 [ 37.898696][ C3] do_idle+0xf9/0x160 [ 37.898920][ C3] cpu_startup_entry+0x54/0x60 [ 37.899209][ C3] start_secondary+0x21c/0x2b0 [ 37.899502][ C3] common_startup_64+0x12c/0x138 [ 37.899787][ C3] INITIAL USE at: [ 37.899959][ C3] __lock_acquire+0x797/0x1570 [ 37.900249][ C3] lock_acquire.part.0+0xeb/0x330 [ 37.900548][ C3] _raw_write_lock_bh+0x38/0x50 [ 37.900840][ C3] addrconf_permanent_addr+0x6e/0x670 [ 37.901126][ C3] addrconf_notify+0x749/0xd60 [ 37.901411][ C3] notifier_call_chain+0xcd/0x150 [ 37.901694][ C3] __dev_notify_flags+0xe6/0x250 [ 37.901991][ C3] dev_change_flags+0xec/0x160 [ 37.902277][ C3] do_setlink+0x7d6/0x21e0 [ 37.902565][ C3] __rtnl_newlink+0xaa7/0xd80 [ 37.902851][ C3] rtnl_newlink+0x63/0xa0 [ 37.903139][ C3] rtnetlink_rcv_msg+0x2fb/0xbf0 [ 37.903423][ C3] netlink_rcv_skb+0x130/0x360 [ 37.903708][ C3] netlink_unicast+0x449/0x710 [ 37.903994][ C3] netlink_sendmsg+0x723/0xbe0 [ 37.904275][ C3] ____sys_sendmsg+0x7b2/0xa10 [ 37.904557][ C3] ___sys_sendmsg+0xee/0x170 [ 37.904857][ C3] __sys_sendmsg+0xcd/0x170 [ 37.905125][ C3] do_syscall_64+0xc6/0x1e0 [ 37.905405][ C3] entry_SYSCALL_64_after_hwframe+0x6d/0x75 [ 37.905733][ C3] INITIAL READ USE at: [ 37.905900][ C3] __lock_acquire+0x797/0x1570 [ 37.906167][ C3] lock_acquire.part.0+0xeb/0x330 [ 37.906569][ C3] _raw_read_lock_bh+0x44/0x80 [ 37.906827][ C3] inet6_fill_ifla6_attrs+0x516/0x740 [ 37.907145][ C3] inet6_fill_link_af+0x5e/0xe0 [ 37.907551][ C3] rtnl_fill_ifinfo+0x1612/0x29e0 [ 37.907815][ C3] rtmsg_ifinfo_build_skb+0x13c/0x240 [ 37.908129][ C3] rtmsg_ifinfo_event.part.0+0x2d/0x120 [ 37.908578][ C3] rtmsg_ifinfo+0x5b/0xa0 [ 37.908842][ C3] register_netdevice+0x1177/0x1690 [ 37.909166][ C3] register_netdev+0x20/0x40 [ 37.909572][ C3] sit_init_net+0x250/0x4f0 [ 37.909834][ C3] ops_init+0x9b/0x560 [ 37.910101][ C3] register_pernet_operations+0x2db/0x710 [ 37.910566][ C3] register_pernet_device+0x2a/0x60 [ 37.910881][ C3] sit_init+0x23/0xe0 [ 37.911154][ C3] do_one_initcall+0x8d/0x1e0 [ 37.911562][ C3] do_initcalls+0x1b2/0x3e0 [ 37.911831][ C3] kernel_init_freeable+0x232/0x310 [ 37.912149][ C3] kernel_init+0x20/0x200 [ 37.912411][ C3] ret_from_fork+0x31/0x70 [ 37.912815][ C3] ret_from_fork_asm+0x1a/0x30 [ 37.913081][ C3] } [ 37.913190][ C3] ... key at: [] __key.36+0x0/0x40 [ 37.913653][ C3] ... acquired at: [ 37.913811][ C3] mark_lock+0x28d/0x3e0 [ 37.914021][ C3] mark_usage+0xd9/0x2a0 [ 37.914237][ C3] __lock_acquire+0x797/0x1570 [ 37.914450][ C3] lock_acquire.part.0+0xeb/0x330 [ 37.914805][ C3] _raw_write_lock+0x30/0x40 [ 37.915016][ C3] addrconf_rs_timer+0xa9/0x730 [ 37.915224][ C3] call_timer_fn+0x13b/0x230 [ 37.915441][ C3] __run_timers+0x545/0x810 [ 37.915790][ C3] timer_expire_remote+0x9b/0xe0 [ 37.916003][ C3] tmigr_handle_remote_cpu+0x270/0x430 [ 37.916212][ C3] tmigr_handle_remote_up+0x19f/0x2e0 [ 37.916422][ C3] tmigr_handle_remote+0x21d/0x3a0 [ 37.916769][ C3] __do_softirq+0x1f8/0x5df [ 37.916983][ C3] irq_exit_rcu+0x97/0xc0 [ 37.917194][ C3] sysvec_apic_timer_interrupt+0x75/0x80 [ 37.917463][ C3] asm_sysvec_apic_timer_interrupt+0x1a/0x20 [ 37.917863][ C3] default_idle+0xf/0x20 [ 37.918080][ C3] default_idle_call+0x6d/0xb0 [ 37.918295][ C3] cpuidle_idle_call+0x1f4/0x280 [ 37.918506][ C3] do_idle+0xf9/0x160 [ 37.918667][ C3] cpu_startup_entry+0x54/0x60 [ 37.919021][ C3] start_secondary+0x21c/0x2b0 [ 37.919248][ C3] common_startup_64+0x12c/0x138 [ 37.919477][ C3] [ 37.919591][ C3] [ 37.919591][ C3] stack backtrace: [ 37.920112][ C3] CPU: 3 PID: 0 Comm: swapper/3 Not tainted 6.9.0-rc1-virtme #1 [ 37.920439][ C3] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS rel-1.16.3-0-ga6ed6b701f0a-prebuilt.qemu.org 04/01/2014 [ 37.920903][ C3] Call Trace: [ 37.921212][ C3] [ 37.921328][ C3] dump_stack_lvl+0x82/0xd0 [ 37.921546][ C3] print_irq_inversion_bug.part.0+0x3d9/0x570 [ 37.921818][ C3] ? kernel_text_address+0x17/0xe0 [ 37.922161][ C3] ? __pfx_print_irq_inversion_bug.part.0+0x10/0x10 [ 37.922427][ C3] ? __pfx_usage_skip+0x10/0x10 [ 37.922640][ C3] ? __pfx_usage_match+0x10/0x10 [ 37.922876][ C3] ? arch_stack_walk+0x8c/0xf0 [ 37.923236][ C3] mark_lock_irq+0x4cd/0xa10 [ 37.923448][ C3] ? common_startup_64+0x12c/0x138 [ 37.923663][ C3] ? __pfx_mark_lock_irq+0x10/0x10 [ 37.923878][ C3] ? __pfx_stack_trace_save+0x10/0x10 [ 37.924094][ C3] ? save_trace+0x8f/0x5b0 [ 37.924448][ C3] mark_lock+0x28d/0x3e0 [ 37.924607][ C3] ? __pfx_stack_trace_save+0x10/0x10 [ 37.924819][ C3] mark_usage+0xd9/0x2a0 [ 37.924977][ C3] __lock_acquire+0x797/0x1570 [ 37.925322][ C3] ? __pfx_validate_chain+0x10/0x10 [ 37.925537][ C3] ? __pfx_addrconf_rs_timer+0x10/0x10 [ 37.925747][ C3] lock_acquire.part.0+0xeb/0x330 [ 37.925960][ C3] ? addrconf_rs_timer+0xa9/0x730 [ 37.926177][ C3] ? __pfx_lock_acquire.part.0+0x10/0x10 [ 37.926532][ C3] ? trace_lock_acquire+0x135/0x1c0 [ 37.926751][ C3] ? addrconf_rs_timer+0xa9/0x730 [ 37.926959][ C3] ? lock_acquire+0x32/0xc0 [ 37.927174][ C3] ? addrconf_rs_timer+0xa9/0x730 [ 37.927534][ C3] _raw_write_lock+0x30/0x40 [ 37.927750][ C3] ? addrconf_rs_timer+0xa9/0x730 [ 37.927959][ C3] addrconf_rs_timer+0xa9/0x730 [ 37.928171][ C3] ? __pfx_lock_acquire.part.0+0x10/0x10 [ 37.928517][ C3] ? __pfx_addrconf_rs_timer+0x10/0x10 [ 37.928738][ C3] ? call_timer_fn+0xe8/0x230 [ 37.928958][ C3] ? lock_acquire+0x32/0xc0 [ 37.929170][ C3] ? call_timer_fn+0xe8/0x230 [ 37.929525][ C3] ? __pfx_addrconf_rs_timer+0x10/0x10 [ 37.929743][ C3] call_timer_fn+0x13b/0x230 [ 37.929961][ C3] ? call_timer_fn+0xe8/0x230 [ 37.930178][ C3] ? call_timer_fn+0xe8/0x230 [ 37.930404][ C3] ? __pfx_call_timer_fn+0x10/0x10 [ 37.930760][ C3] ? mark_lock+0x38/0x3e0 [ 37.930938][ C3] __run_timers+0x545/0x810 [ 37.931155][ C3] ? __pfx_addrconf_rs_timer+0x10/0x10 [ 37.931377][ C3] ? __pfx___run_timers+0x10/0x10 [ 37.931727][ C3] ? __lock_release+0x103/0x460 [ 37.931949][ C3] ? do_raw_spin_lock+0x131/0x270 [ 37.932164][ C3] ? __pfx_do_raw_spin_lock+0x10/0x10 [ 37.932385][ C3] ? lock_acquire+0x32/0xc0 [ 37.932742][ C3] ? timer_expire_remote+0x93/0xe0 [ 37.932998][ C3] timer_expire_remote+0x9b/0xe0 [ 37.933215][ C3] tmigr_handle_remote_cpu+0x270/0x430 [ 37.933436][ C3] ? __pfx_tmigr_handle_remote_cpu+0x10/0x10 [ 37.933868][ C3] tmigr_handle_remote_up+0x19f/0x2e0 [ 37.934129][ C3] tmigr_handle_remote+0x21d/0x3a0 [ 37.934349][ C3] ? __pfx_tmigr_handle_remote+0x10/0x10 [ 37.934565][ C3] __do_softirq+0x1f8/0x5df [ 37.934790][ C3] irq_exit_rcu+0x97/0xc0 [ 37.935085][ C3] sysvec_apic_timer_interrupt+0x75/0x80 [ 37.935315][ C3] [ 37.935433][ C3] [ 37.935546][ C3] asm_sysvec_apic_timer_interrupt+0x1a/0x20 [ 37.935817][ C3] RIP: 0010:default_idle+0xf/0x20 [ 37.936179][ C3] Code: 4c 01 c7 4c 29 c2 e9 72 ff ff ff 90 90 90 90 90 90 90 90 90 90 90 90 90 90 90 90 f3 0f 1e fa 66 90 0f 00 2d c3 c1 2b 00 fb f4 c3 cc cc cc cc 66 66 2e 0f 1f 84 00 00 00 00 00 90 90 90 90 90 [ 37.936957][ C3] RSP: 0018:ffffc9000016fdf8 EFLAGS: 00000242 [ 37.937506][ C3] RAX: 000000000007d701 RBX: 1ffff9200002dfc1 RCX: ffffffffb8da58b5 [ 37.937827][ C3] RDX: 0000000000000000 RSI: 0000000000000000 RDI: ffffffffb66aa4a4 [ 37.938154][ C3] RBP: 0000000000000000 R08: 0000000000000001 R09: ffffed1006c3709c [ 37.938613][ C3] R10: ffff8880361b84e3 R11: ffff8880361ad9b0 R12: 0000000000000000 [ 37.938930][ C3] R13: ffff888001b90040 R14: dffffc0000000000 R15: 0000000000000000 [ 37.939400][ C3] ? ct_kernel_exit.constprop.0+0xc5/0xf0 [ 37.939623][ C3] ? cpuidle_idle_call+0x1f4/0x280 [ 37.939836][ C3] default_idle_call+0x6d/0xb0 [ 37.940050][ C3] cpuidle_idle_call+0x1f4/0x280 [ 37.940282][ C3] ? __pfx_cpuidle_idle_call+0x10/0x10 [ 37.940639][ C3] ? tsc_verify_tsc_adjust+0x5e/0x2b0 [ 37.940857][ C3] do_idle+0xf9/0x160 [ 37.941015][ C3] cpu_startup_entry+0x54/0x60 [ 37.941248][ C3] start_secondary+0x21c/0x2b0 [ 37.941613][ C3] ? __pfx_start_secondary+0x10/0x10 [ 37.941835][ C3] common_startup_64+0x12c/0x138 [ 37.942054][ C3] [ 256.022289][ T2134] netem: version 1.3