[ 59.973888][ T489] br0: port 1(veth1) entered blocking state [ 59.974505][ T489] br0: port 1(veth1) entered disabled state [ 59.975107][ T489] veth1: entered allmulticast mode [ 59.976997][ T489] veth1: entered promiscuous mode [ 60.084667][ T491] br0: port 2(veth3) entered blocking state [ 60.084936][ T491] br0: port 2(veth3) entered disabled state [ 60.085206][ T491] veth3: entered allmulticast mode [ 60.087160][ T491] veth3: entered promiscuous mode [ 60.461815][ T7] br0: port 1(veth1) entered blocking state [ 60.462197][ T7] br0: port 1(veth1) entered forwarding state [ 60.748744][ T7] br0: port 2(veth3) entered blocking state [ 60.748991][ T7] br0: port 2(veth3) entered forwarding state [ 65.583275][ C2] [ 65.583447][ C2] ======================================================== [ 65.583681][ C2] WARNING: possible irq lock inversion dependency detected [ 65.583892][ C2] 6.9.0-rc1-virtme #1 Not tainted [ 65.584042][ C2] -------------------------------------------------------- [ 65.584247][ C2] swapper/2/0 just changed the state of lock: [ 65.584426][ C2] ffff8880065e6d80 (&ndev->lock){++-.}-{2:2}, at: addrconf_rs_timer+0xa9/0x730 [ 65.584676][ C2] but this lock took another, SOFTIRQ-unsafe lock in the past: [ 65.584883][ C2] (&p->alloc_lock){+.+.}-{2:2} [ 65.584887][ C2] [ 65.584887][ C2] [ 65.584887][ C2] and interrupts could create inverse lock ordering between them. [ 65.584887][ C2] [ 65.585417][ C2] [ 65.585417][ C2] other info that might help us debug this: [ 65.585624][ C2] Possible interrupt unsafe locking scenario: [ 65.585624][ C2] [ 65.585846][ C2] CPU0 CPU1 [ 65.585989][ C2] ---- ---- [ 65.586134][ C2] lock(&p->alloc_lock); [ 65.586248][ C2] local_irq_disable(); [ 65.586436][ C2] lock(&ndev->lock); [ 65.586624][ C2] lock(&p->alloc_lock); [ 65.586802][ C2] [ 65.586910][ C2] lock(&ndev->lock); [ 65.587020][ C2] [ 65.587020][ C2] *** DEADLOCK *** [ 65.587020][ C2] [ 65.587242][ C2] 1 lock held by swapper/2/0: [ 65.587395][ C2] #0: ffffc90000230d68 ((&ndev->rs_timer)){+.-.}-{0:0}, at: call_timer_fn+0xe8/0x230 [ 65.587652][ C2] [ 65.587652][ C2] the shortest dependencies between 2nd lock and 1st lock: [ 65.587905][ C2] -> (&p->alloc_lock){+.+.}-{2:2} { [ 65.588052][ C2] HARDIRQ-ON-W at: [ 65.588161][ C2] __lock_acquire+0x797/0x1570 [ 65.588343][ C2] lock_acquire.part.0+0xeb/0x330 [ 65.588524][ C2] _raw_spin_lock+0x30/0x40 [ 65.588709][ C2] set_mems_allowed+0x1d/0x210 [ 65.588891][ C2] kernel_init_freeable+0x72/0x310 [ 65.589074][ C2] kernel_init+0x20/0x200 [ 65.589260][ C2] ret_from_fork+0x31/0x70 [ 65.589481][ C2] ret_from_fork_asm+0x1a/0x30 [ 65.589647][ C2] SOFTIRQ-ON-W at: [ 65.589749][ C2] __lock_acquire+0x797/0x1570 [ 65.589919][ C2] lock_acquire.part.0+0xeb/0x330 [ 65.590085][ C2] _raw_spin_lock+0x30/0x40 [ 65.590251][ C2] set_mems_allowed+0x1d/0x210 [ 65.590415][ C2] kernel_init_freeable+0x72/0x310 [ 65.590579][ C2] kernel_init+0x20/0x200 [ 65.590744][ C2] ret_from_fork+0x31/0x70 [ 65.590916][ C2] ret_from_fork_asm+0x1a/0x30 [ 65.591085][ C2] INITIAL USE at: [ 65.591191][ C2] __lock_acquire+0x797/0x1570 [ 65.591354][ C2] lock_acquire.part.0+0xeb/0x330 [ 65.591525][ C2] _raw_spin_lock+0x30/0x40 [ 65.591690][ C2] set_mems_allowed+0x1d/0x210 [ 65.591853][ C2] kernel_init_freeable+0x72/0x310 [ 65.592033][ C2] kernel_init+0x20/0x200 [ 65.592199][ C2] ret_from_fork+0x31/0x70 [ 65.592368][ C2] ret_from_fork_asm+0x1a/0x30 [ 65.592533][ C2] } [ 65.592601][ C2] ... key at: [] __key.387+0x0/0x40 [ 65.592798][ C2] ... acquired at: [ 65.592896][ C2] __lock_acquire+0xaf0/0x1570 [ 65.593035][ C2] lock_acquire.part.0+0xeb/0x330 [ 65.593170][ C2] _raw_spin_lock+0x30/0x40 [ 65.593304][ C2] __get_task_comm+0x27/0x70 [ 65.593445][ C2] ref_tracker_alloc+0x2ee/0x490 [ 65.593579][ C2] netdev_get_by_index+0x5e/0x80 [ 65.593714][ C2] fib6_nh_init+0x3d8/0x15b0 [ 65.593850][ C2] ip6_route_info_create+0xf18/0x17d0 [ 65.593982][ C2] addrconf_f6i_alloc+0x265/0x560 [ 65.594127][ C2] __ipv6_dev_ac_inc+0x25f/0xb10 [ 65.594264][ C2] addrconf_join_anycast+0x1c0/0x220 [ 65.594403][ C2] __ipv6_ifa_notify+0x6b3/0xaf0 [ 65.594535][ C2] add_addr+0x1fb/0x2d0 [ 65.594671][ C2] add_v4_addrs+0x400/0x800 [ 65.594810][ C2] addrconf_gre_config+0x69/0x140 [ 65.594952][ C2] addrconf_notify+0xb0e/0xd60 [ 65.595098][ C2] notifier_call_chain+0xcd/0x150 [ 65.595240][ C2] __dev_notify_flags+0xe6/0x250 [ 65.595393][ C2] dev_change_flags+0xec/0x160 [ 65.595536][ C2] do_setlink+0x7d6/0x21e0 [ 65.595677][ C2] __rtnl_newlink+0xaa7/0xd80 [ 65.595821][ C2] rtnl_newlink+0x63/0xa0 [ 65.595961][ C2] rtnetlink_rcv_msg+0x2fb/0xbf0 [ 65.596106][ C2] netlink_rcv_skb+0x130/0x360 [ 65.596256][ C2] netlink_unicast+0x449/0x710 [ 65.596402][ C2] netlink_sendmsg+0x723/0xbe0 [ 65.596545][ C2] ____sys_sendmsg+0x7b2/0xa10 [ 65.596687][ C2] ___sys_sendmsg+0xee/0x170 [ 65.596832][ C2] __sys_sendmsg+0xcd/0x170 [ 65.596973][ C2] do_syscall_64+0xc6/0x1e0 [ 65.597121][ C2] entry_SYSCALL_64_after_hwframe+0x6d/0x75 [ 65.597300][ C2] [ 65.597380][ C2] -> (&ndev->lock){++-.}-{2:2} { [ 65.597528][ C2] HARDIRQ-ON-W at: [ 65.597639][ C2] __lock_acquire+0x797/0x1570 [ 65.597820][ C2] lock_acquire.part.0+0xeb/0x330 [ 65.598006][ C2] _raw_write_lock_bh+0x38/0x50 [ 65.598186][ C2] addrconf_permanent_addr+0x6e/0x670 [ 65.598363][ C2] addrconf_notify+0x749/0xd60 [ 65.598541][ C2] notifier_call_chain+0xcd/0x150 [ 65.598728][ C2] __dev_notify_flags+0xe6/0x250 [ 65.598906][ C2] dev_change_flags+0xec/0x160 [ 65.599084][ C2] do_setlink+0x7d6/0x21e0 [ 65.599266][ C2] __rtnl_newlink+0xaa7/0xd80 [ 65.599465][ C2] rtnl_newlink+0x63/0xa0 [ 65.599643][ C2] rtnetlink_rcv_msg+0x2fb/0xbf0 [ 65.599827][ C2] netlink_rcv_skb+0x130/0x360 [ 65.600010][ C2] netlink_unicast+0x449/0x710 [ 65.600196][ C2] netlink_sendmsg+0x723/0xbe0 [ 65.600373][ C2] ____sys_sendmsg+0x7b2/0xa10 [ 65.600557][ C2] ___sys_sendmsg+0xee/0x170 [ 65.600738][ C2] __sys_sendmsg+0xcd/0x170 [ 65.600916][ C2] do_syscall_64+0xc6/0x1e0 [ 65.601095][ C2] entry_SYSCALL_64_after_hwframe+0x6d/0x75 [ 65.601307][ C2] HARDIRQ-ON-R at: [ 65.601413][ C2] __lock_acquire+0x797/0x1570 [ 65.601589][ C2] lock_acquire.part.0+0xeb/0x330 [ 65.601774][ C2] _raw_read_lock_bh+0x44/0x80 [ 65.601955][ C2] inet6_fill_ifla6_attrs+0x516/0x740 [ 65.602134][ C2] inet6_fill_link_af+0x5e/0xe0 [ 65.602316][ C2] rtnl_fill_ifinfo+0x1612/0x29e0 [ 65.602501][ C2] rtmsg_ifinfo_build_skb+0x13c/0x240 [ 65.602683][ C2] rtmsg_ifinfo_event.part.0+0x2d/0x120 [ 65.602900][ C2] rtmsg_ifinfo+0x5b/0xa0 [ 65.603081][ C2] register_netdevice+0x1177/0x1690 [ 65.603258][ C2] register_netdev+0x20/0x40 [ 65.603440][ C2] sit_init_net+0x250/0x4f0 [ 65.603624][ C2] ops_init+0x9b/0x560 [ 65.603767][ C2] register_pernet_operations+0x2db/0x710 [ 65.603974][ C2] register_pernet_device+0x2a/0x60 [ 65.604156][ C2] sit_init+0x23/0xe0 [ 65.604297][ C2] do_one_initcall+0x8d/0x1e0 [ 65.604482][ C2] do_initcalls+0x1b2/0x3e0 [ 65.604660][ C2] kernel_init_freeable+0x232/0x310 [ 65.604837][ C2] kernel_init+0x20/0x200 [ 65.605011][ C2] ret_from_fork+0x31/0x70 [ 65.605190][ C2] ret_from_fork_asm+0x1a/0x30 [ 65.605369][ C2] IN-SOFTIRQ-W at: [ 65.605476][ C2] __lock_acquire+0x797/0x1570 [ 65.605662][ C2] lock_acquire.part.0+0xeb/0x330 [ 65.605836][ C2] _raw_write_lock+0x30/0x40 [ 65.606015][ C2] addrconf_rs_timer+0xa9/0x730 [ 65.606197][ C2] call_timer_fn+0x13b/0x230 [ 65.606373][ C2] __run_timers+0x545/0x810 [ 65.606555][ C2] run_timer_softirq+0xe8/0x1b0 [ 65.606731][ C2] __do_softirq+0x1f8/0x5df [ 65.606909][ C2] irq_exit_rcu+0x97/0xc0 [ 65.607090][ C2] sysvec_apic_timer_interrupt+0x75/0x80 [ 65.607300][ C2] asm_sysvec_apic_timer_interrupt+0x1a/0x20 [ 65.607515][ C2] default_idle+0xf/0x20 [ 65.607689][ C2] default_idle_call+0x6d/0xb0 [ 65.607870][ C2] cpuidle_idle_call+0x1f4/0x280 [ 65.608061][ C2] do_idle+0xf9/0x160 [ 65.608204][ C2] cpu_startup_entry+0x54/0x60 [ 65.608384][ C2] start_secondary+0x21c/0x2b0 [ 65.608562][ C2] common_startup_64+0x12c/0x138 [ 65.608737][ C2] INITIAL USE at: [ 65.608842][ C2] __lock_acquire+0x797/0x1570 [ 65.609019][ C2] lock_acquire.part.0+0xeb/0x330 [ 65.609196][ C2] _raw_write_lock_bh+0x38/0x50 [ 65.609373][ C2] addrconf_permanent_addr+0x6e/0x670 [ 65.609574][ C2] addrconf_notify+0x749/0xd60 [ 65.609752][ C2] notifier_call_chain+0xcd/0x150 [ 65.609930][ C2] __dev_notify_flags+0xe6/0x250 [ 65.610111][ C2] dev_change_flags+0xec/0x160 [ 65.610293][ C2] do_setlink+0x7d6/0x21e0 [ 65.610477][ C2] __rtnl_newlink+0xaa7/0xd80 [ 65.610656][ C2] rtnl_newlink+0x63/0xa0 [ 65.610835][ C2] rtnetlink_rcv_msg+0x2fb/0xbf0 [ 65.611011][ C2] netlink_rcv_skb+0x130/0x360 [ 65.611190][ C2] netlink_unicast+0x449/0x710 [ 65.611366][ C2] netlink_sendmsg+0x723/0xbe0 [ 65.611547][ C2] ____sys_sendmsg+0x7b2/0xa10 [ 65.611729][ C2] ___sys_sendmsg+0xee/0x170 [ 65.611905][ C2] __sys_sendmsg+0xcd/0x170 [ 65.612077][ C2] do_syscall_64+0xc6/0x1e0 [ 65.612249][ C2] entry_SYSCALL_64_after_hwframe+0x6d/0x75 [ 65.612443][ C2] INITIAL READ USE at: [ 65.612545][ C2] __lock_acquire+0x797/0x1570 [ 65.612792][ C2] lock_acquire.part.0+0xeb/0x330 [ 65.612965][ C2] _raw_read_lock_bh+0x44/0x80 [ 65.613131][ C2] inet6_fill_ifla6_attrs+0x516/0x740 [ 65.613407][ C2] inet6_fill_link_af+0x5e/0xe0 [ 65.613575][ C2] rtnl_fill_ifinfo+0x1612/0x29e0 [ 65.613741][ C2] rtmsg_ifinfo_build_skb+0x13c/0x240 [ 65.614012][ C2] rtmsg_ifinfo_event.part.0+0x2d/0x120 [ 65.614217][ C2] rtmsg_ifinfo+0x5b/0xa0 [ 65.614379][ C2] register_netdevice+0x1177/0x1690 [ 65.614654][ C2] register_netdev+0x20/0x40 [ 65.614821][ C2] sit_init_net+0x250/0x4f0 [ 65.614989][ C2] ops_init+0x9b/0x560 [ 65.615156][ C2] register_pernet_operations+0x2db/0x710 [ 65.615426][ C2] register_pernet_device+0x2a/0x60 [ 65.615622][ C2] sit_init+0x23/0xe0 [ 65.615784][ C2] do_one_initcall+0x8d/0x1e0 [ 65.616028][ C2] do_initcalls+0x1b2/0x3e0 [ 65.616209][ C2] kernel_init_freeable+0x232/0x310 [ 65.616409][ C2] kernel_init+0x20/0x200 [ 65.616654][ C2] ret_from_fork+0x31/0x70 [ 65.616818][ C2] ret_from_fork_asm+0x1a/0x30 [ 65.616980][ C2] } [ 65.617046][ C2] ... key at: [] __key.37+0x0/0x40 [ 65.617392][ C2] ... acquired at: [ 65.617493][ C2] mark_lock+0x28d/0x3e0 [ 65.617622][ C2] mark_usage+0xd9/0x2a0 [ 65.617752][ C2] __lock_acquire+0x797/0x1570 [ 65.617964][ C2] lock_acquire.part.0+0xeb/0x330 [ 65.618103][ C2] _raw_write_lock+0x30/0x40 [ 65.618232][ C2] addrconf_rs_timer+0xa9/0x730 [ 65.618373][ C2] call_timer_fn+0x13b/0x230 [ 65.618577][ C2] __run_timers+0x545/0x810 [ 65.618709][ C2] run_timer_softirq+0xe8/0x1b0 [ 65.618842][ C2] __do_softirq+0x1f8/0x5df [ 65.618979][ C2] irq_exit_rcu+0x97/0xc0 [ 65.619114][ C2] sysvec_apic_timer_interrupt+0x75/0x80 [ 65.619360][ C2] asm_sysvec_apic_timer_interrupt+0x1a/0x20 [ 65.619543][ C2] default_idle+0xf/0x20 [ 65.619672][ C2] default_idle_call+0x6d/0xb0 [ 65.619877][ C2] cpuidle_idle_call+0x1f4/0x280 [ 65.620011][ C2] do_idle+0xf9/0x160 [ 65.620115][ C2] cpu_startup_entry+0x54/0x60 [ 65.620249][ C2] start_secondary+0x21c/0x2b0 [ 65.620379][ C2] common_startup_64+0x12c/0x138 [ 65.620584][ C2] [ 65.620648][ C2] [ 65.620648][ C2] stack backtrace: [ 65.620812][ C2] CPU: 2 PID: 0 Comm: swapper/2 Not tainted 6.9.0-rc1-virtme #1 [ 65.621012][ C2] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS rel-1.16.3-0-ga6ed6b701f0a-prebuilt.qemu.org 04/01/2014 [ 65.621382][ C2] Call Trace: [ 65.621484][ C2] [ 65.621565][ C2] dump_stack_lvl+0x82/0xd0 [ 65.621700][ C2] print_irq_inversion_bug.part.0+0x3d9/0x570 [ 65.621934][ C2] ? kernel_text_address+0x17/0xe0 [ 65.622072][ C2] ? __pfx_print_irq_inversion_bug.part.0+0x10/0x10 [ 65.622232][ C2] ? __pfx_usage_skip+0x10/0x10 [ 65.622438][ C2] ? __pfx_usage_match+0x10/0x10 [ 65.622574][ C2] ? arch_stack_walk+0x8c/0xf0 [ 65.622709][ C2] mark_lock_irq+0x4cd/0xa10 [ 65.622838][ C2] ? common_startup_64+0x12c/0x138 [ 65.622968][ C2] ? __pfx_mark_lock_irq+0x10/0x10 [ 65.623173][ C2] ? __pfx_stack_trace_save+0x10/0x10 [ 65.623305][ C2] ? save_trace+0x8f/0x5b0 [ 65.623438][ C2] mark_lock+0x28d/0x3e0 [ 65.623543][ C2] ? __pfx_stack_trace_save+0x10/0x10 [ 65.623755][ C2] mark_usage+0xd9/0x2a0 [ 65.623860][ C2] __lock_acquire+0x797/0x1570 [ 65.623992][ C2] ? __pfx_validate_chain+0x10/0x10 [ 65.624123][ C2] ? __pfx_addrconf_rs_timer+0x10/0x10 [ 65.624251][ C2] lock_acquire.part.0+0xeb/0x330 [ 65.624458][ C2] ? addrconf_rs_timer+0xa9/0x730 [ 65.624592][ C2] ? __pfx_lock_acquire.part.0+0x10/0x10 [ 65.624723][ C2] ? trace_lock_acquire+0x135/0x1c0 [ 65.624857][ C2] ? addrconf_rs_timer+0xa9/0x730 [ 65.625063][ C2] ? lock_acquire+0x32/0xc0 [ 65.625193][ C2] ? addrconf_rs_timer+0xa9/0x730 [ 65.625322][ C2] _raw_write_lock+0x30/0x40 [ 65.625453][ C2] ? addrconf_rs_timer+0xa9/0x730 [ 65.625665][ C2] addrconf_rs_timer+0xa9/0x730 [ 65.625794][ C2] ? __pfx_lock_acquire.part.0+0x10/0x10 [ 65.625927][ C2] ? __pfx_addrconf_rs_timer+0x10/0x10 [ 65.626056][ C2] ? call_timer_fn+0xe8/0x230 [ 65.626265][ C2] ? lock_acquire+0x32/0xc0 [ 65.626397][ C2] ? call_timer_fn+0xe8/0x230 [ 65.626529][ C2] ? __pfx_addrconf_rs_timer+0x10/0x10 [ 65.626662][ C2] call_timer_fn+0x13b/0x230 [ 65.626791][ C2] ? call_timer_fn+0xe8/0x230 [ 65.626996][ C2] ? call_timer_fn+0xe8/0x230 [ 65.627128][ C2] ? __pfx_call_timer_fn+0x10/0x10 [ 65.627259][ C2] ? mark_lock+0x38/0x3e0 [ 65.627367][ C2] __run_timers+0x545/0x810 [ 65.627646][ C2] ? __pfx_addrconf_rs_timer+0x10/0x10 [ 65.627776][ C2] ? __pfx___run_timers+0x10/0x10 [ 65.627910][ C2] ? do_raw_spin_lock+0x131/0x270 [ 65.628041][ C2] ? __pfx_do_raw_spin_lock+0x10/0x10 [ 65.628245][ C2] ? lock_acquire+0x32/0xc0 [ 65.628377][ C2] ? run_timer_softirq+0xe0/0x1b0 [ 65.628510][ C2] run_timer_softirq+0xe8/0x1b0 [ 65.628647][ C2] __do_softirq+0x1f8/0x5df [ 65.628788][ C2] irq_exit_rcu+0x97/0xc0 [ 65.628962][ C2] sysvec_apic_timer_interrupt+0x75/0x80 [ 65.629090][ C2] [ 65.629157][ C2] [ 65.629224][ C2] asm_sysvec_apic_timer_interrupt+0x1a/0x20 [ 65.629384][ C2] RIP: 0010:default_idle+0xf/0x20 [ 65.629618][ C2] Code: 4c 01 c7 4c 29 c2 e9 72 ff ff ff 90 90 90 90 90 90 90 90 90 90 90 90 90 90 90 90 f3 0f 1e fa 66 90 0f 00 2d a3 30 35 00 fb f4 c3 cc cc cc cc 66 66 2e 0f 1f 84 00 00 00 00 00 90 90 90 90 90 [ 65.630081][ C2] RSP: 0018:ffffc9000015fdf8 EFLAGS: 00000242 [ 65.630322][ C2] RAX: 0000000000180ce3 RBX: 1ffff9200002bfc1 RCX: ffffffffb490e9d5 [ 65.630514][ C2] RDX: 0000000000000000 RSI: 0000000000000000 RDI: ffffffffb22aa4a4 [ 65.630708][ C2] RBP: 0000000000000000 R08: 0000000000000001 R09: ffffed1006c2709c [ 65.630978][ C2] R10: ffff8880361384e3 R11: ffff88803612d9b0 R12: 0000000000000000 [ 65.631176][ C2] R13: ffff888001b7c5c0 R14: dffffc0000000000 R15: 0000000000000000 [ 65.631446][ C2] ? ct_kernel_exit.constprop.0+0xc5/0xf0 [ 65.631577][ C2] ? cpuidle_idle_call+0x1f4/0x280 [ 65.631706][ C2] default_idle_call+0x6d/0xb0 [ 65.631834][ C2] cpuidle_idle_call+0x1f4/0x280 [ 65.631964][ C2] ? __pfx_cpuidle_idle_call+0x10/0x10 [ 65.632170][ C2] ? tsc_verify_tsc_adjust+0x5e/0x2b0 [ 65.632301][ C2] do_idle+0xf9/0x160 [ 65.632400][ C2] cpu_startup_entry+0x54/0x60 [ 65.632534][ C2] start_secondary+0x21c/0x2b0 [ 65.632738][ C2] ? __pfx_start_secondary+0x10/0x10 [ 65.632869][ C2] common_startup_64+0x12c/0x138 [ 65.633001][ C2] [ 67.301477][ T516] br0: port 2(veth3) entered disabled state [ 67.496487][ T518] br0: port 1(veth1) entered disabled state [ 67.691845][ T520] veth3: left allmulticast mode [ 67.692009][ T520] veth3: left promiscuous mode [ 67.692266][ T520] br0: port 2(veth3) entered disabled state [ 67.693268][ T520] veth1: left allmulticast mode [ 67.693403][ T520] veth1: left promiscuous mode [ 67.693631][ T520] br0: port 1(veth1) entered disabled state