[ 10.846380][ T249] ip (249) used greatest stack depth: 24336 bytes left [ 12.664588][ T286] ip (286) used greatest stack depth: 24080 bytes left [ 14.652881][ T313] gre: GRE over IPv4 demultiplexor driver [ 14.667738][ T313] ip_gre: GRE over IPv4 tunneling driver [ 15.387649][ T323] GACT probability NOT on [ 15.647158][ T325] ip6_gre: GRE over IPv6 tunneling driver [ 15.803911][ T282] ip6_tunnel: h3-gt6 xmit: Local address not yet configured! [ 15.934880][ T282] ip6_tunnel: h3-gt6 xmit: Local address not yet configured! [ 16.012638][ T11] ip6_tunnel: h3-gt6 xmit: Local address not yet configured! [ 16.074916][ T282] ip6_tunnel: h3-gt6 xmit: Local address not yet configured! [ 16.802755][ T339] br1: port 1(veth1) entered blocking state [ 16.803262][ T339] br1: port 1(veth1) entered disabled state [ 16.804046][ T339] veth1: entered allmulticast mode [ 16.806226][ T339] veth1: entered promiscuous mode [ 16.907732][ T45] br1: port 1(veth1) entered blocking state [ 16.908218][ T45] br1: port 1(veth1) entered forwarding state [ 16.985660][ T341] br1: port 2(veth2) entered blocking state [ 16.985986][ T341] br1: port 2(veth2) entered disabled state [ 16.986287][ T341] veth2: entered allmulticast mode [ 16.988288][ T341] veth2: entered promiscuous mode [ 17.034766][ T11] ip6_tunnel: h3-gt6 xmit: Local address not yet configured! [ 17.035632][ T11] ip6_tunnel: h3-gt6 xmit: Local address not yet configured! [ 17.081232][ T45] br1: port 2(veth2) entered blocking state [ 17.081572][ T45] br1: port 2(veth2) entered forwarding state [ 17.154564][ T251] ip6_tunnel: h3-gt6 xmit: Local address not yet configured! [ 17.594587][ T62] ip6_tunnel: gt6 xmit: Local address not yet configured! [ 17.899095][ T62] ip6_tunnel: gt6 xmit: Local address not yet configured! [ 18.379651][ T11] ip6_tunnel: gt6 xmit: Local address not yet configured! [ 21.066633][ C2] ip6_tunnel: h3-gt6 xmit: Local address not yet configured! [ 24.228257][ T380] Mirror/redirect action on [ 29.643069][ C1] ip6_tunnel: h3-gt6 xmit: Local address not yet configured! [ 38.222555][ C2] tc mirred to Houston: device gt4 is down [ 38.327140][ C2] tc mirred to Houston: device gt4 is down [ 38.428303][ C2] tc mirred to Houston: device gt4 is down [ 38.529539][ C2] tc mirred to Houston: device gt4 is down [ 38.630885][ C2] tc mirred to Houston: device gt4 is down [ 38.732114][ C2] tc mirred to Houston: device gt4 is down [ 38.833334][ C2] tc mirred to Houston: device gt4 is down [ 38.934594][ C2] tc mirred to Houston: device gt4 is down [ 39.035676][ C2] tc mirred to Houston: device gt4 is down [ 39.136836][ C2] tc mirred to Houston: device gt4 is down [ 47.051722][ C1] ip6_tunnel: h3-gt6 xmit: Local address not yet configured! [ 47.509401][ C0] net_ratelimit: 10 callbacks suppressed [ 47.509412][ C0] tc mirred to Houston: device gt6 is down [ 47.611418][ C0] tc mirred to Houston: device gt6 is down [ 47.712880][ C0] tc mirred to Houston: device gt6 is down [ 47.814336][ C0] tc mirred to Houston: device gt6 is down [ 47.927052][ C0] tc mirred to Houston: device gt6 is down [ 48.028641][ C0] tc mirred to Houston: device gt6 is down [ 48.130014][ C0] tc mirred to Houston: device gt6 is down [ 48.231111][ C0] tc mirred to Houston: device gt6 is down [ 48.332303][ C0] tc mirred to Houston: device gt6 is down [ 48.433564][ C0] tc mirred to Houston: device gt6 is down [ 57.802607][ C2] ip6_tunnel: gt6 xmit: Local address not yet configured! [ 71.534451][ C2] ip6_tunnel: gt6 xmit: Local address not yet configured! [ 71.636595][ C2] ip6_tunnel: gt6 xmit: Local address not yet configured! [ 71.738194][ C2] ip6_tunnel: gt6 xmit: Local address not yet configured! [ 71.839599][ C2] ip6_tunnel: gt6 xmit: Local address not yet configured! [ 71.940766][ C2] ip6_tunnel: gt6 xmit: Local address not yet configured! [ 72.042049][ C2] ip6_tunnel: gt6 xmit: Local address not yet configured! [ 72.143285][ C2] ip6_tunnel: gt6 xmit: Local address not yet configured! [ 72.244709][ C2] ip6_tunnel: gt6 xmit: Local address not yet configured! [ 72.346066][ C2] ip6_tunnel: gt6 xmit: Local address not yet configured! [ 72.447411][ C2] ip6_tunnel: gt6 xmit: Local address not yet configured! [ 76.746854][ C1] ip6_tnl_xmit_ctl: 11 callbacks suppressed [ 76.747430][ C1] ip6_tunnel: gt6 xmit: Local address not yet configured! [ 83.402620][ C3] ip6_tunnel: h3-gt6 xmit: Local address not yet configured! [ 109.092450][ C2] tc mirred: target device is gone [ 153.034722][ C2] ip6_tunnel: h3-gt6 xmit: Local address not yet configured! [ 160.562672][ T1039] br1: port 1(veth1) entered disabled state [ 160.631521][ T1040] br1: port 2(veth2) entered disabled state [ 160.717838][ T1041] veth2: left allmulticast mode [ 160.718147][ T1041] veth2: left promiscuous mode [ 160.718505][ T1041] br1: port 2(veth2) entered disabled state [ 160.721098][ T1041] veth1: left allmulticast mode [ 160.721301][ T1041] veth1: left promiscuous mode [ 160.721634][ T1041] br1: port 1(veth1) entered disabled state [ 160.996281][ T1043] [ 160.996484][ T1043] ============================================ [ 160.996707][ T1043] WARNING: possible recursive locking detected [ 160.996911][ T1043] 6.13.0-rc5-virtme #1 Not tainted [ 160.997077][ T1043] -------------------------------------------- [ 160.997279][ T1043] ip/1043 is trying to acquire lock: [ 160.997440][ T1043] ffffffff8b7d6630 ((netdev_chain).rwsem){++++}-{4:4}, at: blocking_notifier_call_chain+0x50/0x90 [ 160.997775][ T1043] [ 160.997775][ T1043] but task is already holding lock: [ 160.998014][ T1043] ffffffff8b7d6630 ((netdev_chain).rwsem){++++}-{4:4}, at: blocking_notifier_call_chain+0x50/0x90 [ 160.998370][ T1043] [ 160.998370][ T1043] other info that might help us debug this: [ 160.998613][ T1043] Possible unsafe locking scenario: [ 160.998613][ T1043] [ 160.998875][ T1043] CPU0 [ 160.999000][ T1043] ---- [ 160.999117][ T1043] lock((netdev_chain).rwsem); [ 160.999279][ T1043] lock((netdev_chain).rwsem); [ 160.999461][ T1043] [ 160.999461][ T1043] *** DEADLOCK *** [ 160.999461][ T1043] [ 160.999700][ T1043] May be due to missing lock nesting notation [ 160.999700][ T1043] [ 160.999933][ T1043] 2 locks held by ip/1043: [ 161.000101][ T1043] #0: ffffffff8b7e7a48 (rtnl_mutex){+.+.}-{4:4}, at: rtnl_dellink+0x228/0xa40 [ 161.000399][ T1043] #1: ffffffff8b7d6630 ((netdev_chain).rwsem){++++}-{4:4}, at: blocking_notifier_call_chain+0x50/0x90 [ 161.000745][ T1043] [ 161.000745][ T1043] stack backtrace: [ 161.000952][ T1043] CPU: 2 UID: 0 PID: 1043 Comm: ip Not tainted 6.13.0-rc5-virtme #1 [ 161.001203][ T1043] Hardware name: Bochs Bochs, BIOS Bochs 01/01/2011 [ 161.001415][ T1043] Call Trace: [ 161.001544][ T1043] [ 161.001630][ T1043] dump_stack_lvl+0x82/0xd0 [ 161.001800][ T1043] print_deadlock_bug+0x40a/0x650 [ 161.001975][ T1043] validate_chain+0x5bf/0xae0 [ 161.002138][ T1043] ? __pfx_validate_chain+0x10/0x10 [ 161.002309][ T1043] ? hlock_class+0x4e/0x130 [ 161.002471][ T1043] ? mark_lock+0x38/0x3e0 [ 161.002601][ T1043] __lock_acquire+0xb9a/0x1680 [ 161.002763][ T1043] lock_acquire.part.0+0xeb/0x330 [ 161.002925][ T1043] ? blocking_notifier_call_chain+0x50/0x90 [ 161.003130][ T1043] ? netconsole_process_cleanups_core+0x2e/0x3d0 [ 161.003357][ T1043] ? __pfx_lock_acquire.part.0+0x10/0x10 [ 161.003523][ T1043] ? trace_lock_acquire+0x14c/0x1f0 [ 161.003696][ T1043] ? lock_acquire+0x32/0xc0 [ 161.003865][ T1043] ? blocking_notifier_call_chain+0x50/0x90 [ 161.004074][ T1043] down_read+0x9f/0x340 [ 161.004200][ T1043] ? blocking_notifier_call_chain+0x50/0x90 [ 161.004403][ T1043] ? __pfx_down_read+0x10/0x10 [ 161.004576][ T1043] ? __lock_release+0x103/0x460 [ 161.004738][ T1043] ? netconsole_netdev_event+0x2c1/0x620 [ 161.004905][ T1043] ? __pfx___lock_release+0x10/0x10 [ 161.005084][ T1043] blocking_notifier_call_chain+0x50/0x90 [ 161.005246][ T1043] __netdev_upper_dev_unlink+0xd9/0x220 [ 161.005415][ T1043] ? __pfx___netdev_upper_dev_unlink+0x10/0x10 [ 161.005618][ T1043] ? __mutex_unlock_slowpath+0x14a/0x3d0 [ 161.005781][ T1043] ? __pfx___mutex_unlock_slowpath+0x10/0x10 [ 161.005983][ T1043] ? rtnl_is_locked+0x15/0x20 [ 161.006157][ T1043] netdev_upper_dev_unlink+0x71/0xa0 [ 161.006318][ T1043] ? __pfx_netdev_upper_dev_unlink+0x10/0x10 [ 161.006515][ T1043] ? netconsole_netdev_event+0x2c1/0x620 [ 161.006678][ T1043] ? mutex_is_locked+0x1c/0x60 [ 161.006846][ T1043] ? rtnl_is_locked+0x15/0x20 [ 161.007009][ T1043] ? netdev_master_upper_dev_get+0x13/0x120 [ 161.007211][ T1043] vrf_device_event+0x78/0xe0 [vrf] [ 161.007382][ T1043] notifier_call_chain+0xcd/0x150 [ 161.007547][ T1043] blocking_notifier_call_chain+0x66/0x90 [ 161.007709][ T1043] unregister_netdevice_many_notify+0xa7b/0x1580 [ 161.007910][ T1043] ? __pfx_unregister_netdevice_many_notify+0x10/0x10 [ 161.008113][ T1043] ? rtnl_dellink+0x228/0xa40 [ 161.008276][ T1043] ? sock_def_readable+0x11e/0x300 [ 161.008439][ T1043] ? __pfx___mutex_lock+0x10/0x10 [ 161.008605][ T1043] ? rtnl_is_locked+0x15/0x20 [ 161.008767][ T1043] ? unregister_netdevice_queue+0x70/0x410 [ 161.008969][ T1043] ? __pfx_unregister_netdevice_queue+0x10/0x10 [ 161.009169][ T1043] ? __nla_validate_parse+0x48/0x3d0 [ 161.009335][ T1043] rtnl_dellink+0x348/0xa40 [ 161.009500][ T1043] ? __pfx_rtnl_dellink+0x10/0x10 [ 161.009677][ T1043] ? find_held_lock+0x2c/0x110 [ 161.009855][ T1043] ? __lock_release+0x103/0x460 [ 161.010017][ T1043] ? rtnetlink_rcv_msg+0x6ef/0xc10 [ 161.010178][ T1043] ? __pfx___lock_release+0x10/0x10 [ 161.010339][ T1043] ? __pfx_lock_acquire.part.0+0x10/0x10 [ 161.010501][ T1043] ? rtnetlink_rcv_msg+0x6ef/0xc10 [ 161.010662][ T1043] ? __pfx_rtnl_dellink+0x10/0x10 [ 161.010825][ T1043] rtnetlink_rcv_msg+0x712/0xc10 [ 161.010989][ T1043] ? __pfx_rtnetlink_rcv_msg+0x10/0x10 [ 161.011151][ T1043] ? hlock_class+0x4e/0x130 [ 161.011320][ T1043] ? mark_lock+0x38/0x3e0 [ 161.011446][ T1043] ? __lock_acquire+0xb9a/0x1680 [ 161.011608][ T1043] netlink_rcv_skb+0x130/0x360 [ 161.011769][ T1043] ? __pfx_rtnetlink_rcv_msg+0x10/0x10 [ 161.011928][ T1043] ? __pfx_netlink_rcv_skb+0x10/0x10 [ 161.012096][ T1043] ? netlink_deliver_tap+0x13e/0x340 [ 161.012260][ T1043] ? netlink_deliver_tap+0xc3/0x340 [ 161.012421][ T1043] netlink_unicast+0x44b/0x710 [ 161.012581][ T1043] ? __pfx_netlink_unicast+0x10/0x10 [ 161.012748][ T1043] ? find_held_lock+0x2c/0x110 [ 161.012914][ T1043] netlink_sendmsg+0x723/0xbe0 [ 161.013077][ T1043] ? __pfx_netlink_sendmsg+0x10/0x10 [ 161.013245][ T1043] ____sys_sendmsg+0x7ac/0xa10 [ 161.013420][ T1043] ? __pfx_____sys_sendmsg+0x10/0x10 [ 161.013582][ T1043] ? __pfx_copy_msghdr_from_user+0x10/0x10 [ 161.013785][ T1043] ___sys_sendmsg+0xee/0x170 [ 161.013954][ T1043] ? __pfx____sys_sendmsg+0x10/0x10 [ 161.014117][ T1043] ? kasan_save_stack+0x34/0x50 [ 161.014281][ T1043] ? kasan_save_stack+0x24/0x50 [ 161.014441][ T1043] ? __kasan_record_aux_stack+0x8e/0xa0 [ 161.014609][ T1043] ? __call_rcu_common.constprop.0+0xa1/0x4b0 [ 161.014823][ T1043] ? __x64_sys_close+0x7c/0xd0 [ 161.014986][ T1043] ? do_syscall_64+0xc1/0x1d0 [ 161.015147][ T1043] ? entry_SYSCALL_64_after_hwframe+0x77/0x7f [ 161.015351][ T1043] ? __lock_acquire+0xb9a/0x1680 [ 161.015513][ T1043] ? find_held_lock+0x2c/0x110 [ 161.015685][ T1043] ? __lock_release+0x103/0x460 [ 161.015861][ T1043] ? __virt_addr_valid+0x22b/0x430 [ 161.016028][ T1043] ? __pfx___lock_release+0x10/0x10 [ 161.016199][ T1043] ? __pfx_lock_acquire.part.0+0x10/0x10 [ 161.016361][ T1043] __sys_sendmsg+0x109/0x1a0 [ 161.016532][ T1043] ? __pfx___sys_sendmsg+0x10/0x10 [ 161.016693][ T1043] ? __pfx_slab_free_after_rcu_debug+0x10/0x10 [ 161.016897][ T1043] do_syscall_64+0xc1/0x1d0 [ 161.017073][ T1043] entry_SYSCALL_64_after_hwframe+0x77/0x7f [ 161.017273][ T1043] RIP: 0033:0x7f87f6fff9a7 [ 161.017445][ T1043] Code: 0a 00 f7 d8 64 89 02 48 c7 c0 ff ff ff ff eb b9 0f 1f 00 f3 0f 1e fa 64 8b 04 25 18 00 00 00 85 c0 75 10 b8 2e 00 00 00 0f 05 <48> 3d 00 f0 ff ff 77 51 c3 48 83 ec 28 89 54 24 1c 48 89 74 24 10 [ 161.018017][ T1043] RSP: 002b:00007ffc5ac54078 EFLAGS: 00000246 ORIG_RAX: 000000000000002e [ 161.018273][ T1043] RAX: ffffffffffffffda RBX: 00007ffc5ac547a0 RCX: 00007f87f6fff9a7 [ 161.018515][ T1043] RDX: 0000000000000000 RSI: 00007ffc5ac540e0 RDI: 0000000000000005 [ 161.018757][ T1043] RBP: 0000000000000002 R08: 0000000000000003 R09: 0000000000000078 [ 161.019000][ T1043] R10: 00007f87f6ebdef8 R11: 0000000000000246 R12: 0000000000000002 [ 161.019254][ T1043] R13: 0000000067795583 R14: 0000000000498600 R15: 0000000000000000 [ 161.019515][ T1043] [ 161.037031][ T1043] ip (1043) used greatest stack depth: 23832 bytes left