View Issue Details
ID | Project | Category | View Status | Date Submitted | Last Update |
---|---|---|---|---|---|
0003136 | Rocky-Linux-9 | General | public | 2023-05-02 21:15 | 2025-06-06 14:20 |
Reporter | Graham Leggett | Assigned To | Neil Hanlon | ||
Priority | normal | Severity | major | Reproducibility | random |
Status | acknowledged | Resolution | open | ||
OS | Rocky | OS Version | 9.1 | ||
Summary | 0003136: IPv6 unreliable - Failed to download metadata for repo 'baseos' | ||||
Description | IPv6 sometimes works, sometimes does not. From time to time on an IPv6 only host, dnf fails as follows: Rocky Linux 9 - BaseOS 0.0 B/s | 0 B 00:40 Errors during downloading metadata for repository 'baseos': - Curl error (6): Couldn't resolve host name for https://mirrors.rockylinux.org/mirrorlist?arch=x86_64&repo=BaseOS-9 [Could not resolve host: mirrors.rockylinux.org] Error: Failed to download metadata for repo 'baseos': Cannot prepare internal mirrorlist: Curl error (6): Couldn't resolve host name for https://mirrors.rockylinux.org/mirrorlist?arch=x86_64&repo=BaseOS-9 [Could not resolve host: mirrors.rockylinux.org] | ||||
Steps To Reproduce | Run "dnf update" on an IPv6 only host. | ||||
Tags | No tags attached. | ||||
Looks like the failure is limited to when the unbound resolver is locally installed and used. | |
We've had some intermittent reports of this but have not been able to nail down a root cause. How is your IPv6 traffic handled? Is it a direct allocation, or via a 646 tunnel broker like HE.net? Do you have any firewall rules on your network that would be preventing ICMP6 and/or PMTU traffic from flowing? |
|
It's an IPv6 only host in a datacentre (specifically Hetzner in DE). IPv6 works if the DNS points at a Mikrotik router. As soon as DNS is changed to point at a locally installed copy of the unbound nameserver, we fail. Looking at https://dnssec-analyzer.verisignlabs.com/mirrors.rockylinux.org it seems fastly fails DNSSEC checks. I wonder if resolvers are interpreting this as "DNS name not configured securely, pretend it does not exist". |
|
Thanks for the info, that's helpful. Let me reach out to Fastly and see if they have some thoughts here. DNSSEC is an ongoing project for us as an infrastructure team, too, but I'm not able to give an ETA at this time. |
|
In theory DNSSEC missing isn't an error, but if DNSSEC is misconfigured then resolution attempts will throw an error. Very odd that a CDN doesn't do DNSSEC. |
|
We have also been working on an ipv6 only network and had this problem on all of our rockylinux installations. We were running Nat64 with ipv6 on the inside and only ipv4 on the external network connection. By simply using the "-6" parameter with dnf, we succesfully updated without any further issues. I.E. "dnf -6 update" Hope this helps someone. |
|
The issue has reappeared. Last week it was still working. Now it's broken: Error: Failed to download metadata for repo 'devel': Cannot prepare internal mirrorlist: No URLs in mirrorlist |
|
ignore my last comment - this was supposed for a different issue report | |
Some more info after this happened again. As per https://dnsviz.net/d/mirrors.rockylinux.org/dnssec/ the DNS name mirrors.rockylinux.org is hosted on DNS server that support both IPv6 and IPv4. The CNAME however points at Fastly, which for some bizarre reason has DNS that only supports IPv4 in 2025. The IPv6 only host cannot reach the IPv4 only DNS, and so we get "Could not resolve host". Fastly needs to fix their DNS. |
|
I recall this being an issue in CentOS 6 and/or 7. The preferred addressing is IPv6 when it's enabled. Is there something in the path that isn't IPv6-enabled, which is dropping the requests before they reach Fastly. If you run ''mtr -6 mirrors.rockylinux.org'', it should show when the break is happening. If the break is happening with Unbound locally, I'd love to see how it's configured. I notice that Fastly has IPv6 addresses when querying public resolvers. I'm quite curious about what you see in ''mtr'' and how DNS is configured in your environment. Can you share any of that info? |
|
Look closely at the very last entry on the bottom right of https://dnsviz.net/d/mirrors.rockylinux.org/dnssec/ - you see the name rockylinux.map.fastly.net has an AAAA record, but this AAAA record is only present on a series of four DNS servers, all four of which have IPv4 addresses, and zero of which have IPv6 addresses like everything else. An IPv6-only name server cannot contact the IPv4-only name servers of rockylinux.map.fastly.net, and so the lookup returns SERVFAIL. Lots of people don't see this because while the host might be IPv6-only, and the network might be IPv6 only, if the DNS is dual stack then the IPv4 addresses are reachable and the request "works". If you have a pure IPv6 host, a pure IPv6 network, AND a pure IPv6 DNS server, you get SERVFAIL. |
|
One further detail - has Rockylinux followed the instructions here: https://www.fastly.com/documentation/guides/full-site-delivery/domains-and-origins/enabling-dualstack-connections/ "Enabling dualstack on customer-specific hostnames If you have a customer-specific hostname, contact support and we'll provide you with a parallel IPv6 map or enable dualstack on your current one." |
|
Date Modified | Username | Field | Change |
---|---|---|---|
2023-05-02 21:15 | Graham Leggett | New Issue | |
2023-05-03 10:09 | Graham Leggett | Note Added: 0003169 | |
2023-05-03 12:37 | Neil Hanlon | Assigned To | => Neil Hanlon |
2023-05-03 12:37 | Neil Hanlon | Status | new => acknowledged |
2023-05-03 12:37 | Neil Hanlon | Note Added: 0003170 | |
2023-05-03 13:32 | Graham Leggett | Note Added: 0003171 | |
2023-05-03 14:03 | Neil Hanlon | Note Added: 0003172 | |
2023-05-03 14:50 | Graham Leggett | Note Added: 0003173 | |
2023-05-16 15:23 | Zulu Echo | Note Added: 0003334 | |
2025-05-05 10:43 | Ronald Wahl | Note Added: 0010033 | |
2025-05-05 10:46 | Ronald Wahl | Note Added: 0010034 | |
2025-06-04 12:28 | Graham Leggett | Note Added: 0010363 | |
2025-06-04 15:06 | Chris Short | Note Added: 0010396 | |
2025-06-04 17:14 | Graham Leggett | Note Added: 0010429 | |
2025-06-06 14:20 | Graham Leggett | Note Added: 0010462 |