View Issue Details

IDProjectCategoryView StatusLast Update
0003136Rocky-Linux-9Generalpublic2025-06-06 14:20
ReporterGraham Leggett Assigned ToNeil Hanlon  
PrioritynormalSeveritymajorReproducibilityrandom
Status acknowledgedResolutionopen 
OSRockyOS Version9.1 
Summary0003136: IPv6 unreliable - Failed to download metadata for repo 'baseos'
DescriptionIPv6 sometimes works, sometimes does not.

From time to time on an IPv6 only host, dnf fails as follows:

Rocky Linux 9 - BaseOS 0.0 B/s | 0 B 00:40
Errors during downloading metadata for repository 'baseos':
  - Curl error (6): Couldn't resolve host name for https://mirrors.rockylinux.org/mirrorlist?arch=x86_64&repo=BaseOS-9 [Could not resolve host: mirrors.rockylinux.org]
Error: Failed to download metadata for repo 'baseos': Cannot prepare internal mirrorlist: Curl error (6): Couldn't resolve host name for https://mirrors.rockylinux.org/mirrorlist?arch=x86_64&repo=BaseOS-9 [Could not resolve host: mirrors.rockylinux.org]
Steps To ReproduceRun "dnf update" on an IPv6 only host.
TagsNo tags attached.

Activities

Graham Leggett

Graham Leggett

2023-05-03 10:09

reporter   ~0003169

Looks like the failure is limited to when the unbound resolver is locally installed and used.
Neil Hanlon

Neil Hanlon

2023-05-03 12:37

administrator   ~0003170

We've had some intermittent reports of this but have not been able to nail down a root cause.

How is your IPv6 traffic handled? Is it a direct allocation, or via a 646 tunnel broker like HE.net?

Do you have any firewall rules on your network that would be preventing ICMP6 and/or PMTU traffic from flowing?
Graham Leggett

Graham Leggett

2023-05-03 13:32

reporter   ~0003171

It's an IPv6 only host in a datacentre (specifically Hetzner in DE).

IPv6 works if the DNS points at a Mikrotik router. As soon as DNS is changed to point at a locally installed copy of the unbound nameserver, we fail.

Looking at https://dnssec-analyzer.verisignlabs.com/mirrors.rockylinux.org it seems fastly fails DNSSEC checks. I wonder if resolvers are interpreting this as "DNS name not configured securely, pretend it does not exist".
Neil Hanlon

Neil Hanlon

2023-05-03 14:03

administrator   ~0003172

Thanks for the info, that's helpful.

Let me reach out to Fastly and see if they have some thoughts here. DNSSEC is an ongoing project for us as an infrastructure team, too, but I'm not able to give an ETA at this time.
Graham Leggett

Graham Leggett

2023-05-03 14:50

reporter   ~0003173

In theory DNSSEC missing isn't an error, but if DNSSEC is misconfigured then resolution attempts will throw an error.

Very odd that a CDN doesn't do DNSSEC.
Zulu Echo

Zulu Echo

2023-05-16 15:23

reporter   ~0003334

We have also been working on an ipv6 only network and had this problem on all of our rockylinux installations. We were running Nat64 with ipv6 on the inside and only ipv4 on the external network connection.

By simply using the "-6" parameter with dnf, we succesfully updated without any further issues.
I.E. "dnf -6 update"

Hope this helps someone.
Ronald Wahl

Ronald Wahl

2025-05-05 10:43

reporter   ~0010033

The issue has reappeared. Last week it was still working. Now it's broken:

Error: Failed to download metadata for repo 'devel': Cannot prepare internal mirrorlist: No URLs in mirrorlist
Ronald Wahl

Ronald Wahl

2025-05-05 10:46

reporter   ~0010034

ignore my last comment - this was supposed for a different issue report
Graham Leggett

Graham Leggett

2025-06-04 12:28

reporter   ~0010363

Some more info after this happened again.

As per https://dnsviz.net/d/mirrors.rockylinux.org/dnssec/ the DNS name mirrors.rockylinux.org is hosted on DNS server that support both IPv6 and IPv4.

The CNAME however points at Fastly, which for some bizarre reason has DNS that only supports IPv4 in 2025.

The IPv6 only host cannot reach the IPv4 only DNS, and so we get "Could not resolve host".

Fastly needs to fix their DNS.
Chris Short

Chris Short

2025-06-04 15:06

reporter   ~0010396

I recall this being an issue in CentOS 6 and/or 7. The preferred addressing is IPv6 when it's enabled. Is there something in the path that isn't IPv6-enabled, which is dropping the requests before they reach Fastly. If you run ''mtr -6 mirrors.rockylinux.org'', it should show when the break is happening. If the break is happening with Unbound locally, I'd love to see how it's configured.

I notice that Fastly has IPv6 addresses when querying public resolvers. I'm quite curious about what you see in ''mtr'' and how DNS is configured in your environment. Can you share any of that info?
Graham Leggett

Graham Leggett

2025-06-04 17:14

reporter   ~0010429

Look closely at the very last entry on the bottom right of https://dnsviz.net/d/mirrors.rockylinux.org/dnssec/ - you see the name rockylinux.map.fastly.net has an AAAA record, but this AAAA record is only present on a series of four DNS servers, all four of which have IPv4 addresses, and zero of which have IPv6 addresses like everything else.

An IPv6-only name server cannot contact the IPv4-only name servers of rockylinux.map.fastly.net, and so the lookup returns SERVFAIL.

Lots of people don't see this because while the host might be IPv6-only, and the network might be IPv6 only, if the DNS is dual stack then the IPv4 addresses are reachable and the request "works".

If you have a pure IPv6 host, a pure IPv6 network, AND a pure IPv6 DNS server, you get SERVFAIL.
Graham Leggett

Graham Leggett

2025-06-06 14:20

reporter   ~0010462

One further detail - has Rockylinux followed the instructions here:

https://www.fastly.com/documentation/guides/full-site-delivery/domains-and-origins/enabling-dualstack-connections/

"Enabling dualstack on customer-specific hostnames
If you have a customer-specific hostname, contact support and we'll provide you with a parallel IPv6 map or enable dualstack on your current one."

Issue History

Date Modified Username Field Change
2023-05-02 21:15 Graham Leggett New Issue
2023-05-03 10:09 Graham Leggett Note Added: 0003169
2023-05-03 12:37 Neil Hanlon Assigned To => Neil Hanlon
2023-05-03 12:37 Neil Hanlon Status new => acknowledged
2023-05-03 12:37 Neil Hanlon Note Added: 0003170
2023-05-03 13:32 Graham Leggett Note Added: 0003171
2023-05-03 14:03 Neil Hanlon Note Added: 0003172
2023-05-03 14:50 Graham Leggett Note Added: 0003173
2023-05-16 15:23 Zulu Echo Note Added: 0003334
2025-05-05 10:43 Ronald Wahl Note Added: 0010033
2025-05-05 10:46 Ronald Wahl Note Added: 0010034
2025-06-04 12:28 Graham Leggett Note Added: 0010363
2025-06-04 15:06 Chris Short Note Added: 0010396
2025-06-04 17:14 Graham Leggett Note Added: 0010429
2025-06-06 14:20 Graham Leggett Note Added: 0010462