In this episode of PING, APNIC Chief Scientist Geoff Huston and I discuss a pattern of DNS traffic that Geoff has observed through APNIC Labs' advertising-based measurement system.

Nearly every DNS query Geoff sees appears twice, sometimes more. On a typical day, APNIC Labs observes around 150 million unique DNS labels, yet the system records approximately 270 million incoming DNS queries. That's a significant amount of duplication.

So what's going on?

The advertising-based measurement system relies on issuing unique DNS queries, which, in turn, generate unique web page serves. This allows APNIC Labs to measure Internet-wide behaviour across around 30 million browsers, games, and devices each day.

That said, these DNS names and web URLs are not necessarily seen by only a single entity. Intermediary systems such as caches and proxies can observe and reissue requests, and previously opened browser tabs may revisit older URLs. As a result, it is not uncommon for the same DNS name or web URL to be seen more than once.

The point is that after the 'first' fetch, the subsequent fetches can usually be held to be re-presentations of the same experiment and can therefore often be discarded (in the case of the web). In contrast, DNS has always operated with an element of unreliable transport. Features such as DNSSEC can generate SERVFAIL responses and force repeated attempts to obtain a name-to-address mapping. In this environment, re-fetching is not merely repetition; it is itself an aspect of the system under test. How many resolvers lie behind a given user's system? How many kinds of resolver (DNSSEC enabled, or not) does the user depend on?

Finding that almost all DNS queries are repeated, often within a very short interval, prompted Geoff to investigate what was happening under the covers. The results revealed several recurring patterns, which we discuss in this episode of PING.

Read more about the findings:

You can stream and subscribe to PING via the following channels:

If you're interested in sharing your insights or research, please get in touch - we're always looking for great stories from the community. Please let us know what you think of the podcast and the APNIC Blog so we can keep improving.

The views expressed by the authors of this blog are their own and do not necessarily reflect the views of APNIC. Please note a Code of Conduct applies to this blog.